IEEE Robotics & Automation Magazine - March 2012 - 100

*
for researchers is to present an evolution of a solution in distinct conference papers, culminating in a concise
final paper in a journal. Such papers
can be evaluated against the same
checklist as the previous category, with
the addition that the authors should
provide references to their earlier works
and explain its evolution. However,
although such papers appear to be a
summary of prior work, they should
present new insights or improvements;
the paper cannot be just a collection of
conference papers. The authors should
present clearly the innovation, but you,
as reviewer, will need to confirm this.
Some online reviewing management
systems instantly offer a search for
similar publications based on either the
title, keywords, or the authors.
Each point on the checklist is, in
fact, an indicator of the quality of the

contribution. An incomplete literature
overview or an overrepresentation of
the authors' own work provides weak
support for the paper. If the authors
fail to give a sufficient explanation of
why their solution is an improvement
of existing solutions, you should be
critical in assessing the contributing
value of the paper. Also, when the
paper seems to be just a summary of
previous works, you should consider
whether the paper contributes significantly to the conference or journal.
Furthermore, if the proposed solution
does not seem to have a solid theoretical basis, the proposed solution can,
and probably will, be questioned by
readers. Naturally, all this requires a
deep knowledge of your field and that
is why reviewing is a challenging task.
This leads back to the recommendations, in the previous SRP article, on

turning down a review request. In
assessing all of these points, the reference list obviously plays an important,
but often overlooked, role as its contents give you necessary insights. In a
future article, we will therefore give
you an overview of some important
tools for checking references.
Comments, concerns, and contributions are welcomed at SRP@ieeeras.org, or visit our Web site at http://
wiki.ieee-ras.org/ras srp.
Ludo Visser is a doctoral candidate
at the University of Twente, The
Netherlands, and was 2011 chair of
the RAS SAC Student Reviewer
Project. Tamas Haidegger is an
adjunct professor at the Budapest
Institute of Technology and Economic, Hungary, and was the 2010-
2011 Student Member of the RAS
AdCom.

The Reviewing Process: Validating Results
By Ignacio Galiana and Pablo Cerrada

Any scientific publication aims to
advance the field of knowledge that it
deals with, and therefore the editorial
staff will always be seeking the most
revolutionary papers among all of
those received. On the other hand, the
reviewers' task is usually a much more
conservative one, as reviewers are
responsible for verifying the realism
of the methods proposed and the
veracity of the claimed results.
As the prestige of any scientific
publication relies on both novelty and
veracity of the papers published in it,
validating the results presented in a
paper is a major concern in the reviewing process, and it is crucial to the
future of the publication. In this article,
we focus on how to assess whether the
results claimed by the authors are true,
if they are exaggerated or fake, or if they
have been copied from other works,
i.e., plagiarism. Not appreciating the
Digital Object Identifier 10.1109/MRA.2012.2190360
Date of publication: 26 March 2012

100

IEEE ROBOTICS & AUTOMATION MAGAZINE

importance of this can lead to one of
two extreme situations:
l an excellent article is rejected because
its results are unbelievable to the
reviewer
l an article that exaggerates the results
obtained, or in which these results
are made up or copied from another
author's work, is published.
Editors and reviewers should be
aware of the former, as a rejected paper
can be published in a competitor journal. A similar situation like this occurred
when Enrico Fermi first submitted his
work on the weak interaction theory of
beta decay. Nature rejected the paper
because it was considered unbelievable.
Five years later, it was finally accepted
when the results had already been
proved [1].
The latter scenario usually has a very
strong negative impact on the journal's
prestige and credibility, and often ends
in scandals when other researchers try to
replicate the results claimed by the
authors. Fortunately, these situations do

MARCH 2012

not occur very often, but there are some
recent examples that revealed some weak
points of the editorial system. Kazuhiro
Kosuge, past president of RAS, reported
in the March 2011 issue of this magazine
that, in 2010, serious plagiarism cases
occurred and announced that the IEEE
would take actions on the basis of IEEE
ethics policies in the future.
Because they erode the credibility of
the academic community, new methodologies should be implemented to
avoid these issues. With some publications, authors can attach multimedia
files to the submitted paper to support
the presented results or methodologies.
Although this can be a very helpful
resource for evaluating the realism of a
contribution, it is not very widely used;
however, in some cases, it may be insufficient. During the reviewing process,
the reviewer may encourage the authors
to add a multimedia file to their submission to validate their results.
(continued on page 102)

http://www.ras.org http://http:// http://wiki.ieee-ras.org/ras

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2012