IEEE Robotics & Automation Magazine - September 2015 - 111

the accomplishment of the goal of a motor skill, and human
likeness represents in what manner the task is executed, which
can or cannot be correlated with the level of accomplishment
of the goal. Each ability is associated to one or more benchmarks, which allows the quantitative measurement of the corresponding ability. To allow for truthful application across a
wide variety of bipedal systems, all benchmarks should be
made independent from weight and size.
Benchmarks of Performance
In our view, two features can describe performance: stability
and efficiency. We define stability as the ability to maintain
equilibrium during the execution of a motor skill. Loss of
equilibrium can be easily detected by the occurrence of a falling event. To assess stability within a single trial, we identified two benchmarks: time until falling and cycles until
falling. The time until falling should be used in all static postural conditions (e.g., quiet standing on a static surface) because the detection of a cycle cannot be easily determined.
The number of cycles (e.g., walking stride cycle or tilting
platform cycle) is more suitable during dynamic conditions,
or when robots with different sizes are considered, because of
the influence of speed and size on time. To measure stability
across different trials, the success rate should also be measured. Another benchmark of the stability is the ability of
maintaining the center of mass (CoM) above the polygon of
support, reflecting what Fleishman referred to as gross body
equilibrium. The ability can be measured analytically by the
energy stability margin (ESM) [11], or by identifying the
maximum accepted disturbance, in terms of amplitude and
frequency. Measuring the energy efficiency of robots and humans can be done by the specific cost of transport ^c t h [12],
[13], defined as the ratio of the energy consumed and the
weight times the distance traveled. In robotics, to isolate the
effectiveness of the mechanical design and controller from
the efficiency of the actuators, the specific energetic cost of
transport ^c eth , comprises the total energy consumed, and
specific mechanical cost of transport ^c mth, which only considers the positive mechanical work of the actuation system,
have been introduced. Another way to assess the energetic
aspects of locomotion has been recently introduced with the
concept of passivity, defined as the ability of optimizing the
use of gravity and inertia to move the body forward. The resulting passive gait measure (PGM) [14] appears to be a potential benchmark because of its practical use in robotic and
human scenarios. Another aspect of efficiency is the ability
of reacting promptly to an external command or perturbation, usually referred to as reaction time.
Benchmarks of Human Likeness
Human likeness is a term widely used in humanoid robot
community to define the similarity with human behavior.
The concept of healthy behavior is used instead in the fields
of wearable robotics and human biomechanics. In
our scheme, we propose to maintain the term human likeness, due to its conciseness. To translate this concept into a

number of abilities and related benchmark, we divided
human likeness into two categories, kinematics and dynamics (see Figure 5).
Under the kinematics category, we included all the abilities that can be analyzed by observing only the motion
of the body. We have identified three further subcategories
as follows.
● Whole body motion can be generally described by the
motion of the CoM and compared with humans through
correlation techniques, such as dynamic time warping
[15]. Recently, other techniques for global movement assessment have been introduced, such as the gait harmony
[16]. In the specific case of posture, the human-like wholebody sway is commonly considered [17], [18]. Global motion can be also assessed by visual inspection from human
observers [19].
● Individual joint motion can be easily measured and compared with healthy humans [15]. Foot motion is also a crucial aspect in walking. In the ideal benchmarking scheme,
the assessment of basic wheel-like mechanisms of the
foot-i.e., heel, ankle, and forefoot rockers-should be included [20], [21].
● Coordination, which includes interlimb coordination, such
as symmetry [22] and trunk/arm motions used for regulating body momentum [23]; and intralimb coordination, i.e.,
ankle-knee-foot synergies [24].
The category of dynamics
includes all the abilities
that are correlated with
Benchmarking not only
forces behind movements. The ground reacallows the assessment
tion forces are mostly
used as a descriptor of the
and comparison of the
global kinetics of the
body. Beyond the direct
performance of different
measurement of forces,
some other interesting
technologies but also
features related to dynamics can be considered
defines and supports
and assessed. One of
them is the dynamic simthe standardization and
ilarity, introduced by Alexander et al. [25], which
regulation processes.
is defined with the following criteria: 1) geometric similarity, 2) equal phase relationships, 3) equal duty
factors, 4) equal relative stride lengths, 5) equal relative
ground reaction forces, and 6) equal relative mechanical
power outputs. They verified experimentally that differentsized animals meet these six criteria when they move with
the same Froude number. Therefore, the Froude number can
be taken as a compact way to describe dynamic similarity between a robot and human, irrespective to size [26]. Mummolo et al. [14] recently proposed an indicator of dynamicity,
i.e., the dynamic gait measure (DGM), defined as the ability
of a legged system to maintain dynamic stability while
September 2015

IEEE ROBOTICS & AUTOMATION MAGAZINE

111

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2015