IEEE Robotics & Automation Magazine - March 2013 - 78

sible (i.e., open) a person is to the activity and interaction on
the basis of the trunk orientations and leans of these static
poses. Herein, a body pose is defined to be static if it is held
for at least 4 s [20].
The 3-D Kinect skeleton model [16] is used to track the
trunk orientations of users. We use the planes generated
from the three joint points of the skeleton model representing the left and right shoulders and the spine (in the middle
of the torso, along the back) to determine the orientation of
(a)
(b)
(c)
the upper trunk, and the three joint points that represent the
Figure 4. Face orientation examples: (a) centered horizontal,
left, right, and center hip locations to define the orientation
(b) left and away, and (c) right and away.
of the lower trunk. Figure 5(a) shows an
example of the skeleton for a person sitting
at the card-game scenario. Trunk lean is
determined from the hip and shoulder joint
points. If the right/left shoulder joint is less
Upper
than 30% of the hip width (defined as the
Trunk
distance between the left and right hip
Lower
joints) to the right/left of the right/left hip
Trunk
joint, then the participant is considered to
be in an upright stance. If the angle between
(a)
(b)
(c)
the upper and lower trunk planes is more
than 10°, the user is considered to be leaning
Figure 5. Example Kinect skeletons overlaid on Kinect 2-D images of a person with
forward. These parameters have been veridifferent trunk orientations/leans: (a) upper- and lower-trunk orientations toward
the activity, (b) upper-and lower-trunk orientations away from the activity, and
fied by the authors through empirical testing
(c) sideways lean.
of numerous different individuals in different leaning poses. These trunk orientations
and leans are then categorized utilizing the
Table 1. Accessibility levels.
Davis Nonverbal State Scale (DNSS) [20].
Trunk Orientation
Accessibility Level
The DNSS is a coding method developed to
Upper/lower trunk: T/N or N/T combined with
IV
investigate body poses displayed by a person
upright or forward leans, T/T with all possible leans
during an interaction to directly correlate a
Upper/lower trunk: T/N or N/T except positions that III
person's body language to his/her reaction
involve upright or forward leans
during one-on-one conversations [20]. The
Upper/lower trunk: N/N, A/N, N/A, T/A, A/T with all II
possible leans
DNSS relates the upper and lower trunk orientations/leans to a person's level of accessiUpper/lower trunk: A/A with all possible leans
I
bility. Using DNSS, the trunk orientations
are defined as: toward (T), when oriented
between 0° and 3° from the robot; neutral
Table 2. User state.
(N), when oriented between 3° and 15° from
Face Orientation
Accessibility
User State
the robot; and away (A), when oriented
Toward robot or activity
I
Distracted
more than 15° from the robot. Figure 5(b)
Toward robot or activity
II
Distracted
and (c) shows examples of the trunk orienToward robot or activity
III
Attentive
tations and leans. Table 1 shows the combiToward robot or activity
IV
Attentive
nations of the trunk orientations and leans
Away from robot or activity
I
Distracted
for each accessibility level from level I (least
accessible static pose) to level IV (most
Away from robot or activity
II
Distracted
accessible static pose).
Away from robot or activity
III
Short-term distracted
Once the face and trunk orientations
Away from robot or activity
IV
Short-term distracted
have been classified individually, a person's
overall user state (distracted or attentive to
Body Language
the robot or activity) is determined based on the combination
A person's fluctuations in rapport, stress, involvement, and of body language and face orientations (Table 2). These user
affective state can be determined by analyzing their static states are based on studies that have shown that the lower
body poses during one-on-one interactions [20]. During the body parts such as the lower/upper trunk define the domiproposed HRI scenarios, the robot can determine how acces- nant direction of involvement of a person rather than the
ren

78

*

len

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

March 2013
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2013
