IEEE Robotics & Automation Magazine - June 2019 - 81

Response Prompting for Decision Making
To make intelligent decisions in the context of providing instruction, a cognitive framework is proposed that makes use of
response-prompting methodologies borrowed from the education domain [17]. Response prompting is a well-defined, evidencebased collection of teaching practices that involve assessing the
environment and acting or providing assistance to stimulate a
targeted behavior. By incorporating these proven methodologies
into our approach, an intelligent system is able to use the same
methods a teacher would use to provide instruction.
Prompt/response strategies allow an IRI to perform in multiple modalities. Common prompt modalities are vocal, visual,
gestural, modeling (i.e., demonstrations), or physical prompts.
Another useful feature of prompting is the ability to modify
the intrusiveness of prompts, called fading; more intrusive
prompts provide more support, whereas less intrusive prompts
allow students to perform the desired behavior independently.
Importantly, these methods go far beyond simple instructional systems: they involve a complex hierarchy of prompts, with
multiple paradigms for arranging, traversing, and fading those
prompts based on instructional needs
and student performance. These structured designs assure that student performance moves toward independence.
Introduction
Leveraging response-prompting
instructional strategies in the cognitive
domain represents a novel approach to
an intelligent robotic and AR system
for instruction. We believe that examStop Monitor
ining this approach provides valuable
and constructive evidence for instruction by autonomous robots.
Cognitive Process Overview
Figure 2, inspired by [18], illustrates
the cognitive framework at a high
level. In the cognitive process, it is by
taking sensory information from the

world and perceiving information salient to the task at hand
that we create an interpretation of the states and actions of the
world. Then, using contextual knowledge, that interpretation
is reasoned about to generate a higher-level representation of
the situation: i.e., a decision is made. In the intelligent instruction application, this involves first using the responseprompting methodology to select the correct instruction
response (e.g., present stimulus, prompt, consequence, reinforcement) and then performing the selected action.
The general instruction process is shown in Figure 3. The
instructional intervention begins with an introduction and
general instructions for the scenario. In task instruction, the
task is introduced and the target stimulus presented. Next, the
prompt is selected. The intrusiveness of the prompt depends
on the methodology being used (see the "Response-Prompting Types" section) and the previous prompt/response history
(e.g., prior incorrect responses). Next is the student response
period, which ends when the student is idle, the task is complete, or the response interval has elapsed.
The response evaluation determines the outcome of the
interaction: a correct answer results in positive reinforcement.
For a correct but nonoptimal answer, a correction occurs before

Reasoning
Sensory

Situational
Knowledge

Evaluation

Perception

Instruction
Strategy

Instruction
Selection

Effector
System

Decision Making

The World

Unique aspects of robotic and intelligent systems
would strongly benefit populations in need of educational support.
This work attempts to address these open issues by adapting
proven methods used for instruction in the education field to
the decision-making process of an IRI. The approach is then
employed to teach students with intellectual disabilities in a
one-on-one setting.
We have already introduced the response-prompting concept for instruction in preliminary work with an IRI [15] and
an AR device [16]. Here, we present new prompting strategies, additional experiments, expanded results and discussion,
and observations and lessons learned over this broader study.
We hope that this article and the presented findings will help
drive further research into the creation of intelligent instruction systems, particularly for students with elevated needs,
such as those with disabilities.
●

Action

Figure 2. The cognitive process flowchart. The information flows
from lighter- to darker-shaded process steps.

Prompt
Selection and
Delivery

Task
Instruction

Interval
Elapsed

Next
Lesson

Student
Response and
Correct
Response Observation

Differential
Reinforcement

Incorrect
Answer/
Prompt
Escalation

Corrected
Response
Correction

Correct but
Nonoptimal
Response

Response
Evaluation

Figure 3. The overall instruction process based on the response-prompting methodology.

JUNE 2019

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2019