Tech Briefs Magazine - June 2024 - MD19

ChatGPT Enables Roboticist to Choreograph
Flying Robots
The research shows potential for reprogramming industrial robots without the need for expert
knowledge or programming skills.
Technical University of Munich, Germany
P
rofessor Angela Schoellig from the
Technical University of Munich
(TUM) uses ChatGPT to develop choreographies
for swarms of drones to perform
along to music. An additional
safety filter prevents mid-air collisions.
The researchers' results demonstrate
that large language models (LLMs) such
as ChatGPT can be used in robotics.
The web interface is easy to use: Doctoral
student Martin Schuck selects a music
track and enters a text requesting a suggested
choreography. Additional instructions
can be given to the drone swarm via
another prompt in the ChatGPT tool before
an algorithm check whether the suggested
flight paths are feasible.
The screen in the Learning Systems and
Robotics Lab now shows a simulated airfield
with six drones flying in circles to the
music. If the scientist from the Chair of
Safety, Performance and Reliability for
Learning Systems at TUM likes this choreography,
he logs it in. Soon afterward, six
palm-sized drones take off from the floor
of the robotics lab.
In Schoellig's laboratory, the research
team has installed six ceiling-mounted
cameras in a room measuring around 40
square meters and three meters high.
Crosses are marked on the floor with insulating
tape. These indicate the starting positions
of the drones. Once the computer
has verified a possible choreography, they
can take off.
The cameras detect the position of the
quadrocopters, which are equipped with
four propellers and motors, 200 times a
second. The system compares them with
the desired position. The " airshows " in the
Learning Systems and Robotics Lab, which
the research team realizes with up to nine
drones, are 100 percent safe today. Without
the special safety filter, only one in four
demonstrations is accident-free.
Schoellig combined ChatGPT with
the safety filter for the " Dance of the
Flying Robots. " According to her, the
ChatGPT AI tool was primarily created
to generate texts, but it can also suggest
Motion Design, June 2024
Professor Angela Schoellig in her Learning Systems and Robotics Lab, where she developed the choreographies
with the flying robots. (Image: Astrid Eckert/TUM)
choreographies. " However, it initially
knows nothing about the properties of
drones and physical limits for the flight
paths. So, it is clear that ChatGPT makes
mistakes, " she added.
The additional safety algorithm closes
this gap by mapping out flight paths for
the proposed choreography so precisely
that mid-air collisions are completely
avoided. Drones can even approach each
other diagonally. Schoellig calls the overall
concept of ChatGPT and security filter designed
to use several flying robots
" SwarmGPT. " The tool generates the processes
in the air and at the same time
serves as an interface between the robot
and the human, who does not require any
expert knowledge.
When Schoellig began her drone research
almost 15 years ago, choreographies
were hand-crafted. It took more
than three years to develop the first six
choreographies for six drones and get
them up and running. " ChatGPT has
brought about a quantum leap, " she said.
Over the past three months, the researchers
have experimented with over 30 choreographies
for up to nine drones. Today, it
takes the researchers only around five
minutes to develop a safe choreography
for 30-second music clips with three
drones. The more drones are added, the
more time ChatGPT needs for its calculations
and the longer it takes to propose a
choreography. But Schoellig is certain:
" The concept is scalable. "
Can other robots be used with a similar
interface via ChatGPT? For robots
that use voice control to pick up objects,
lay cables or open doors, the success
rate of those actions is currently only 63,
56, and 80 percent. So far, applications
in other robotics scenarios have been
somewhat unreliable. For Schoellig, this
is an incentive, " I assume that our approach
will keep getting better in other
scenarios, too. " It may soon be possible
to reprogram suction-based and industrial
robots simply by voice command
without the need for expert knowledge
or programming skills.
For more information, contact
Andreas Schmitz at andreas.schmitz@
tum.de; +49 162-27-46-193.
19

Tech Briefs Magazine - June 2024

Table of Contents for the Digital Edition of Tech Briefs Magazine - June 2024