IEEE Robotics & Automation Magazine - September 2023 - 13

which the module will focus [7]. The limited working space,
positioning, and heading requirements for operations and the
marine traffic nearby constitute a complex spatial environment.
How best to achieve efficient maneuvering in such an
environment by taking certain optimal metrics like minimal
time or energy is worth studying. Furthermore, the maneuver
should also comply with maneuvering regulations such as the
International Regulations for Preventing
Collisions at Sea to make it more applicable
in real situations.
Taking results from the aforementioned
modules, the " human-in-the-loop/automatic
control " part will focus on either
assisting operators or taking the operation
in an automatic manner. For example, to
assist with the training of operators in
simulation, a focus-attention model can
be established through expert knowledge
and training data in simulation. Further,
autodocking is an example of automatic
control. The controller makes use of current sensor data to
produce control commands to thrusters, thus realizing docking
operation. The module is close to a high-level application and
therefore needs to take specific requirements from the application
into account.
As a result, information from all the different modules will
for this application lie in uncertainties from the ship model and
environmental effects and the nonlinear control.
Given the historical operation data and ship motion data,
"
DYNAMIC OPTIMIZATION
REFERS TO THE STATE OF
THE SHIP AND THE MISSION
BEING EXECUTED.
„
be gathered to establish an onboard support center, forming
various tools ranging from risk assessment, maneuvering evaluation,
and sensor diagnosis to real-time planning. Note that
the onboard supporting tools can feed back to each module,
forcing it to update. If a predictive path shows a potential failure
of path followed by the autocontrol due to an environmental
condition change, a feedback to the optimization module will
force it to replan.
To summarize, by effectively combining the aforementioned
modules, it is possible to develop efficient onboard
support tools for either assisting the operator during marine
operation or achieving automatic control of the operation.
GUNNERUS TWIN-SHIP VALIDATION
To demonstrate how the Gunnerus digital
twin works, several related applications
are presented as validation cases.
All the demos can be found at https://
org.ntnu.no/intelligentsystemslab/
gunnerus/digitaltwin.html.
HARBOR DOCKING
Here we take harbor docking operation
as the first example to illustrate possible
solutions using the digital twin system,
as shown in Figure 5. In recent
years, the maritime industry, including
Kongsberg, Wärtsilä, and Det Norske
Veritas has put effort into implementing
autodocking systems [15]. The challenges
the OSP framework introduced in Figure 2 can be applied
to simulate ship dynamics. As long as the fidelity of the
model is acceptable, we can build up docking scenarios in
simulation and invite shipmasters to perform the operation
under different levels of sea conditions.
The simulated data could then be fed into
the tools to train the controller for ship
docking, and into the predictor for online
estimation of ship motion. This process
will be run in a short period of time if
there is enough simulated data for training.
Further validation can be achieved
and compared between the docking op -
eration in the field and in a digital twin
simulation. If validated, the twin ship in
simulator can work in two modes. The
first mode is to provide onboard support
for predicting ship motion. By feeding both the field operation
and ship motion data, the predicted ship motion can be
fed back to the shipmaster during operation. The other mode
is automatic control, which takes only filed ship motion data
as input and produces operation commands for ship docking.
In Figure 6, each block in " digital space " represents an
FMU with the signal communication specified. The involved
components are provided by different stakeholders, as listed
in Table 2.
A full-scale ship docking experiment is conducted in Ålesund
harbor, Norway. When Gunnerus is operated to dock to a
berth, the implementation of twin-ship docking in simulator is
going on simultaneously. By feeding it the in-field ship operation
and environmental sensor data, the twin ship produces
highly resemble trajectories and speeds, as shown in Figures 7
and 8. Although the twin ship is not a complete duplicate
of the physical ship, we are looking to make them as indistinguishable
as possible. Slight mismatches are observed, especially
in speeds, as shown in Figure 8. They might be caused
Early
Data
Transmission
and
Visualization
Data
Sensitivity
Analysis
Performance
Optimization
Warning and
Prediction
Human-inthe-Loop
Control
Onboard
Support
* Risk Assessment
* Online Planning
* Status Diagnosis
* Motion Prediction
* . . .
FIGURE 4. Digital twins for maritime prediction and maintenance.
SEPTEMBER 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
13
https://org.ntnu.no/intelligentsystemslab/gunnerus/digitaltwin.html https://org.ntnu.no/intelligentsystemslab/gunnerus/digitaltwin.html https://org.ntnu.no/intelligentsystemslab/gunnerus/digitaltwin.html
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
