Machine Learning, Safety & Security

LEAS – shore-side decision support for traffic situations with highly automated or autonomous vessels using AI.

Introduction of LEAS

Maritime transport routes are currently characterized by increasing digitalization and automation on board ships. Further development from partially to highly to fully automated and autonomous ships is currently being researched worldwide and implemented in prototype developments. So far, little attention has been paid to the technical and operational integration of such vessels into the existing system of conventional ships and shore-side traffic control services. The merging of traffic and monitoring regimes is particularly important near coasts with high traffic density, where decisions are sometimes required in the shortest possible time.

Summary of the Development

LEAS is focused on the design, implementation and demonstration of shore-side support services based on artificial intelligence (AI) for conventional to autonomously navigating and possibly unmanned vessels (mixed traffic). Thus, a maritime transport scenario is addressed for which – apart from some simulation studies – no practical experience on operational requirements and procedures is available. The developed advisory components and modules shall be based on elements working with artificial intelligence (AI) and implemented in an innovative human-machine interface (HMI). The latter is to indicate to the human operator the intention or goals pursued by the automated system and the means employed.

Exemplary situation

The following example of a fictitious safety scenario is used to illustrate the issue of shore-side support to improve the safety and efficiency of coastal maritime transportation for traffic situations involving highly automated or autonomous vessels in relation to vessel traffic services (VTS) centers.

Thomas recently successfully completed his training as an operator in a VTS traffic center. He is one of the first to practice this profession without having gone to sea himself. It is October 2025, an autumn night with showers and gusty winds is approaching. Besides Thomas, there are two other employees on duty, so the minimum manning requirement is met.

He masters the 12 monitors and diverse support systems with confidence because he is well prepared through training. Routinely, he observes the traffic until the voice radio intensity increases. An alarm appears, indicating that a ship has left the traffic separation scheme (TTS) near the wind farm north of Helgoland. Another, obviously highly automated ship is approaching. Is there someone on the bridge there? Does the shipping company’s own monitoring center have an eye on the situation? A collision hazard is beginning to develop with the ship that has just left the TTS. There is no evidence of an evasive maneuver. The automated ship as the course holder continues to sail in accordance with the rules. The other ship, as the one obliged to take evasive action, now touches the safety limit of the wind farm. There it is not deep, which is why the alarm for the danger of grounding is triggered. In the meantime, six different alarms have been raised in the monitoring area. Which one has the highest priority, which one is the most urgent? Thomas has to decide. It’s a good thing that an AI-based decision support system is now available to him for this purpose.

LEAS scientific/technical work objectives

Can anomalies in the driving behavior of ships be detected primarily from AIS data using AI-based methods for traffic monitoring?
How can AI be used to counteract an escalation of danger in complex traffic situations?
How can AI be used to detect and eliminate the influence of errors that occur when determining a ship’s position data?
Can AI be used to achieve faster and more reliable prediction of a vessel’s maneuvering characteristics for traffic monitoring?
How should the human-machine interface (HMI) of a VTS system equipped with AI-based methods be designed?
How must a simulation system or demonstrator be structured to incorporate AI functionalities and be used for evaluation?
Is the current legal framework sufficient to use AI in the maritime domain to support VTS operators?

Main work packages

Traffic monitoring and data streams

The national and international regulations and their dependencies (e.g., IMO/IALA) will be summarized and analyzed. In particular, the emerging regulations for MASS (Maritime Autonomous Surface Ships) will be considered. The activities for the further development of VTS operations will be compared in an international context.

Traffic management concept and system architecture

The future processes for mixed transports are specified and modeled. Furthermore, proposals for VTS centers in dealing with MASS are developed. The specifications include data formats, transmission rates and distances as well as output formats and a database model. In addition, the developed data requirements and proposed data formats will be aligned with guidelines and standards currently being developed (e.g., IMO Compendium, IALA specifications). Traffic scenarios will be specified to verify the technical capabilities of the modules.

The aggregation of all data streams is essential for the development of the AI modules. Secure and structured data recording allows offline training of the AI modules and later verification. In particular, inconsistencies in the data and outliers are identified and removed. Further, a concept for the handover between back- and frontend for the web-based VTS will be developed.

The demonstrator requires the development of a robust software and hardware architecture. Interfaces and technical requirements will be defined. Also, the system modules will be defined and the exchange of sensor, situational awareness and route information between ship and shore as well as the later integration of the system into today’s VTS solutions will be defined.

Human-machine interface for AI-based VTS support

The basis is ISO 9241-210 “Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems”. The design process is an iterative procedure with continuous optimization and adaptation of design solutions on the basis of user-centered evaluations. The design process is an iterative procedure with continuous optimization and adaptation of design solutions on the basis of user-centered evaluations. User involvement takes place in the form of workshops and interviews.

AI-based decision support systems and modules

The work package provides all necessary data required for the development of AI modules. Furthermore, the AI algorithms are developed and verified. Thus, this represents the main work package of LEAS.
Also, the AI modules required for the web-based portion of the project will be developed here.

AI Module 1 – Decision support for selection of position data

Definition of training and test data set
Implementation of comparison models based on classical statistical methods, e.g.: Density determination in histograms, Gaussian Mixture Models or Single Class Support Vector Machines
Analysis of the data set for different projections in parameter space such as: Elevation, Azimuth Angle, Signal to Noise Density Ratio, Satellite Type (GPS Block 2 / 3), Satellite No, Vessel Position, Vessel Orientation.
Selection of at least two methods of deep learning, e.g. normalizing flows
Iterative improvement of the models by hyperparameter optimization
Comparison with classical comparison models regarding consistency of all models
Implementation of the interface to the HMI

AI Module 2 – Hazard escalation and situation assessment

Extraction of historical encounter situations
Determination of the danger of a situation with the help of ship domains and derived limit values
Determination of the last possible moment of an intervention
Resolution of dangerous situations by maneuver prediction with Deep Learning
Verification of the learning procedure in post-processing
Identification and safe management of complex traffic situations

AI Module 3 – Intelligent Anomaly Detection

Identification of anomalous ship movements and situations under various conditions
Trajectory investigation with regard to course and speed
Determination of the earliest possible point of intervention on the ship’s course
Verification of the learning procedure in post-processing

AI Module 4 – Motion prediction of autonomous Vessels

Classification of vessels as a basis for AI development.
Parameter estimation for vessel classes based on historical and statistical AIS data and other data sources using Deep Learning
Verification of parameter sets for ship classes

Demonstrator AI based decision support modules with innovative HMI.

In this part of the work, the two demonstrators are developed. The HMI mock-ups will be implemented, interfaces between the AI modules and the HMI front ends will be realized and integrated into the simulation system to create a realistic user environment that can be evaluated.
For this purpose, the AI modules are brought together in the demonstrators. This includes the integration of the AI modules into the simulation system, creation of interfaces between real-time data or simulator and AI prototypes to provide realistic sensor data and position information.
Furthermore, the basic structure for a web-based VTS infrastructure is generated.

In three places, the findings of LEAS will be integrated as a prototype

MSCW infrastructure (Maritime Simulation Center Warnemünde | HSW | Germany)
Simulator system at the Fraunhofer Center for Mairitme Logistics (CML | Hamburg | Germany)
Full-Web-Based VTS at FleetMon (Rostock | Germany)

Within this work package, it is planned to continue cyber security load tests for web-based VTS infrastructure.

The Demonstrator | Outcome

With the completion of the project, it is possible to evaluate the effect of MASS in traffic centers. Recommendations for action can also be made as to how future highly automated ships must be displayed in the front end. The driving behavior of these special ships will be predicted using AI to provide operators with a reliable situation picture.

Status meetings

Digital kickoff meeting held in 02-2022

On Friday, February 11, 2022, the official kickoff meeting for LEAS took place. Due to Corona, this had to be carried out digitally. In 5 hours, the individual work contents were discussed and the next steps were derived from this. All partners, all associated partners and the project executing agency were present. In total, 25 people were involved in the meeting. In the coming weeks, the talks will continue in smaller rounds. All in all, it was a very successful meeting.

KI Workshop in Warnemuende on HSW 05-2022

The work on the application of artificial intelligence is one of the essential work aspects in the project. For the activities assessed in AP4, a workshop was held on May 19, 2022 under the leadership of DLR Neustrelitz. For this purpose, all partners and persons actively involved in the development met in Warnemünde. After presenting the individual AI parts, strategies for implementation were discussed. Interfaces to the demonstrators were also reviewed and further work steps were derived from this. It became apparent that the goals set are very ambitious, but at the same time seem achievable. After a very intensive seven hours of discussion, the workshop was declared successfully completed.

2nd annual meeting 2022 from 16 to 17 November in Rostock and Warnemünde

From November 16 to 17, the second LEAS Annual Meeting for 2022 took place at the University of Applied Sciences Wismar with its campus in Rostock-Warnemünde. The agenda for the two days included workshops and presentations by project members and project partners, as well as an evening event organized by FleetMon at the Radisson Blu Hotel in Rostock.

Under the topic “Developments in VDES as a follow-up to the AIS and S100 / S200 standard for highly automated or autonomous vessels”, Carsten Hilgenfeld (Head of Research & Development FleetMon) presented the development of FleetMon’s “Full Web Based VTS (Vessel Traffic Service)” for the Zanzibar Maritime Authority. Michael Bergmann (Executive Advisor at BM Bergmann-Marine) continued by talking about the regulatory status of VDES (VHF Data Exchange System) and the implications for the AIS market, and Sebastian Belz (Secretary General at European Platform of Transport Sciences, EPTS Foundation) closed the workshop with his presentation on the International VTS Conference with the support of the European Platform of Transport Sciences in 2024.

For the project participants and the interested attending parties from the industry, the evening was an excellent opportunity to get to know each other personally and to exchange information about the current developments of the new VDES standards.