OTC-SmartFishing

Background and Objective

The aim of the project was to develop an optical underwater sensor system for the in-situ detection and classification of marine organisms for use in fisheries and marine ecology research. In addition, it should also form the basis for further applications and technology developments in the field of underwater technology and environmental monitoring. As part of the ‘OTC-smartFishing’ project, an initial prototype was to be developed by the project partners and tested in fisheries applications.

The need for this task arose because unwanted bycatch in fisheries is increasingly the focus of social attention. Up to now, approaches to bycatch reduction have been based primarily on the physical modification of nets by exploiting body shape and size (e.g. through changes in mesh size and mesh shape of the nets), or by exploiting differences in the behaviour of species in the net. However, despite decades of research and development, this strategy for optimising net selection has its limits. This means that new strategies are needed to effectively separate species and sizes. In fisheries and marine research, too, the aim is to reduce the number of animals caught.

Until now, optical methods such as underwater photos and videos have been used in places that are difficult to access, such as coral reefs or the deep sea. They are usually analysed manually, while AI-supported image analysis is still in its infancy. Initial work on camera-based analyses in trawl nets was carried out in Norway, for example. However, these approaches could only be used on very few ships due to their size, for example.

Approach

At the start of the project, all project partners worked on creating the specifications and requirements as well as designing the hardware and software components. In addition to the technical requirements for a complex camera system, the project focussed on the mechanical requirements under the extremely harsh conditions in a trawl net on deck and in the water. Among other things, an innovative concept was developed for the installation and positioning of a camera system in a trawl net.

During the project, the developed prototype was successfully tested on several sea voyages with a fishing research vessel. The system was further improved and data was obtained to improve the AI models.

The project was presented to the public in a variety of ways in the form of lectures and media reports. In addition, one of the sea voyages took place with the participation of 10 fishermen from Mecklenburg-Western Pomerania, with whom the prototype could be used and discussed.

Results

The project successfully developed a prototype of the optical underwater sensor system for the in situ detection and classification of marine organisms and tested it in practice.

The overall result consists of hardware and software components and annotated image data sets for the further development of AI models. In addition, valuable insights were gained that can be used for the further development of the system and future developments by the project partners.

The centrepiece of the developed camera system is an underwater unit with an innovative camera and lighting. In addition to autonomous operation of the underwater unit, this camera system also offers the option of communicating with a topside unit. This makes it possible both to transmit the video images from the underwater camera(s) live to a computer on board and to control the camera and lights directly. For the underwater unit to communicate with the topside unit, an existing winch had to be extensively modified.

In the project, various software components (camera node, core node, AI node, user interface) were developed and object recognition and tracking algorithms for fish in a trawl net were successfully implemented. By using the developed prototype, important image data sets with fish in trawls could also be created, which are the basis for further improvements and developments of AI models.

The software components developed and the recognition algorithms can be reused extensively. There is a need for further research and optimisation in some sub-aspects, which could not be realised to the necessary extent within the scope of this research project. The results of the OTC-smartFishing project provide a solid basis for the next phase of the Future Cluster Ocean Technology Campus Rostock. The findings on the challenges of recognising and tracking fish in the trawl make it possible to work specifically on existing problems.