UFOTriNet - sustainable fish monitoring in Kiel Bight: continuous, automated and non-invasive

Background and Objective

For years, many fish stocks in almost all marine EU waters, including the Baltic Sea and the North Sea, have been severely or even overexploited. Strong demand and the associated economic incentive to increase fish production are primarily responsible. The efficient implementation of EU marine and fisheries policies for the long-term and sustainable use of our marine resources requires the provision of temporally and spatially highly resolved data on the diversity of fish species, their seasonal dynamics and on the causal relationships with their animate (biotic) and inanimate (abiotic) environment.

In the UFOTriNet-project, a Germany-wide consortium of five project partners and more than 20 employees from various disciplines, we are working in an interdisciplinary innovation’s project to provide an automated, continuous, and non-invasive monitoring of fish populations.

For application in different areas and to cover different functions, three complementary variants of Underwater-Fish-Observatories (“UFOs”) have been designed and built, originating and further developed from an already existing (stationary) prototype: a stationary, a portable and a mobile UFO. All three systems are modularly equipped with the same core set of opto-acoustic sensors: a stereo camera and a sonar device for monitoring fish. These three "basic types" of UFOs represent the cornerstones of the trilateral network tested in Kiel Bay as part of UFOTriNet.

Modularity, compactness, structural integrity and multiple deployment options are decisive advantages of the trilateral UFO array compared to point-based vessel-based fish stock surveys. The three developed UFO variants are freely combinable and allow an area-wide monitoring of fish stocks as a cost-effective, standardized, environmentally friendly and permanently measuring alternative to the classical fish stock survey. The targeted goal is the autonomy of the systems, among other things by increasing the energy efficiency and reducing the maintenance effort, especially by nano-coating and UVC-LED irradiation as novel technologies to prevent biological growth on the devices.

Approach

The focus of UFOTriNet is the development and empirical testing of innovative non-invasive measurement and control systems – essentially based on imaging opto-acoustic sensor technology – for synchronous and high-resolution monitoring of fish and their environment. To also monitor the environment synchronously, additional sensors have been integrated to measure, among others, temperature, salinity, pressure, oxygen content, chlorophyll, turbidity and currents. Complementary research aspects are, among others, the optimization of automatic fish identification and size classification based on AI-algorithms as well as autonomization by increasing energy efficiency (LED-UVC experiments, windshield wipers) and reducing maintenance efforts (nano-antifouling coating).

The different UFO-systems can be deployed in different areas (marine hotspots) or for different purposes (e.g. wind farm or protected area monitoring). This is the most important difference of the UFO concept compared to all other systems worldwide.

The stationary and portable UFO (a miniaturized version of the stationary UFO) are designed either for continuous operation or for flexible routine operation - as needed in different key areas (e.g. marine protected areas, spawning, nursery or recruitment areas, wind farms, etc.) or for temporary fixation (e.g. at bridge pylons, rocks, lighthouses or aquaculture facilities). Both rely on a land connection. In contrast, the mobile UFO is designed for shorter location-independent missions. In a remotely operated or an autonomous mode, it provides spatially highly scaled and flexible observations and can also be used as an accompanying support to fishing vessels or research ships.

As part of the test network of UFOTriNet, the stationary UFO was deployed for data acquisition in May 2020 on the eastern shore of the Kiel Fjord near the aquaculture company "Kieler Lachsforelle" at a depth of about 5 m. The portable UFO was launched in July 2021 in Eckernförde Bay at the Ostsee-Info Center (OIC). The actual equipment carrier was attached to a specially constructed rung ladder mounted on the sheet pile wall of the outer harbour area there. A first test run of the mobile UFO was carried out in Kiel Bay in June 2022.

Additional sampling campaigns with research vessels were used to collect comparative data for calibration, blending, and conversion, while providing important process understanding for developing optimal hardware design and optimizing the pattern recognition algorithms of both the mobile and the stationary systems.

In the course of Corona-related shortages and closings of laboratories and workshops - particularly during the construction and building of the mobile UFO - the project was extended by 12 months. During this extension period, the focus is on final completion, testing and operating of the mobile UFO, and the related data collection and analysis.

Data and Methods

All three UFO systems combine modern acoustic and optical methods (pattern recognition) for monitoring fish stocks. Here, the optical near-field component located in the overlap area is considered a sub-sample of the spatially more extensive acoustic far-field component. By coupling the optical near-field and acoustic far-field, meaningful hybrid data can be provided. The biological hybrid data from acoustics and stereo-optics (abundance, species ID, size measurements) as well as collected data from environmental sensors (temperature, salinity, chlorophyll, current velocity, turbidity, etc.) will be fed into an integrated, newly built data- and information-system with online access. Within the framework of the International Council for the Exploration of the Sea (ICES) (e.g. for stock assessment and quota calculation), HELCOM (for the implementation of the International Marine Strategy Framework Directive) or the National Water Framework Directive, the data will be processed and made available.

Between 2019 and 2022, the workflow focused on design and development of the three UFO-units in a total of four work packages. This includes the integration of the hardware and the associated software development as well as the implementation of opto-acoustic sensors and an optimized energy supply. It also includes optimal processing of large data sets, improvement of opto-acoustic pattern recognition for fish and other marine organisms, optimization of maintenance intervals and their implementation, and controlled experiments on the effectiveness of UV-C irradiation as an antifouling tool for the optical units. For quality assurance, a series of standard tests to ensure international standards for integrated assessment of fish stocks are developed and implemented as part of the data handling: calibration and validation of acquired data, conversion of 2D information to 3D information, scientifically adequate assessment of data quality and provision of adequate data processing options. Last but not least, project management and coordination focus on the technical-administrative management, the use of modern communication infrastructure (two digital interactive posters) for the dissemination of the results, the effective implementation of the work packages as well as the close cooperation and communication between the partners and the assurance of achieving the project’s objectives.

Results

Core of the project is the automatic fish identification and size classification of the recorded data from the camera and the sonar - based on AI algorithms (automatic pattern recognition). Depending on depth and visibility, fish are continuously recorded in the near field (camera) within approx. 8-12 m and in the far field (sonar) in up to approx. 200 m distance.

The combination (hybridization) of information from camera and sonar provides:

• detection of individual fish (in shoals) and species identification
• counts of all fish per time unit
• size classification of individual fish
• biomass calculation per time unit

Accompanying environmental parameters, such as temperature (shown below), salinity, oxygen, chlorophyll content, turbidity, and currents are measured directly from the stationary UFO or from other monitoring stations. Comparisons of these data with fish dynamics provide insight into causal relationships on the dynamics of their environment.

Links and Downloads

• 06.2019: project start, report on the funding for the further development of the “Underwater-Fish-Observatory” (UFO) by the Federal Ministry of Food and Agriculture (BMEL) as part of its innovation support program (www.geomar.de/news/article/meeres-ufos-erkunden-die-lebensgemeinschaft-unter-wasser/)
• 05.2020: NDR contribution following the deployment of the stationary UFO in the Kiel Fjord (https://www.ndr.de/nachrichten/schleswig-holstein/Fischbestand-der-Meere-Ein-UFO-soll-helfen,geomar274.html)
• 06.2021: The re-deployment of the stationary UFO after maintenance work was accompanied by filming and interviews by the NDR. These were carried out for an episode of the NDR QUIZSHOW – “Die Leuchte des Nordens” with Jörg Pilawa. The broadcast is to take place in the course of 2022.
• 11.2021: „Leer gefischt? – Wie wir die Fische im Meer schützen können“, children‘s show „neun ½“ oft he WDR (https://kinder.wdr.de/tv/neuneinhalb/av/video-leer-gefischt--wie-wir-die-fische-im-meer-schuetzen-koennen-100.html
• Publication: Böer G, Veeramalli R, Schramm H (2021) Segmentation of Fish in Realistic Underwater Scenes using Lightweight Deep Learning Models, ROBOVIS 2021: Computer Science, Environmental Science (https://www.semanticscholar.org/paper/Segmentation-of-Fish-in-Realistic-Underwater-Scenes-B%C3%B6er-Veeramalli/d36bc554bb0b39ff8ec420039899a4c842a10a2c)
• Publication: Bouzouit A, Fietz D, Badri-Hoeher S (2021) Fish Tracking Based on Sonar Images by Means of a Modified Optical Flow, Proc. MTS/IEEE Oceans Conference (OCEANS '21), San Diego - Porto, September 2021 (https://ieeexplore.ieee.org/document/9705868)
• Publication: Fietz D, Bouzaouit A, Badri-Hoeher S (2021) Background Noise Cancellation from Active Sonar Images Utilizing Transmission-free Recordings, Proc. MTS/IEEE Oceans Conference (OCEANS '21), San Diego - Porto, September 2021 (https://ieeexplore.ieee.org/document/9705811)