Search

Developing high‐performing machine learning algorithms requires large amounts of annotated data. Manual annotation of data is labour‐intensive, and the cost and effort needed are an important obstacle to the development and deployment of automated analysis. In a previous work, we have shown that deep learning classifiers can successfully be trained on synthetic images and annotations. Here, we provide a curated set of fish image data and backgrounds, the necessary software tools to generate synthetic images and annotations, and annotated real datasets to test classifier performance. The dataset is constructed from images collected using the Deep Vision system during two surveys from 2017 and 2018 that targeted economically important pelagic species in the Northeast Atlantic Ocean. We annotated a total of 1,879 images, randomly selected across trawl stations from both surveys, comprising 482 images of blue whiting, 456 images of Atlantic herring, 341 images of Atlantic mackerel, 335 images of mesopelagic fishes and 265 images containing a mixture of the four categories.

One of the leading causes of overfishing is the catch of unwanted fish and marine life in commercial fishing gears. Echosounders are nowadays routinely used to detect fish schools and make qualitative estimates of the amount of fish and species present. However, the problem of estimating sizes using acoustic systems is still largely unsolved, with only a few attempts at real-time operation and only at demonstration level. This paper proposes a novel image-based method for individual fish detection, targeted at drastically reducing catches of undersized fish in commercial trawling. The proposal is based on the processing of stereo images acquired by the Deep Vision imaging system, directly placed in the trawl. The images are pre-processed to correct for nonlinearities of the camera response. Then, a Mask R-CNN architecture is used to localize and segment each individual fish in the images. This segmentation is subsequently refined using local gradients to obtain an accurate estimate of the boundary of every fish. Testing was conducted with two representative datasets, containing in excess of 2600 manually annotated individual fish, and acquired using distinct artificial illumination setups. A distinctive advantage of this proposal is the ability to successfully deal with cluttered images containing overlapping fish.

Fish counts and species information can be obtained from images taken within trawls, which enables trawl surveys to operate without extracting fish from their habitat, yields distribution data at fine scale for better interpretation of acoustic results, and can detect fish that are not retained in the catch due to mesh selection. To automate the process of image-based fish detection and identification, we trained a deep learning algorithm (RetinaNet) on images collected from the trawl-mounted Deep Vision camera system. In this study, we focused on the detection of blue whiting, Atlantic herring, Atlantic mackerel, and mesopelagic fishes from images collected in the Norwegian sea. To address the need for large amounts of annotated data to train these models, we used a combination of real and synthetic images, and obtained a mean average precision of 0.845 on a test set of 918 images. Regression models were used to compare predicted fish counts, which were derived from RetinaNet classification of fish in the individual image frames, with catch data collected at 20 trawl stations. We have automatically detected and counted fish from individual images, related these counts to the trawl catches, and discussed how to use this in regular trawl surveys.

Acoustic-trawl surveys are an important tool for marine stock management and environmental monitoring of marine life. Correctly assigning the acoustic signal to species or species groups is a challenge, and recently trawl camera systems have been developed to support interpretation of acoustic data. Examining images from known positions in the trawl track provides high resolution ground truth for the presence of species. Here, we develop and deploy a deep learning neural network to automate the classification of species present in images from the Deep Vision trawl camera system. To remedy the scarcity of training data, we developed a novel training regime based on realistic simulation of Deep Vision images. We achieved a classification accuracy of 94% for blue whiting, Atlantic herring, and Atlantic mackerel, showing that automatic species classification is a viable and efficient approach, and further that using synthetic data can effectively mitigate the all too common lack of training data.

In-trawl camera systems promise to improve the resolution of trawl sampling used to ground-truth the interpretation of acoustic survey data. In this study, the residence time of fish in front of the Deep Vision camera system, used to identify, measure and count fish inside the trawl, was analysed to determine the reliability of spatial distribution recorded by the system. Although Atlantic herring (Clupea harengus), haddock (Melanogrammus aeglefinus), and most Atlantic cod (Gadus morhua) moved quickly back through the aft part of the pelagic trawl, saithe (Pollachius virens) spent up to 4 min in front of the system. The residence time increased for saithe and cod when other individuals were present, and cod swimming in the low water flow close to the trawl netting spent longer there than cod at the centre of the trawl. Surprisingly, residence time was not related to the size of the fish, which may be explained by the collective behaviour of shoaling fish. Our findings suggest that while in-trawl images can be used to identify, measure and count most species, when sampling fast-swimming species such as saithe the position inferred from when they were imaged may not reflect the actual spatial distribution prior to capture.

An in-trawl stereo camera system (DeepVision) collected continuous, overlapping, images of organisms ranging from krill and jellyfish to large teleost fishes, including saithe ( Pollachius virens ) and Atlantic cod ( Gadus morhua ) infected with parasitic copepods. The four-dimensional position (latitude, longitude, depth, time) of individuals was recorded as they passed the camera, providing a level of within-haul spatial resolution not available with standard trawl sampling. Most species were patchily distributed, both vertically and horizontally, and occasionally individuals were observed at significant vertical and horizontal separation from conspecifics. Acoustically visible layers extending off the continental rise at 250 m depth and greater were verified as primarily blue whiting ( Micromesistius poutassou ), but also included a small proportion of evenly distributed golden redfish ( Sebastes marinus ) and greater Argentines ( Argentina silus ). Small, but statistically significant, differences in length by depth were observed for blue whiting within a single haul. These results demonstrate the technology can greatly increase the amount and detail of information collected with little additional sampling effort.

Rosen, S., Engås, A., Fernö, A., and Jörgensen, T. 2012. The reactions of shoaling adult cod to a pelagic trawl: implications for commercial trawling. – ICES Journal of Marine Science, 69: 303–312. The reactions of shoaling adult Atlantic cod to a pelagic trawl were measured during fishing off the north coast of Norway. Cod remaining in the trawl track dived at rates as fast as 0.35 m s−1 following vessel passage and swam away from the vessel, in the direction of the approaching trawl, at an average rate of 0.6 m s−1. They did not attempt to swim ahead of the trawl as documented previously, but passed into the lower half of the trawl entrance and swam slowly in the direction of trawling at a rate of 0.2–0.5 m s−1 as the trawl's greater speed through water carried them deeper into the trawl. Shoals compressed vertically once inside the trawl, suggesting that packing density increased at least fourfold. Fish remained in the lower part of the trawl as they moved through its tapered section towards the codend, with little to no clearance above the bottom panel, but significant clearance beneath the top panel. Catches were sufficient to support commercial harvest, and the behaviour observed suggests that changes in trawl design and fishing strategy might improve fuel economy and species selectivity.

Small offshore banks may be sites of intense feeding by upper trophic level predators. We studied the distribution of cetaceans, seabirds, pelagic fish, euphausiids and zooplankton over a 9 × 15 km bank to determine the conditions and processes that concentrated prey there and to exam- ine the relative importance of bottom-up or top-down controls. Euphausiids were the primary prey during most foraging activity. While these were widespread in subsurface waters, foraging was con- centrated on dense surface swarms that formed during daylight hours over 2 small crests. Internal wave passage resulted in upward movement and concentration of euphausiids in these areas through a coupling of physical processes and euphausiid behavior, resulting in surface swarms. Thus, inter- nal waves appear to provide a critical mechanism enhancing trophic energy transfer. The formation of dense, localized and accessible prey concentrations was more important to foraging than was the overall available prey biomass. The estimated maximum daily consumption of euphausiids by cetaceans, seabirds and herring combined was

Ecosystem surveys are carried out annually in the Barents Sea by Russia and Norway to monitor the spatial distribution of ecosystem components and to study population dynamics. One component of the survey is mapping the upper pelagic zone using a trawl towed at several depths. However, the current technique with a single codend does not provide finescale spatial data needed to directly study species overlaps. An in-trawl camera system, Deep Vision, was mounted in front of the codend in order to acquire continuous images of all organisms passing. It was possible to identify and quantify of most young-of-the-year fish (e.g. Gadus morhua, Boreogadus saida and Reinhardtius hippoglossoides) and zooplankton, including Ctenophora, which are usually damaged in the codend. The system showed potential for measuring the length of small organisms and also recorded the vertical and horizontal positions where individuals were imaged. Young-of-the-year fish were difficult to identify when passing the camera at maximum range and to quantify during high densities. In addition, a large number of fish with damaged opercula were observed passing the Deep Vision camera during heaving; suggesting individuals had become entangled in meshes farther forward in the trawl. This indicates that unknown numbers of fish are probably lost in forward sections of the trawl and that the heaving procedure may influence the number of fish entering the codend, with implications for abundance indices and understanding population dynamics. This study suggests modifications to the Deep Vision and the trawl to increase our understanding of the population dynamics. publishedVersion