Content-Aware Segmentation of Objects Spanning a Large Size Range: Application to Plankton Images

Type Article
Date 2022-06
Language English
Author(s) Panaïotis Thelma1, Caray–counil Louis1, Woodward Ben2, Schmid Moritz S.3, Daprano Dominic4, Tsai Sheng Tse4, Sullivan Christopher M.4, Cowen Robert K.3, Irisson Jean-Olivier1
Affiliation(s) 1 : Laboratoire d’Océanographie de Villefranche, Sorbonne Université, Villefranche-sur-Mer, France
2 : CVision AI, Medford, MA, United States
3 : Hatfield Marine Science Center, Oregon State University, Newport, OR, United States
4 : Center for Quantitative and Life Science, Oregon State University, Corvallis, OR, United States
Source Frontiers In Marine Science (2296-7745) (Frontiers Media SA), 2022-06 , Vol. 9 , P. 870005 (16p.)
DOI 10.3389/fmars.2022.870005
WOS© Times Cited 6
Keyword(s) plankton images, ISIIS, image processing, image segmentation, object detection, convolutional neural network, computer vision

As the basis of oceanic food webs and a key component of the biological carbon pump, planktonic organisms play major roles in the oceans. Their study benefited from the development of in situ imaging instruments, which provide higher spatio-temporal resolution than previous tools. But these instruments collect huge quantities of images, the vast majority of which are of marine snow particles or imaging artifacts. Among them, the In Situ Ichthyoplankton Imaging System (ISIIS) samples the largest water volumes (> 100 L s-1) and thus produces particularly large datasets. To extract manageable amounts of ecological information from in situ images, we propose to focus on planktonic organisms early in the data processing pipeline: at the segmentation stage. We compared three segmentation methods, particularly for smaller targets, in which plankton represents less than 1% of the objects: (i) a traditional thresholding over the background, (ii) an object detector based on maximally stable extremal regions (MSER), and (iii) a content-aware object detector, based on a Convolutional Neural Network (CNN). These methods were assessed on a subset of ISIIS data collected in the Mediterranean Sea, from which a ground truth dataset of > 3,000 manually delineated organisms is extracted. The naive thresholding method captured 97.3% of those but produced ~340,000 segments, 99.1% of which were therefore not plankton (i.e. recall = 97.3%, precision = 0.9%). Combining thresholding with a CNN missed a few more planktonic organisms (recall = 91.8%) but the number of segments decreased 18-fold (precision increased to 16.3%). The MSER detector produced four times fewer segments than thresholding (precision = 3.5%), missed more organisms (recall = 85.4%), but was considerably faster. Because naive thresholding produces ~525,000 objects from 1 minute of ISIIS deployment, the more advanced segmentation methods significantly improve ISIIS data handling and ease the subsequent taxonomic classification of segmented objects. The cost in terms of recall is limited, particularly for the CNN object detector. These approaches are now standard in computer vision and could be applicable to other plankton imaging devices, the majority of which pose a data management problem.

Full Text
File Pages Size Access
Publisher's official version 16 8 MB Open access
Supplementary Material 14 KB Open access
Top of the page

How to cite 

Panaïotis Thelma, Caray–counil Louis, Woodward Ben, Schmid Moritz S., Daprano Dominic, Tsai Sheng Tse, Sullivan Christopher M., Cowen Robert K., Irisson Jean-Olivier (2022). Content-Aware Segmentation of Objects Spanning a Large Size Range: Application to Plankton Images. Frontiers In Marine Science, 9, 870005 (16p.). Publisher's official version : , Open Access version :