Reviews in Aquaculture (2017) 9, 369–387 doi: 10.1111/raq.

12143

Application of machine vision systems in aquaculture with

emphasis on fish: state-of-the-art and key issues
Mohammadmehdi Saberioon1, Asa Gholizadeh2, Petr Cisar1, Aliaksandr Pautsina1 and Jan Urban1
1 Laboratory of Signal and Image Processing, Institute of Complex Systems, South Bohemian Research Center of Aquaculture and Biodiversity of

e Bud
ejovice, Nove Hrady, Czech Republic
Hydrocenoses, Faculty of Fisheries and Protection of Waters, University of South Bohemia in Cesk
2 Department of Soil Science and Soil Protection, Faculty of Agrobiology, Food and Natural Resources, Czech University of Life Sciences Prague,
Prague, Czech Republic

Correspondence
Mohammadmehdi Saberioon, Laboratory of
Signal and Image Processing, Institute of
Complex Systems, South Bohemian Research
Center of Aquaculture and Biodiversity of
Hydrocenoses, Faculty of Fisheries and
Protection of Waters, University of South


Bohemia in Cesk e Bud

ejovice, Z
amek 136, 37
333 Nov e Hrady, Czech Republic. Emails:
msaberioon@frov.jcu.cz; saberioon@gmail.com
Received 5 October 2015; accepted 28 Decem-
ber 2015.
Abstract
Demands of aquatic products are increasing dramatically during past decades.
Also quality assurance has gradually received more attention by both producers
and consumers. Thus, fish producers are exploring all possible approaches for
improving the productivity and profitability. Monitoring of fish state and beha-
viour during cultivation may help to improve profitability for producers and also
reduce the threat of severe loss because of disease and stress incidents. It is neces-
sary to evaluate and measure quality of fish products in accurate, fast and objec-
tive way for meeting the different demands of the fish-processing industry after
harvesting. Traditional methods are usually time-consuming, expensive, laborious
and invasive. Using rapid, inexpensive and noninvasive methods is therefore
important and desirable. Optical sensors and machine vision system provide the
possibility of developing faster, cheaper and noninvasive methods for in situ and
after harvesting monitoring of quality in aquaculture. This review describes the
most recent technologies and the suitability of different optical sensors for the fish
farming management and also assessment, measurement and prediction of fish
products quality. Two major areas of optical sensors applications in aquaculture
are discussed in this review: (i) preharvesting and during cultivation; and (ii)
post-harvesting. Finally, accuracy and uncertainty of optical sensors applications
in aquaculture are discussed. This review showed that MVSs and optical sensors
have found real-world application based on tremendous possibility offered by
digital camera development and increasing the speed of computer-based process-
ing; however, still new algorithms, methods and re-engineered sensors need to be
developed to meet real-world requirements.
Key words: accuracy and uncertainty, aquaculture, aquaculture management, fish quality,
machine vision systems, optical sensors.

producers and also decreases the risk of severe defeat and

Introduction
Nowadays, aquatic products’ quality assurance has increas-
ingly received attention by both producers and consumers.
Quality, freshness and authenticity of fish products would
determine the value, price and best-used-before date
(Menesatti et al. 2010; Costa et al. 2011a; Sun 2011). Fish
farmers are thus exploring all possible approaches for
improving the productivity and profitability of their prod-
ucts (Mathiassen et al. 2011).
Fish and fish products monitoring during different stages
may lead to higher productivity and profitability for
vertebral malformations because of disease and stress inci-
dents (Boglione & Costa 2011). In other words, by selecting
a suitable method to evaluate fish and fish products status,
it is possible to attain efficient management during differ-
ent growth stages and improve fish products quality in
aquaculture. Conventional techniques for monitoring and
assessment of fish quality are usually time-consuming,
invasive, costly and persistent. Therefore, using fast and
economical methods which would prevent any stress to
reared fish during different cultivation stages seems
essential.

© 2016 Wiley Publishing Asia Pty Ltd 369

M. Saberioon et al.
In the past three decades, optical sensor and machine
vision system (MVS) inventions are growing dramatically
and they are becoming more sensitive, powerful and
cheaper day by day. This trend provides possibility of
developing faster, cheaper and noninvasive methods for
in situ monitoring and after harvesting for quality monitor-
ing in aquaculture.
The intention of this review was to explore the literature
related to available tools such as optical sensors, MVSs and
their needs and limitations for fish farming management
and also assessment, measurement and prediction of fish
quality products.
This review investigates two major areas of optical sen-
sors and MVSs application in aquaculture, namely: (i) pre-
harvesting and during fish growth; and (ii) post-harvesting.
Accuracy and uncertainty of optical sensors and MVSs
application in aquaculture are considered as well.
Preharvest
Application of optical sensors, digital imaging methods and
computer processors in aquaculture has been increased
considerably during past years. As a result, many research-
ers have oriented more towards using optical sensors and
MVSs for analysing the individual fish features and quan-
tification of behavioural studies, despite the fact that fish
are sensitive and free to move in their environment in
which lighting, visibility and stability are not controllable
in most scenarios.
There are two types of optical sensors available for moni-
toring aquatic organisms, nonunderwater and underwater.
Nonunderwater optical sensors have been used for continu-
ous monitoring of fish features and behaviours such as fish
eating activity, detecting excess feeding and automatically
controlling the feeding process from a small aquarium (Lee
et al. 2014) to a big tank (Atoum et al. 2015). Nevertheless,
using nonunderwater camera has disadvantages such as a
high number of reflections on the water surface. Solar
reflections often result in camera saturation, which avoid
the correct performance of the image-processing algo-
rithms. These reflections can be compensated in indoor
scenarios (ponds with thermal covers and inside buildings)
by applying artificial lighting. However, in outdoor envi-
ronments, they are very difficult to compensate (Dios et al.
2003). Thus, usually optical sensors which are operating
underwater or in a wet environment are highly desired.
Another nonunderwater camera is the modulated light
detecting and ranging (LiDAR) systems. LiDAR system has
been used for fisheries application to overcome the limita-
tion of passive observation systems (Liang et al. 2006). A
LiDAR transmits and receives electromagnetic radiation at
high frequency and operates in the ultraviolet, visible and
infrared region. This system has also been employed to
370
study fish behaviour such fish schooling (Churnside 2003)
and study the correlation between fish species and their
environment (Reese et al. 2011) or species identification
and population estimation (Carrera et al. 2006).
Based on a recent review by Mallet and Pelletier (2014),
underwater video techniques have been increasingly used
for observing aquatic animals in the latest years. Typically,
underwater optical systems consist of a camera in visible
(Vis) (Shortis et al. 2007; Pelletier et al. 2012) or near-
infrared (NIR) (Gleason et al. 2007; Butler & Rowland
2009) range which placed within a waterproof housing
attached to a recorder and a power source. For instance,
Pelletier et al. (2012) developed a video technique called
STAVIRO which utilizing high-definition (HD) camera
enclosed in a rotating unbaited system and abiotic sea bot-
tom cover. The camera range of application can be
expanded by adding underwater light sources and laser-
based measurement system. Usually, colour cameras are
used for species identification in optimal environments
such as freshwater and underwater environments with
enough light.
Another type of underwater cameras is acoustic cameras,
which are the high-frequency multibeam sonar with a
unique acoustic lens system designed to focus the beam to
created high-resolution images (Belcher et al. 2002). These
cameras are much more suitable for turbid and low ambi-
ent light water. Mueller et al. (2006) used acoustic cameras
for detecting fish and sizing at camera range up to 16 m
under ice. Mizuno et al. (2015) also utilized acoustic cam-
era for fish classification.
Stereovision system, which includes two cameras in a
side-by-side arrangement, can provide simultaneous view
of fish from different positions (Watson et al. 2005). It has
been applied for measuring different fish features such as
size (Harvey et al. 2002, 2003; Costa et al. 2006) and also
fork length and length distribution in net cage (Torisawa
et al. 2011). Stereovision systems have also been used for
counting tuna stock during their transfer to aquaculture
cages (Costa et al. 2009). Application of stereovision sys-
tem as a low-cost solution for automation of some fish
farm operations, for example estimation of fish length and
weight of Gilthead sea bream (Sparus aurata) in sea cages
and tanks, has been reported as well (Dios et al. 2003).
Table 1 shows a summary of advantages and disadvantages
of different sensors in aquaculture.
Behaviour analysis
Fish are very sensitive to factors present within their captive
environment, being in small tanks or in sea cages, and they
respond to these factors with a range of behaviours (Little
& Finger 1990; Mancera et al. 2008). Monitoring fish beha-
viours would provide a better understanding of the coping
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

Table 1 Advantages and disadvantages of underwater cameras in aquaculture
Technique Principle Application
Single camera Vis-NIR Behavioural studies,
individual feature
analysis
Acoustic High-frequency Behavioural studies,
multibeam sonar fish classification
Stereovision Two cameras in Vis range Fish feature
measurement (size
and mass), counting
stocks
LiDAR Electromagnetic radiation Behaviour studies
at high frequency and (schooling, species
operates in the UV, Vis identification)
and IR region
and motivational mechanism of fish with environmental
cues. For instance, increasing leaping behaviour when fish
is infested with parasites (Furevik et al. 1993), and break-
down of schooling during feeding (Juell et al. 1993) or
change in vertical distribution when artificial light is used
(Oppedal et al. 2011). Monitoring of individual fish beha-
viours such as individual food intake or swimming speed in
response to environmental changes such as stocking, feed-
ing, net cleaning and grading can not only provide useful
information for improving production management
(Oppedal et al. 2011) but also it would help farmers to
observe fish behaviour as welfare indicators for the better
and effective management of aquaculture facilities (Brown
et al. 2006). Observing individual fish behaviour may also
assist in better understanding of group-based measure-
ment, because it is still unclear that group behaviour
depends on factors such as leadership or emerged from
self-organized behaviour (Sumpter 2010). Moreover, it can
help to realize schooling behaviours and communication
methods between individuals in a group more efficiently
(Delcourt et al. 2012). Fish behaviour analysis can also be
used for environmental risk assessment such as the pres-
ence of chemical agents in water (Kane et al. 2004; Masud
et al. 2005). Kane et al. (2004) showed the behavioural
changes in fish when exposed with m-amino-benzoic acid
ethyl ester methanesulfonate (MS222). Xiao et al. (2015)
also exhibited change in swimming speed and tail beat fre-
quently (TBF) as a result of the presence of chemicals in
water. Laboratory analysis in case of water quality monitor-
ing provides accurate information, but it is time-consum-
ing and expensive. Therefore, it is highly desired to provide
the complementary monitoring system that would be fast,
accurate and inexpensive. Continuous monitoring and
quantifying behavioural response of fish are potential
methods of assessing stress, disease and water pollution.
Although a number of different conventional monitoring
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
Advantages Disadvantages References
Easy to use, inexpensive, Cannot be used at Butler and Rowland (2009),
high spatial resolution night, labour-intensive Gleason et al. (2007),
Pelletier et al. (2012),
Shortis et al. (2007)
Useful for low ambient Low spatial Belcher et al. (2002)
light water resolution Mueller et al. (2006)
Accurate, high spatial Require extensive Harvey et al. (2002, 2003),
resolution calibration and Costa et al. (2006, 2009),
expertise Torisawa et al. (2011),
Dios et al. (2003)
Can be used at night, Expensive, Churnside (2003)
accurate expertise Carrera et al. (2006)
required
methods such as human vision inspection (Niizato & Gunji
2011), telemetry (Bridger & Booth 2003; Conti et al. 2006)
and marking or tagging (Block et al. 2005) are available,
they do not provide fast and accurate information. In some
cases, these methods are also invasive and can change and
modify fish behaviours, which do not represent all natural
interaction rules in fish groups (Dahlbom et al. 2011). For
instance, hand scoring which has conventionally been used
for analysis of fish position in a tank has low spatial and
temporal resolution (Bjerselius et al. 2001; Wibe et al.
2001). In other words, manual registration and analysis of
models to study fish behaviours are time-consuming and
not accurate, especially when large volumes of data are
available.
During past decade, MVS as an alternative to conven-
tional method has been used for real-time and offline mon-
itoring of fish (Salierno et al. 2008), which can provide
automate, inexpensive, noninvasive and accurate informa-
tion about fish behavioural parameters. MVS can be
defined as construction of explicit information and mean-
ingful descriptions of physical objects via image analysis
(Gli
nski et al. 2011). Video tracking, which provides the
possibility to track animals in a group, has attracted
researchers because of its high temporal and spatial resolu-
tion. Based on Maggio and Cavallaro (2011), video tracking
is the tracking of moving object(s) and monitoring their
activities by processing the image sequences obtained from
video cameras. Video analysis also plays a key role in auto-
matic measurement and analysis without human presence.
Different methods such as image analysis techniques
(Duarte et al. 2009) and motion analysis (Patullo et al.
2007; Grubich et al. 2008) have been used to quantify the
behaviour parameters automatically.
To date, a number of video tracking systems have been
developed for providing a trajectory of fish in their envi-
ronment. Kato et al. (2004) developed a computer vision
371
M. Saberioon et al.
system for quantifying Zebrafish (Brachydanio rerio, Cypri-
nidae) behaviour. Mirat et al. (2013) expanded an auto-
mated program called ZebraZoom to detect Zebrafish
(Brachydanio rerio, Cyprinidae) movement. Perez-Escudero
et al. (2014) also developed a visual system called idTracker
for tacking individuals, even siblings, in a group based on
finding fingerprint of each animal in a video recording of a
group. Recently, Fukunaga et al. (2015) built up a video
tracking system called GroupTracker by adapting a Gaus-
sian mixture model-based system for tracking fish even
under severe occlusion. Video tracking system has also been
applied to measure parameters such as velocity, swimming
distance, trace map and turning directions for studying
individual fish behaviour (Kato et al. 2004).
All above-mentioned studies conducted the tracking
system under the laboratory conditions, but some tried to
develop tracking system based on natural fish habitat where
condition may vary drastically by unavoidable and uncon-
trolled interferences such as murky water, algae in camera
lens and unknown objects. For example, Spampinato et al.
(2008) developed a fish tracking system based on CamShift
algorithm with overall accuracy of 85% for tracking fish in
the natural underwater environment.
Researchers believe that fish have social behaviour with
each other under cultural conditions; therefore, some beha-
vioural activities such as schooling should be studied
among more than one fish under the similar to real condi-
tion (Xu et al. 2006). Delcourt et al. (2011) demonstrated
the feasibility of multitracking glass eel (Anguilla anguilla)
as a transparent species using MVS and fluorescent visible
implant elastomer (VIE). They used this system for analys-
ing the swimming behaviour and weight losses according
to the swimming behaviours of multiple glass eel
(Anguilla anguilla). Hemelrijk et al. (2010) used several
video cameras and mirrors to determine the three-dimen-
sional (3D) position of fish (from 10 to 60 individuals) and
to measure the shape and internal structure of mullet
schools as a common habit of fish for reduction in preda-
tion.
Machine vision systems have been also employed for
monitoring various fish behaviours to find the major stress
factors that alter their reactions. Israeli and Kimmel (1996)
showed the correlation between fish speed and oxygen and
hypoxia stress, respectively. Nucci et al. (2010) recorded
Bluefin tuna (Thunnus thynnus) activity using digital video
acquisition system to study the correlation between fish
speed and potential stresses. They showed that not only
speed of swimming accelerated but also harmonious swim-
ming behaviour changed during the presence of divers,
which would have harmful effect on physical characteristics
of and mortality among fish stock. Xu et al. (2006) devel-
oped an algorithm based on alteration in projected area
caused by the movement of individual fish during frame
372
sequence captured at given time intervals. They showed
that their image-processing algorithm has the capability to
monitor tilapia (Oreochromis niloticus) response to hypoxic
condition as one of the serious fish mortalities in aquacul-
ture. Papadakis et al. (2012) expanded an inexpensive sys-
tem based on computer vision to quantify behavioural
variations of fish under stocking density as an important
stress factor in fish farms. They also showed a connection
between fish behaviour and stocking density and/or net
condition using MVS. In other words, they successfully
monitored fish inspection and biting of net due to stress
caused by stocking density.
Abnormality trajectory detection has also studied using
MVS. Beyan and Fisher (2013) employed underwater video
recording for detecting abnormal fish behaviours. They
proposed clustering and labelling data with 71% accuracy.
Pinkiewicz et al. (2011) also developed an algorithm for
detecting and tracking of fish in sea cage and automatically
analysing the fish movement and behaviour during the day
based on human activity or environmental changes.
Fish behaviours can be used for water quality assessment.
Actually, fish can be used as bio-indicators because of their
high sensitivity to changes in their aquatic environment
especially to pollution. Serra-Toro et al. (2010) developed a
video-based system for monitoring fish behaviour in clean
water and polluted water. Recently, Pautsina et al. (2014)
expanded a noninvasive crayfish (Orconectes virilis) cardio
monitoring system for monitoring water quality. They uti-
lized NIR optical sensors for monitoring cardiac activity of
crayfish (Orconectes virilis) as a promising biological sensor
for the detection of chemicals in water.
The tropical Zebrafish (Danio rerio) is an important ani-
mal model for the developmental and behavioural analysis
in biomedical research, disease modelling and drug devel-
opment (Liu 2007). It has an important role in investigat-
ing developmental biology as well. All above-mentioned
roles have made Zebrafish important for developing and
examining methods to monitor and quantify their beha-
viours precisely. Optical sensors together with a video ana-
lytic tool are able to provide the precise quantitative
measurement of Zebrafish behavioural abnormalities,
which can be seen in mutant fish for understanding the
nervous system function. Kato et al. (2004) developed a
MVS for quantifying Zebrafish behaviours. They exhibited
successful tracking of individual Zebrafish in tank with
97% accuracy. Tian et al. (2014) also built up a Zebrafish
motion analysis system using high-frame-rate videos based
on the fine motor movement and multiview trajectory.
Their system could distinguish between movement of wild-
type and transgenic Zebrafish.
Information about the feeding behaviour would be a
valuable input into the process of developing efficient
feeding management strategies. Parsonage and Petrell
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

(2003) created an automatic method using MVS to detect
sea cages’ pellets in images obtained from upward facing
video cameras. Atoum et al. (2015) lately developed an
automatic system based on video signal processing to
understand fish behaviour during feeding in highly dense
tank. However, many studies monitored increasing fish
activity during feeding, but still there is little information
about behavioural dynamics fish population during feeding
(Alfredsen et al. 2007; Oppedal et al. 2011).
One of the most common technical difficulties which
researchers encounter during studying multiple individual
fish in a same area using optical sensors is occlusion.
Occlusion occurs when there are many individuals moving
together or some individuals swimming close to each other
at the same time or in several time units in succession. It
can also be defined as the phenomenon of two or more
tracked target objects becoming one during a time period
(Delcourt et al. 2012). Typically, there are two types of
occlusion: (i) when two fish are swimming so close that can
be falsely perceived as a single longer fish; and (ii) when
two trajectories cross and two fish are perceived as a single
T- or X-shaped individual (Dolado et al. 2014). Detecting
occlusion and tracking individuals during and after occlu-
sion have been studied by some researchers. For instance,
Kato et al. (2004) suggested using erosion and dilatation
system for solving occlusion problem when more than one
fish is tracked. Perez-Escudero et al. (2014) expanded an
algorithm by identifying individual fish, using fingerprints
from occlusion-free portions of video and then used the
signature to resolve the occlusions and identify switches.
Dolado et al. (2014) developed an image-processing
method for the detection of occlusion in video and separate
individuals involved in two-dimensional (2D) space. More-
over, Baum et al. (2014) created an algorithm for overcom-
ing occlusion problem by employing hybrid filter and game
theory-related reinforcement. Another suggested method
for overcoming occlusion problem is studying fish in 3D
environment. Tracking animals in 3D using MVSs has been
studied by different approaches such as light field video
cameras in which composite optics that are used to simul-
taneously capture images focused at different distances
from lens, therefore, allow to reconstruct scene in 3D (Mat-
sumoto et al. 2013). 3D tracking system has also been used
in single image of reflections or shadows from the surface
of a 3D object such as spherical mirror (Kanbara et al.
2006; Chen et al. 2011). Nevertheless, multiple cameras are
usually employed to reconstruct 3D scene for tracking
objects. For example, Spitzen et al. (2013) used two mono-
chrome charge-coupled-device (CCD) video cameras to
study in-flight behaviour of individual malaria mosquito to
human odour and heat in 3D space. Cachat et al. (2011)
reconstructed 3D environment using images from two
video cameras, and manually recorded the position of
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
individual Zebrafish to understand the neurophenotyping
of adult Zebrafish in 3D environment. Similarly, some
other researchers tried to track multiple objects using mul-
tiple cameras. Viscido et al. (2004) employed stereovision
system to track 4 to 6 groups of giant danios (Danio ae-
quipinnatus). Veeraraghavan et al. (2006) proposed a
method based on motion information to track multiple
bees using two video cameras. Wu et al. (2011) developed
algorithm by solving three linear assignment problems for
tracking multiple fruit fly using two video cameras auto-
matically. Usually, synchronizing multiple cameras needs
different hardware and complicated software; it also
requires more handlings, which may affect animal beha-
viours (Dell et al. 2014). Besides, spatial resolution of
images dramatically drops when objects move away from
sensors (Gokturk et al. 2004).
Recently, a new hardware based on single-point 3D
imaging technology (e.g. Microsoft Kinect or Asus’s Xtion
Pro) has been introduced. This hardware can provide 3D
single points in real time by combining a colour video cam-
era, an IR video camera with an IR projector to create
defined IR laser light pattern which leads to gain depth
information. Single-point 3D imaging technology offers
information with less computational and power consump-
tion in comparison with other methods such as stereovi-
sion system for obtaining information in 3D space, which
makes it an ideal sensor for monitoring in real-time with
less handling process. Saberioon and Cisar (2016) have
introduced a novel method for tracking multiple fish in 3D
using Microsoft Kinect. They successfully (98%) tracked
multiple fish in an aquarium. Their system can provide
high spatial resolution at relatively high frame rate, which
allows researchers to have much more detailed information
for applications such as behavioural studies where precision
information is required.
Individual fish features analysis
Fish farmers need to have information about individual fish
features such as length, weight, sex, maturity and skin col-
our during different growth stages to monitor growth sta-
tus for better stock management. These individual fish
features are more crucial for quality assessment of orna-
mental fish because breeding and production of high-qual-
ity fish is based on selection, by repeated sorting and
grading of many fish according to gender and quality dur-
ing grow-out (Gomelsky et al. 1995; Zion et al. 2008). The
main common conventional techniques used for studying
and monitoring individual fish features are extractive tech-
niques (Trenkel & Cotter 2009; Jouffre et al. 2010), which
are providing accurate information about individual fish,
but they are used more for inventories and small-scale
observations than for monitoring. Furthermore, extractive
373
M. Saberioon et al.
techniques such as net casting are invasive, which is not
popular in the context of monitoring.
Skin colour has an important role in displaying physio-
logical, behavioural and ecological status of aquatic organ-
isms (Pavlidis et al. 2006). Skin coloration pattern in fish is
not only representative of growth rates but also it exhibits
the welfare of fish. When fish are stressed, they undergo
various metabolic changes, all of which are expressed exter-
nally by various parameters including skin colour changes.
Moreover, skin colour provides useful information for
planning the accurate feeding. Skin colour in ornamental
fish even determines their market price. Researchers have
used different optical sensors for measuring and determin-
ing skin colour in fish. Pavlidis et al. (2006) used portable
spectrocolormeter to measure the 3D characteristics of col-
our appearance (CIE L, a*, b*) of wild (Pagrus pagrus,
Pagrus caeruleostictus and Dentex gibbosus) and farmed red
skin Sparidae. They introduced new index called Entire
Colour Index (ECI) for exhibiting fish skin colour pattern.
Wallat et al. (2005) demonstrated how a MVS could be
employed for objective measurement of the skin colour of
live goldfish (Carassius auratus) and to optimize the feed-
ing scheme in order to achieve the most desirable skin
colour. Za
tkova et al. (2011) utilized a MVS to estimate
changes in skin colour of wels catfish (Silurus glans). They
showed the feasibility of MVS for monitoring skin colour
changes due to diet alteration. Colihueque et al. (2011)
developed a method for estimating skin colour, spottiness
and darkness using computer-based image analysis for cate-
gorizing cultured rainbow trout (Oncorhynchus mykiss).
Urban et al. (2012) created an algorithm for measuring fish
skin colour in Vis bands. Their algorithm can also provide
the dominant wavelength of fish skin. Moreover, recently
Segade et al. (2014) showed the effect of diet on seahorse
(Hippocampus hippocampus) body colour using MVS.
Another important fish feature, which farmers need to
know, is fish size. Fish size can help farmers to have better
decision about grading, harvesting and ultimately time to
harvest the stock (Beddow et al. 1996). Farmers need fish
size and mass information during different growth stages to
monitor growth rate and plan accurate feeding to avoid
excess of food as one of the most contributors to waste and
water pollution. The most conventional method for mea-
suring size is to catch and anaesthetize fish. This method is
accurate but it is labour-intensive, time-consuming, inva-
sive and just representing small portion of population
(Lines et al. 2001).
Image processing in aquaculture as noninvasive tech-
nique is attracting researchers for estimating fish mass and
size in past three decades (Zion 2012). MVS provides the
possibility of measuring size and length of fish, remotely
without causing stress in fish. Dios et al. (2003) reported
application of stereovision system to predict mass by
374
measuring fin-to-fin length, height and length dimension
with 96% accuracy in nursery and 95% accuracy in sea
cage, respectively. Moreover, salmon biomass with less than
4% error (Beddow et al. 1996) and reef fish with 95% accu-
racy (Harvey et al. 2001) have been predicted using stereo-
vision system.
Researchers have found different image parameters as
promising indices for measuring mass in fish. For example,
Zion et al. (1999) used fish area to estimate fish mass of
grey mullet (Mugil cephalus), carp (Cyprinus carpio) and
St. Peter (Oreochromis) with 95%, 99% and 99% accuracy,
respectively. Odone et al. (2001) tried to find the relation-
ship between shape and weight using support vector
machine (SVM) as a machine learning system, and exhib-
ited a significant accuracy of weight prediction with only
3% error rate. de Verdal et al. (2014) studied the correla-
tion between body weight of European sea bass (Dicentrar-
chus labrax) larvae with information from image. They
revealed reliable correlation (r2 = 0.98) between parameters
such as area, perimeter, length, height and volume, which
can be acquired from image with body weight. Moreover,
Viazzi et al. (2015) demonstrated the possibility of estimat-
ing mass of Jade perch (S. barcoo) by measuring area of fish
from side view. Their model could predict the mass with
significant correlation using area and length information
from image.
Researchers estimated fish length by fitting Point-Distri-
bution Mode (PDM) manually (Tillett et al. 2000) or auto-
matically (Lines et al. 2001) for segmenting fish images,
which ultimately has been used to measure size and mass.
They successfully achieved 95% and 98% overall accuracy
in comparison with manual measurement. However, accu-
racy of these methods would be decreased when live fish
images acquired from commercial fish cages due to poor
image condition and large fish density. Torisawa et al.
(2011) used stereovision system positioned at different
depths in commercial sea cage to estimate length of Pacific
bluefin tuna (Thunnus orientalis). They reported 5% fish
length estimation error for tuna allocated up to 5.5 m away
from the cameras.
Dual-camera optical ranging system has also been used
for estimating fish size and mass. Costa et al. (2006) devel-
oped a system to measure size and mass using two under-
water cameras and artificial neural network (ANN). They
showed high accuracy for measuring fish size. However,
accuracy is dramatically decreased when small fish are sub-
jected for measurement.
Fish species identification is important for fishery man-
agement. Farmers need to sort harvested fish according to
species and size for better marketing in polyculture fish
farming system. Besides, farmers can use the fish species
identification for feeding strategies (Gerking 1994; Alcaraz
et al. 2015) and stock assessment (Hoggarth 2006). Fish
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

species identification also can be used to have more accu-
rate fish disease diagnosis. Fish diseases may manifest dif-
ferent symptoms in different species of fish. To address this
problem, fish species must be properly classified first before
the disease can be identified (Hu et al. 2012).
Traditionally, decimation is a common practice for sort-
ing and preparing fish for harvesting at specified intervals
during the season. Decimation increases overall fish pro-
duction by decreasing population density and reduces
growth disturbance caused by competitive interactions
between big and small fish. However, it is labour-expensive
operation and it is harmful due to causing stress for fish
during different growth stages (Spampinato et al. 2008;
Zion 2012). Development of means and methods for con-
tinuous and automated underwater selective fish harvesting,
thus would be useful to overcome such problems.
Machine vision system and image processing from
underwater images have been used for fish species identifi-
cation. Researchers used two feature-based systems for
species identification, namely geometry-based features and
skin-based features. Geometry-based features such as area,
width, height and shape are first approach for identifying
the species. Zion et al. (2000) used moment invariants to
extract typical features from side-view images of fish tails
and used them for grey mullet (Mugil cephalic), St. Peter’s
fish (Sarotherodon galilaeus) and carp (Cyprinus carpio)
with accuracies of 100%, 91% and 91%, respectively. They
took images from fish when they were swimming through a
narrow channel with their sides to the camera. They also
used the background illumination to overcome water
opaqueness and to generate high image contrast. Storbeck
and Daan (2001) measured a number of geometric features
such as width and length at various locations along the
body of the fish as seen by a camera located perpendicular
to a conveyor belt simultaneously. Then, they applied ANN
for identifying species. Zion et al. (2007) built up a real-
time underwater computer vision system for common carp
(Cyprus carpio), tilapia (Oreochromis spillers) and grey
mullet (Mugil cephalic) by extracting size and orientation-
invariant features from fish silhouettes. Aguzzi et al. (2009)
utilized K-nearest neighbour’s classification algorithm to
classify the fixed segmented area of fish shape from video
footage of deep-sea camera (1100 m). They could identify
Zoarcid fish (eelpouts) with 78% accuracy, red crabs
(Paralomis multispina) with 43% accuracy and snail
(Buccinum soyomaruae) with 63% accuracy. Rodrigues
et al. (2010) created an image-processing technique based
on scale-invariant feature transform and principal compo-
nent analysis (PCA) for parameterizing shape, appearance
and motion. They used artificial immune network and
adaptive radius immune algorithm for clustering individu-
als of the same species. They reported 92% accuracy using
their system for the classification of fish species. del Rıo
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
et al. (2013) introduced new automated video-imaging
protocol to count fish at day and night in a continuous
fashion; however, their protocol can be used to extract fish
geometry parameters. They implemented colorimetric cali-
bration that could efficiently discriminate suitable image
for fish feature extraction. Their protocol contributes to an
idea for developing a tool for long-lasting monitoring of
marine animals communities.
Skin-based features such as colour and texture are also
studied for species identification. White et al. (2006) used
10-shape features (grid-line lengths) and 114-colour fea-
tures to classify seven trawl-catch fish by species while being
conveyed on belt with accuracy of 99.8% for the identifica-
tion of seven species. Hu et al. (2012) created a method of
species classification based on colour and texture features
and using a multiclass SVM. They could classify seven spe-
cies with accuracy from 93.33% to 100%. Table 2 shows
summary of optical sensors’ application in aquaculture
before harvesting and during cultivation.
Post-harvest
Fish and fish products usage has been increased in recent
decades and covers 16% of human diet all around the
world (Alasalvar et al. 2011). The estimated value of total
consumption of fish and fish products only in European
Union is 25 billion Euros, which makes this market very
competitive (European Commission, Trade Statistics 2009,
Fisheries). Furthermore, consumers’ expectation and
awareness about the quality of food products have also
been raised. Quality of fish production enhances competi-
tiveness among the producers and leads them to explore
technological solution for improving productivity and
profitability.
Fish and fish products assessment can be categorized into
two groups: physical and intrinsic chemical attributes;
however, physical attributes such as size, shape, weight, col-
our and texture are attended highly by consumers as they
play the most significant role in purchase decision (Shew-
felt & Bruckner 2000; Alasalvar et al. 2011). Other physical
characteristics such as surface defects are also usually set as
a basic for fish products sorting. In other words, external
attributes are always the visual impressions that directly
involve the acceptance and satisfactory of fish products for
consumers. Therefore, it is necessary to evaluate and mea-
sure external attributes to meet the different market
demands for fish-processing industry.
Traditionally, the visual quality of fish products is deter-
mined by expert inspection; however, it is time-consuming,
labour-expensive, and imprecise (Balaban et al. 2008).
Automated visual inspection such as computer vision
system is undergoing substantial growth, due to its cost-
effectiveness, consistency, accuracy and superior speed.
375
M. Saberioon et al.

Table 2 Summary of optical sensors applications preharvesting

Application Principal Technique Species References

Individual fish Fish skin colour Portable Pagrus pagrus, Pavlidis et al. (2006)
features spectrophotometer Pagrus caeruleostictus and
analysis Dentex gibbosus and farmed
red skin Sparidae
Single camera Live goldfish, Cultured Wallat et al. (2005),
rainbow trout Colihueque et al.
(Oncorhynchus mykiss), (2011), Segade et al.
Seahorse (Hippocampus (2014)
hippocampus)
Fish size Stereovision Salmon, Reef fish, Grey mullet Odone et al. (2001), Beddow et al.
(Mugil cephalus), Jade perch (1996), Harvey et al. (2001),
S. barcoo, Pacific bluefin Dios et al. (2003), Torisawa et al.
tuna (Thunnus orientalis) (2011)
Dual camera Not specified Costa et al. (2006)
Single camera Grey mullet (Mugil cephalus), Zion et al. (1999), Viazzi et al.
Carp (Cyprinus carpio) and (2015), Odone et al. (2001)
St. Peter (Oreochromis)
Fish species Single camera Grey mullet (Mugil cephalic), Zion et al. (2000, 2007), Storbeck
identification St. Peter’s fish and Daan (2001), Aguzzi et al.
(Sarotherodon galilaeus) and (2009), Rodrigues et al. (2010),
Carp (Cyprinus carpio), Red Hu et al. (2012), White et al. (2006)
crabs (Paralomis multispina),
Zoarcid fish (Eelpouts), Snail
(Buccinum soyomaruae)
Behavioural Individual fish Single camera Zebrafish (Brachydanio rerio, Kato et al. (2004), Mirat et al.
studies behaviour Cyprinidae) (2013), Liu (2007), Tian et al. (2014)
Multiple fish Single camera – Delcourt et al. (2011),
behaviour Hemelrijk et al. (2010)
Study stress factor Single camera – Israeli and Kimmel (1996),
on fish Xu et al. (2006),
Papadakis et al. (2012)
Abnormal trajectory Single camera – Beyan and Fisher (2013),
studies Pinkiewicz et al. (2011)
Water quality Single camera – Serra-Toro et al. (2010),
assessment Pautsina et al. (2014)
Feeding behaviour Single camera – Parsonage and Petrell (2003),
studies Lee et al. (2014)

There is increasing evidence that MVS is being adapted at
commercial level (Brosnan & Sun 2004) not only because
of its reliability and accuracy (Misimi et al. 2008) but also
because it is inexpensive, fast and not labour in compare to
traditional methods. In this context, optical sensors play
important role in several major areas of fish quality assess-
ment including automated sorting, grading and processing.
Machine vision system
Machine vision system has been promoted by researchers
for food quality inspections during past decades. Exclu-
sively, many researchers have used MVS for quality assess-
ment of aquatic products (Hong et al. 2014). MVS can
recognize objects, it can also extract and analyse quantita-
tive information from digital images. It has the ability to
improve the quality of human vision by electrically perceiv-
ing and understanding of an image (Brosnan & Sun 2004).
It opens up the possibility of studying objects in electro-
magnetic spectrum regions in which the human eye is not
sensitive such as ultraviolet (UV) or IR regions (Zude
2008). This system offers image information at the pixel
level for explaining quality features of food products which
permits availability of more accurate information (Sun
2011).
Typical MVS consists of two steps, namely image acqui-
sition and image processing. Optical sensor, image acquisi-
tion board, frame grabber and lighting system are main
components of the image acquisition system (Chen et al.
2002), which are used to obtain images and convert them
to digital form. Optical sensors which are used in MVS
are usually based on solid-state CCD or complementary

Reviews in Aquaculture (2017) 9, 369–387

376 © 2016 Wiley Publishing Asia Pty Ltd

metal–oxide–semiconductor (CMOS) camera technology
of the line-scan type using thermionic tube device (Shortis
et al. 2007). Both monochrome and polychrome cameras
have been applied throughout the fish industry (Brosnan &
Sun 2004). Then digital images are processed in three
phases: (i) preprocessing for removing noises and improv-
ing images; (ii) segmentation and feature extraction (Sun
2011); and finally (iii) classification, which is performed for
identifying objects of different clusters for final decision
(Du & Sun 2004). MVS capabilities have been compared to
human inspection (Misimi et al. 2008), sensorial panel
(Quevedo & Aguilera 2010), colorimeter (Yagiz et al. 2009)
and spectrophotometer (Kong et al. 2007). It was con-
cluded that MVS can outperform other methods in some
cases (i.e. sensory panel and human inspection) or at least
as reliable as other instruments (i.e. colorimeter and spec-
trophotometer) for quality assessment of food products.
One of the important steps, which improve the prof-
itability, is sorting or grading of fish products. Fish should
be graded for economical reasons such as increasing the
value of crop and fish products. Producers also need to
record information about species, weight and size to reduce
the required labour time on-board vessels and increase the
traceability.
Automate fish sorting techniques using MVS have been
investigated intensively to reduce the cost and increase the
precision. MVS uses fish features namely morphological
features such as size, volume, weight and shape for com-
mercial sorting, grading fish and fish products after har-
vesting (White et al. 2006; Quevedo et al. 2008; Veliyulin
et al. 2011).
Nakai et al. (2008) extended a MVS for fish sorting. They
used web camera to capture images and adopting a flexible
sorting system based on ANN model. They have used 2D
fast Fourier transform (2DFFT) for feature extraction and
generate input for ANN. White et al. (2006) created a MVS
called CatchMeter to sort fish with 99.8% accuracy. Their
system could determine orientation of the fish, identifying
the flatness or roundness of fish, measure the length and
determine the fish species. However, their system is limited
to fish with large aspect rations where only few species are
available.
Ibrahim and Sultana (2006) developed a system for sort-
ing fish based on size. They used K-means for clustering
and segmentation of fish images from the background and
measured the length for final sorting. Costa et al. (2013b)
utilized shape analysis of digital images of cultured seabass
(Dicentrarchus labrax, L.) to sort them based on size, sex
and the presence of abnormalities.
The determination of fish length is leading to calculate
the weight. As Martınez-Palacios et al. (2002) revealed the
linear relationship between weight and length with correla-
tion determination of 99%. Balaban et al. (2010) also
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
proved that there is a significant correlation between view
area, obtained using MVS and weight. Moreover, G€ um€ ußs
and Balaban (2010) predicted the rainbow trout
(Oncorhynchus mycosis) weight based on view area. Stereo-
vision systems are also used for accurate high-speed weight
estimation of herring (Clupea harengus) by employing the
linear regression of multiple 2D and 3D features. Skin col-
our has also been used for fish sorting based on gender,
which is important for marketing and breeding pro-
grammes. For instance, Zion et al. (2008) generated an
image-processing algorithm for sorting guppy fish (Poecil-
ia reticulata) based on gender. The applied algorithms have
been derived from shape and colour differences between
female and male guppies.
Surface colour of fish or fish products is always the first
parameter which consumers perceive and use in their deci-
sion. Colihueque et al. (2011) showed the importance of
Blue Black (BB) phenotype in adult rainbow trout
(Oncorhynchus mycosis) as attractive trait in some markets,
which may due to higher growth rate (23%) in BB than
other skin colour phenotypes which makes them more
demanded in market; therefore, analysis of skin colour of
fish products is important. Kause et al. (2008) used MVS
for analysis of cutlets as a selection tool to genetically
improve flesh composition and colour in large rainbow
trout (Oncorhynchus mykiss). Yagiz et al. (2009) compared
the performance of MVS and Minolta colorimeter for mea-
suring colour of irradiated Atlantic salmon (Salmo salar)
fillets. They pointed that colours measured by MVS are
much closer to average real colour of Atlantic salmon
(Salmo salar) fillet than those measured by Minolta col-
orimeter. Quevedo et al. (2008) showed the correlation
between CIE La*b* colours and quality of salmon
(Salmo salar) fillets. They also compared their results with
sensory panel and proved no difference in measurement of
the salmon (Salmo salar) fan scores between both methods.
G€ um€ ußs and Balaban (2010) revealed that colour informa-
tion such as CIE La*b* can be used as an additional param-
eter in sorting system. Costa et al. (2013a) developed a tool
based on image processing and partial least squares (PLS)
to discriminate European sea bass reared under organic
condition from conventional products. They found that
fish under organic protocol are lighter than ones raised
under conventional conditions.
Changes in area, shape and thickness of Atlantic cod
(Gadus morhua) and Atlantic salmon (Salmo salar) fillets
during rigor mortis and ice storage, as affected by peri-
mortem handling stress, also measured by imaging technol-
ogy in 2D and 3D modes (Erikson & Misimi 2008). Two
extremes of perimortem handling stress had a large effect
on the onset time of rigor mortis, although they had a neg-
ligible influence on the appearance of the fillets a few days
after slaughter. It was reported that exposure of salmon and
377
M. Saberioon et al.
cod against two extremes of perimortem stress and subse-
quent ice storage would cause significant changes in fillet
length, roundness, area and cross-sectional height profiles
during storage.
Stien et al. (2006) used MVS for automated measure-
ment of the same approach performed for rainbow trout
(Oncorhynchus mykiss) fillets, compared to other methods,
concluded that the imaging-based method was more accu-
rate and had higher flexibility than measuring isometric
and isotonic tension, on isolated muscle strips, using a
rigor meter.
Machine vision system has also been applied for deter-
mining the freshness as one of the factors in the quality
assessment of aquatic products. Conventionally, chemical
analysis has been used for determining the quality and
freshness accurately, but they are time-consuming, expen-
sive and destructive. Jarmin et al. (2012) displayed the fea-
sibility of freshness assessment using MVS, they also
compared the Torrymeter and MVS for detecting fish
freshness. They found better freshness determination by
Torrymeter compared to MVS; however, MVS showed
more accurate spoiling time in fish. Quevedo and Aguilera
(2010) developed an algorithm for estimation of firmness,
as an index for exhibiting the fish freshness in Atlantic sal-
mon (Salmon salar) filets based on colour analysis. Costa
et al. (2013c) proposed a method for evaluating freshness
based on the colorimetric imaging of the whole external
body of gilthead seabream (Sparus aurata) using PLS. They
found a significant colorimeter difference between fresh
and nonfresh fish.
Fat content as one of the most important characteristics
of fish fillet quality and freshness is also needed to be mea-
sured. Some researchers employed MVS and image-proces-
sing techniques to estimate fat content of Atlantic salmon
(Salmo salar) fillets (Stien et al. 2007b), rainbow trout
(Oncorhynchus mykiss W.) cutlets (Stien et al. 2006) and
brown trout (Salmo trutta) cutlets (Marty-Mahe et al.
2004).
Hyperspectral imaging
Hyperspectral imaging (HSI) technique, regarded as an
emerging technology, combines spectroscopy with imaging
technology into one system to characterize tested products
with detailed spectral and spatial distribution information
simultaneously (ElMasry & Sun 2010). Using HSI, some
chemical and physical attributes of fish samples can be
visualized and mapped, which enables HSI to better expla-
nation of the quality distribution in aquatic products.
In the HSI, the spectral reflectance of each pixel is
acquired for a range of wavelengths in the electromagnetic
spectra. The wavelengths may include the Vis and IR
regions of the electromagnetic spectra. The HSI is similar
378
to multispectral imaging, the difference is a broader range
of wavelengths (more number of spectral bands) which is
been scanned for each pixel in the HSI. The resulting infor-
mation is a set of pixel values (intensity of the reflectance)
at each wavelength of the spectra in the form of an image
(Kim et al. 2001; Mahesh et al. 2008; Gowen et al. 2010;
Manickavasagan & Jayasuriya 2014).
Gowen et al. (2007) reviewed the application of HSI
for food quality control and food safety applications.
The authors discussed the components of HSI, different
image-processing techniques and various applications in
food quality and safety. Liu et al. (2013) recently
reviewed feasibility of NIR spectroscopy and HIS to
quantify and evaluate of chemical composition (fat,
protein and moisture), microbiological (freshness, spoi-
lage and nematodes) and sensory (flavour, texture and
colour) attributes of fish and fishery products. More-
over, HSI has been used for quantifying many fish and
fish products features such as fat distribution (ElMasry
& Sun 2010), texture (Wu et al. 2012), pH (He et al.
2012) and moisture (He et al. 2013) with high signifi-
cant prediction ability (r > 0.8). Fat and water content
in fillets of multiple fish species, including Atlantic hal-
ibut, catfish, cod, saithe, mackerel and herring were
also estimated using HSI (ElMasry & Wold 2008). Fur-
thermore, Costa et al. (2011b) has used HSI to discrim-
inate between tank-cultured and sea cage-cultured
European sea bass (Dicentrarchus labrax). They could
successfully discriminate (87%) individual animals at
48 h post-mortem.
The usefulness of HSI for food security assessment has
been proved as well. For instance, Cheng and Sun (2015)
used HSI for determining total viable counts to evaluate
microbial spoilage of fish fillets as a rapid and noninvasive
manner. He et al. (2014) assessed the feasibility of HSI for
evaluating surface lactic acid bacteria spoilage of farmed
Atlantic salmon (Salmo salar) during cold storage time.
They showed that HSI has the potential to estimate lactic
acid bacteria in Atlantic salmon (Salmo salar). Moreover,
high-resolution imaging spectroscopy in interactance mode
has been used for the detection of blood and melanin spots
in fresh and smoked salmon fillets (Mathiassen et al. 2011).
The imaging spectroscopy set-up (e.g. QVision QMoni-
tor) has been developed and commercialized to be adopted
by industry sector. This system is capable of analysing fish
fillets with respect to fat content, water content, salt con-
tent, colour, pigment (astaxanthin) content, blood spots,
black spots, melanin spots and other discolorations (Segt-
nan et al. 2009). Another online industrial HSI system has
also been expanded by Sivertsen et al. (2012) for assessing
freshness of fish. Their system can evaluate the freshness
with r between 0.82 and 0.91 on individual fillet per
second.
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

Thermal imaging
Thermal imaging (TI) or IR thermography is an emerging
analytical technique, suitable for the food industry. It is a
2D, noncontact diagnostic technique for measuring tem-
perature emitted by a body surface of materials, which can
be employed in nondestructive quality evaluation (Giorleo
& Meola 2002; Arora et al. 2008). TI utilizes the radiation
emitted to produce a pseudo image of the thermal distribu-
tion of a body surface. In thermography, a large number of
point temperatures are measured over an area and pro-
cessed to form a thermal map or thermogram of the target
surface. The basic principle of TI is based on the fact that
all materials emit IR radiation. The IR is a band of invisible
light found on the electromagnetic spectrum in the wave-
length region of 0.75–100 mm; this region comprises the
near (0.75–2.5 mm), short-wave (1.4–3 mm), mid- (3–
8 mm), long-wave (>8 mm) and extreme (15–100 mm) IR
regions.
Thermal imaging systems are suitable for a wide range of
applications due to their portability, real-time imaging, and
noninvasive and noncontact temperature measurement
capability. They can also be applied to real food systems
without alteration (Gowen et al. 2010). Unlike multispec-
tral and hyperspectral imagery, TI does not require an illu-
mination source; however, integrated systems for active
thermography measurements (also known as pulsed ther-
mography or thermal wave imaging systems) incorporate a
heating or cooling unit to provide a thermal differential.
Kong et al. (2007) utilized TI to investigate quality changes
of salmon fillet muscle during thermal sterilization. TI usu-
ally can be used as a supplementary method for detecting
foreign materials, which could not be distinguished by
mechanical and optical methods (Meinlschmidt &
Maergner 2002; Ginesu et al. 2004).
X-rays
X-rays are a form of electromagnetic radiations with wave-
lengths between 0.01 and 10 nm, corresponding to energies
in the range 120 eV to 120 keV. Imaging systems for X-rays
are two main types: planar and computed tomography
(CT). Planar X-ray imaging generates an image of the
X-ray absorption between the X-ray generator (source) and
a line or area sensor, thus resulting in flat (2D) images. CT,
on the other hand, uses digital geometry processing to gen-
erate a 3D image of the inside of an object from a large ser-
ies of one or 2D X-ray images taken around a single axis of
rotation (Veliyulin et al. 2011).
Kolstad et al. (2004) applied CT for analysing of fat con-
tent and distribution in whole Atlantic halibut (Hippoglos-
sus hippoglossus L.). Romvari et al. (2002) used CT
scanning with a high number of scans in spiral-scanning
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
mode for successful estimation of fat and protein content
of live anaesthetized carp and pike-perch. Segtnan et al.
(2009) demonstrated the feasibility of using CT scans at
one, two and three voltages 80, 110 and 130 kV to predict
the content of NaCl, but deemed the use of CT impractical
for real-time monitoring. H aseth et al. (2009) showed how
CT scanning at several voltages (80, 110 and 130 kV)
enables a quantitative and distributional analysis of salt,
protein and water content of dried/salted cod fillets.
Mery et al. (2010) and Jamieson (2002) showed detec-
tion of bones using a planar X-ray imaging system at 40 kV
and advanced image-processing methods. Their systems
managed to identify bones with 99% accuracy. Veliyulin
et al. (2011) revealed that planar X-ray imaging at 40 kV
enabled accurate estimation of the weight of whole herring
and recommended that further research should be carried
out using online high-speed conveyor-based linear X-ray
imaging. Table 3 shows the summary of optical sensor
applications after harvesting.
Accuracy and uncertainty
Human knowledge about the environment variables is lim-
ited, instruments cannot measure the variables with ideal
accuracy, samples are not exhaustive, and simplifications of
the real world are essential when resources are restricted
(Heuvelink et al. 2007). Environmental data are rarely cer-
tain, and these errors may be difficult to quantify in prac-
tice. In the absence of such knowledge, we would be
uncertain about the true state of the environment. Uncer-
tainty is an expression of confidence about knowledge, and
therefore, it is subjective. Quantifying uncertainty is essen-
tial for establishing the value of data as input to decision
making (Brown 2004) and for judging the reliability of
decisions that are informed by the data (Beven 2000).
Determining the causes of uncertainty in environmental
research as well as directing resources towards improving
data quality is also important (Brown & Heuvelink 2006).
The predictive ability of MVS and data acquired from
optical sensors are limited due to number of factors, which
negatively influence the estimated prediction which
requires more attention. Moreover, because the capturing
and analysing steps of optical information are frequently
expanded, there is an essential need to quantify and evalu-
ate the associated uncertainties in the context of the appli-
cation examples.
Some factors such as illumination negatively influence
the accuracy of the recorded data using optical sensors.
Illumination plays an important role in the overall effi-
ciency and accuracy of the system; it greatly affects the
quality of image. An appropriate light system can reduce
the effect of reflection, shadow and some noise-making
parameters, thereby reducing the required time of image
379
M. Saberioon et al.

Table 3 Summary applications of optical sensors after harvesting

System Purpose Technique References

Machine Fish sorting Using morphological White et al. (2006), Quevedo et al. (2008), Veliyulin et al. (2011)
vision system features (size, volume, weight)
Size and fish area image Nakai et al. (2008)
Measuring length, shape and White et al. (2006)
determining fish species
Using size based on K-mean Ibrahim and Sultana (2006)
Weight Balaban et al. (2010), Gu€mu€ßs and Balaban (2010),
Martınez-Palacios et al. (2002)
Fish quality Skin colour Kause et al. (2008), Yagiz et al. (2009), Quevedo et al. (2008),
Gu€ mu
€ ßs and Balaban (2010), Costa et al. (2013a)
Changes in area, shape and Erikson and Misimi (2008), Stien et al. (2007a)
thickness
Freshness Jarmin et al. (2012), Quevedo and Aguilera (2010), Costa et al. (2013c)
Fat content Stien et al. (2007b), Marty-Mahe et al. (2004)
Hyperspectral Physical Fat distribution ElMasry and Sun (2010), ElMasry and Wold (2008)
imaging system attributes Texture Wu et al. (2012)
Freshness Sivertsen et al. (2012)
Chemical pH He et al. (2012)
attributes Fat content Segtnan et al. (2009)
Moisture He et al. (2013)
Food security Microbial spoilage Cheng and Sun (2015)
Lactic acid bacteria He et al. (2014)
Blood and melanin spots Mathiassen et al. (2011)
X-rays Physical Fat distribution Kolstad et al. (2004)
attributes Weight Veliyulin et al. (2011)
Chemical Fat and protein content Romvari et al. (2002)
attributes Salt, protein and water content Haseth et al. (2009)
Food security Detection of bones Mery et al. (2010), Jamieson (2002)

processing. The aspects to be considered in an illumination
system are location, lamp type and colour quality.
Furthermore, in a tank or cage measurement, there are
more uncertainties than in the laboratory, such as atmo-
spheric attenuation, sun angle, large pixel sample and vary-
ing quality of sensor’s stability and sensitivity. Besides, fish
are living in heterogeneous environment which affects the
accuracy of measurements. Moreover, cage or tank mea-
surement can provide only statistical average of population
parameters and not all which increase uncertainty. Water
also adds more complexity for deriving correct information
from aquatic’s optical information from afar when it is in
the sensor’s field of view.
Another important concern about uncertainty in optical
information is changing behaviour due to the presence of
artificial and unnatural objects in aquatic natural environ-
ment, as Oppedal et al. (2011) demonstrated artificial light
or presence and handling of multiple cameras in stereovi-
sion would cause changes in fish behaviour which affect on
prediction accuracy.
High cost of optical sensors, limited availability and the
complexity involved in processing the raw data into a final
reflectance or image product are additional restrictions of
using these high-tech instruments in aquaculture. These
factors deter ordinary users from using these technologies.
To overcome these sort of limitations, a number of
strategies have been recommended. Some studies suggested
that standardization of the rules for assessing the accuracy
and precision of predictions is a prerequisite for the com-
parison of different optical sensors and their applications
across different studies. For instance, preparation of sam-
ples should be standardized as it affects the accuracy of
MVS as well as HSI prediction. Lack of standardization in
sample preparation leads to the heterogeneous nature of
the material such as fish features, which influence the qual-
ity of spectra and image accordingly. Another standardiza-
tion technique would be developing the specific image-
processing protocols for more accurate results (Aguzzi
et al. 2009; Costa et al. 2013b). So, standardization would
help to eliminate or at least reduce this sort of restrictions.
Others recommended spiking – the complementation
and recalibration of a calibration set using known samples
representative of the studied area – of local samples into
general calibration models which can improve predictive
accuracy under laboratory or in situ measurement condi-
tions. It has also been suggested to develop algorithms and

Reviews in Aquaculture (2017) 9, 369–387

380 © 2016 Wiley Publishing Asia Pty Ltd

techniques for automating the measurement process with
the possibility of robust feature matching and verification,
under variable conditions of lighting and perspective, to
avoid delays in data processing. Besides, automated system
will overcome the greatest source of error in the manual
selection by an operator of the same feature in images
(Harvey et al. 2003).
Conclusion
This review covers different applications of optical sensors
and MVSs with their basic concept and principles in fish-
ery. Recent functions of image analysis in fish industry pre-
sented in the current review can be used to consider the
fundamental methods and technologies associated with
computer vision, which would provide a useful approach
for automated, objective, rapid and hygienic inspection of
fish and fish products. The general trend seems to point
towards use of optical and imaging technologies to opti-
mize the cost of fish production and improve their quality
in many aspects. These systems have the potential to
become a vital component of automated behavioural stud-
ies, feature analysis, aquatic food-control and processing
operations. Optical sensors also provide the possibility of
increasing cultural efficiency for farmers by decreasing
probable source of stresses. However, in order to verify the
capabilities and to increase the speed of computer-based
processing, new algorithms, methods and re-engineered
sensors need to be developed to meet the real-world
requirements. For instance, occlusion is still a major draw-
back for application of optical sensors and MVSs in study-
ing fish population behavioural dynamics such as feeding.
Stereovision systems and single-point 3D imaging technol-
ogy can also be applicable to improve information acquisi-
tion accuracy and overcome problems such as occlusion by
adding another dimension to monitor fish behaviours,
inexpensively.
Machine vision system has also found real-world applica-
tions based on the tremendous possibility offered by the
digital camera development, so we expect in the near
future, new potential for both industry and consumers
could be presented to provide faster and cheaper alterna-
tives for evaluating as many fish characteristics as possible.
Actually, new perspectives for rapid and noninvasive assess-
ment of fish behaviours, quality and freshness as well as fish
features are anticipated. For instance, because of strong
pressure on the fish welfare improvement, high-spatial-
resolution cameras will be widely needed to be used for fish
behaviour studies. Furthermore, active vision methods such
as LiDAR seem promising for actively monitoring of fish
schooling; however, they need more development.
Inspection of fish products is required due to high
demand of aquatic products during past decades (FAO,
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
2014), but some available technologies and methods such
as HSI and TI were discussed here are still at the experi-
mental stages; so further studies should be conducted to
improve the aforementioned technologies for commercial-
ization and adoption by industrial sectors. Moreover, as
Wu and Sun (2013) pointed out, most studies, which used
HSI, have employed linear calibration methods such as
PLS; however, nonlinear calibration models such as data
mining methods (e.g. SVM and memory based learning)
and also ANN may provide more reliable information that
need to be addressed in future studies. In other words, by
improvement and development in chemometric analysis
methods, HSI and TI will be used as dominant analytical
tools for fish quality monitoring in fish industry.
Acknowledgments
The authors acknowledge the financial support of the Min-
istry of Education, Youth and Sport of the Czech Republic –
projects ‘CENAKVA’ (No. CZ.1.05/2.1.00/01.0024),
‘CENAKVA II’ (No. LO1205 under the NPU I program).
References
Aguzzi J, Costa C, Fujiwara Y, Iwase R, Ramirez-Llorda E, Mene-
satti P (2009) A novel morphometry-based protocol of auto-
mated video-image analysis for species recognition and
activity rhythms monitoring in deep-sea fauna. Sensors 9(11):
8438–8455.
Alasalvar C, Miyashita K, Shahidi F, Wanasundara U (2011)
Handbook of Seafood Quality, Safety and Health Applications.
Wiley-Blackwell, Oxford.
Alcaraz C, Gholami Z, Esmaeili HR, Garcıa-Berthou E (2015)
Herbivory and seasonal changes in diet of a highly endemic
cyprinodontid fish (Aphanius farsicus). Environmental Biology
of Fishes 98(6): 1541–1554.
Alfredsen JA, Holand B, Solvang-Garten T, Uglem I (2007)
Feeding activity and opercular pressure transients in Atlantic
salmon (Salmo salar L.): application to feeding management
in fish farming. Hydrobiologia 582(1): 199–207.
Arora N, Martins D, Ruggerio D, Tousimis E, Swistel AJ,
Osborne MP et al. (2008) Effectiveness of a noninvasive digi-
tal infrared thermal imaging system in the detection of breast
cancer. The American Journal of Surgery 196(4): 523–526.
Atoum Y, Srivastava S, Xiaoming L (2015) Automatic feeding
control for dense aquaculture fish tanks. IEEE Signal Process-
ing Letters 22(8): 1089–1093.
Balaban MO, Odabasi AZ, Damar S, Oliveira ACM (2008) Qual-
ity evaluation of seafood. Computer Vision Technology for
Food Quality Evaluation, pp. 189–209. Academic Press,
Amsterdam.
Balaban MO, Unal€ Sßeng€
or GF, Soriano MG, Ruiz EG (2010)
Using image analysis to predict the weight of Alaskan sal-
381
M. Saberioon et al.
mon of different species. Journal of Food Science 75(3):
E157–E162.
Baum T, Izhaki I, Rivlin E, Katzir G (2014) Active tracking and
pursuit under different levels of occlusion: a two-layer
approach. Machine Vision and Application 25(1): 173–184.
Beddow TA, Ross LG, Marchant JA (1996) Predicting salmon
biomass remotely using a digital stereo-imaging technique.
Aquaculture 146(3): 189–203.
Belcher E, Hanot W, Burch J. (2002) Dual-Frequency Identifica-
tion Sonar (DIDSON). 2002 International Symposium on
Underwater Technology, pp. 187–192. IEEE. http://doi.org/
10.1109/UT.2002.1002424
Beven K (2000) On model uncertainty, risk and decision mak-
ing. Hydrological Processes 14(14): 2605–2606.
Beyan C, Fisher RB (2013) Detecting abnormal fish trajectories
using clustered and labeled data. Presented at the 2013 20th
IEEE International Conference on Image Processing (ICIP),
pp. 1476–1480. IEEE. http://doi.org/10.1109/ICIP.2013.673
8303
Bjerselius R, Lundstedt-Enkel K, Olsen H, Mayer I, Dimberg K
(2001) Male goldfish reproductive behaviour and physiology
are severely affected by exogenous exposure to 17b-estradiol.
Aquatic Toxicology 53(2): 139–152.
Block BA, Teo SLH, Walli A, Boustany A, Stokesbury MJW, Far-
well CJ et al. (2005) Electronic tagging and population struc-
ture of Atlantic bluefin tuna. Nature 434(7037): 1121–1127.
Boglione C, Costa C (2011) Skeletal deformities and juvenile
quality. Sparidae Biology and Aquaculture of Gilthead Sea
Bream and Other Species, Biology and Aquaculture of Gilthead
Sea Bream and Other Species, pp. 233–294. Wiley-Blackwell,
Oxford, UK.
Bridger CJ, Booth RK (2003) The effects of biotelemetry trans-
mitter presence and attachment procedures on fish physiology
and behavior. Reviews in Fisheries Science 11(1): 13–34.
Brosnan T, Sun D-W (2004) Improving quality inspection of
food products by computer vision – a review. Journal of Food
Engineering 61(1): 3–16.
Brown JD (2004) Knowledge, uncertainty and physical geogra-
phy: towards the development of methodologies for question-
ing belief. Transactions of the Institute of British Geographers
29(3): 367–381.
Brown JD, Heuvelink GBM (2006) Assessing Uncertainty Propa-
gation through Physically Based Models of Soil Water Flow and
Solute Transport. John Wiley and Sons Ltd, Chichester, UK.
Brown GE, Rive AC, Ferrari MCO (2006) The dynamic nature
of antipredator behavior: prey fish integrate threat-sensitive
antipredator responses within background levels of predation
risk. Behavioral Ecology and Sociobiology 61(1): 9–16.
Butler GL, Rowland SJ (2009) Using underwater cameras to
describe the reproductive behaviour of the endangered eastern
freshwater cod Maccullochella ikei. Ecology of Freshwater Fish
18(3): 337–349.
Cachat J, Stewart A, Utterback E, Hart P, Gaikwad S, Wong K
et al. (2011) Three-dimensional neurophenotyping of adult
zebrafish behavior. PLoS ONE 6(3): e17597.
382
Carrera P, Churnside JH, Boyra G, Marques V, Scalabrin C, Uri-
arte A (2006) Comparison of airborne lidar with echosoun-
ders: a case study in the coastal Atlantic waters of southern
Europe. ICES Journal of Marine Science 63: 1736–1750.
Chen YR, Chao K, Kim MS (2002) Machine vision technology
for agricultural applications. Computers and Electronics in
Agriculture 36(2): 173–191.
Chen Z, Wong KYK, Liu M, Schnieders D. (2011) Single-view
reconstruction from an unknown spherical mirror. Presented
at the 2011 18th IEEE International Conference on Image
Processing (ICIP 2011), pp. 2677–2680. IEEE. http://doi.org/
10.1109/ICIP.2011.6116219
Cheng J-H, Sun D-W (2015) Rapid and non-invasive detection
of fish microbial spoilage by visible and near infrared hyper-
spectral imaging and multivariate analysis. LWT –Food Science
and Technology 62(2): 1060–1068.
Churnside JH (2003) A comparison of lidar and echosounder
measurements of fish schools in the Gulf of Mexico. ICES
Journal of Marine Science 60(1): 147–154.
Colihueque N, Parraguez M, Estay FJ, Diaz NF (2011) Skin color
characterization in rainbow trout by use of computer-based
image analysis. North American Journal of Aquaculture 73(3):
249–258.
Conti SG, Roux P, Fauvel C, Maurer BD, Demer DA (2006)
Acoustical monitoring of fish density, behavior, and growth
rate in a tank. Aquaculture 251(2–4): 314–323.
Costa C, Loy A, Cataudella S, Davis D, Scardi M (2006) Extract-
ing fish size using dual underwater cameras. Aquaculture Engi-
neering 35: 218–227.
Costa C, Scardi M, Vitalini V, Cataudella S (2009) A dual cam-
era system for counting and sizing Northern Bluefin Tuna
(Thunnus thynnus; Linnaeus, 1758) stock, during transfer to
aquaculture cages, with a semi automatic Artificial Neural
Network tool. Aquaculture 291(3–4): 161–167.
Costa C, Antonucci F, Pallottino F, Aguzzi J, Sun D-W, Mene-
satti P (2011a) Shape analysis of agricultural products: a
review of recent research advances and potential application
to computer vision. Food and Bioprocess Technology 4(5):
673–692.
Costa C, D’Andrea S, Russo R, Antonucci F, Pallottino F, Mene-
satti P (2011b) Application of non-invasive techniques to dif-
ferentiate sea bass (Dicentrarchus labrax, L. 1758) quality
cultured under different conditions. Aquaculture International
19(4): 765–778.
Costa C, Menesatti P, Rambaldi E, Argenti L, Bianchini ML
(2013a) Preliminary evidences of colour differences in Euro-
pean sea bass reared under organic protocols. Aquacultural
Engineering 57: 82–88.
Costa C, Antonucci F, Boglione C, Menesatti P, Vandeputte M,
Chatain B (2013b) Automated sorting for size, sex and skeletal
anomalies of cultured seabass using external shape analysis.
Aquacultural Engineering 52: 58–64.
Costa C, Antonucci F, Menesatti P, Pallottino F, Boglione C,
Cataudella S (2013c) An advanced colour calibration method
for fish freshness assessment: a comparison between standard
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

and passive refrigeration modalities. Food and Bioprocess
Technology 6(8): 2190–2195.
Dahlbom SJ, Lagman D, Lundstedt-Enkel K, Sundstr€ om LF,
Winberg S (2011) Boldness predicts social status in zebrafish
(Danio rerio). PLoS ONE 6(8): e23565.
Delcourt J, Ylieff M, Bolliet V, Poncin P, Bardonnet A (2011)
Video tracking in the extreme: a new possibility for tracking
nocturnal underwater transparent animals with fluorescent
elastomer tags. Behavior Research Methods 43(2): 590–600.
Delcourt J, Deno€el M, Ylieff M, Poncin P (2012) Video multi-
tracking of fish behaviour: a synthesis and future perspectives.
Fish and Fisheries 14(2): 186–204.
Dell AI, Bender JA, Branson K, Couzin ID, de Polavieja GG,
Noldus LPJJ et al. (2014) Automated image-based tracking
and its application in ecology. Trends in Ecology and Evolution
29(7): 417–428.
Dios JRM-D, Serna C, Ollero A (2003) Computer vision and
robotics techniques in fish farms. Robotica 21(3): 233–243.
Dolado R, Pertusa JF, Gimeno E, Quera V, Beltran FS (2014) A
method for resolving occlusions when multitracking individu-
als in a shoal. Behavior Research Methods 47(4): 1032–1043.
Du C-J, Sun D-W (2004) Recent developments in the applica-
tions of image processing techniques for food quality evalua-
tion. Trends in Food Science and Technology 15(5): 230–249.
Duarte S, Reig L, Oca J (2009) Measurement of sole activity by
digital image analysis. Aquaculture Engineering 41(1): 22–27.
ElMasry G, Sun DW (2010) Principles of hyperspectral imaging
technology. Hyperspectral Imaging for Food Quality Analysis
and Control, pp. 3–43. Academic Press, San Diego.
ElMasry G, Wold JP (2008) High-speed assessment of fat and
water content distribution in fish fillets using online imaging
spectroscopy. Journal of Agricultural and Food Chemistry 56
(17): 7672–7677.
Erikson U, Misimi E (2008) Atlantic salmon skin and fillet color
changes effected by perimortem handling stress, rigor mortis,
and ice storage. Journal of Food Science 73(2): C50–C59.
FAO (2014) The State of World Fisheries and Aquaculture 2014,
pp. 1–243. FAO, Rome, Italy.
Fukunaga T, Kubota S, Oda S, Iwasaki W (2015) GroupTracker:
video tracking system for multiple animals under severe
occlusion. Computational Biology and Chemistry 57: 39–45.
Furevik DM, Bjordal  A, Huse I, Fern€ o A (1993) Surface activity
of Atlantic salmon (Salmo salar L.) in net pens. Aquaculture
110(2): 119–128.
Gerking SD (1994) Feeding Ecology of Fish. Academic Press, San
Diego.
Ginesu G, Giusto DD, Margner V, Meinlschmidt P (2004)
Detection of foreign bodies in food by thermal image process-
ing. IEEE Transactions on Industrial Electronics 51(2): 480–
490.
Giorleo G, Meola C (2002) Comparison between pulsed and
modulated thermography in glass–epoxy laminates. NDT and
E International 35(5): 287–292.
Gleason ACR, Reid RP, Voss KJ (2007) Automated classification
of underwater multispectral imagery for coral reef monitor-
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
ing. Presented at the Oceans 2007, pp. 1–8. IEEE, Vancouver.
http://doi.org/10.1109/OCEANS.2007.4449394
Gli
nski J, Horabik J, Lipiec J (2011) Encyclopedia of Agrophysics.
Springer Science and Business Media, Berlin, Heidelberg.
Gokturk SB, Yalcin H, Bamji C (2004) A time-of-flight depth
sensor – system description, issues and solutions. Presented
at the Conference on IEEE Computer Vision and Pattern
Recognition Workshop, p. 35. http://doi.org/10.1109/
CVPR.2004.17
Gomelsky B, Cherfas NB, Hulata G, Ben-Dom N (1995) Color
variability in normal and gynogenetic progenies of ornamen-
tal (Koi) common carp. Aquaculture 137: 99–102.
Gowen A, O’Donnell C, Cullen P, Downey G, Frias J (2007)
Hyperspectral imaging – an emerging process analytical tool
for food quality and safety control. Trends in Food Science and
Technology 18(12): 590–598.
Gowen AA, Tiwari BK, Cullen PJ, McDonnell K, O’Donnell CP
(2010) Applications of thermal imaging in food quality and
safety assessment. Trends in Food Science and Technology 21
(4): 190–200.
Grubich JR, Rice AN, Westneat MW (2008) Functional mor-
phology of bite mechanics in the great barracuda (Sphyraena
barracuda). Zoology 111(1): 16–29.
G€
um€ ußs B, Balaban MO (2010) Prediction of the weight of
aquacultured rainbow trout (Oncorhynchus mykiss) by image
analysis. Journal of Aquatic Food Product Technology 19(3–4):
227–237.
Harvey E, Fletcher D, Shortis M (2001) A comparison of the
precision and accuracy of estimates of reef-fish lengths deter-
mined visually by divers with estimates produced by a stereo-
video system. Fishery Bulletin 99(1): 63–71. Available from
URL: http://www.geomsoft.com/markss/papers/Harvey_
etal2001_comp_precacc.pdf
Harvey E, Fletcher D, Shortis M (2002) Estimation of reef fish
length by divers and by stereo-video. Fisheries Research 57(3):
255–265.
Harvey E, Cappo M, Shortis M, Robson S, Buchanan J, Speare P
(2003) The accuracy and precision of underwater measure-
ments of length and maximum body depth of southern blue-
fin tuna (Thunnus maccoyii) with a stereo–video camera
system. Fisheries Research 63: 315–326.
H
aseth TT, Høy M, Egelandsdal B, Sørheim O (2009) Nonde-
structive analysis of salt, water, and protein in dried salted
cod using computed tomography. Journal of Food Science 74
(3): E147–E153.
He H-J, Wu D, Sun D-W (2012) Application of hyperspectral
imaging technique for non-destructive pH prediction in sal-
mon fillets. Biosystem Engineeering Research Review 17: 5–8.
He H-J, Wu D, Sun D-W (2013) Non-destructive and rapid
analysis of moisture distribution in farmed Atlantic salmon
(Salmo salar) fillets using visible and near-infrared hyperspec-
tral imaging. Innovative Food Science and Emerging Technolo-
gies 18: 237–245.
He H-J, Sun D-W, Wu D (2014) Rapid and real-time prediction
of lactic acid bacteria (LAB) in farmed salmon flesh using near-
383
M. Saberioon et al.
infrared (NIR) hyperspectral imaging combined with chemo-
metric analysis. Food Research International 62: 476–483.
Hemelrijk CK, Hildenbrandt H, Reinders J, Stamhuis EJ (2010)
Emergence of oblong school shape: models and empirical data
of fish. Ethology 116(11): 1099–1112.
Heuvelink GBM, Brown JD, van Loon EE (2007) A probabilistic
framework for representing and simulating uncertain envi-
ronmental variables. International Journal of Geographical
Information Science 21(5): 497–513.
Hoggarth DD (2006) Stock Assessment for Fishery Management.
FAO, Rome, Italy.
Hong H, Yang X, You Z, Cheng F (2014) Visual quality detec-
tion of aquatic products using machine vision. Aquaculture
Engineering 63(3): 62–71.
Hu J, Li D, Duan Q, Han Y, Chen G, Si X (2012) Fish species
classification by color, texture and multi-class support vector
machine using computer vision. Computers and Electronics in
Agriculture 88(C): 133–140.
Ibrahim MY, Sultana S (2006) Study on fresh fish sorting tech-
niques. Presented at the IEEE International Conference on
Mechatronics, pp. 462–467. IEEE. http://doi.org/10.1109/
ICMECH.2006.252571
Israeli D, Kimmel E (1996) Monitoring the behavior of hypoxia-
stressed Carassius auratus using computer vision. Aquaculture
Engineering 15(6): 423–440.
Jamieson V (2002) Physics raises food standards. Physics World
15(1): 21–22.
Jarmin R, Khuan LY, Hashim H, Rahman NHA (2012) A com-
parison on fish freshness determination method. Presented at
the 2012 International Conference on System Engineering and
Technology (ICSET), pp. 1–6. IEEE. http://doi.org/10.1109/
ICSEngT.2012.6339329
Jouffre D, Borges MDF, Bundy A, Coll M, Diallo I, Fulton EA
et al. (2010) Estimating EAF indicators from scientific trawl
surveys: theoretical and practical concerns. ICES Journal of
Marine Science 67(4): 796–806.
Juell JE, Furevik DM, Bjordal  A (1993) Demand feeding in sal-
mon farming by hydroacoustic food detection. Aquaculture
Engineering 12(3): 155–167.
Kanbara M, Ukita N, Kidode M, Yokoya N (2006) 3D scene
reconstruction from reflection images in a spherical mirror.
Presented at the 18th International Conference on Pattern
Recognition (ICPR’06), Vol. 4, pp. 874–879. IEEE.
http://doi.org/10.1109/ICPR.2006.32
Kane AS, Salierno JD, Gipson GT, Molteno TCA, Hunter C
(2004) A video-based movement analysis system to quantify
behavioral stress responses of fish. Water Research 38(18):
3993–4001.
Kato S, Kato S, Nakagawa T, Nakagawa T, Ohkawa M, Mura-
moto K et al. (2004) A computer image processing system for
quantification of zebrafish behavior. Journal of Neuroscience
Methods 134(1): 1–7.
Kause A, Stien LH, Rungruangsak-Torrissen K, Ritola O, Ruo-
honen K, Kiessling A (2008) Image analysis as a tool to facili-
384
tate selective breeding of quality traits in rainbow trout.
Livestock Science 114(2–3): 315–324.
Kim HY, Lieffering M, Miura S, Kobayashi K, Okada M (2001)
Growth and nitrogen uptake of CO2-enriched rice under field
conditions. New Phytologist 150: 223–229.
Kolstad K, Vegusdal A, Baeverfjord G, Einen O (2004) Quantifi-
cation of fat deposits and fat distribution in Atlantic halibut
(Hippoglossus hippoglossus L.) using computerised X-ray
tomography (CT). Aquaculture 229(1–4): 255–264.
Kong F, Tang J, Rasco B, Crapo C, Smiley S (2007) Quality
changes of salmon (Oncorhynchus gorbuscha) muscle during
thermal processing. Journal of Food Science 72(2): S103–S111.
Lee WP, Osman MA, Talib AZ, Ogier JM, Yahya K (2014)
Tracking multiple fish in a single tank using an improved par-
ticle filter. Advances in Computer Science and Its Applications,
Vol. 279, pp. 799–804. Springer, Berlin, Heidelberg.
Liang J, Yang K, Xia M, Zhang X, Lei X, Zheng Y et al. (2006)
Monte Carlo simulation for modulated pulse bathymetric
light detecting and ranging systems. Journal of Optics 8(5):
415–422.
Lines JA, Tillett RD, Ross LG, Chan D, Hockaday S, McFarlane
NJB (2001) An automatic image-based system for estimating
the mass of free-swimming fish. Computers and Electronics in
Agriculture 31(2): 151–168.
Little EE, Finger SE (1990) Swimming behavior as an indicator
of sublethal toxicity in fish. Environmental Toxicology and
Chemistry 9(1): 13–19.
Liu T (2007) A quantitative zebrafish phenotyping tool for
developmental biology and disease modeling [life sciences].
IEEE Signal Processing Magazine 24(1): 126–129.
Liu D, Zeng XA, Sun DW (2013) NIR spectroscopy and imaging
techniques for evaluation of fish quality – a review. Applied
Spectroscopy Reviews 48: 609–628.
Maggio E, Cavallaro A (2011) Video Tracking. John Wiley and
Sons Ltd, Chichester, UK.
Mahesh S, Manickavasagan A, Jayas DS, Paliwal J, White NDG
(2008) Feasibility of near-infrared hyperspectral imaging to
differentiate Canadian wheat classes. Biosystems Engineering
101(1): 50–57.
Mallet D, Pelletier D (2014) Underwater video techniques for
observing coastal marine biodiversity: a review of sixty years
of publications (1952–2012). Fisheries Research 154: 44–62.
Mancera JM, Vargas-Chacoff L, Garcıa-L opez A, Kleszczy nska
A, Kalamarz H, Martınez-Rodrıguez G et al. (2008) High
density and food deprivation affect arginine vasotocin, iso-
tocin and melatonin in gilthead sea bream (Sparus auratus).
Comparative Biochemistry and Physiology Part A: Molecular
and Integrative Physiology 149(1): 92–97.
Manickavasagan A, Jayasuriya H (2014) Imaging with Electro-
magnetic Spectrum. Springer, Berlin, Heidelberg
Martınez-Palacios CA, Tovar EB, Taylor JF, Duran GR, Ross
LG (2002) Effect of temperature on growth and survival of
Chirostoma estor estor, Jordan 1879, monitored using a
simple video technique for remote measurement of length
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

and mass of larval and juvenile fishes. Aquaculture 209:
377–389.
Marty-Mahe P, Loisel P, Fauconneau B, Haffray P (2004)
Quality traits of brown trouts (Salmo trutta) cutlets
described by automated color image analysis. Aquaculture
232(1): 225–240.
Masud S, Singh IJ, Ram RN (2005) Behavioural and hematologi-
cal responses of Cyprinus carpio exposed to mercurial chlo-
ride. Journal of Environmental Biology/Academy of
Environmental Biology, India 26(2 Suppl): 393–397.
Mathiassen JR, Misimi E, Bondø M, Veliyulin E, Østvik SO
(2011) Trends in application of imaging technologies to
inspection of fish and fish products. Trends in Food Science
and Technology 22(6): 257–275.
Matsumoto J, Urakawa S, Takamura Y, Malcher-Lopes R, Hori
E, Tomaz C et al. (2013) A 3D-video-based computerized
analysis of social and sexual interactions in rats. PLoS ONE
8(10): e78460.
Meinlschmidt P, Maergner V (2002) Detection of foreign sub-
stances in food using thermography. In Maldague XP, Rozlos-
nik AE (eds) Presented at the AeroSense 2002, Vol. 4710, pp.
565–571. SPIE. http://doi.org/10.1117/12.459608
Menesatti P, Costa C, Aguzzi J (2010) Quality evaluation of fish
by hyperspectral imaging. Hyperspectral Imaging for Food
Quality Analysis and Control, 1st edn, pp. 273–294. Academic
Press, San Diego.
Mery D, Lillo I, Loebel H, Riffo V, Soto A, Cipriano A et al.
(2010) Automated detection of fish bones in salmon fillets
using X-ray testing. 2010 Fourth Pacific-Rim Symposium on
Image and Video Technology (PSIVT), pp. 46–51.
http://doi.org/10.1109/PSIVT.2010.15
Mirat O, Sternberg JR, Severi KE (2013) ZebraZoom: an auto-
mated program for high-throughput behavioral analysis and
categorization. Frontiers in Neural Circurrs 7: 1–12.
Misimi E, Erikson U, Skavhaug A (2008) Quality grading of
Atlantic salmon (Salmo salar) by computer vision. Journal of
Food Science 73(5): E211–E217.
Mizuno K, Liu X, Asada A, Ashizawa J, Fujimoto Y, Shimada T
(2015) Application of a high-resolution acoustic video camera
to fish classification: an experimental study. 2015 IEEE
Underwater Technology (UT), pp. 1–4. IEEE. http://doi.org/
10.1109/UT.2015.7108250
Mueller RP, Brown RS, Hop H, Moulton L (2006) Video and
acoustic camera techniques for studying fish under ice: a
review and comparison. Reviews in Fish Biology and Fisheries
16(2): 213–226.
Nakai M, Satoh H, Takeda F (2008) Proposal of an automatic
fish sorting system by intelligent image processing. World
Automation Congress 2008 (WAC 2008), pp. 1–6.
Niizato T, Gunji YP (2011) Metric–topological interaction
model of collective behavior. Ecological Modelling 222(17):
3041–3049.
Nucci ME, Costa C, Scardi M, Cataudella S (2010) Preliminary
observations on bluefin tuna (Thunnus thynnus, Linnaeus
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
1758) behaviour in captivity. Journal of Applied Ichthyology
25: 95–96.
Odone F, Trucco E, Verri A (2001) A trainable system for
grading fish from images. Applied Artificial Intelligence 15(8):
735–745.
Oppedal F, Dempster T, Stien LH (2011) Environmental drivers
of Atlantic salmon behaviour in sea-cages: a review. Aquacul-
ture 311(1–4): 1–18.
Papadakis VM, Papadakis IE, Lamprianidou F, Glaropoulos A,
Kentouri M (2012) A computer-vision system and methodol-
ogy for the analysis of fish behavior. Journal of Aquaculture
Engineering 46: 53–59.
Parsonage KD, Petrell RJ (2003) Accuracy of a machine-vision
pellet detection system. Aquaculture Engineering 29(3–4):
109–123.
Patullo BW, Jolley-Rogers G, Macmillan DL (2007) Video track-
ing in the extreme: video analysis for nocturnal underwater
animal movement. Behavior Research 39(4): 783–788.
Pautsina A, Kuklina I,
Stys D, Cısar P, Kozak P (2014) Noninva-
sive crayfish cardiac activity monitoring system. Limnology
and Oceanography: Methods 12: 670–679.
Pavlidis M, Papandroulakis N, Divanach P (2006) A method for
the comparison of chromaticity parameters in fish skin: pre-
liminary results for coloration pattern of red skin Sparidae.
Aquaculture 258(1–4): 211–219.
Pelletier D, Leleu K, Mallet D, Mou-Tham G, Herve G (2012)
Remote high-definition rotating video enables fast spatial sur-
vey of marine underwater macrofauna and habitats. PLoS
ONE 7(2): e30536.
Perez-Escudero A, Vicente-Page J, Hinz RC, Arganda S, de Pola-
vieja GG (2014) idTracker: tracking individuals in a group by
automatic identification of unmarked animals. Nature Meth-
ods 11(7): 743–748.
Pinkiewicz TH, Purser GJ, Williams RN (2011) A computer
vision system to analyse the swimming behaviour of farmed
fish in commercial aquaculture facilities: a case study using
cage-held Atlantic salmon. Aquaculture Engineering 45(1): 20–
27.
Quevedo R, Aguilera JM (2010) Computer vision and stere-
oscopy for estimating firmness in the salmon (Salmon salar)
fillets. Food and Bioprocess Technology 3(4): 561–567.
Quevedo RA, Aguilera JM, Pedreschi F (2008) Color of salmon
fillets by computer vision and sensory panel. Food and Biopro-
cess Technology 3(5): 637–643.
Reese DC, O’Malley RT, Brodeur RD, Churnside JH (2011) Epi-
pelagic fish distributions in relation to thermal fronts in a
coastal upwelling system using high-resolution remote-sen-
sing techniques. ICES Journal of Marine Science 68(9): 1865–
1874.
del Rıo J, Aguzzi J, Costa C, Menesatti P, Sbragaglia V, Nogueras
M et al. (2013) A new colorimetrically-calibrated automated
video-imaging protocol for day-night fish counting at the
OBSEA coastal cabled observatory. Sensors 13(11): 14740–
14753.
385
M. Saberioon et al.
Rodrigues MTA, Padua FLC, Gomes RM, Soares GE (2010)
Automatic fish species classification based on robust feature
extraction techniques and artificial immune systems. Pre-
sented at the 2010 IEEE Fifth International Conference on
Bio-Inspired Computing: Theories and Applications (BIC-
TA), pp. 1518–1525. IEEE. http://doi.org/10.1109/BICTA.
2010.5645273
Romvari R, Hancz CS, Petrasi ZS, Molnar T, Horn P (2002)
Non-invasive measurement of fillet composition of four
fresh-water fish species by computer tomography. Aquacul-
ture International 10: 231–240.
Saberioon MM, Cisar P (2016) Automated multiple fish tracking
in three-dimension using a structured light sensor. Computers
and Electronics in Agriculture 121: 215–221.
Salierno JD, Gipson GT, Kane AS (2008) Quantitative move-
ment analysis of social behavior in mummichog, Fundulus
heteroclitus. Journal of Ethology 26(1): 35–42.
Segade A, Robaina L, Otero Ferrer F, Garcıa Romero J, Molina
Domınguez L (2014) Effects of the diet on seahorse (Hip-
pocampus hippocampus) growth, body colour and biochemical
composition. Aquaculture Nutrition 21: 807–813.
Segtnan VH, Høy M, Sørheim O, Kohler A, Lundby F, Wold JP
et al. (2009) Noncontact salt and fat distributional analysis in
salted and smoked salmon fillets using X-ray computed
tomography and NIR interactance imaging. Journal of Agricul-
tural and Food Chemistry 57(5): 1705–1710.
Serra-Toro C, Montoliu R, Traver VJ, Hurtado-Melgar IM,
Nunez-Red o M, Cascales P (2010) Assessing Water Quality by
Video Monitoring Fish Swimming Behavior, pp. 428–431. IEEE
Computer Society. http://doi.org/10.1109/ICPR.2010.113
Shewfelt RL, Bruckner B (2000) Fruit and Vegetable Quality: An
Integrated View. CRC Press, Lancaster.
Shortis M, Harvey E, Seager J (2007) A review of the status and
trends in underwater videometric measurement. Presented at
the SPIE Videometrics IX, San Jose, California.
Sivertsen AH, Heia K, Hindberg K, Godtliebsen F (2012) Automatic
nematode detection in cod fillets (Gadus morhua L.) by hyper-
spectral imaging. Journal of Food Engineering 111(4): 675–681.
Spampinato C, Chen-Burger YH, Nadarajan G (2008) Detecting,
tracking and counting fish in low quality unconstrained
underwater videos. Visapp 2: 514–519.
Spitzen J, Spoor CW, Grieco F, ter Braak C, Beeuwkes J, van
Brugge SP et al. (2013) A 3D analysis of flight behavior of
Anopheles gambiae sensu stricto malaria mosquitoes in
response to human odor and heat. PLoS ONE 8(5): e62995.
Stien LH, Manne F, Ruohonene K, Kause A, Rungruangsak-Tor-
rissen K, Kiessling A (2006) Automated image analysis as a
tool to quantify the colour and composition of rainbow trout
(Oncorhynchus mykiss W.) cutlets. Aquaculture 261(2): 695–
705.
Stien LH, Bratland S, Austevoll I, Oppedal F, Kristiansen TS
(2007a) A video analysis procedure for assessing vertical fish
distribution in aquaculture tanks. Aquaculture Engineering 37
(2): 115–124.
386
Stien LH, Kiessling A, Manne F (2007b) Rapid estimation of fat
content in salmon fillets by colour image analysis. Journal of
Food Composition and Analysis 20(2): 73–79.
Storbeck F, Daan B (2001) Fish species recognition using com-
puter vision and a neural network. Fisheries Research 51(1):
11–15.
Sumpter JP (2010) Pharmaceuticals in the environment: moving
from a problem to a solution. Green and Sustainable Phar-
macy, pp. 11–22. Springer, Berlin, Heidelberg.
Sun D-W (2011) Computer Vision Technology for Food Quality
Evaluation. Academic Press, San Diego.
Tian J, Hunziker W, Satpathy A, Burgunder JM, Ng ES, Ong SG
et al. (2014) Motion analytics of zebrafish using fine motor
kinematics and multi-view trajectory. Multimedia Systems S:
1–11.
Tillett R, McFarlane N, Lines J (2000) Estimating dimensions of
free-swimming fish using 3D point distribution models.
Computer Vision and Image Understanding 79(1): 123–141.
Torisawa S, Kadota M, Komeyama K, Suzuki K, Takagi T (2011)
A digital stereo-video camera system for three-dimensional
monitoring of free-swimming Pacific bluefin tuna, Thunnus
orientalis, cultured in a net cage. Aquatic Living Resources 24
(2): 107–112.
Trenkel VM, Cotter J (2009) Choosing survey time series for
populations as part of an ecosystem approach to fishery man-
agement. Aquatic Living Resources 22(02): 121–126.
Urban J,
Stys D, Sergejevova M, Masojıdek J (2012) Expertomica
Fishgui: comparison of fish skin colour. Journal of Applied
Ichthyology 29(1): 172–180.
Veeraraghavan A, Srinivasan M, Chellappa R, Baird E, Lamont
R (2006) Motion based correspondence for 3D tracking of
multiple dim objects. Presented at the 2006 IEEE Interna-
tional Conference on Acoustics Speed and Signal Processing,
Vol. 2, pp. II-669–II-672. IEEE. http://doi.org/10.1109/
ICASSP.2006.1660431
Veliyulin E, Misimi E, Bondø M, Vebenstad PA, Østvik SO
(2011) A simple method for weight estimation of whole her-
ring (Clupea harengus) using planar X-ray imaging. Journal of
Food Science 76(3): E328–E331.
de Verdal H, Vandeputte M, Pepey E, Vidal M-O, Chatain B
(2014) Individual growth monitoring of European sea bass
larvae by image analysis and microsatellite genotyping. Aqua-
culture 434(C): 470–475.
Viazzi S, Van Hoestenberghe S, Goddeeris BM, Berckmans D
(2015) Automatic mass estimation of Jade perch Scortum bar-
coo by computer vision. Aquacultural Engineering 64: 42–48.
Viscido SV, Parrish JK, Gr€ unbaum D (2004) Individual behavior
and emergent properties of fish schools: a comparison of obser-
vation and theory. Marine Ecology Progress Series 273: 239–249.
Wallat G, Lazur A, Chapman F (2005) Carotenoids of different
types and concentrations in commercial formulated fish diets
affect color and its development in the skin of the red oranda
variety of goldfish. North American Journal of Aquaculture 67
(1): 42–51.
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd

Watson DL, Harvey ES, Anderson MJ, Kendrick GA (2005) A
comparison of temperate reef fish assemblages recorded by
three underwater stereo-video techniques. Marine Biology 148
(2): 415–425.
White DJ, Svellingen C, Strachan NJC (2006) Automated mea-
surement of species and length of fish by computer vision.
Fisheries Research 80(2–3): 203–210.
Wibe AE, Nordtug T, Jenssen BM (2001) Effects of bis(tributyl-
tin)oxide on antipredator behavior in threespine stickleback
Gasterosteus aculeatus L. Chemosphere 44(3): 475–481.
Wu D, Sun D-W (2013) Advanced applications of hyperspectral
imaging technology for food quality and safety analysis and
assessment: a review — part II: applications. Innovative Food
Science and Emerging Technologies 19: 15–28.
Wu HS, Zhao Q, Zou D, Chen YQ (2011) Automated 3D trajec-
tory measuring of large numbers of moving particles. Optics
Express 19(8): 7646–7663.
Wu D, He HJ, Sun DW (2012) Non-destructive texture analysis
of farmed salmon using hyperspectral imaging technique. Pre-
sented at the Proceedings of the Third CIGR International
Conference of Agriculture Engineering, Valencia, Spain.
Xiao G, Feng M, Cheng Z, Zhao M, Mao J, Mirowski L (2015)
Water quality monitoring using abnormal tail-beat frequency
of crucian carp. Ecotoxicology and Environmental Safety 111:
185–191.
Xu J, Liu Y, Cui S, Miao X (2006) Behavioral responses of tilapia
(Oreochromis niloticus) to acute fluctuations in dissolved oxy-
Reviews in Aquaculture (2017) 9, 369–387
© 2016 Wiley Publishing Asia Pty Ltd
Machine vision systems in aquaculture
gen levels as monitored by computer vision. Acquaculture
Engineering 35(3): 207–217.
Yagiz Y, Balaban MO, Kristinsson HG, Welt BA, Marshall MR
(2009) Comparison of Minolta colorimeter and machine
vision system in measuring colour of irradiated Atlantic sal-
mon. Journal of the Science of Food and Agriculture 89(4):
728–730.
Za
tkova I, Sergejevova M, Urban J, Vachta R,
Stys D, Masojıdek
J (2011) Carotenoid-enriched microalgal biomass as feed sup-
plement for freshwater ornamentals: albinic form of wels cat-
fish (Silurus glanis). Aquaculture Nutrition 17(3): 278–286.
Zion B (2012) The use of computer vision technologies in aqua-
culture – a review. Computers and Electronics in Agriculture 88
(C): 125–132.
Zion B, Shklyar A, Karplus I (1999) Sorting fish by computer
vision. Computers and Electronics in Agriculture 23(3): 175–
187.
Zion B, Shklyar A, Karplus I (2000) In-vivo fish sorting by com-
puter vision. Aquaculture Engineering 22(3): 165–179.
Zion B, Alchanatis V, Ostrovsky V, Barki A, Karplus I (2007)
Real-time underwater sorting of edible fish species. Computers
and Electronics in Agriculture 56(1): 34–45.
Zion B, Alchanatis V, Ostrovsky V, Barki A, Karplus I (2008)
Classification of guppies’ (Poecilia reticulata) gender by com-
puter vision. Aquaculture Engineering 38(2): 97–104.
Zude M (2008) Optical Monitoring of Fresh and Processed Agri-
cultural Crops. CRC Press, Boca Raton.
387