Professional Documents
Culture Documents
1*
Roseleena, J., 2Nursuriati, J., 1Ahmed, J. and 1Low, C. Y.
1
Faculty of Mechanical Engineering, Universiti Teknologi MARA, 40450 Shah Alam,
Selangor, Malaysia
2
Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, 40450
Shah Alam, Selangor, Malaysia
Abstract: The agricultural industry scenario in many industrialized countries has adopted an image processing
system as a solution to automate the grading process in order to provide accurate, reliable, consistent and
quantitative information in addition to the large volumes, which human graders are not able to perform. In
Malaysia, the grading of palm oil Fresh Fruit Bunches (FFB) is still performed manually through visual inspection
using the surface color as the main quality attribute. It is the intention here to introduce an automated grading
system for palm oil FFB using a computer assisted photogrammetric methodology which correlate the surface
color of fruit bunches, not the fruitlets, to their ripeness and eventually sorts the fruit to two predefined fruit
categories. The methodology consists of five main phases, i.e. image acquisition, image pre-processing, image
segmentation, calculation of color Digital Numbers (DN) (data manipulation) and finally the classification of
ripeness. This computerized photogrammetric image processing technique using MATLAB® package which is
integrated to a sorting system differs in various aspects from other digital imaging technique or machine vision
system adopted for classifying fruit ripeness. A comprehensive discussion will be presented based on the results
achieved through actual fruit testing on the prototype grading system. The main concern was to ensure the
reliability of the computerized photogrammetric technique achievable and the system’s mechanism working as
intended. The fruit classification ability of the system yields above 90% accuracy and taking not more than 25
seconds to classify and sort each fruit.
Keywords: Palm oil FFB, surface color, image processing, automated grading system, ripeness index
value
*Corresponding author.
Email: rosel714@salam.uitm.edu.my © All Rights Reserved
Tel: +603 55436266; Fax: +603 55435160
1000 Roseleena, J., Nursuriati, J., Ahmed, J. and Low, C. Y.
for the measurement of fruit ripeness. However, the manipulation) and finally the classification of
disadvantage of using such method on palm oil FFB the FFB ripeness. The analysis is carried using
is that the destructive testing can only be done on MATLAB® image processing toolboxes running on
the fruitlets of the fruit bunches and it requires the Windows® platform. The image pre-processing phase
fruits to be sliced and the surface of the mesocarp comprises of three steps, i.e. image binarization,
exposed (Idris et al., 2003). However, studies have morphological processing and the extraction of FFB
found that the oil content of the flesh of the mesocarp properties. Similarly, the image segmentation phase
has direct relationship with the fruit surface color comprises of another three different steps, i.e. image
and this can be used as an indicator of palm oil cropping, conversion from RGB to L*a*b* color space
quality (Balasundram et al., 2006). Hence, in order and the segmentation of FFB image using K-means
to increase the efficiency and quality of grading clustering. The schematic flow of the methodology is
FFB in palm oil mills, a fruit grading system based represented in Figure 1.
on computer-based technologies such as machine For comparison purposes, the average digital
vision (Yud-Rec et al., 2002; Abdullah et al., 2005; numbers of the unsegmented and segmented FFB
Meftah Salem et al., 2008; Narendra and Hareesham, images are computed. The calculation process will
2010) are necessary to replace the traditional grading consider both the masked and the FFB image as the
performed by trained human inspectors. Up to date, input to calculate the average RGB color intensities
most of the work carried out in capturing the fruit or digital number (Meftah Salem et al., 2008).
images was done using a stationary fruit sample and Each layer of RGB was totaled-up, and divided by
tested out manually. The image processing technique the total numbers of pixels in the masked image to
is commonly programmed to run on specific image obtain the average DN value. The color values were
vision software. Whereas this novel image processing then computed to get the values for R/G and R/B.
methodology is assisted by customized hardware and To emphasize the difference between ripe and unripe
software systems that would carry out the on-line classification, the maximum values of these ratio
grading and sorting process which would hopefully were then used to calculate R/G*R/B. The output
increase the FFB grading consistency and reduce value obtained from this stage of data manipulation is
sorting time. known as the ripeness index. The ripeness index will
The application of machine vision system is be the main parameter used to classify the ripeness
widely studied by researchers from local institutions of the fruits.
and had been applied in grading fruits such as starfruit Image cropping is one of the most important
(Abdullah et al., 2005), papaya (Slamet et al., 2007) steps in the image segmentation phase. The aim of
and Jatropha Curcas (Zulham et al., 2000). Different image cropping is to reduce the image size for further
approaches were used to determine the fruit ripeness analysis in order to increase the computational speed.
index or maturity, size and quality with color remains In order to obtain better cropped image, it is very
as one of the important factors to determine the grade much dependent on the background color that is
and quality of the fruits. The maturity or ripeness used during the image acquisition process. In order
index was based on color intensity. It was recorded to enhance the image quality, it is recommended that
that, a computer program was developed and the mean the FFB is clean from dirt to minimize or eliminate
color intensity was used to differentiate between the the noises or disturbances generated during the image
different color and ripeness of the fruits such as palm pre-processing stage. However, during harvesting,
oil FFB. Most of the machine vision grading system it is difficult to control the cleanliness of the FFB
employs a standard methodology such as image and thus the image processing algorithm has to be
acquisition, image processing and computing the designed to overcome this problem.
result. Image acquisition is one of the most important
processes for the performance of a machine vision
system, because with a high-quality image obtained,
the following processing and analysis of the image
would be easily feasible.
The novel methodology that has been developed
for this research is known as the photogrammetric
grading of oil palm fresh fruit bunches and primarily
consists of five main phases, i.e. image acquisition,
image pre-processing, image segmentation,
calculation of color Digital Numbers (DN) (data Figure 1. A novel methodology for photogrammetric grading
of palm oil FFB
FFB
Webcams
Figure 2. Integrated photogrammetric grading system Conveyor
by the algorithm which outputs a digital actuating the output values attained after the image processing
signal to the mechanical sorter. The GUI is designed stage are referred to as the ripeness index. There is
to allow the user to adjust the threshold value that will a distinct difference in the range of values between
distinguish the ripe from the unripe fruits. In total, it the two ripeness categories and the threshold value
will take less than 30 seconds to complete the whole for this sampling batch is approximately 3.5. FFB
sequences of the photogrammetric grading system, samples having greater than ripeness index number
from the feeding to the sorting process. The GUI of 3.5 will be categorized as ripe and samples with
was developed to offer users two different modes of lesser value will be unripe.
processing which are off-line and on-line. For the off- Further tests were done to evaluate the overall
line processing mode, the FFB images are manually system’s functionality. The next test was to focus
uploaded by pressing or clicking the ‘load image’ on the system’s capability to feed the FFB into the
button. The images are first captured manually, either inspection chamber for the image acquisition and
under controlled environment condition as in the use processing stages, classify the fruits correctly to their
of an inspection chamber or taken on-site, and saved ripeness and sort the fruits accordingly. A total of
in a JPEG format. Once the images are uploaded, 30 fruit samples of the tenera type were taken from
the photogrammetric grading system will perform a local plantation. The quality of this species may
automatically. A resize factor function is added in vary slightly from fruits that are harvested in big
order to reduce the image pixel during the image commercial plantations due to the way the fruits are
cropping and hence increase the processing speed. soiled and irrigated.
Before the test, the fruits were visually graded
by the local graders so that the ripeness of the
fruits would be known prior to the image analyzing
process. From the manual grading process, 20 of the
FFB are classified as ripe fruits and the rest are found
to be unripe. The ripe fruits are fed into the system
continuously followed by the unripe ones. The total
weight of the FFB samples is 350 kg with an average
of 11.7 kg each and the maximum weight reaching
20.5 kg. The maximum weight is noted for checking
Figure 4. GUI for FFB grading on the rollers and conveyor capacity and at the
During the on-line processing, all the same time the dimensions of the FFB are measured
photogrammetric sequences are automatically ensuring there is sufficient space for the FFB to be fed
actuated when the presence of the fruit is detected and pass through the inspection chamber. The results
by the sensor which is located at the entrance of the from the manual grading process are then compared
inspection chamber. The on-line processing mode to the output from the photogrammetric grading for
requires the use of a DAQ card to integrate the image each individual fruit.
processing algorithm to the grading system via the The ripeness index of the images are recorded
control unit. As compared to the machine vision and tabulated as in Table 1. The table shows the result
systems designed by other researchers to grade palm for the 30 sample fruits which contain information on
oil fruit, this photogrammetric system provides an the fruit number, ripeness index from camera 1 and 2,
on-line inspection and grading without any human average ripeness index number, ripeness category and
intervention and in addition, it is integrated to an respective comments. From the results, the graph of
automatic fruit sorter unit which is lacking and not the average ripeness index against the fruit sample is
available in most systems. plotted and illustrated in Figure 5. The fruit samples
which are not correctly graded are highlighted in
Results and Discussion circles. The threshold value was set to 3.5 and hence
the index ripeness values for all ripe fruits should
For the initial test on the photogrammetric exceed this value. The range of ripe values obtained
grading system, a total of 34 fruit samples were taken varies from a minimum value of 3.56 to a maximum
in order to determine the value of the threshold. of 5.83. Whereas the highest ripeness index for the
The attainment of the threshold value will be the unripe fruits is 2.49. It can be concluded here that the
predefined value for the system to distinguish the ripe threshold value of 3.5 can actually be reduced to 3.0
from the unripe fruit values. The testing was done because the highest unripe value will not exceed 2.5
off-line under controlled environment condition and for this particular sampling batch.
Table 1. Ripeness Index Results ripe. If manually inspected, most of the fruit surface
was actually ripe and the result from the grading
Fruit Camera
1
Camera
2
Average
Ripeness
Ripeness
Category
Status
Comment
system is actually correct. In order to clarify that the
Sample
No. Ripeness Ripeness Index (Manual (Automated ripeness category processed by the system is correct,
Index Index Number grading) grading)
the fruit should be manually graded again to avoid
1 1.92 2.83 2.38 Unripe OK
confusion. In addition, fruit sample number 29 as
2 2.5 2.48 2.49 Unripe OK
3 2.21 2.07 2.14 Unripe OK
shown in Figure 7, also faced similar problem with
4 3.5 3.84 3.67 Unripe Misclassified
the purplish black surface color of the fruit exposed
5 2.01 2.56 2.29 Unripe OK and captured by the webcam which resulted in the
6 1.99 2.29 2.14 Unripe OK fruit to be classified as unripe instead of ripe. The best
7 1.91 2.01 1.96 Unripe OK solution would be to rotate the fruit so that all sides
8 2.01 2.60 2.31 Unripe OK can be captured. However, to design the mechanism
9 1.68 2.86 2.27 Unripe OK to rotate may not be easy due to the big sampling
10 1.91 2.55 2.23 Unripe OK
number and weight of the fruits.
11 5.51 6.14 5.83 Ripe OK
Another problem faced during the testing
12 4.64 5.84 5.24 Ripe OK
OK but process was the difficulty to control the movement
13 1.06 7.68 4.37 Ripe FFB out of
webcam range of the fruits from the feeder section to the inspection
14 3.65 4.82 4.24 Ripe OK
chamber. Manual assistance and human intervention
15 2.99 5.34 4.17 Ripe OK
16 2.35 5.04 3.70 Ripe OK is required for the fruits to be properly aligned so that
17 4.42 6.47 5.45 Ripe OK the fruits are able to roll down, stop and transport to
18 5.55 9.56 7.56 Ripe OK the desired position which is located approximately
19 3.09 4.45 3.77 Ripe OK in the middle of the inspection chamber.
20 3.16 5.28 4.22 Ripe OK
21 5.10 10.00 7.55 Ripe OK
22 3.12 3.99 3.56 Ripe OK
23 7.05 4.38 5.72 Ripe OK
24 4.83 3.50 4.17 Ripe OK
Technical
25 - - - - Error. System
hang.
26 2.50 4.69 3.60 Ripe OK
27 3.38 5.54 4.46 Ripe OK
28 3.19 4.03 3.61 Ripe OK
Figure 6. Fruit sample number 4
29 2.08 3.20 2.64 Ripe Misclassified
30 4.98 4.41 4.70 Ripe OK
Threshold Value