Mango is one of the most traded fruits in the world. Therefor several pests and diseases which reduce the production and quality of mangoes and their price in the local and international markets. Several solutions for automatic diagnosis of these pests and diseases have been proposed by researchers in the last decade. These solutions are based on Machine Learning (ML) and Deep Learning (DL) algorithms. In recent years, Convolutional Neural Networks (CNNs) have achieved impressive results in image classification and are considered as th classification. However, one of the most significant issues facing mango pests and diseases classification solutions is the lack of availability of large and labeled datasets. Data augmentation is one of solutions that has been successfully reported in the literature namely blur, contrast, flip, noise, zoom and affine transformation to know, on the one hand, the impact of each technique on the performance of a ResNet50 CNN using a the combination between them which gives the best performance to the DL network. Results show that the best combination classifying mango leaf diseases is ‘Contrast & Flip & Affine transformation’ which gives to the model a training accuracy of 98.54% and testing accuracy of 97.80% with an f1_score > 0.9.
## I. INTRODUCTION
Mango or Magnifera Indica L. (scientific name) is a lucrative fruit widely cultivated in tropical countries. It belongs to the family anacardiaceous. Its overall consumption in 2017 was estimated at 50.65 million metric tons [1]. This fruit was in 2021, in terms of quantities exported, the third most traded tropical fruit after pineapple and avocado [2]. Mango fruit is very appreciated because of its richness in nutrients (vitamins A, B, C, K,...), flavorful pulp and alluring aroma [3,4]. This fruit contributes enormous economic benefits to exporting countries and mango growers.
However, mango production suffers severely from pests and diseases witch lead to a reduction of both quality and quantity. This influence mango price in the international market.
In the last decade, several solutions for automatic diagnosis of these pests and diseases have been proposed by researchers. These solutions are first based on image processing (IP) and machine learning (ML) techniques and finally, in the last five years, on deep learning (DL) algorithms DL based solutions have achieved state-of-the-art performance on Image Net and other benchmark datasets [5]. In recent years, Convolutional Neural Networks (CNNs) have achieved impressive results in image classification and are considered as the leading methods for object detection in computer vision [5,6].
However, one of the biggest issues facing mango pests and diseases identification solutions is the lack of availability of large and labeled datasets [7,8,9,10]. The limited training data inhibits performance of DL based models which need big data on which to train well to avoid overfitting and improves the model's generalization ability. Overfitting happens when the training accuracy is higher than the accuracy on the validation/test set. The generalizability of a model is the difference in performance it exhibits when evaluated on training data (known data) versus test data (unknown data). The use of data augmentation process is one of solutions that has been successfully reported in the literature [1]. This overfitting solution generates a more comprehensive set that minimizes the distance between training and validation sets.
A data augmentation process based on image manipulation is presented in this paper for improving the quality of a small dataset of mango leaves presented in [1]. The specific contributions of the paper include:
- Generate a dataset for every data augmentation strategy except affine transformation. The DL model is trained in each generated dataset to know the impact of each data augmentation technique in the performance of the model.
- Generate multiple datasets from pair wise sequential combination of data augmentation techniques, namely blur, contrast, flip, noise and
zoom. This is to know the combinations which give the best performance to the DL model.
- Apply affine transformation technique to the previous best combinations to determine the final combination which is better to classify diseased mango leaves.
The rest of the paper is organized as follows: Section 2 is an overview of the literature review, Section 3 deals with the data acquisition and data augmentation techniques and the CNN model used, Section 4 presents and discusses the results of the data augmentation techniques. The last section concludes the paper and announces the futures works of the authors.
## II. RALATED WORKS
The literature review presented in this paper concerns only data augmentation strategies used for mango pest or diseases classification and mango or other fruits quality grading.
Shorten et al. [11] presented a survey dealing with image data augmentation algorithms such as color space augmentations, geometric transformations, mixing images, kernel filters, random erasing, adversarial training, feature space augmentation, generative adversarial networks (GAN), meta-learning and neural style transfer. They also discussed the application of augmentation methods based on GANs and others characteristics of data augmentation such as curriculum learning, test-time augmentation, resolution impact, and final dataset size. Dandavate et al. [12] applied data augmentation techniques namely rotation, scaling and image translation to a fruit dataset to avoid overfitting and obtain better performances with their simple CNN model. Agastya et al. [13] used VGG-16 and VGG-19 for an automatic batik classification. Applying random rotation in a certain degree, scaling and shearing, they improve the accuracy of their models up to $10\%$. Bargoti et al. [14] presented a fruit (mangoes, apples, and almonds) detection system using Faster R-CNN. They used image flipping and scaling to improve the performance of their model with an F1-score of $>0.9$ achieved for mangoes and apples. Wu et al. [15] investigated several deep learning-based methods for mango quality grading. VGG-16 is found to be the best model for this task. During the training of their models, authors applied, at each epoch, randomly data augmentation strategies such as horizontal or vertical image flipping, rotation, brightness, contrast and zoom in/out. Zang et al. [16] developed a fruit category identification by using a 13-layer CNN and three data augmentation strategies namely noise injection, image rotation and Gamma correction. The final obtained overall accuracy is $94.94\%$, at least 5 percentage points higher than state-of-the-art approaches. Supekar et al. [17] performed a mango grading system based on ripeness, size, shape and defects. They used K-means clustering for defect segmentation and Random Forest Classifiers. To avoid overfitting with an initial training dataset of 69 images, authors applied image rotation on angle of 90,180 and 270. The final training dataset obtained consists of 522 images which allows their model to obtain an overall accuracy of $88,88\%$.
## III. METHODOLOGY AND MODEL
### a) Data acquisition
The dataset used in this paper is a part of 'MangoLeafBD' dataset produced by Ahmed et al. [18] and downloadable from 'Mendeley Data' platform (https://data.mendeley.com/datasets/hxsnwty3r).
MangoLeafBD dataset contains height classes, seven of which correspond to mango leaf diseases and one contains healthy leaves.
In this paper, four diseases namely anthracnose, Gall Midge, Powdery Mildew and Sooty Mold are treated as they are among the most mango leaf diseases treated by researchers during the last five years [19](Fig.1andFig.2). The dataset used contains four classes corresponding respectively to these diseases and a class of healthy leaves. There are 500 RGB leaf images of $240 \times 320$ pixels in each class making a total of 2,500. Images are in.JPG format.
### b) Data augmentation
Data augmentation is a powerful solution against overfitting. It allows a model with a small dataset to become robust and generalizable. There are two categories of data augmentation: the first is based on image manipulations and the second on DL (generative adversarial networks (GANs), feature space augmentations, adversarial training, Neural Style Transfer, Meta Learning Data Augmentation) [11].
This research focuses on the first category because i) the second is generally used to generate synthetics images from quite a large dataset, ii) mango leaf images taken under real-world conditions suffer mainly from the problems of temperature variation, shadowing, overlapping of leaves, and presence of multiple objects. The first category can allow us to generate images in these cases.
This papers deals with following techniques:
- Noise injection
Image noise is a random disturbance in the brightness and color of an image. Noise injection is an effective way to avoid overfitting and improves the test ability of a machine learning model [13]. There are several ways to add noise to an image (e.g. Gaussian noise, Salt and Pepper noise, Speckle noise,...). Gaussian noise is performed fixing mean parameter to 0 and sigma parameter to 0.05.
- Blur
Blurring an image means make it less sharp. Photographic blur occurs with movement in the model or scene relative to the camera, and vice versa. To realize this, Gaussian blur was carried out using a kernel size (5,15).
#### - Contrast and Brightness
The Contrast and Brightness function improves the appearance of an image. Brightness improves the overall clarity of the image and contrast adjusts the difference between the darkest and lightest colors. Contrast parameters used are $\{0.5;2;2.5\}$ and brightness parameters are $\{1;4;5\}$. For each original image, three new images are generated with respectively the following parameters contracts, brightness $\{c;b\}: \{0.5;1\}, \{2;4\}, \{2.5;5\}$.
#### Zoom
Zooming an image means enlarging it in a sense that the details in the picture became more visible and clear. Each image is zoomed three times and from the center using zoom parameters $\{3;5;7\}$.
#### - Image flipping
To flip or mirror an image means to turn it horizontally (horizontal flip) or vertically (vertical flip). Flip function generates an image so that the left side becomes the right side or the top becomes the bottom. The images are vertically and horizontally flipped using flip parameter 0 and 1 respectively.
#### - Affine transformation
An affine transformation is, in general a combination of translations, rotations, shears and dilations [12]. It is used to simulate images captured from different camera projections and positions. Affine transformation is performed using an input matrix (In) of size 2x3 and an output matrix (Out) of the same size. The input matrix corresponds to three points in the input image and the second matrix is their corresponding locations in the output image. In the training dataset, twenty additional images are randomly generated for each image. But after that, the generated images on which there is no part of mango leaf are removed.
Fig.3 shows an example of a diseased mango leaf (anthracnose) on which all these data augmentation techniques are applied.
 Fig. 1: Ranking of the most common and treated mango diseases [19]
 Anthracnose
The data augmentation process (Fig. 4) is carried out as follow:
First step: For each of the above mentioned data augmentation strategies (except affine transformation), a new dataset for training and validation is generated (Fig. 3, Table 2). Images of the original dataset are added to the generated one. This is to know the impact of each data augmentation strategy on the overall performance of the model.
Second step: Every strategy (except affine transformation) is combined respectively by the 4 others sequentially to generate new datasets (Table 2).
Final step: Affine transformation is applied to the best combination that gives the best performance to the DL model (Table 3).
The augmentation techniques are carried out using python Open Source Computer Vision Library (OpenCV).
Table 1: Composition of the datasets in the first step
<table><tr><td></td><td>Original</td><td>Original & Blur</td><td>Original & Contrast</td><td>Original & Flip</td><td>Original & Noise</td><td>Original & Zoom</td></tr><tr><td>Train</td><td>1600</td><td>3200</td><td>6400</td><td>4800</td><td>3200</td><td>6400</td></tr><tr><td>Validation</td><td>400</td><td>800</td><td>1600</td><td>1200</td><td>800</td><td>1600</td></tr><tr><td>Test</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td></tr><tr><td>Total</td><td>2500</td><td>4500</td><td>8500</td><td>6500</td><td>4500</td><td>8500</td></tr></table>
Table 2: Composition of the datasets in the second step
<table><tr><td></td><td>Blur & Contrast</td><td>Blur & Flip</td><td>Blur & Noise</td><td>Blur & Zoom</td><td>Contrast & Flip</td><td>Contrast & Noise</td><td>Contrast & Zoom</td><td>Flip & Noise</td><td>Flip & Zoom</td><td>Noise & Zoom</td></tr><tr><td>Train</td><td>8000</td><td>6400</td><td>4800</td><td>8000</td><td>9600</td><td>8000</td><td>11200</td><td>6400</td><td>9600</td><td>8000</td></tr><tr><td>Validation</td><td>2000</td><td>1600</td><td>1200</td><td>2000</td><td>2400</td><td>2000</td><td>2800</td><td>1600</td><td>2400</td><td>2000</td></tr><tr><td>Test</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td><td>500</td></tr><tr><td>Total</td><td>10500</td><td>8500</td><td>6500</td><td>10500</td><td>12500</td><td>10500</td><td>14500</td><td>8500</td><td>12500</td><td>10500</td></tr></table>
 Original image
 Noised image
 Blurred image
(mean: 0.1, std: 0.5) (std: 0.5, kernel: (5,15))
 Contrast; $\{\mathbf{c},\mathbf{b}\} = \{2,4\}$
 Zoomed image (param: 5)
 Vertical flipped image
 Horizontal flipped image
 Affine Transformation In; Out = [50,70; 230,50; 50,220]; [50,70; 230,50; 50,220]
 Fig. 3: An example of generated images
### c) CNN model
Use Residual neural network (ResNet) is proposed in 2015, by He et al. [20]. ResNet won the first place at the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2015). To preserve knowledge, reduce losses and boost performance during the training phase, ResNet introduced residual connections between layers. A residual connection in a layer means that the output of a layer is convolution of its input plus its input [21]. ReNet50 is used in this research. It consists of 50 layers as it is shown by the Fig.
5.
The model is updated by replacing the number (1000) of nodes of the softmax output layer by 5 (corresponding to the number of treated mango leaf diseases).
### d) Implementation details
The data augmentation process and ResNet50 model are all carried out using respectively, OpenCV and Keras laboratories. Model's training parameters used include Adam optimizer with a learning rate of 0.001, binary cross-entropy (loss function) and epochs of 8.
The model is trained on a server with an NVIDIA GPU and 32 GB of RAM.
## IV. RESULT AND DISCUSSION
The initial small dataset is split as follow: $64\%$ for training, $16\%$ for validation and $20\%$ for testing. After randomly splitting the dataset, we have 1,600 images for training, 400 images for validation and 500 images for testing. Results show that the training accuracy $(87.18\%)$ is greater than the testing accuracy $(39.34\%)$. So the model overfitted as it is shown by the Fig. 6.
 Fig. 4: Workflow of the data augmentation task
 Fig. 5: The architecture of ResNet50 [21]
Since the dataset is not enough to train robustly the DL model, data augmentation process is carried out. This ask concerns only training and validation data [22]. Test data remains equal to 500 images.
In the first step, after training phase, results show that the DL model overfits on all datasets except 'Original & Contrast' which gives a training and testing accuracy of $90.56\%$ and $86.23\%$ respectively (Table 3, Fig. 7).
In the second step, training the model on the combined datasets yielded the results in Table 4. The model is not overfitted on the 'Contrast & Flip' (training: $95.29\%$; testing: $91.39\%$ ) and 'Flip_Zoom' (training: $93.15\%$; testing: $90.59$ ) datasets. These two datasets are the best ones in the second step since they give best results to the DL model.
Finally, affine transformation strategy is applied to 'Contrast & Flip' and 'Flip_Zoom' datasets. Results show that 'Contrast & Flip' gives the best performances with an accuracy of $97.80\%$ and a f1_score $> 0.9$ (Table 5, Fig. 8, Fig. 9).
 Fig. 6: Training result of the original dataset
 Fig. 7: Training result of the dataset 'Original & Contrast'
Following the results presented previously, in the first step, the model overfitted in the generated datasets, except 'Original and Contrast' dataset which resulted in an accuracy of $86.23\%$. Concerning data augmentation strategies namely blur, contrast, noise and zoom, the best cominations for classifying mango leaf diseases are 'Contrast & Flip' and 'Flip & Zoom', according to the results in the second step. These two strategies yielded accuracies of $91.39\%$ and $90.59\%$ respectively. In the final step, applying the 'Affine Transformation' strategy to the datasets generated by these two strategies revealed that the best combination for mango leaf diseases classification is 'Contrast & Flip & Affine Transformation' since it yielded an accuracy of $97.80\%$.
Table 3: Results of the first step
<table><tr><td></td><td>Original & Blur</td><td>Original & Contrast</td><td>Original & Flip</td><td>Original & Noise</td><td>Original & Zoom</td></tr><tr><td>Training Accuracy (%)</td><td>98.25</td><td>90.56</td><td>95.35</td><td>76.36</td><td>92.76</td></tr><tr><td>Testing Accuracy (%)</td><td>84.21</td><td>86.23</td><td>80.60</td><td>34.84</td><td>84.80</td></tr><tr><td>Result</td><td>overfitting</td><td>ok</td><td>overfitting</td><td>overfitting</td><td>overfitting</td></tr></table>
Table 4: Results of the second step
<table><tr><td>Result</td><td>Testing Accuracy (%)</td><td>Training Accuracy (%)</td><td></td><td></td></tr><tr><td>overfitting</td><td>65.45</td><td>87.28</td><td>Blur & Contrast</td><td></td></tr><tr><td>overfitting</td><td>85.30</td><td>94.14</td><td>Blur & Flip</td><td></td></tr><tr><td>overfitting</td><td>63.32</td><td>78.29</td><td>Blur & Noise</td><td></td></tr><tr><td>overfitting</td><td>65.82</td><td>92.08</td><td>Blur & Zoom</td><td></td></tr><tr><td>ok</td><td>91.39</td><td>95.29</td><td>Contrast & Flip</td><td></td></tr><tr><td>overfitting</td><td>31.74</td><td>84.25</td><td>Contrast & Noise</td><td></td></tr><tr><td>overfitting</td><td>58.23</td><td>88.71</td><td>Contrast & Zoom</td><td></td></tr><tr><td>overfitting</td><td>645.85</td><td>78.80</td><td>Flip & Noise</td><td></td></tr><tr><td>ok</td><td>90.59</td><td>93.15</td><td>Flip & Zoom</td><td></td></tr><tr><td>overfitting</td><td>78.47</td><td>82.48</td><td>Noise & Zoom</td><td></td></tr></table>
Table 5: Results of the final step
<table><tr><td></td><td>Original & Blur</td><td>Original & Contrast</td></tr><tr><td>Training dataset</td><td>50 056</td><td>51 315</td></tr><tr><td>Validation dataset</td><td>12 514</td><td>12 828</td></tr><tr><td>Test dataset</td><td>500</td><td>500</td></tr><tr><td>Total</td><td>63 070</td><td>64 643</td></tr><tr><td>Training Accuracy (%)</td><td>98.54</td><td>97.44</td></tr><tr><td>Testing Accuracy (%)</td><td>97.80</td><td>93.98</td></tr></table>
 Training and Validation Accuracy Fig. 8: Training result of the 'Contrast & Flip & Affine Transformation' dataset
 Training and Validation Accuracy Fig. 9: Training result of the 'Flip & Zoom & Affine Transformation' dataset
## V. CONCLUSION AND FUTURE WORKS
This paper presented three contributions. The first allowed us to know the impact of data augmentation techniques namely blur, contrast, flip, noise and zoom in mango leaf diseases classification. The second is to know the best combinations between these techniques which give the best performance to the deep learning model. The last one reveals that applying 'affine transformation' technique to the combination 'Contrast & Flip' gives the best performance to the Resnet50 CNN with an accuracy of $97.80\%$.
This solution can be used to improve the performance of DL models for image classification with small datasets.
Our future work, is to propose a dataset of mango leaf diseases with images captured in mango orchards of a sahelian country like Senegal. Applying this combination as a data augmentation technique to this dataset will allow us to achieve excellent results in mango leaf disease classification using a deep learning model such as ResNet50. Then, this model will be deployed in mobile and web applications to allow mango growers to diagnose diseases in their crops without expert intervention.
## ACKNOWLEDGEMENTS
The authors would like to thank IRD (Institut de Recherche pour le Développement) SENEGAL for access to their server which was used in this study.
Generating HTML Viewer...
References
22 Cites in Article
Kusrini Kusrini,Suputa Suputa,Arief Setyanto,I Agastya,Herlambang Priantoro,Krishna Chandramouli,Ebroul Izquierdo (2020). Data augmentation for automated pest classification in Mango farms.
N Rosman,N Asli,S Abdullah,M Rusop (2019). Review: Some common disease in mango.
Shih-Lun Wu,Hsiao-Yen Tung,Yu-Lun Hsu (2020). Deep Learning for Automatic Quality Grading of Mangoes: Methods and Insights.
Alvaro Fuentes,Sook Yoon,Sang Kim,Dong Park (2017). A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition.
Christian Szegedy,Vincent Vanhoucke,Sergey Ioffe,Jon Shlens,Zbigniew Wojna (2016). Rethinking the Inception Architecture for Computer Vision.
Nabodip Sutrodhor,Molla Rashied,M Firoz,Prokash Karmokar,Tasrifa Nur (2018). Mango Leaf Ailment Detection using Neural Network Ensemble and Support Vector Machine.
S Arivazhagan,S Vineth,Ligi (2018). Mango Leaf Diseases Identification Using Convolutional Neural Network.
Rabia Saleem,Jamal Shah,Muhammad Sharif,Mussarat Yasmin,Hwan-Seung Yong,Jaehyuk Cha (2021). Mango Leaf Disease Recognition and Classification Using Novel Segmentation and Vein Pattern Technique.
Md. Mia,Sujit Roy,Subrata Das,Md. Rahman (2020). Mango leaf disease recognition using neural network and support vector machine.
K Simonyan,A Zisserman (2014). A8_ Large-Scale Image Recognition. Very Deep Convolutional Networks for Large-Scale Image Recognition.
C Shorten,T Khoshgoftaar (2019). A survey on Image Data Augmentation for Deep Learning.
Rucha Dandavate,Vineet Patodkar (2020). CNN and Data Augmentation Based Fruit Classification Model.
I Agastya,Arief Setyanto (2018). Classification of Indonesian Batik Using Deep Learning Techniques and Data Augmentation.
Suchet Bargoti,James Underwood (2017). Deep fruit detection in orchards.
Shih-Lun Wu,Hsiao-Yen Tung,Yu-Lun Hsu (2020). Deep Learning for Automatic Quality Grading of Mangoes: Methods and Insights.
Yu-Dong Zhang,Zhengchao Dong,Xianqing Chen,Wenjuan Jia,Sidan Du,Khan Muhammad,Shui-Hua Wang (2019). Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation.
Parag Thakurta,Sujoy Sett (2020). Priority driven Call Scheduling in Mobile Networks: A MOGA based approach.
(2021). FAO elearning Academy: major achievements for 2021.
Demba Faye,Idy Diop,Doudou Dione (2022). Mango Diseases Classification Solutions Using Machine Learning or Deep Learning: A Review.
Kaiming He (2015). Deep Residual Learning for Image Recognition.
Luqman Ali,Fady Alnajjar,Hamad Jassmi,Munkhjargal Gocho,Wasif Khan,M Serhani (2021). Performance Evaluation of Deep CNN-Based Crack Detection and Localization Techniques for Concrete Structures.
C Chiagoziem,Qin Ukwuoma,Md Zhiguang,Belal Bin,Liaqat Heyat,Zahra Ali,Happy Almaspoor,Monday (2022). Recent Advancements in Fruit Detection and Classification Using Deep Learning Techniques.
Explore published articles in an immersive Augmented Reality environment. Our platform converts research papers into interactive 3D books, allowing readers to view and interact with content using AR and VR compatible devices.
Your published article is automatically converted into a realistic 3D book. Flip through pages and read research papers in a more engaging and interactive format.
Our website is actively being updated, and changes may occur frequently. Please clear your browser cache if needed. For feedback or error reporting, please email [email protected]
Thank you for connecting with us. We will respond to you shortly.
