Abstract

Unmanned Aerial Vehicles (UAVs), often called drones, have gained progressive prevalence for their swift operational ability as well as their extensive applicability in diverse real-world situations. Of late, UAV usage in precision agriculture has attracted much interest from scientific community. This study will look at drone aid in precise farming. Big data has the ability to analyze enormous amounts of data. Due to this, it is one of the diverse crucial technologies of Information and Communication Technology (ICT) which had applied in precision agriculture for the abstraction of critical information as well as for assisting agricultural practitioners in the comprehension of the most feasible farming practices, and also for better decision-making. This work analyses communication protocols, as well as their application toward the challenge of commanding a drone fleet for protecting crops from infestations of parasites. For computer-vision tasks as well as data-intensive applications, the method of deep learning has shown much potential. Due to its vast potential, it can also be used in the field of agriculture. This research will employ several schemes to assess the efficacy of models includes Visual Geometry Group (VGG-16), the Convolutional Neural Network (CNN) as well as the Fully-Convolutional Network (FCN) in plant disease detection. The methods of Artificial Immune Systems (AIS) can be used in order to adapt deep neural networks to the immediate situation. Simulated outcomes demonstrate that the proposed method is providing superior performance over various other technologically-advanced methods.

Keywords:
Artificial Immune System (AIS); Big Data; Convolutional Neural Network (CNN); Deep Learning; Drones; Fully Convolutional Network (FCN); Pests; Precision Agriculture; Visual Geometry Group (VGG16)

Resumo

Os veículos aéreos não tripulados (UAVs), muitas vezes chamados de drones, ganharam prevalência progressiva por sua rápida capacidade operacional, bem como por sua ampla aplicabilidade em diversas situações do mundo real. Ultimamente, o uso de UAV na agricultura de precisão tem atraído muito interesse da comunidade científica. Este estudo analisará a ajuda de drones na agricultura de precisão. O big data tem a capacidade de analisar enormes quantidades de dados. Por isso é uma das diversas tecnologias cruciais de tecnologia da informação e comunicação (TIC) que foram aplicadas na agricultura de precisão para a abstração de informações críticas, bem como para auxiliar os praticantes agrícolas na compreensão das práticas agrícolas mais viáveis, e também para uma melhor tomada de decisão. Este trabalho analisa protocolos de comunicação, bem como sua aplicação no desafio de comandar uma frota de drones para proteção de lavouras contra infestações de parasitas. Para tarefas de visão computacional, bem como para aplicações com uso intensivo de dados, o método de aprendizagem profunda mostrou muito potencial. Devido ao seu vasto potencial, também pode ser utilizado na área agrícola. Esta pesquisa empregará vários esquemas para avaliar a eficácia de modelos, incluindo o Grupo de Geometria Visual (VGG-16), a Rede Neural Convolucional (CNN), bem como a Rede Totalmente Convolucional (FCN) na detecção de doenças de plantas. Os métodos de sistemas imunológicos artificiais (AIS) podem ser utilizados para adaptar redes neurais profundas à situação imediata. Os resultados simulados demonstram que o método proposto oferece desempenho superior em relação a vários outros métodos tecnologicamente avançados.

Palavras-chave:
Sistema Imunológico Artificial (AIS); Big Data; Rede Neural Convolucional (CNN); Aprendizado Profundo; Drones; Rede Totalmente Convolucional (FCN); Pragas; Agricultura de Precisão; Grupo de Geometria Visual (VGG-16)

1. Introduction

Agriculture, a key income source for many nations, is able to fulfill humanity’s two most essential requisites: food as well as fiber. Over the earlier decades, agriculture had drastically changed due to technological developments such as the Green Movement. The focus of agricultural research is on a myriad of topics such as water depth, commodities, and management of livestock. These responsibilities can be performed by means of drones with the utilization of an extensive range of devices as well as sensors. During the past few decades, sector managers have been revolutionized by modern developments for handling a broad range of risks with the inclusion of abrupt variations in climate as well as pests, both of which has a detrimental impact on the crop amounts as well as the agricultural product quality. Since drones are constrained by their individual supplies of fuel as well as pesticide, they are able to achieve the elimination of pests by asking for aid from other drones. Attempts have been made to address these aforementioned concerns by the use of certain bio-inspired techniques of enrolment (Refaai et al., (2022)REFAAI, M.R., DATTU, V.S., GIREESH, N., DIXIT, E., SANDEEP, C.H. and CHRISTOPHER, D., 2022. Application of IoT-based drones in precision agriculture for pest control. Advances in Materials Science and Engineering, vol. 2022, pp. 1160258. http://dx.doi.org/10.1155/2022/1160258.
http://dx.doi.org/10.1155/2022/1160258... .

Green Revolution had taken place from the 1960s till the 1980s. It led to increased production of food crops as well as food stability, especially in impoverished nations. Therefore, despite the doubling of the global population, and tripling of food production during the 1960s, the demands have already been fulfilled by agricultural production with an increase of 25% in the agricultural land. The prediction is that there will be an increase of over 75% in the consumption of agriculture as well as food goods by 2050.

Precision agriculture will involve the aggregation of historically generated as well as real-time data into unstructured and structured datasets. Since the majority of the precision agriculture-generated data is in the unstructured form, the present research trend involves this data’s utilization for the acquisition of knowledgeable information. Big data can enable an extensive variety of precision agricultural tasks for the extraction of knowledge and ideas from information in order to solve many unique in addition to major agricultural challenges & choices. Innovative technologies will give a structure for extracting insights from data in order to make earlier judgments on increasing productivity while preventing unnecessary expenses (Bendre et al., 2015BENDRE, M.R., THOOL, R.C. and THOOL, V.R., 2015. Big data in precision agriculture: Weather forecasting for future farming. In2015 1st International Conference on Next Generation Computing Technologies (NGCT), Dehradun, India. USA: IEEE, pp. 744-750. http://dx.doi.org/10.1109/NGCT.2015.7375220.
http://dx.doi.org/10.1109/NGCT.2015.7375... ).

With agricultural “big data,” it is essential to have huge investments in data storage as well as data processing infrastructures, some of which will have to operate in almost real-time (for example, forecasting of weather, monitoring of crop pests as well as animal diseases). As a result, the word "big data analysis" may refer to an innovative set of practices that are being developed to enable farmers along with organizations to remove financial worth from massive volumes of diverse data through the facilitation of high-velocity capture, discovery, and/or analysis (Kamilaris et al., 2017KAMILARIS, A., KARTAKOULLIS, A. and PRENAFETA-BOLDÚ, F.X., 2017. A review on the practice of big data analysis in agriculture. Computers and Electronics in Agriculture, vol. 143, pp. 23-37. http://dx.doi.org/10.1016/j.compag.2017.09.037.
http://dx.doi.org/10.1016/j.compag.2017.... ).

With big data analysis, certain huge-scale agricultural corporations had accordingly tailored advice to the farmers and thus, had achieved an almost USD 20 billion increase in the annual global profits from agricultural crops.

The below dimensions are used for the characterization of the big data (Evstatiev and Gabrovska-Evstatieva, 2021EVSTATIEV, B.I. and GABROVSKA-EVSTATIEVA, K.G., 2021. A review on the methods for big data analysis in agriculture. IOP Conference Series. Materials Science and Engineering, vol. 1032, no. 1, pp. 012053. http://dx.doi.org/10.1088/1757-899X/1032/1/012053.
http://dx.doi.org/10.1088/1757-899X/1032... ): Volume, Velocity, and Variety. Big data may have additional characterizations as Veracity and Valorization. While big data can be typically defined in terms of the aforementioned attributes as well as dimensions, all of them do not have to be mandatorily fulfilled as such a data analysis will be highly complex.

Human beings are always in competition with pests for accessing the available natural resources, and most significantly, the production of food. For all developing nations, plant pests, inclusive of weeds, pathogens as well as insects, continue to be one of the key constraints in food and agricultural crop production (Talebpour et al., 2015TALEBPOUR, B., TÜRKER, U. and YEGÜL, U., 2015. The role of precision agriculture in the promotion of food security. Irish Journal of Agricultural and Food Research, vol. 4, no. 1, pp. 1-23. http://dx.doi.org/10.24102/ijafr.v4i1.472.
http://dx.doi.org/10.24102/ijafr.v4i1.47... ).

The occurrence of agricultural crop losses is often caused by either abiotic factors like nutrients, temperature, water, and irradiation or biotic factors like pathogens, pests, and weeds, commonly termed environmental actors. In general, these losses will result in the minimization of yield efficiency. Potential loss, as well as actual loss, will constitute the two distinct rates of loss (Patrício and Rieder, 2018)PATRÍCIO, D.I. and RIEDER, R., 2018. Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review. Computers and Electronics in Agriculture, vol. 153, pp. 69-81. http://dx.doi.org/10.1016/j.compag.2018.08.001.
http://dx.doi.org/10.1016/j.compag.2018.... . The potential loss will signify losses that happened without the utilization of any protection systems in comparison with yields having similar crop production intensity within a no-loss scenario. On the other hand, the actual loss will constitute losses that took place in spite of the utilization of practices of crop production. Along with the crop quantity losses, there are also qualitative crop losses that may have been caused due to reduction in the valuable ingredients as well as the market quality. Hence, healthy crop growth is only possible with early detection of diseases as well as pests. With an early incidence of pests as well as its swift development, crop vegetation will suffer an adverse effect, and also will have decreased growth. Meanwhile, pest incidence in the crop’s late-stage development will result in decreased yields as well as market values.

DL, an extension of standard ML, is deployed via the addition of into the model, and also data which can get transmitted with the use of numerous functions that permit the data’s hierarchical representation via the number of abstraction levels. Diverse fields of applications are increasingly employing deep learning due to its capacity for feature learning. The deep learning algorithm will utilize the composition of the lower-level features to yield the hierarchy of the higher-level features, and thereby, accomplish the automatic extraction of features from the raw dataset which has been provided (Ganatra and Patel, (2021)GANATRA, N. and PATEL, A., 2021. Deep learning methods and applications for precision agriculture. In: Joshi, A., Khosravy, M. and Gupta, N., eds. Machine Learning for Predictive Analysis: Proceedings of ICTIS 2020. Singapore: Springer, pp. 515-527. https://doi.org/10.1007/978-981-15-7106-0_51
https://doi.org/10.1007/978-981-15-7106-... .

With DL algorithms, it is possible to resolve complicated problems with more swiftness as well as precision since these algorithms will employ highly-complex model structures so as to perform tremendous parallelization. The DL will constitute convolutions, pooling layers, fully-connected layers, activation functions, gates, memory cells, and so on. Each constituent’s utilization is dependent on the employed network architecture, namely, Recursive Neural Networks, the CNN, and the RNN. In DL, the most oft-used approach is the CNN due to it being accounted for as a deep, feed-forward Artificial Neural Network (ANN).

AlexNet, LeNet, ZFNet, GoogLeNet, etc. are some of the architectures which are frequently used by researchers. All the deep learning architectures will have their own strengths as well as weaknesses. All these architectures have pre-trained weights, that is, their networks have already undergone training with a certain dataset. As a result, these architectures can accurately classify certain problem domains. This work proposes the VGG-16, the 8-layer CNN, the 12-layer CNN, and the 2- layer FCN as well as parameter optimization by means of immune system approaches. Section Two discusses the associated literary works. Section Three has a description of the many employed approaches. Section Four has explanations of the experimental outcomes, and Section Five has the work’s conclusions.

2. Literary works

Tantalaki et al. (2019)TANTALAKI, N., SOURAVLAS, S. and ROUMELIOTIS, M., 2019. Data-driven decision making in precision agriculture: the rise of big data in agricultural systems. Journal of Agricultural & Food Information, vol. 20, no. 4, pp. 344-380. http://dx.doi.org/10.1080/10496505.2019.1638264.
http://dx.doi.org/10.1080/10496505.2019.... submitted an overview of the research which was focused on the practical use of data science approaches, particularly the ML approaches, in suitable farming structures. With the utilization of big data technologies, the data-intensive process of decision-making had an abundance of new opportunities. The authors examined literary works related to agricultural practices which used big data analysis for the resolution of numerous problems and thereby uncovered opportunities as well as up-and-coming areas of applicability. Nevertheless, the successful deployment of this technology in precision agriculture was hindered by the generated data’s high volume as well as complexity. Despite machine learning’s potential in handling agricultural big data, it had to undergo reinvention to fulfillment of its present difficulties.

Segalla et al. (2020)SEGALLA, A., FIACCO, G., TRAMARIN, L., NARDELLO, M. and BRUNELLI, D., 2020. Neural networks for pest detection in precision agriculture. In2020 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Trento, Italy. USA: IEEE, pp. 7-12. http://dx.doi.org/10.1109/MetroAgriFor50201.2020.9277657. presented an efficient solution for the automation of Codling Moths’ detection. The system had taken pictures of the trapped insects within an orchard, had used a Deep Neural Network (DNN) algorithm for the image analysis, and also, in the event of a positive detection, had delivered alarms to the farmer. The system was wholly autonomous, and also had operated on its own for the overall crop season. There was the utilization of detection reports for optimization of the chemical treatment upon threat identification. An energy-neutral balance was accomplished through the design of a prototype with a small solar panel-powered embedded platform.

Albanese et al. (2021)ALBANESE, A., NARDELLO, M. and BRUNELLI, D., 2021. Automated pest detection with DNN on the edge for precision agriculture. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 11, no. 3, pp. 458-467. http://dx.doi.org/10.1109/JETCAS.2021.3101740.
http://dx.doi.org/10.1109/JETCAS.2021.31... submitted an embedded system that was improved with functionalities of ML for assurance of continuous recognition of pest infestation within the fruit orchards. The basis for the proposed solution was a low-power embedded sensing system as well as a Neural Accelerator which could take as well as process images within the oft-used pheromone-based traps. The platform’s various abilities were highlighted via the training as well as the implementation of three distinct machine learning algorithms. Additionally, with the integration of the functionalities of energy harvesting, the proposed approach ensured the battery life’s extension. The experimental outcomes proved automation of the pest infestation task for an unlimited time without any necessity for intervention from the farmer.

Lippi et al. (2021)LIPPI, M., BONUCCI, N., CARPIO, R.F., CONTARINI, M., SPERANZA, S. and GASPARRI, A., 2021. A YOLO-based pest detection system for precision agriculture. In:2021 29th Mediterranean Conference on Control and Automation (MED), Puglia, Italy. USA: IEEE, pp. 342-347. http://dx.doi.org/10.1109/MED51440.2021.9480344.
http://dx.doi.org/10.1109/MED51440.2021.... had drawn inspiration from the requirements of PANTHEON, H2020 European project for accuracy agricultural of hazelnut orchards, and thus, had proposed a data-driven system for detection of pests. Primary-step pest finding was an essential step in the design of efficient strategies for crop defense in Precision Agriculture (PA) settings. Out of all possible pests, the authors had concentrated on trues due to hazelnut production being seriously jeopardized by these pests. For this purpose, the authors gathered a custom dataset within a realistic outdoor environment, and also trained a You Only Look Once (YOLO)-based CNN which had accomplished an average precision of ≈ 94.5% on a holdout dataset. Moreover, they performed an extensive assessment of the detector performance via analysis of the influence of data augmentation approaches as well as the depth information. Eventually, the authors implemented it on an NVIDIA Jetson Xavier. Upon its arrival at ≈ 50 fps, there was the facilitation of online processing on-board of any robotic stage.

A difficult challenge in a PA scenario was identifying bugs with protective color features within the complex field surroundings. To this end, Hu et al. (2022)HU, Z., XIANG, Y., LI, Y., LONG, Z., LIU, A., DAI, X., LEI, X. and TANG, Z., 2022. Research on Identification technology of field pests with protective color characteristics. Applied Sciences (Basel, Switzerland), vol. 12, no. 8, pp. 3810. http://dx.doi.org/10.3390/app12083810.
http://dx.doi.org/10.3390/app12083810... had devised a technique of field pest identification on the basis of near-infrared imaging technology as well as YOLOv5. Initially, there was a selection of an appropriate infrared filter as well as ring light source for an image acquisition system’s construction as per the wavelength that had the greatest spectral reflectance difference between the pest’s (Pieris rapae) spectral curves and that of its host plant (cabbage), that were created by certain spectral attributes. Afterwards, there was a collection of the field pest images for the construction of a data set, which was then trained as well as tested by means of YOLOv5. The simulated outcomes had shown that 0.56 s was average required time for a single pest image’s detection, and 99.7% was reached by the mAP.

Melgar-García et al. (2022)MELGAR-GARCÍA, L., GUTIERREZ-AVILES, D., GODINHO, M.T., ESPADA, R., BRITO, I.S., MARTÍNEZ-ÁLVAREZ, F., TRONCOSO, A. and RUBIO-ESCUDERO, C., 2022. A new big data triclustering approach for extracting three-dimensional patterns in precision agriculture. Neurocomputing, vol. 500, pp. 268-278. http://dx.doi.org/10.1016/j.neucom.2021.06.101.
http://dx.doi.org/10.1016/j.neucom.2021.... had introduced a novel big data tri-clustering scheme that was on the basis of evolutionary algorithms. The innovative approach was capable of detecting 3d patterns based on the vegetation indicators of vine crops. Testing was done of diverse vegetation indices for identification of the crops’ distinct patterns. Reporting of the experimental outcomes was done with the utilization of a Portuguese vineyard crop which depicted four distinct areas with diverse moisture stress particularities which could cause changes in the vineyard’s management. With the performance of scalability studies, it was found that the proposed scheme was feasible for the management of a huge dataset.

Ullah et al. (2022)ULLAH, N., KHAN, J.A., ALHARBI, L.A., RAZA, A., KHAN, W. and AHMAD, I., 2022. An efficient approach for crops pests recognition and classification based on novel deeppestnet deep learning model. IEEE Access : Practical Innovations, Open Solutions, vol. 10, pp. 73019-73032. http://dx.doi.org/10.1109/ACCESS.2022.3189676.
http://dx.doi.org/10.1109/ACCESS.2022.31... had put forward an innovative end-to-end DeepPestNet structure for pest identification as well as classification. This DeepPestNet design was made up of eleven learnable layers, with the inclusion of 8 convolutional layers as well as three Fully-Connected (FC) layers. The authors employed image rotation schemes for increasing the dataset’s size as well as image augmentation schemes for testing DeepPestNet’s generalization ability. The proposed DeepPestNet framework’s assessment was performed with the well-known Deng’s crop dataset. This framework was employed for recognition as well as classification of the crop pests into 10-class pests, namely, LocustaMigratoria, EuproctisPseudoconspersa Strand, ChrysochusChinensis, EmpoascaFlavescens, SpodopteraExigua, larva of laspeyresiapomonella, parasalepida, acridacinerea, larva of S. exigua as well as L. pomonella types of insect pests. The suggested DeepPestNet framework accomplished a 100% optimal accuracy. Furthermore, the authors conducted a comparison study of the suggested DeepPestNet framework against the standard pre-trained models of deep learning.

Zangina et al. (2021)ZANGINA, U., BUYAMIN, S., AMAN, M.N., ABIDIN, M.S.Z. and MAHMUD, M.S.A., 2021. A greedy approach to improve pesticide application for precision agriculture using model predictive control. Computers and Electronics in Agriculture, vol. 182, pp. 105984. http://dx.doi.org/10.1016/j.compag.2021.105984.
http://dx.doi.org/10.1016/j.compag.2021.... took a proactive approach, introducing insecticide demand control using an active mass-spring suspension mechanism. In addition, with the usage of a controller that depends on model predictive control which employed the model of active demand management, this work effectively resolved issue of identification of right time, quantity as well as location for the application of pesticide in an agricultural field. Afterward, the proposal of a greedy algorithm was made for a resolution of the problem of vehicle routing after identification of the pesticide application’s optimal time as well as location. With due consideration of the models of pest risk predictions, the proposed approach mitigated the pest infestation risk. It was evident from the simulation outcomes that the proposed approach maximized crop protection against pests. In addition, when compared with the current approaches, the proposed approach’s performance analysis had shown its substantially lower complexity in computation as well as it is almost 78% quicker convergence toward the optimal solution.

3. Methodology

This section had discussions on the VGG-16, the CNN, the 2- layer FCN as well as the proposed PLANET-parameter optimization that employed the immune system approaches.

3.1. VGG-16

The model of VGG-Net was devised by Simonyan et al., with the least amount convolution within the network. Despite its structural simplicity, the VGG-Net’s extensive application in the CNN models was due to its more in-depth arrangement which was monitored by the layers of associations of double or triple convolution layers. On the other hand, the earlier models had the layers of sharing trailed by the convolution and so on. VGG provided a suitable feature representation for over a million photos (the ImageNet dataset) from 1,000 different groups, so as a result, the framework served as an effective feature extractor for potentially fresh pictures. The ImageNet dataset was capable to gather related attributes from photographs, including new images that did not previously exist or that were in completely different groups within the dataset. As a result, employing pre-trained models as an effective feature removal proved advantageous (Sahinbas and Catak, 2021SAHINBAS, K. and CATAK, F.O., 2021. Transfer learning-based convolutional neural network for COVID-19 detection with X-ray images. In U. KOSE, D. GUPTA, V.H.C. DE ALBUQUERQUE and A. KHANNA, eds. Data science for COVID-19. Cambridge: Academic Press, pp. 451-466. http://dx.doi.org/10.1016/B978-0-12-824536-1.00003-4.
http://dx.doi.org/10.1016/B978-0-12-8245... ).

Figure 1 RepresentsVGG-16’s design. The VGG-16 design comprises three convolution filters with thirteen convolution layers each for feature extraction, with every convolution layer preceding a ReLU layer, and extreme pooling layers for sampling. It will include three distinct levels which are completely linked for classification purposes, two of them will operate as hidden layers, and third classification layer with 1,000 units will reflect the picture categories in the ImageNet database.

Even though the VGG-16 is a structural replica of a bigger filter, it will also conserve the merits of smaller-sized filters. When compared to the earlier models, the VGGNet has better perform better with the utilization of less number of parameters. Moreover, it will employ two distinct ReLU layers rather than a single ReLU layer for two layers of convolution. Because of the decrease in spatial size of every layer’s input volumes (the convolution as well as partnering layers’ end result), there will be an increase in the volume depths due to the increasing filter numbers. It will show good performance for problems of object classification as well as edge detection.

It will fine-tune the model duty via the application of the VGG-16 network model. Suppose, it has a dataset with 𝑚 samples {(𝑥 (1), 𝑦 (1)),…,(𝑥 (𝑚), 𝑦 (𝑚))} for training purposes. The definition of network’s complete cost function will be according to the below Equation 1:

For the above, 𝐾𝑤, 𝑏 ((𝑖)) will indicate neural network design, will indicate the connection weight among jth element of layer 1 as well as the ith element of layer 𝑙+1 while b will indicate hidden layer neuron’s bias term. Equation 1 will be a regulation item on right-hand side that will avoid over-fitting, will significantly minimize the weight, and also will adjust the two terms’ relative importance before as well as after cost function, 𝜆. Upon resolution of the minimum values of Equation 1 as well as the minimum value of 𝐽(𝑊, 𝑏), the reverse conduction algorithm will adopt the accepted batch gradient descent optimization procedure, also will assess the partial derivative of 𝑊 as well as 𝑏.

3.2. Convolutional Neural Network (CNN)

Being a generic DL type, a CNN structure has more amount of hidden layers as well as neurons compared to either ANN or traditional Multi-Layer Perceptron (MLP). Moreover, CNN is in fact a type of supervised learning which is able to self-learn as well as to self-organize on the basis of the input data as well as its associated output labels. It is able to remove dependence on the hand-crafted features, and also is able to directly learning useful features from the data. Over the past decades, successful application of the CNNs has been done in diverse areas such as classification of images, localization of objects, and face recognition. Due to its high-efficiency, the CNNs are comprehensively employed for design of screening tools to assist clinicians in the field of medicine (Zhao et al., 2019a, bZHAO, Z., DENG, Y., ZHANG, Y., ZHANG, Y., ZHANG, X. and SHAO, L., 2019a. DeepFHR: intelligent prediction of fetal Acidemia using fetal heart rate signals based on convolutional neural network. BMC Medical Informatics and Decision Making, vol. 19, no. 1, pp. 1-15. http://dx.doi.org/10.1186/s12911-019-1007-5. PMid:31888592.
http://dx.doi.org/10.1186/s12911-019-100... ).

Upon comparison with the conventional schemes of machine learning, a CNN’s key difference is its direct ignorance of the requisite for the methods of feature extraction as well as selection. Thus, for majority of the physiological signals, CNN utilization is able to avoid valuable information loss as well as mitigate the computational burden associated with the best features’ extraction and selection during the training procedure for the pathological conditions’ accurate classification. Furthermore, a CNN will employ receptive fields as well as weight sharing for substantial reduction of the parameter numbers required by the neural networks for training purposes. The following appealing benefits became the main justifications for selecting a CNN for objective forecasting of fetal academia.

The CNN will incorporate the feature extractor as well as the classifier, and the Figure 2 will offer the illustration of this work’s 8-layer deep 2D CNN style which will contain input layer, the convolution-activation-normalization pooling layers, fully-connected-dropout layers as well as last classification layer. From input to output, various computational neural nodes build connections between one layer and another, as the input information is transferred layer by layer. The continuous convolution pooling design would decode, interpret, converge, and transfer the distinctive information of the original input onto hidden feature space. Following that, an entirely connected layer will do classification based on the collected characteristics. The spatial size description of each layer's output feature maps is going to be indicated by the output shape, while the parameter will represent the overall amount of weights including the biases.Figure 2 has detailed descriptions of the CNN model’s employed layers.

• Image input layer (layer 1) - Upon utilization of a random crop approach for the image transformation, there will be the image dataset’s enrichment as well as enhancement of the model’s generalization capacity;

• Convolution layer (layer 2) - A feature map, whereby the hidden layers are linked together, can be used for the layer of convolution to retrieve pixel-level abstracted picture characteristics using convolution processes performed by one or more convolution kernels (that are stated to as filters). Every convolution kernel will use a sliding window technique to traverse the entire feature map, gathering & incorporating data from every little area to finish capturing the input image's partial feature. The filter parameters utilized in every convolution layer of a CNN are often similar for a pair of factors: (i) sharing allows the picture content to be unaffected by location; (ii) this consistency considerably reduces parameters of optimization. A critical as well as appealing attribute of the CNN algorithm is its parameter sharing mechanism;

• Activation layer (layer 3) - The output of the convolution layer will be mapped using an Activation Function (AF) to produce feature mapping connection. Usually, the AF is employed in the middle of each layer of a neural network to perform the mapping modification of the input information as well as to provide the network's non-linear modeling capability. The element-by-element computations will not change the dimension of initial information during this method. In comparison to alternative linear functions, our CNN algorithm picked the ReLU due to the following advantages: (i) faster convergence; and (ii) just a single threshold is essential for activation value acquisition, eliminating the requirement for complicated calculations;

• Normalization layer (layer 4)-The Batch Normalization (BN) layer will standardize each layer's input data throughout the neural network's training technique, causing the gradient to grow in size and so avoiding the gradient disappearance issue while also significantly accelerating training speed;

• Pooling layer (layer 5) - The CNN structure will typically insert a pooling layer (also named a sub-sampling layer) on a periodic basis between the consecutive convolution layers. As image features which are feasible in a region may have equivalent applicability in yet another region, pooling layer will incorporate the features which are semantically similar. The operation of pooling will minimize the eigenvectors of the convolution output as well as the parameter numbers. Thus, the pooling has the ability to mitigate the model complexity as well as increase the computation speed whilst performing over-fitting prevention. Akin to the convolution layer, feature mapping for every sub-region on input feature map will be carried out by the operation of pooling in steps of stride. The popular pooling procedures are max pooling, average pooling, and the randomized pooling. This CNN model will employ the former operation to evaluate the image area’s maximum value as the pooled result;

• Fully-connected layer (layer 6) -Location of the fully-connected layer will be at network structure’s end. And it is a standard MLP network. This network layer’s final output will be the input images’ high-level features that are then statistically measured as per a classifier. Further, there is assessment of probability of the input image’s corresponding class label. It is assumed that the input image information has been abstracted into more information-intensive features after numerous rounds of convolution as well as pooling processing. Thus, the convolution layer as well as the pooling layer are taken in account as essential methods for the automatic image feature extraction. Upon completion of feature transformation, the task of final classification will be carried out by the fully-connected layer;

• Dropout layer (layer 7) -For the classification task, it will typically try to avoid the over-fitting from occurring. Over-fitting is said to have occurred when in spite of the trained model’s acquisition of high accuracy on the training data, its test data will have a relatively huge generalization error. That is to say, over-fitting is referred to as a particular situation where a defined model is capable of memorization of the training data’s random noise yet is unable to learn the training data’s general trend. Over-fitting can occur due to multiple factors, and in this work, listed below are some of available as well as proposed specified solutions:

  1. a

     Regularization: This robust method will introduce additional information so as to resolve an ill-posed problem and thus, prevent over-fitting. For this work, there is application of the L2 regularization for a regularizes addition to cost function;

  2. b

     Dropout technique: Typically, dropout layer will be organized after fully-connected layer. At the time of training procedure, various neural units will get temporarily dropped from network with a specific probability.

• Classification layer (layer 8) - Eventually, a softmax function is employed by the classification layer for separation of the output classes.

3.3. Two-Layer Fully Convolutional Networks (FCN)

For a convent, every layer output will be a $h\times w\times d$-sized three-dimensional array, in which h, as well as w will indicate the dimensions while d will indicate the feature or channel dimension. The image’s initial layer will have a $h\times w$ pixel size as well as d number of channels. The higher-layers’ locations will be associated with the image’s locations they have path-connections with, also referred to as their receptive fields (Long et al., 2015LONG, J., SHELHAMER, E. and DARRELL, T., 2015. Fully convolutional networks for semantic segmentation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA. USA: IEEE, pp. 3431-3440.).

On an intrinsic level, the convents are translation-invariant. Its essential components (i.e., convolution, pooling as well as activation functions) will function on the local input regions, also are only dependent on relative spatial coordinates. Suppose that x_ij will denotes data vector at a location (i; j) in a certain layer, and y_ij will denote the same for the succeeding layer, these functions will use Equation 2 to evaluate the outputs, y_ij:

In this case, k represents kernel size, s represents stride or subsampling factor, and f_ks represents layer type as follows: a matrix multiplication for convolution or average pooling, a spatial max for max pooling, an activation function's element-wise non-linearity, along with so on for other layer forms.

This functional structure will be preserved throughout composition, with kernel size and stride obeying the transformation rule Equation 3:

Upon computation of a general non-linear function by a general net, a net having layers only of this form will assess a non-linear filter that it will refer to as a deep filter or fully-convolutional network. Typically, an FCN is able to operate on any-sized inputs to yield corresponding (possibly resampled) spatial dimensions as its output.

A real-valued loss function with an FCN is used to define a job. If the loss function is a sum of the spatial dimensions of the last layer,$l(x;\theta )={\displaystyle {\sum }_{ij}l\text{'}(x_{ij};\theta )}$ its parameter gradient will be a sum over each of its spatial components’ parameter gradients. After evaluating the whole picture, the stochastic gradient descent on l shall be comparable to the stochastic gradient descent on l', while all of the receptive fields of last layer will be treated as a mini-batch.

Upon significant overlap of these receptive fields, the feed-forward computation as well as the back-propagation will have more efficacy when they are evaluated layer-by-layer over the complete image rather than patch-by-patch in an independent manner.

The classification nets are going to be converted into fully-convolutional nets, that will offer coarse output maps. However, these coarse outputs will have to be connected back to the pixels for pixel-wise predictions.

The generic recognition nets, inclusive of LeNet, AlexNet as well as its deeper successors, will supposedly use inputs of fixed sizes to yield outputs that are non-spatial in nature. The layers that are fully interconnected in these nets will have fixed dimensions & will not use geographic coordinates. Nonetheless, these fully interconnected layers may be visualized as convolutions with kernels that embrace their whole input areas (Wang et al., 2016WANG, L., QIAO, Y., TANG, X. and VAN GOOL, L., 2016. Actionness estimation using hybrid fully convolutional networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. USA: IEEE, pp. 2708-2717. http://dx.doi.org/10.1109/CVPR.2016.296.
http://dx.doi.org/10.1109/CVPR.2016.296... ). Thus, these nets will get cast into fully-convolutional networks which have the ability to use any-sized inputs to yield output maps that are spatial in nature.

Moreover, although the resultant maps share equivalence with the original net’s evaluation for certain input patches, there is high amortization of the evaluation over those patches’ overlapping regions. As an example, the AlexNet will need 1:2 ms (on a generic GPU) for inferring a 227 $\times$ 227 image’s classification scores. Contrastingly, for a 500 $\times$ 500 image, the fully-convolutional net will only need 22 ms to yield an output grid of 10 $\times$ 10 grid, that is more than five times quicker than the naive approach1.

Due to these convolutionalized models’ spatial output maps, they are the obvious choice for dense difficulties includes semantic segmentation. With availability of ground truth at every output cell, forward pass as well as the backward pass are quite straightforward, and also will make full use of the convolution’s intrinsic computational efficacy (as well as aggressive optimization). For the AlexNet example, the corresponding backward time for a single image is 2:4 ms while for a fully-convolutional 10 $\times$ 10 output map it is 37 ms, and thus will result in a speedup that is same as that of the forward pass.

When the classification nets are reinterpreted as being fully-convolutional, they can yield output maps for all sizes of inputs. However, subsampling is normally used to minimize their output dimensions. Subsampling is employed by the classification nets to maintain small filters as well as viable requisites for computation. By doing so, the output of these networks' fully-convolutional version will be coarsened by reducing its input size by a factor equal to the pixel stride of output units' corresponding fields.

3.4. Proposed PLANET - Parameter Optimization Using Immune System

Computer research’s key inspiration for problem-solving is the biology as well as its diverse concepts. Of all the various natural concepts, emergence of the immune system has become a vital design aspect in the optimization algorithm development. A problem of optimization will seek the optimum element from a potential solution set. Since every scientific area will have diverse problems of optimization, there has been the development of various algorithms as well as techniques. Heuristics are the most feasible ones from among these algorithms, and are inclusive of diver’s algorithms like the genetic algorithms, the ant colony optimization, the firefly swarm optimization, the artificial bee colony, and so on.

The concept of population-based heuristics is employed by Decastro’s AIS. Computer researchers had developed algorithms that were influenced by the biological immune system’s defense mechanisms against diverse pathogen types. The resolution of various problems of optimization in areas of science technology as well as science was made possible with these algorithm types. Nevertheless, the biological immune system’s high-complexity has made its usage quite difficult in areas of technology. In addition, while there is on-going comprehensive research on the biological immune system, the adoption of the AIS has been done for only certain mechanisms that are influenced by the human immune structure, inclusive of clonal selection (Touami et al., 2018TOUAMI, M.A.C., SALEM, M. and KHELFI, M.F., 2018. Optimizing Planet Wars bot using an immune-based algorithm. In2018 International Conference on Applied Smart Systems (ICASS), USA: IEEE, pp. 1-7. http://dx.doi.org/10.1109/ICASS.2018.8652061. ).

3.4.1. Natural Immune System

The biological immune system, with inclusion of human body system, will have a structure made up of cells, molecules as well as organs for providing the body defense against diseases. For the purpose of generating an immune response, the immune system will categorize body cells into self-cells, and the foreign cells into non-self-cells (i.e., antigens). For defeating the non-self-cells, a feasible mechanism will activate the immune system response. Now, this mechanism’s definition will be offered by the antigen type since a specific antigen will signify a specific response as well as a specific procedure. In the event of similar antigens, the immune system will develop memory cells are developed by the immune system so as to yield a rapid response (De Castro and Von Zuben, 2002DE CASTRO, L.N. and VON ZUBEN, F.J., 2002. Learning and optimization using the clonal selection principle. IEEE Transactions on Evolutionary Computation, vol. 6, no. 3, pp. 239-251. http://dx.doi.org/10.1109/TEVC.2002.1011539.
http://dx.doi.org/10.1109/TEVC.2002.1011... ).

Figure 3 depicts the procedure of biological clonal selection as well as the steps taken for defending the body, beginning with antigen detection till its elimination.

Clonal selection is one among the most critical defense mechanisms against the non-self-cells. For clonal selection, the immune response will describe the procedure of how an immune system will offer stimulus against a particular antigen through proliferation of a specified cell type which is the only one capable of recognizing that antigen.

Upon an antigen’s entry into the body, the immune response will be to enable the B-cells’ antibody secretion capability. Later, the T-cells will signal the B-cells to proliferate as well as to mature into the plasma cells, i.e., cells which will secrete the terminal antibodies. Since the B-cells’ proliferation will be in accordance with the level of affinity, a higher affinity will signify the generation of more number of clones, and in turn, affinity maturation will be the term used to describe this overall procedure.

Below are the steps of the procedure of clonal selection:

1. The cloned cells will pass through a procedure of mutation;

2. There will be elimination of the self-reactive receptor;

3. There will be proliferation of mature cells which are capable of antigen detection.

3.4.2. Artificial Immune System (AIS)

The approach known as AIS was created by drawing inspiration from the biological immune system. Its search strategy ought to be similar to the natural immune system because of the connection with the fitness function and affinity maturation in the natural system.

Size of pooling, size of filter, rate of learning, and number of filters are few examples of the AIS-optimized CNN parameters. Random antibodies initialization will involve these CNN parameters being initialized in a random manner.

1. Initialization: There will be generation of N, a random population within the search space just like the procedure in other heuristic algorithms. Due consideration will be given to this population as antibodies;

2. Clonal proliferation: The antibodies will clone (proliferate) as per their fitness (affinity) in this step;

3. Maturation: The technique of maturation will be akin to the mutation procedure with a P mutation probability. Equation 4 will be applied with this mutation as below:

   $x_{id}=x_{id}+k(x_{d_{\max }}-x_{d_{\min }}).N(\mathrm{0,1})$(4)

For the above, k will indicate the scale factor, N (0, 1) will indicate the standard distribution,$x_{id}$ will indicate antibody i’s d-dimension while $x_{id}=x_{id}+k(x_{d_{\max }}-x_{d_{\min }}).N(\mathrm{0,1})$will indicate variable i’s max bound, and $x_{id}=x_{id}+k(x_{d_{\max }}-x_{d_{\min }}).N(\mathrm{0,1})$will indicate variable i’s min bound.

1. Evaluation: This step will calculate the affinity values so as to assess every antibody’s fitness function;

2. Aging operator: It will remove individuals lost more. Thus, Aging operator will result in the initial population’s upgrade;

3. Selection procedure: It will be employed to pick N individual to the subsequent generation;

Figure 4 Depicts the flowchart of proposed PLANET-parameter optimization which employs AIS.

4. Results and discussion

In this unit, VGG 16, CNN 8 layer - 2 layers FCN, CNN 12 layer - 2 layer FCN and proposed PLANET - parameter optimization using immune system methods are used. The accuracy, sensitivity, specificity, negative predictive value, f measure, and misclassification rate as shown in Table 1 to Table 6 and Figure 5 to Figure 10.

From Figure 5, it can be observed that the proposed PLANET - parameter optimization using immune system has higher accuracy by 6.46% for VGG16, by 3.15% for CNN 8 layer - 2 layer FCN and by 1.91% for CNN 12 layer - 2 layer FCN respectively.

From Table 2 and Figure 6 It is clear that the projected PLANET- parameter optimization using immune system has higher average sensitivity by 6.22% for VGG16, by 3.14% for CNN 8 layer - 2 layer FCN and by 1.98% for CNN 12 layer - 2 layer FCN respectively.

From Table 3 and Figure 7 It is clear that the projected PLANET - parameter optimization using immune system has higher average specificity by 1.16% for VGG16, by 0.56% for CNN 8 layer - 2 layer FCN and by 0.34% for CNN 12 layer - 2 layer FCN respectively.

From Table 4 and Figure 8, it can be observed that the proposed PLANET - parameter optimization using immune system has higher average negative predictive value by 1.18% for VGG16, by 0.57% for CNN 8 layer - 2 layer FCN and by 0.34% for CNN 12 layer - 2 layer FCN respectively.

From Table 5 and Figure 9, it can be observed that the proposed PLANET - parameter optimization using immune system has higher average f measure by 6.64% for VGG16, by 3.25% for CNN 8 layer - 2 layer FCN and by 2.01% for CNN 12 layer - 2 layer FCN respectively.

From Table 6 and Figure 10, It is clear that the projected PLANET - parameter optimization using the immune system has a lower misclassification rate by 60.7% for VGG16, by 35.48% for CNN 8 layer - 2 layer FCN and, by 23.33% for CNN 12 layer - 2 layer FCN respectively.

5. Conclusion

This work has conducted a survey for analyzing the impact of deep learning’s application in the agricultural field. Being robust models of visualization, CNNs can offer feature hierarchies. Hence, improvements on the prior best results can be offered by the convolutional networks themselves through their training which is from end-to-end, then pixels-to-pixels. The prime focus was on the construction of an FCN which will employ an arbitrarily-sized input to yield an output having a relative size with inference as well as learning which is highly effective. As an evolutionary algorithm, the AIS’s inspiration is based on how the human body is defended from pathogens by the biological immune system. With the AIS, the objective function’s maximization handled to the enhancement of the optimized CNN parameter. It is demonstrated from the simulated outcomes that the proposed PLANET-parameter optimization with immune system usage have high accuracies by 6.46% as that of VGG-16, by 3.15% as that of the CNN 8 layer - 2 layer FCN, and by 1.91% for the CNN 12 layer - 2 layer FCN.

Publication Dates

History