Expert Recommender System for Mapping Optimum Crop Type

Over the centuries many revolutions have happened, lot of innovations took place, several discoveries changed the way the world would have been, these may be credited to discovery of paper, compass, steel, steam engine, anti-biotics, lenses, transistors, electricity, or computer. Civilizations have evolved but, in every era, agriculture played a major role in fueling the prosperity of the nation and establishing its growth. Agriculture in India provides livelihood to almost about 60% of India’s population.

Percentage contribution of agriculture and allied sectors in gross value added for India at present prices stood at 17.8% in 2020 Shankarnarayan et al., (2020). Investment through consumer in India should grow in 2021, post losses due to COVID-19 contraction. Food sector in India is set for huge growth owing to its immense potential for diversity and value addition. Food processing sectors contributes approximately 32% of total market share, supposedly one of biggest sectors and stands fifth in terms of production, consumption, export and expected growth. Farming as a sector is a vulnerable investment owing to dependency on diversified geographical locations, environmental conditions and pest attacks, it is of prime importance to devise technological assisted methods to monitor and provide early remedial actions for the damage and infections to the crop. Damages caused due to erratic environmental conditions, diseases and pest attacks cause substantial losses in the yield and quality of vegetables produced worldwide Soil Health Card (2019). Food and Agriculture Organization, Fao (2021) report says that more than half world’s population depends on agriculture for their survival. With innovations in technology, this is the best time to connect all the missing links to make farming a sustainable activity for all farmers. Forecasting of crop production is most important aspect of agricultural statistics system. As an effective measure towards understanding weather cycles, crop diseases and crop patterns that would benefit and empower farmers for better decision-making from sowing to marketing. Banerjee et al., (2018).

Briefly there are numerous ways in which agriculture impacts economy, can be summarized as:
 
Resource for food
 
Healthy production establishes availability of food supply to all, thereby reducing pressure for imports.
 
National income
 
prosperity in production can result in exports, increase in demand and supply of exports can strengthen economy.
 
Requirements for raw materials
 
Farming involves requirement of certain kind of initial infrastructure and products for maintaining the continuity, Healthy growth results in increase in earnings and so does in availability of infrastructure and products.
 
Agriculture in rural development
 
With healthy growth rural families can survive with agriculture as livelihood and this would even motivate young generation to consider farming as carrier option. More than half of the world’s population come from underdeveloped and developing countries. And a vast majority of that population lives in rural areas.

Agriculture’s essential role is one of production, both for food and other raw materials for the rural and urban populations. The land is a basic resource for agriculture and rural or developing areas have lots of it. The exploitation of this resource has helped families in rural areas turn agriculture into a revenue source and develop.

With innovations in agriculture techniques and technology, future ahead seems quite promising, From the above excerpt, it is evident that the importance of agriculture cannot be overstated. As scientists continue to discover new procedures to increase crop and livestock yields, increase overall food quality and reduce loss due to insects and diseases, we can safely say that agricultural research still has a long way to go.

It’s been observed that agriculture is being practiced in India based on year old extensive knowledge of ancestors and not on factual basis. This practice was successful in the early years but now with climate and soil undergoing extensive changes it has been observed that areas which were suitable for growth of a particular crops since years now has slowly and slowly stopped to support the growth of the same crop as discussed by Reddy (2017) and Arora (2019). As a result of this farmers now must grow different crops without any assurance that whether the soil and climatic conditions of their area would facilitate the cultivation of the crop or not Thornthwaite (1948). Thus, a more logical means of support is required by the farmers over which they can rely regarding which crops they could harvest in their regions and also help them in understanding the weather conditions of the area in near future so that they are prepared for the worst beforehand.

Proposed work is an intelligent solution, a recommender system that employs a statistical and learning algorithm to recommend the best possible crops to the farmers, application is a data driven approach which makes use of several amount of previous data to make the best predictions.  System brings three cutting edge technologies that is IoT, Machine Learning and Cloud Computing together over a same platform. It employs diversified category of sensor networks at its ground level for collection of the raw data. The data which is collected is then sent to the cloud for storage so that it can be accessed from anywhere. Moreover, it allows remote sensing of the local environment as data collected is then and there stored over the cloud. Once the data is collected which consists of data such soil nutrient values; ph. contents; moisture contents; the rainfall in that area and etc. Based on the collected data analysis is done to extract out crucial chunks of information using several data mining techniques and then the data is passed through a machine learning model to make predictions related to which crops would be best to cultivate in the area under consideration. Recommender proposed offers a platform where the user would just be needed to be enter basic details such as the name and place where he wishes to do farming and based on the entered details the Machine Learning model which already has access to the soil contents and climatic conditions data recommends the best possible crops suitable for that region.

The paper is arranged as follows, section II, presents the motivation for conducting the study, section III presents the materials and method employed for implementing the algorithm adopted for recommending suitable crop, section IV discusses in brief about the results obtained from the implementation conducted and finally conclusion is presented in section V.
 
Motivation-literature survey
 
With innovations in agriculture techniques and methods, a substantial number of modifications have happened in the way farmers cultivate crops. Technologies have a gone a long way in assisting farmers enhance their yield, suggestions has been in every domain, ranging from types of crops, irrigation support, fertilizers, sowing distance, disease identification, stress level and healthy growth. Learning algorithms have been instrumental in finding gaps and analysing efficacy of integrating technologies Pudumalar et al., (2016) and Jha et al., (2019) discussed same in there work. Employing sensors, cameras and drones for efficient farming have involved technologies such as IoT, cloud computing and big data analysis for the cause Wolfert et al., (2017). Extensive work has been done on predicting the yield, with variations on certain set of dependent parameters like climate, soil types and fertilizer quantity Gaitan (2015). A novel learning technique for estimating crop yield and effect of climate on cultivation of crops presented by Crane-droesch (2018). The technique suggested employed classifying, clustering, detecting and predicting diversified environmental conditions affecting agricultural operations. Mehnatkesh et al., (2012)  employed ANN and MLR to estimate biomass yield of winter wheat by identifying input features such as soil, precipitation, topographic and management factors, the amount of (nitrogen, phosphorus and potash) fertilizers consumed and efficiency of water usage. Model achieved an accuracy of 90%. Kumar et al., (2019) presented a technique that suggest a crop taking temperature, rainfall and soil pH into consideration, classification technique employed was decision tree and regression. Pudumalar et al., (2016) presented a mining technique for recommending crop on basis of soil characteristics only. Technique employed was ensemble and nave Bayes algorithm for classification. Jha et al., (2019) employed K-nn and random forest to develop a recommendation model, taking pH value, moisture and temperature as input.  Ransom et al., (2019) utilized soil and climate data for suggesting nitrogen level for corn cultivation, performance of 8 learning algorithms were tested for the model.  Model was assessed based on the nitrogen fertilizer recommendation. Duraisamy Vasu et al., (2017) presented an assessment of variability of soil characteristics employing geospatial algorithms for farm level nutrient management. From the literature survey we concluded that a cross validation of results with experimental parameters is must, as with mutation sometimes ideal parameters for the crop cultivation may change from time to time and from location to location as discussed in Sirsat et al., (2017). Work proposed considered suitability of soil properties, NPK level, climatic properties and moisture content for recommending a particular crop, model was classified employing random forest algorithm in refernce to work by Arindam (2021).

Majority of farmers are not aware and are not educated in the agriculture domain, therefore they practice agriculture in traditional techniques, these are generally not efficient and leads to low yield. Furthermore, the knowledge of choices of suitable crops and favorable soil types are difficult for farmers to decide. However traditional techniques will give results but with innovations and recent solutions yield can increase the yield multifold. Farming is subjected to diversified challenges such as: climatic hazards, degradation of natural resources, desertification, difficulty for accessing agronomists, lack of comprehensive information on crops, heavy agricultural manual operations, the difficulty of mobilizing and capitalizing on knowledge in the field. Number of solutions are being proposed to address above challenges as techniques to guide the cultivators on the cultivation techniques, How to guide farmers on the choice of seed varieties, to guide farmers to choose soil types, seasons for a crop and further to guide farmers about crop rotation.

Work proposed presents a diagnostic cyber physical assisted recommender system-based quality of soil contents and climatic conditions best possible crops suitable for that region. System proposed has been categorized in four verticals.

Monitoring of soil parameters related to health of crop.

Acquire climatic predictions for the region.

Mapping and classification crops with data acquired.

Periodical update of Control Unit algorithms via cloud support.

Flow diagram of the model proposed is depicted in Fig 1, model implements mentioned verticals employing wireless data transfer to avoid delay.



Parameters acquired via sensors will be subjected to calibration process and will get transmitted from the remote site to the centrally maintained control unit.

Climatic predictions are acquired from registered forecast sites along with previous patterns, for mapping favorable crops on region being considered.
 
Physical layer in any IOT system consists of sensors
 
Ph sensor (SKU 235871), temperature sensor (DHT-11), NPK sensor (DIY), moisture sensor (SKU DSM058) and rainfall sensor (YL 83), these sensors were put at the site identified at Safedabad, Lucknow at crop field of maize, crop gave us a small window of six months from June to November 2020 for the experiment. Sensors were interfaced with Arduino further with ESP-8266 module to ensure internet connectivity, as Arduino does not support internet connectivity. [3-7,20].

Data acquired is then uploaded on the cloud through internet connectivity for work proposed, Google drive was used, the data is then processed to remove outliers and spurious readings and eventually an excel sheet of this data is generated. Data assimilated on cloud is then downloaded to control unit, which is equipped with logic for mapping favorable conditions defined for different crops, data obtained over a period was mapped on the ideal conditions for a particular crop and suggestions for enhancing production or sowing a different crop will be provided. Mapping has been achieved via random forest algorithm on a python platform, the algorithm was trained on database obtained from Crop_recommendatio.csv from Kaggle, it had 2200 rows of data, depicted in Fig 2. Data generated from the experiment was put to testing from the trained random forest algorithm on a labeled database for cross validation. Fig 3 depicts the experimental setup, 3(a) depicts interfacing of temperature (DHT 11) and moisture sensor (SKU DSM058) with Arduino, 3(b) depicts interfacing of NPK sensor (DIY) with Arduino and 3(c) depicts sensor deployment. [8-10, 20, 21].

​

​

Training phase involved collection of data; data was accumulated for various cities of the country. The data had details about the rainfall patterns max. and min. temperature; humidity percentage and the kind of environmental conditions required for the heathy cultivation of various crops. Fig 4 depicts number of months required for cultivation of different crops; from the statistics it can be concluded more than 80% per cent of variety takes nearly 4 months for cultivation.

​

Fig 5 depicts a general statistic for minimal temperature required for cultivation of crops and Fig 6 depicts a general statistic for maximum temperature required for cultivation of crops. From the visualization of data it can be concluded that out of all the majority crops which are being grown in India about 35% of them require a minimum temperature of around 21 degrees and thus out of all the crops grown in India about 90% of them require a minimum temperature of ≤ 24 degree. Further out of all the crops being cultivated in India 42% of them can bear a maximum temperature of upto 33 degree Celsius and out of all the crops being cultivated in India 90% of them can bear a maximum temperature of ≤ 38 degree Celsius.

​

​

Fig 7 depicts a general statistic for ideal Ph value for cultivation of crops and Fig 8 depicts approximate level of Ph values in different states of India. From the visualization it can be concluded that out of the crops being grown in India 50% of them require a pH level of at least up to 6.0 and none of crops require a pH value greater than 10. Soil as standard is categorized as acid if pH value is less than 6, it is categorized as normal to saline if pH ranges between 6 and 8.5, as tending to alkaline if pH values range between 8.5 and 8.9 and as alkaline if pH value increases between 9.

​

​

All crops require nitrogen, phosphorus and potassium for healthy growth, these components supply necessary nutrients to the crops. Fig 9 depicts different standardized levels of nitrogen, phosphorus and potassium suggested for favorable environment.  Test for nitrogen level is generally recommended annually as levels are very much dependent on climatic conditions such as temperature pattern during growth, rainfall witnessed, category of crop sown, time of harvest, fall tillage, quantity of fertilizer put in. Phosphorous and potassium levels are almost constant in soil over the time of few years, consequently there sampling may be conducted every 3 to 3 years or when sowing different variety or crop. These values were employed for training the random forest model as discussed in work by Kumar et al., (2017), Kumar et al., (2018, October) and Verma et al., (2021).

​

Table 1 provides the range for minimum and maximum amount of rainfall required for different crops, rainfall is one of the most important factors which effects the crop production. Out of all the crop being cultivated in India almost 80% of them need a minimum rainfall of ≤ 960 mm and 98% of the crops grown in India need a maximum rainfall of about ≤ 2500 mm. These data were employed to train the random forest learning algorithm.



The model developed was trained to predict the crop farmer should sow in order to have maximum yield, the parameters that were considered included:
1. Maximum and minimum temperature recorded in the region under observation.
2. pH value and variations in the value in region under observation.
3. Maximum and minimum rainfall region under observation might witness.
4. Nitrogen, phosphorous and potassium level of the region under investigation.
5. Characteristics specific to location of region.             

Recommendation for the crop was achieved through random forest algorithm, a front end in the form of webpage was developed employing html to input values, The experiment prefers well-drained, light, deep, loose soil, high in organic matter. Unlike most vegetables, Potatoes have a high-water requirement-roughly 1 in/week during bulking. Thus, for high yields irrigation is usually beneficial, ideal range for rainfall will be between 500 to 700 mm, optimal growth for a healthy potato crop is at temperature which varies around 20 to 30^oC, Potatoes prefer soils with a pH of 5.5 to 7.0 and low salinity. However, in practice potatoes are grown in soil pH’s from 4.5 to 8.5 and this has a distinct impact on the availability of certain nutrients. At lower pH values potatoes can suffer from aluminum and other heavy metal ion toxicity, ideally Low nitrogen (<240 kg/ha) and high sucrose levels in the plant favor the formation of more tubers. The optimum dose for P and K varies from 60 to 100 kg /ha and 100 to150, kg ha, respectively, for plains. major potato rowing states are Himachal Pradesh, Punjab, Uttar Pradesh, Madhya Pradesh, Gujarat, Maharashtra, Karnataka, West Bengal, Bihar and Assam, the experiment was conducted in a location near safedabad, Lucknow. The sensors were installed and readings were collected for period from sowing to reaping of the crop, temperature and rainfall values were taken from registered weather sites and were even cross validated for the experiment period. Table 2 depicts the readings acquired from different sensors at the site T.S. Babalola (2021) and M. Roja (2021).



Heatmap depicted in Fig 10, presents a two dimension correlation matrix between the column and rows employing colored cells to represent relation between. Inter-dependence between two variables, also termed as correlation is estimated using Pearson correlation coefficient which measures how the value of two different variables vary with respect to each other. The correlation coefficient is determined by dividing the covariance by the product of the two variables’ standard deviations.

​

Pearson correlation coefficient between two variables X and Y can be calculated using the following formula:

 

Where,
r = Corelation coefficient.
xi = Values of x in a sample space.
x = Mean value of x.
yi = Values of y in a sample space.
y = Mean value of y.
Coefficient can take any values from -1 to 1.
• r =1, indicates positive correlation between two variables suggesting that when one variable increases, the other variable also increases.
• r = -1, indicates negative correlation between two variables, suggesting that when one variable increases, the other variable decreases.
• r = 0, indicates there is no correlation between two variables, suggesting that the variables change in a  random manner with respect to each other.

Model achieved an accuracy of 96% in the training module for the database acquired from Kaggle, model was cross validated for set of values acquired from experimental setup. Fig 11 depicts values of accuracy, precision, recall, f1-score and support for the complete set of databases for all the crops obtained from Kaggle.

​

Experiment was conducted in Lucknow, database was acquired for Maximum and minimum temperature recorded in the region which was between 15 to 34^oC at that period of time, pH values recorded was between 5.8 to 6.8, maximum rainfall region under observation was approximately 200 mm and nitrogen, phosphorous and potassium level of the region under investigation was 150 kg/ha, 60 kg/ha and 65 kg/ha respectively, with a little variation from day to day.

Setup was put up for a period from February to July and readings were acquired twice a day, the generated data was continuously updated on cloud from where it was downloaded and subjected the learning model developed, Values when fed to the model predicted Maize as an ideal crop, appeared as pop up. Cloud had an additional facility of updating comparison standards without disturbing the experimental setup. Classification was achieved through random forest algorithm trained on labeled dataset, that achieved an accuracy of 93% and the crop predicted was Maize. Fig 12 depicts the classification achieved.  

Food sector in India is set for huge growth owing to its immense potential for diversity and value addition as a manufacturer and supplier to the world, Agriculture forms the backbone of the sector. Food processing sectors contributes approximately 32% of total market share, supposedly one of biggest sectors and stands fifth in terms of production, consumption, export and expected growth. Farming as a sector is a vulnerable investment owing to dependency on diversified geographical locations, environmental conditions and pest attacks, it is of prime importance to devise technological assisted methods to monitor and provide early remedial actions for the damage and infections to the crop. Work proposed is an effort to increase the yield, a farmer would otherwise have; system implements the recommender system assisted by sensor data and weather conditions prevailing to suggest ideal crops to be sown for the region. Model acquires data: moisture content, rainfall estimated, NPK content of soil, temperature conditions and Ph value of the soil in addition to climatic conditions of the region for recommending the suitable crop. Model achieves an accuracy which is at par with existing solutions, further model has been cross validated with readings acquired from calibrated sensors at the site such as: Ph sensor (SKU 235871), temperature sensor (DHT-11), NPK sensor (DIY), moisture sensor (SKU DSM058) and rainfall sensor (YL 83), these sensors were put at the site identified at Safedabad, Lucknow at crop field of maize . Sensors were interfaced with Arduino further with ESP -8266 module to ensure internet connectivity, as Arduino does not support internet connectivity, climatic conditions were acquired from Government weather prediction platform, crop gave us a small window of six months from June to November to conduct the experiment. Classification was achieved through random forest algorithm trained on labeled dataset which achieved an accuracy of 96%, the testing data when fed to the model gave output with an accuracy of 93%. Classification results are promising and provide a strong base for implanting the system for enhancing the crop yield.

None