Crop Yield Prediction using Deep Learning Algorithm based on CNN-LSTM with Attention Layer and Skip Connection

Indian agriculture boasts a rich history, beginning with the Indus Valley Civilization (2600-1600 BCE), which cultivated rice, wheat, barleyand cotton using irrigation systems (Bheemabai, 2017). During the Mughal Empire (1526-1857), new crops like tobacco and potatoes were introduced, alongside advancements in farming techniques such as crop rotation (selfstudyhistory) (Batra et al., 2021). Colonial British policies focused on exporting raw materials, leading to decreased food production and famine in the 19^th century (Shobanadevi et al., 2023). The Green Revolution of the 1960s and 1970s introduced high-yield crops and modern farming methods, significantly boosting food production (Howlett, 2008).
        Despite its historical importance, Indian agriculture faces challenges today, particularly from climate change. Erratic rainfall, droughts and floods are reducing crop yields and productivity (Shook et al., 2021, Dwivedi et al., 2022). Dependence on monsoon rains makes agriculture vulnerable to weather variations, leading to crop failures and reduced farmer income (Mahdi et al., 2020). Rising temperatures are degrading soil quality and affecting crop growth (Elavarasan et al., 2020). While other countries adopt irrigation and modern technologies to mitigate climate impacts, many Indian farmers still use traditional methods, struggling to adapt (Majeed et al., 2021; Durai and Shamili, 2022).
        Machine learning, a subset of artificial intelligence, enables computers to identify patterns and insights from data without explicit programming, improving over time with experience (Keerthana et al., 2021). Deep learning, a subset of machine learning, trains neural networks to understand complex data representations and is used for tasks like image recognition, recommender system, natural language processingand predictive analytics (Shen et al., 2017; Nevavuori et al., 2019; Baek et al., 2020). Recent advances include feature engineering-based LSTM models that enhance crop harvest forecasting by creating new features from existing data (Iniyan et al., 2023).

Related works
 
Jhajharia and Mathur (2022)  conducted a research work in Rajasthan, India, implementing various machine learning techniques to estimate crop yield on five identified crops. The study discovered that the Random Forest, SVM and Lasso Regression models performed better in predicting agricultural yield than deep learning models such as Gradient Descent and LSTM. However, the study suggested that a larger dataset and further investigation into soil and rainfall are required for practical applications of prediction models in crop production Panigrahi et al., (2023) formulated a forecasting model for the Indian state of Telangana from 2016 to 2018 for Bengal gram, groundnutsand maize. It utilized six supervised regression models: gradient boosting regression, random forest regression, linear regression, decision tree regression, XGBoost regression and voting regression. The research found that the XGBoost Regression and Random Forest Regression models were the most precise. A two-step approach to enhance agricultural yield, involving forecasting seasonal rainfall using modular artificial neural networks (MANNs), followed by using the rainfall data and crop-specific land area to forecast the yield of major kharif crops with support vector regression (SVR) (Khosla et al., 2020).
       
In a study by Gopal and Bhargavi (2019), a hybrid MLR-ANN model was developed that utilized the coefficients and bias from a multiple linear regression (MLR) model to initialize the weights and bias in the input layer of the artificial neural network (ANN) model, in place of random weights and bias. This approach improved the accuracy of the model over traditional methods. Nigam et al., (2019) investigated various machine-learning algorithms for forecasting crop yield based on variables such as temperature, rainfall, seasonand area. Simple RNN and LSTM were used to predict temperature and precipitation initially. Keerthana et al., (2021) discussed various machine learning algorithms, including AdaBoost regressor, Random Forest, Gradient Boosting, Decision Treesand KNN classifiers. The study found that an ensemble model consisting of Decision Tree Regressor and AdaBoost Regressor produced the most precise outcomes. Khaki et al., (2020) suggested a novel approach that merges CNNs and RNNs. CNN captures both the spatial relationships among soil data gathered at various depths and the temporal dependencies in meteorological data, while RNN represents the rising trend in crop production over time due to ongoing advancements in plant breeding and management techniques. Sivanantham et al., (2022) developed a new method called QRECF-DFFMPC to improve prediction accuracy while minimizing time consumption. This approach comprises an input layer, hidden layersand an output layer. The empirical orthogonal function in hidden layer 1 is used to select appropriate features. Quantile regression is then applied in hidden layer 2 to evaluate the features and produce the regression result for each data point. Satpathi et al., (2023) conducted a comparative analysis for Chhattisgarh using ANN, LASSO, ELNETand ridge regression with 21 years of historical rice data from three districts: Raipur, Surgujaand Bastar. The study found that ANN performed better with Raipur and Surguja data, while ELNET performed better with Bastar data. Additionally, different ensemble models were used, with performance being comparable for Raipur and Surguja, while Bastar performed better with Random Forest.

Dataset
 
The crop prediction model is trained on a dataset featuring information on rice and wheat, including soil, climate conditionsand yield potential (Fig 1). It also incorporates historical weather and soil data from various regions. (Kaggle, 2024)  this dataset, which covers nearly all Indian states, is essential for training and assessing the model’s accuracy in predicting crop yields based on current conditions.


 
Theoretical background
 
CNN: Convolutional Neural Networks (CNNs) are a class of deep neural networks widely used for image and signal processing tasks, such as image classification, object detectionand speech recognition (Kiranyaz et al., 2021). A 1D CNN, specifically, is designed to process one-dimensional signals like audio or time series data. It can be viewed as a special case of 2D CNNs, where the input is a one-dimensional sequenceand the filters are one-dimensional vectors (Srivastava et al., 2022). The main advantage of 1D CNNs is their ability to learn local features from input signals, such as sound wave shapes or time series patterns, making them effective for tasks like speech recognition or anomaly detection. The core of CNNs is the convolution operation, a mathematical process where a filter slides over an input signal to produce a set of feature maps, crucial for extracting relevant features. To analyse the proposed crop yield prediction, we used the CNN architecture as shown in Fig 2.

    ....(1)   

 
x = Input image.
w = Filter.
b = Bias term.
 
Activation function
 
After each convolution operation, an activation function is applied to the output of the layer. The network may learn more complicated patterns in the input thanks to the activation function, a non-linear function that incorporates non-linearity into the network. Some commonly used activation functions in CNNs include ReLU, sigmoid and tanh.
 

y = f (x)....(2)       

 
x= Input to the activation function.
 
Pooling operation
 
Pooling is a down sampling operation that reduces the spatial dimensions of the feature maps while preserving their important features. The two most commonly used pooling operations in CNNs are max pooling and average pooling.
 

 ....(3)     

x = Input feature map.
y = Output of the pooling operation.
s = Stride (the distance between adjacent pooling regions).
m and n= Indices of the pooling region.
 
Fully connected layer
 
Each neuron in it is a conventional neural network is linked to every other neuron in the layer above it. A matrix multiplication operation is used to calculate the result of a fully connected layer, which continues by a bias term and an activation function.
 

y = f (w_x + b)....(4)
 

x = Input.
W = Weight matrix.
b = Bias term.
 
LSTM
 
It is a form of RNN (Saini and Nagpal, 2022) which incorporates gating mechanisms to selectively retain or discard information from previous time steps. This makes it well-suited for processing long sequences of data and modelling long-term dependencies. The LSTM cell contains a forget gate, an input gate, a candidate activation, a cell state update, an output gateand a hidden state output as shown in Fig 3. The forget gate in an LSTM network decides the extent to which the prior cell state should be preserved, while the input gate regulates the amount of the candidate activation that must be appended in the cell state. The output gate determines the amount of the cell state that should be emitted as the hidden state. The equations for an LSTM cell can be written as:


 

 

 
it → Input gate.
ft → Forget gate.  
Ot → Output gate.
ht → Hidden gate.
Ct → Cell state at timestamp (t).
Ct → Candidate for cell state at timestamp (t).
 
Attention layer
 
An attention layer in neural networks selectively focuses on relevant parts of the input data by computing a weight vector based on the similarity between a query vector and key vectors (Ahmad et al., 2018). This weight vector is used to compute a weighted sum of value vectors, producing the output of the attention layer. Multi-head attention allows the model to capture multiple aspects of the input data (Filippi et al., 2019), commonly used in natural language processing and computer vision.
 
Skip connection
 
Skip connections, also known as residual connections, bypass one or more layers in a neural network to mitigate vanishing gradient problems, making it easier for the network to learn complex representations (Zhang et al., 2020).
 
Addition-based skip connection
 
In this method, the output of a layer is added to the output of a previous layer, preserving original information and aiding in gradient flow during training.
 
Multiplication-based skip connection
 
This approach multiplies the output of a layer elementwise with the output of a previous layer, selectively emphasizing important features. Both methods of skip connections improve deep neural network performance by enabling the simultaneous learning of shallow and deep features.
 
Proposed model
 
A crop yield prediction model that incorporates a 1D CNN, LSTMand attention layer is depicted in Fig 4. The LSTM records temporal patterns, the CNN extracts features from environmental inputand the attention layer improves accuracy by concentrating on important details. By combining LSTM and attention outputs, skip connections enhance pattern recognition.


       
First, we reshape the input data to have the shape (n,t,m), where n is the no. of samples, t is the no. of timesteps and m is the no. of features.
Then, we define the model architecture as follows:
Input layer: X ∊ R^nxtxm
CNN layer: C = CNN (X) ∊ R^nxtxc_units
LSTM layer: (q) = LSTM (C) ∊ R^nxa_units
Attention layer: (a) Attention (q) ∊ R^nxa_units
Skip Connection: q_units * C_units
Flatten layer: (r) = Dense(S) ∊ R^nxtnr_units
Output layer: (y) = Dense(r) ∊ R^nx1
Where,
CNN: Convolutional Neural Network layer with h_units units.
LSTM: Long Short-Term Memory layer with q_units units.
Attention: Attention layer that takes as input the LSTM layers and outputs an attention vector of a_units dimensions.
Dense: Fully connected layer with r_units units.
       
The model is trained to minimize the mean squared error loss function:
       

 ....(11)

 
with 500 epochs using the Adam optimizer with a batch size of 32.
Algorithm 1: CNN-LSTM with Attention.
1: Preprocess and reshape (X, Y).
2: Input: Train_X, Train_Y.
3: Hyper-Parameters: optimizer, rate, pool size, batch size.
4: Initialize ().
5: Model=sequential ().
6: Convolution (filters, kernel size, activation).
7: LSTM (units, activation).
8: Attention (weights, activation)
9: Add skip connections:
9.1: LSTM_Skip = LSTM (units, activation).
9.2: Attention_Skip = Attention (weights, activation).
10: Model. Compile (Train_X, Train_Y, epochs, batch size).

The dataset we used for this study had 12 characteristics and was filtered for rice and wheat, yielding about 5000 rows. Both training and testing datasets were separatedand the CNN layer was in charge of feature selection. Metrics such as the correlation coefficient, R² score, RMSEand MAE were used to assess the performance of the various models that were tested. The CNN-LSTM model was modifiedand the best attention layer was chosen to be combined with skip connections after it was compared to other attention layers. Table 1 demonstrates that the CNN-LSTM with skip connections and multi-head attention performed better than the other models.


       
Also, we have implemented other machine-learning algorithms like decision tree regressor, random forest regressor and support vector regressor using Jupiter. Since, this is a regression model we don’t have accuracy as an evaluation metric therefore for accuracy we have multiplied the correlation coefficient by 100.
       

 ....(12)      
 

 
....(13)

 
n = No. of observations.
yi = Actual value.
yi= Predicted value.
       

 

 
SS_res= Sum of squared residuals.
SS_tot= Total sum of squares.
       
In the Table 2, we evaluated the performance of different models and compared their accuracy. Our modified model CNN-LSTM multi-head attention with multiplication skip connection outperformed other models with 98% accuracy.


       
Root mean square error (RMSE) measures the difference between actual and predicted values, with a lower RMSE indicating higher model accuracy. In Fig 5, the support vector regressor has the highest RMSE, while the CNN-LSTM multi-head attention with multiplication skip connection has the lowest, showing minimal error. Mean Absolute Error (MAE) similarly gauges prediction accuracy, with the CNN-LSTM model having the smallest MAE in Fig 6. The R² score, ranging from 0 to 1, indicates how well the model explains the variance in the target variable. The CNN-LSTM model in Fig 7 achieves the highest R² score of 0.959, reflecting excellent performance.






       
In Fig 8, we can see the line graph of actual data and predicted data. We have taken 100 data points for the comparison. The value in the fig is between 0 to 1 because scaling was performed in the original data. In the graph, the predicted data at some points like 42,49 and 60 have some high spike than the actual data which means still the model is not 100% accurate.


       
In Fig 9, the diagonals are all 1/dark red, representing a perfect correlation since each variable is compared to itself. As correlations strengthen between different variables, the numbers increaseand the colors darken accordingly.


       
A scatter plot visually represents the relationship between two variables by plotting data points on a two-dimensional graph. A trend line (diagonal) indicates the strength of the relationship: tightly clustered points suggest a strong relationship, while scattered points indicate a weaker one. Fig 10 show that data points in the Hybrid 1D CNN-LSTM with attention layer have points more closely grouped, though outliers are present.


       
We trained the Hybrid 1D CNN-LSTM model for 200 epochs using the MSE loss function and Adam optimizer. As shown in Fig 11, the training loss drops quickly during the first 100 epochs and then continues to decrease more gradually. The validation loss follows a similar pattern. The loss function’s downward trend indicates that the model improves over time, but it hasn’t yet reached its optimal performance, as the loss is still above zero.

We developed a modified CNN-LSTM model for predicting crop yields, specifically for rice and wheat, using a dataset containing crop, soiland climate information. We trained and evaluated several models, including decision tree regressor, random forest regressor, support vector regressor and various CNN-LSTM variations (e.g., CNN-LSTM with multi-head attention and different skip connections). Among these, the CNN-LSTM multi-head attention with multiplication skip connection outperformed the others based on metrics like RMSE, MAE, R² score and correlation coefficient. Although the results were promising, the study’s small dataset size is a limitation. Future work with larger datasets may yield even better results. Table 1 highlights that the top two models-Hybrid 1D CNN-LSTM with attention and Random Forest-could potentially deliver higher performance if combined in an ensemble

The authors did not receive support from any organization for the submitted work. No funding was received to assist with the preparation of this manuscript. No funding was received for conducting this study. No funds, grants, or other support was received.

The authors have no relevant financial or non-financial interests to disclose. The authors have no conflicts of interest to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.