In this blog, I will give you a step-by-step guide for building a share price prediction and forecasting model using Long Short Term Memory (LSTM).
Machine learning supervised learning is all about predicting the unlabeled data using the known labelled data. One of the best projects where we can apply our knowledge of supervised learning and LSTM is the Stock Price prediction model.
Disclaimer:The information in this blog is for informational purposes only and not financial advice
Abstract:
Here are the steps that we are going to do for building out the model from scratch,
Importing dataset
Modifying the data for our needs
Scaling the data
Train test split
Stacked LSTM model
Forecasting
1. Importing dataset:
The first step of building any machine learning model is to import the appropriate dataset. There are many websites for downloding the stock price dataset on the internet. I used Yahoo Finance to import the dataset as per my required timeline.
We can go to the Yahoo finance website and search for our desired company with its ticker/symbol. There we can see the historical data column, where we can download the CSV file of the stock price dataset.
The stock with which I have built the model is Microsoft Corporation (MSFT) and named it df.
#dataset @ https://finance.yahoo.com/quote/MSFT/history
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
df=pd.read_csv("/content/MSFT.csv")
df
2. Modifying dataset:
The downloaded CSV file will have many columns like date, open, close, volume, etc. But for simplicity, we can only consider the Close column and make a new dataset out of that Close column namely df1.
df1=df[['Close']]
3.Scaling the data:
We need to scale out the dataset because more deviation in the data makes it hard for the model to make the predictions accurately. So we are scaling the range of the data between 0 and 1. MinMaxScaler is a module of the preprocessing which in turn is present in the scikit-learn library.
from sklearn.preprocessing import MinMaxScaler
scaler=MinMaxScaler(feature_range=(0,1))
df1=scaler.fit_transform(df1)
df1.shape
4.Train test split:
Splitting the dataset into training and test data is a good practice when training a model. For linear regression models, such as those used for GDP calculation, per capita income prediction, and others, you can perform this split using sklearn.model_selection.train_test_split(). This function will randomly partition the data, although this approach is not suitable for time series data.
However, when dealing with stock price prediction models, random splitting is not possible and, in fact, it is illogical. This is because today's share price often depends on the previous day's share price, creating a temporal relationship that cannot be ignored.
Firstly we want to find the length of the length of the training and testing data. Then we can split the dataset accordingly.
train_size=int(len(df1)*0.8)
test_size=int(len(df1)-train_size)
train_data,test_data=df1[0:train_size,:],df1[train_size:len(df1),:1]
train_data,test_data
Now we can split the train and test data for training our model. Let's define a generic function that will take the dataset and time steps(numbers of the previous day's stock price need to be considered for predicting the current day's price) as parameters and will return the x_train and y_train array.
The function basically splits the data into x_train and y_train i.e. the previous day's stock price and the current day's stock price respectively. The obtained y_train of the previous set will act as an x_train now. This will continue until the time_steps.
def create_dataset(dataset, time_step=1):
dataX, dataY = [], []
for i in range(len(dataset)-time_step-1):
a = dataset[i:(i+time_step), 0]
dataX.append(a)
dataY.append(dataset[i + time_step, 0])
return np.array(dataX), np.array(dataY)
time_step = 100
x_train, y_train = create_dataset(train_data,time_step)
x_test, y_test = create_dataset(test_data, time_step)
After passing the train and test data in the function, we have obtained x_train,y_train,x_test and y_test.
5.Stacked LSTM model:
First of all, when working with deep learning models we need to make our dataset 3D since it accepts only 3D arrays.
x_train=x_train.reshape(x_train.shape[0],x_train.shape[1],1)
x_test=x_test.reshape(x_test.shape[0],x_test.shape[1],1)
Now we can import the necessary libraries.
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
Let's prepare our sequential LSTM model.
model=Sequential()
model.add(LSTM(50,return_sequences=True,input_shape=(100,1)))
model.add(LSTM(50,return_sequences=True))
model.add(LSTM(50))
model.add(Dense(1))
model.compile(loss='mean_squared_error')
Now it's time to train our model with the x_train and y_train. For that we want to use fit() function.
model.fit(x_train,y_train)
Now, let's predict and descale the x_train and x_test for plotting using predict() and scaler.inverse_transform() respectively.
train_predict=model.predict(x_train)
test_predict=model.predict(x_test)
train_predict=scaler.inverse_transform(train_predict)
test_predict=scaler.inverse_transform(test_predict)
Then we can plot the obtained predicted value in the actual graph of the stock price. So that we can see our model's accuracy.
look_back=100
train_predict_plot=np.empty_like(df1)
train_predict_plot[:,:]=np.nan
train_predict_plot[look_back:len(train_predict)+look_back,:]=train_predict
test_predict_plot=np.empty_like(df1)
test_predict_plot[:,:]=np.nan
test_predict_plot[len(train_predict)+(look_back*2)+1:len(df1)-1, :]=test_predict
plt.plot(scaler.inverse_transform(df1))
plt.plot(train_predict_plot)
plt.plot(test_predict_plot)
plt.show()
6.Forecasting:
With our trained LSTM stock price prediction model we can use it to predict the stock trend of the company for n number of days.
For this the x_input is the x_test and using it as the time steps data for the upcoming days. For better visualization, we can plot the obtained forecasting data in the actual graph.
Credits: