How to use LSTM with 1D, 2D and 3D Array?

 

Actually LSTM supports three-dimensional input. They are – (samples, time steps, features)

• Samples. One complete sequence is considered as one sample. A batch may contains one or more samples. In NLP, if we are dealing with the text at sentence level (means taking one sentence at a time), then our sample size will be one.
• Time Steps. It can be considered as the number of times, we feed to the model. In the above discussed one-sentence case of NLP, the number of words in the sentence will be equivalent to the time-steps.
• Features. It represents the number of columns of each sample. In the above one sentence case of NLP, we will obtain it through embedding. Generally, we use word2vec, Glove, etc. to get the word embeddings. Thus, in this case the dimension of the embedding will considered as feature size.

Now, we will try to understand, how to reshape 1D and 2D array to 3D Numpy array and apply LSTM on the top of that. We will also see, how LSTM works on 3D Numpy array.

# Using 1D array with LSTM

 

from keras import Model

from keras.layers import Input, Dense, Bidirectional

from keras.layers.recurrent import LSTM

import numpy as np

# define model for simple BI-LSTM + DNN based binary classifier

def define_model():

    input1 = Input(shape=(1,1)) #take the reshape last two values,

#see "data = np.reshape(data,(10,1,1))" which is

# "data/batch-size, row, column"

    lstm1 = Bidirectional(LSTM(units=32))(input1)

    dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1)

    dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1)

    model = Model(inputs=[input1],outputs=[dnn_output])

    # compile the model

    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

    model.summary()

    return model

# Take 1-D array data with 10 records/elements

data = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0])

Y = [1,1,1,1,1,0,0,0,0,0] #define the label for binary classification

print("data = ", data)

# Reshape the data to 3D-Numpy array

data = np.reshape(data,(10,1,1))

print("data after reshape => ",data)

# Call the model

model = define_model()

# Fit the model

model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1)

# Take a test data to test the working of the model

test_data = np.array([[0.29]])

# reshape the test data

test_data = np.reshape(test_data,(1,1,1))

# predict the sigmoid output [0,1] for the 'test_data'

pred = model.predict(test_data)

print("predicted sigmoid output => ",pred)

# Using 2D array with LSTM

 

from keras import Model

from keras.layers import Input, Dense, Bidirectional

from keras.layers.recurrent import LSTM

import numpy as np

# define model for simple BI-LSTM + DNN based binary classifier

def define_model():

    input1 = Input(shape=(2,1)) #take the reshape last two values,

# see "data = np.reshape(data,(10,2,1))" which is

# "data/batch-size, row, column"

    lstm1 = Bidirectional(LSTM(units=32))(input1)

    dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1)

    dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1)

    model = Model(inputs=[input1],outputs=[dnn_output])

    # compile the model

    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

    model.summary()

    return model

# Take a dummy 2D numpy array to train the model

data = np.array([[0.1, 0.15],

                 [0.2, 0.25],

                 [0.3, 0.35],

                 [0.4, 0.45],

                 [0.5, 0.55],

                 [0.6, 0.65],

                 [0.7, 0.75],

                 [0.8, 0.85],

                 [0.9, 0.95],

                 [1.0, 1.5]])

Y = [1,1,1,1,1,0,0,0,0,0] #Class label for the dummy data

print("data = ", data)

# Reshape the data into 3-D numpy array

data = np.reshape(data,(10,2,1)) #Here we have a total of 10 rows or records

print("data after reshape => ",data)

# Call the model

model = define_model()

# fit the model

model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1)

# Take a test data to test the working of the model

test_data = np.array([[0.2, 0.33]])

# reshape the test data

test_data = np.reshape(test_data,(1,2,1))

# predict the sigmoid output [0,1] for the 'test_data'

pred = model.predict(test_data)

print("predicted sigmoid output => ",pred)




# Using 3D array with LSTM

 

from keras import Model

from keras.layers import Input, Dense, Bidirectional

from keras.layers.recurrent import LSTM

import numpy as np

# define model for simple BI-LSTM + DNN based binary classifier

def define_model():

    input1 = Input(shape=(2,2)) #use row and column size as input size

    lstm1 = Bidirectional(LSTM(units=32))(input1)

    dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1)

    dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1)

    model = Model(inputs=[input1],outputs=[dnn_output])

    # compile the model

    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

    model.summary()

    return model

# Take a dummy 3D numpy array to train the model

data = np.array([[[0.1, 0.15], [0.2, 0.25]],

                 [[0.3, 0.35], [0.4, 0.45]],

                 [[0.5, 0.55], [0.6, 0.65]],

                 [[0.7, 0.75], [0.8, 0.85]],

                 [[0.9, 0.95], [1.0, 1.5]]])

Y = [1,1,1,0,0] #define binary class level for this model

print("data = ", data)

# NO NEED TO RESHAPE THE DATA as it is already in 3D format

# Call the model

model = define_model()

# Fit the model

model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1)

# Take a test data to test the working of the model

test_data = np.array([[[0.2, 0.33],[0.2, 0.33]]])

# predict the sigmoid output [0,1] for the 'test_data'

pred = model.predict(test_data)

print("predicted sigmoid output => ",pred)