End to End Amazon Product Sentiment analysis machine Learning model

Deploy a tensorflow based serverless sentiment analysis application using Google App Engine and Streamlit in just few simple steps.

Photo by Kevin Ku on Unsplash

Sentiment Analysis incorporates the fundamental aspects of opinion mining in text based Natural Language processing. Organizations find it useful in dealing with situations related to customer reviews, analysing how satisfactory was their product/services according to the customer. The feedback from users helps organizations to improve their services.

In this article, we are going to learn how to develop a serverless Sentiment analysis application deployed on Google App engine and UI demo running on Streamlit. If implemented successfully, we would be able to run our app in the following way from any part of the world.

Sentiment analysis UI Demo on Streamlit

Before you go through the article, it is important to note that there are Pre-trained Natural language processing API on Google cloud platform which you can use directly for your applications, instead of developing a custom model. It will simplify the process.

What we need to develop the application:

1. Google Colab will be used to build and train our model and generating our saved model files. The saved model can be directly uploaded to Google cloud Platform from colab notebooks.
2. Tensorflow is a widely used machine learning framework. From loading the dataset to doing text vectorization and finally building a sequential neural network, Tensorflow’s API is heavily used in our project.
3. Google AI Platform is used for hosting our trained model in cloud. The AI platform is an end to end platform that can be used to train your model, host trained model in the cloud and use it for making prediction on unseen data.
4. Google Container Registry for hosting our docker image.
5. Google App engine helps us deploy our Sentiment analysis application smoothly using our docker image and run in the cloud.

Model Training

The amazon product reviews dataset is a large dataset provided by amazon for academics and research purpose in the field of Machine learning, Natural language processing. In our application, we are going to use a subset of that dataset which only consists of reviews for Mobile_Electronics products. Tensorflow API provides a simple way to load the dataset directly in your notebook through tensorflow_datasets (tfds).

ds = tfds.load('amazon_us_reviews/Mobile_Electronics_v1_00', split='train', shuffle_files=True)
assert isinstance(ds, tf.data.Dataset)
print(ds)
df = tfds.as_dataframe(ds)

Preparing the dataset

Before we build our sequential model, we will implement the following steps in order to pre-process our dataset and keep only the required information.

• Create a new feature column from the star-rating column in the dataset to form our Label column.
• Keep only the review_body and Label column, remove the rest of the columns.
• Convert text into lower case, split it and remove stopwords.
• Create a new corpus from the new set of filtered sentences.

df["Sentiment"] = df["data/star_rating"].apply(lambda score: "positive" if score >= 3 else "negative")
df['Num_words_text'] = df['data/review_body'].apply(lambda x:len(str(x).split()))
df['Sentiment'] = df['Sentiment'].map({'positive':1, 'negative':0})
#Text Preprocessing
corpus = []
for i in range(0, 104975):
  review = re.sub('[^a-zA-Z]', ' ', df['short_review'][i])
  review = review.lower()
  review = review.split()
  ps = PorterStemmer()
  all_stopwords = stopwords.words('english')
  all_stopwords.remove('not')
  review = [ps.stem(word) for word in review if not word in   
  set(all_stopwords)]
  review = ' '.join(review)
  corpus.append(review)

Text Vectorization

To convert our text into numerical vectors, we are going to use Tokenizer class from Tensorflow. Each sentence will be converted into a sequence of numerical tokens assigned according to the vocabulary size and the frequency of a word occurring in the whole corpus. We will only apply the tokenizer to the training set and not the test set. This is to avoid our model learning from the unseen data too. Save the tokenizer object which we implemented on our text corpus as it is going to be later be used when our hosted model will be used for predicting sentiment on new customer review. The hyper parameters for our tokenizer can be played around with and see if you can get better results with them.

#Define the hyperparameters for tokenization
vocab_size = 20000
embedding_dim = 16
max_length = 100
trunc_type='post'
padding_type='post'
oov_tok = "<OOV>"
training_size = 70000
test_size = 95000
#Convert text into tokens
tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)
tokenizer.fit_on_texts(training_sentences)

word_index = tokenizer.word_index

training_sequences = tokenizer.texts_to_sequences(training_sentences)
training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)

val_sequences = tokenizer.texts_to_sequences(val_sentence)
val_padded = pad_sequences(val_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)

testing_sequences = tokenizer.texts_to_sequences(testing_sentences)
testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)
import io
import json
tokenizer_json = tokenizer.to_json()
with io.open('tokenizer.json', 'w', encoding='utf-8') as f:
    f.write(json.dumps(tokenizer_json, ensure_ascii=False))

Building Model

To build our model, we are going to use Embedding layer, followed with a bi-directional LSTM. I have not focussed much on fine tuning the model, but one can always try with different combination of Neural network layers to see if the accuracy of the model improves.

model = tf.keras.Sequential([
                             
    tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
    tf.keras.layers.Dropout(0.4),
    #tf.keras.layers.LSTM(embedding_dim,dropout=0.2, recurrent_dropout=0.2,return_sequences=True),
    #tf.keras.layers.Flatten(),                         
    #tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
    #tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64,  return_sequences=True)),
    #tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),
    tf.keras.layers.Dense(64, activation='relu'),
    #tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dropout(0.4),
    #tf.keras.layers.Dense(8, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.summary()
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, 100, 16)           480000    
_________________________________________________________________
dropout (Dropout)            (None, 100, 16)           0         
_________________________________________________________________
bidirectional (Bidirectional (None, 128)               41472     
_________________________________________________________________
dense (Dense)                (None, 64)                8256      
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 65        
=================================================================
Total params: 529,793
Trainable params: 529,793
Non-trainable params: 0
_________________________________________________________________

Our model performs fairly well with just 8 epochs giving us an accuracy of 93% on training set and 89% on validation set. Once we have built and trained our model, we can save it so it can be used for prediction of new values.

# Save model
model.save("amazon_review_classification_model2")

Deployment

The saved model can be directly uploaded to google cloud using gsutil commands. You’ll need to first authenticate your colab instance. Once your authentication is done, create a project in your google cloud console where we will create our bucket to store our model.

*Please note that this will incur charges on your account. If you have the free credits available when you created your cloud account, then you are good to go.

from google.colab import auth
auth.authenticate_user()
!curl https://sdk.cloud.google.com | bash 1> /dev/null
!gcloud init
# Copy model to bucket - https://cloud.google.com/storage/docs/uploading-objects#gsutil 
# Use "-r" for folders (r stands for recursive)
# replace with <your model name>  <your bucket name>
!gsutil cp -r amazon_review_classification gs://amazon_review_test

Great! We have successfully uploaded our saved model to the google cloud platform which can be used to predict sentiment on text sentences. The following steps can be either done manually or through gcloud command utility:

• Connect your stored model in the bucket to the AI Platform by first creating a new model in the platform. This will simplify the accessibility of our model through and API call.
• Once you have created a model, Create a version for this model and link it with your trained model in the google storage.
• A service account is needed to be created for security of our application. Create and download the JSON key which is needed to be provided in your Streamlit app file. Without the JSON key, our application will not work. This key grant access to your Cloud account. Keep the JSON key file private.

That’s it! We have set up our model and performed all the required steps to place our trained model in the GCP accurately. We can now deploy our app directly from our desktop to the Google App engine using Docker and make file. To deploy our complete application, run from the terminal in your project directory in a python virtual environment.

make gcloud-deploy

This command will deploy our app.yaml file and put our app in a docker container which is defined in our dockerfile. The following requirements are mentioned in our docker file depicting how we want our app to be deployed.

FROM python:3.7 
## App engine stuff
# Expose port you want your app on
EXPOSE 8080 
# Upgrade pip 
RUN pip install -U pip 
COPY requirements.txt app/requirements.txt
RUN pip install -r app/requirements.txt 
# Create a new directory for app (keep it in its own directory)
COPY . /app
WORKDIR app 
# RunENTRYPOINT [“streamlit”, “run”, “app_classification.py”, “ — server.port=8080”, “ — server.address=0.0.0.0”]

If the above process completes successfully, then our app will be deployed and hosted on Google App Engine and we can now run our app using the https link from any part of the world.

The complete code is available on my Github. If you have any questions, you can connect with me at Linkedin.

Thanks for reading.