Predicting Wine Reviewer Name using Wine Reviews...? Yes!!

Introduction to my Thought Process

The question in its essence is, are Wine Reviews written over time sufficient enough to 'unconsciously' know who was it that wrote the Review?

As a start, I wanted to restrict my temptation to consider other factors such as Country, Region, points, etc; as they would give us more insight on how each Reviewer belonging to a specific region affects their availability to a Wine in order to write a Review or how he/she evaluates a Wine.

This word, 'evaluates' now begs the question, are Wine Reviews enriched enough to help us know which Reviewer they have been written by?

I bilieve so, since it's the journey (Explaination) that tells a lot more than the destination (Quantified Evaluation).

A little bit of background about myself, I am definietly not a Wine-taster, all I have wondered about Alcohol is, does it taste good and gets me the buzz? What I am trying to say is, I do not really Know what is it that a Wine-Reviwer looks for in a Wine.

• As a being, outside the domain of a Subject Matter Expert in Wine, I would begin by looking out for the words Reviewers use to describe a Wine.
  • Which I did so, using the Word Clouds
• As I went through the WordClouds, which generate high frequency words appeared in the document (Reviews), I observed that certain users do like to use lot of Stop_Words, 'with' being an Interesting one.
  • This showed me a glimpse of what a Reviewer looks for,
    • Culmination of tastes and smells of Wine
• Next, I removed the Stop_Words to understand the nature with which a Reviewer approaches the tasting of Wine.
  • For example, some looked for the Aromas more than the Palate taste and vice-versa

These observations as I will talk about as we go through the code, led me to believe that Reviews contained sufficient information to determine who the Reviewer was.

What next?

• Now that the exploration of Reviews led me to believe (for lack of a better word) that we do can use the Reviews to figure of who the Reviewer was, I chose to stick to Machine Learning algorithms.
  • Only stuck to ML algorithms as the data-set was just in 100,000s and ML- algorithms are sufficiently powerful enough to predict the outcome for not so large dataset, as we will see.
• My first Intuition was to use Logistic Regression, with One-vs-Rest Classification since, after we do the Tf-IDF of the reviewes, the words can be considered as dimensions and the Reviewer as the data point in that Manifold created by them dimensions.
• Then after testing with other ML-algorithms such as Random Forests and K-NN, It hit me that SVMs would do a great job of getting to the global-optimum because of their nature of Dual-Optimization of a Paraboloid, which it turns out absolutely does.
• Apart from all the other analysis and prediction, one of the important factors was the imbalance of Classes(Reviewers), hence, used SMOTE analysis.

Loading up the Dataset

import numpy as np
import pandas as pd
import nltk
import re
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.svm import LinearSVC
from sklearn.metrics import classification_report
from imblearn.over_sampling import SMOTE
nltk.download('punkt')
nltk.download('stopwords')

PATH = "C:\\Users\\Rohit\\Deep Learning\\" 

df_raw= pd.read_csv(f'{PATH}winemag-data-130k-v2.csv', low_memory=False)

# generating word frequencies
def gen_freq(text):
    # list of words
    word_list = [] 
    
    # loop over all the text docs and extract words into word_list

    for sentence in text:
        for words in sentence.split():
            word_list.append(words)
    
    # create word freuencies using word_list
    word_freq = pd.Series(word_list).value_counts()
    
    return word_freq

stop_words = set(stopwords.words('english'))
'With' in stop_words

Word Cloud of Each Reviewer - with Stop_Words

for i in df_raw['taster_name'].dropna().unique():
    newb=df_raw[['description','taster_name']][df_raw['taster_name']==i]
    word_freq_view = gen_freq(newb['description'].tolist())
    wc_normal = WordCloud(width = 400, height = 330, max_words = 50, background_color = 'white', stopwords = stop_words).generate_from_frequencies(word_freq_view)
    print(newb['taster_name'].unique())
    plt.figure(figsize = (14, 10))
    plt.imshow(wc_normal, interpolation = 'bilinear')
    plt.axis('off')
    plt.show()

Oberservation:

What I saw in those Word Clouds (with Stop_words) is that some Reviewers look for multiple flavours as well as aromas and clumination of them two.

• Which led to words such as 'and','with', as well as, 'this' to have high frequency.

Word Cloud of Each Reviewer without Stop_Words

for i in df_raw['taster_name'].dropna().unique():
    newb=df_raw[['description','taster_name']][df_raw['taster_name']==i]
    newb['description'] = newb['description'].apply(lambda x: ' '.join([w for w in nltk.word_tokenize(x) if w.isalpha() == True and len(w)>3 and not w in stop_words]))
    word_freq_view = gen_freq(newb['description'].tolist())
    wc_normal = WordCloud(width = 400, height = 330, max_words = 50, background_color = 'white', stopwords = stop_words).generate_from_frequencies(word_freq_view)
    print(newb['taster_name'].unique())
    plt.figure(figsize = (14, 10))
    plt.imshow(wc_normal, interpolation = 'bilinear')
    plt.axis('off')
    plt.show()

Observation:

After removing the Stop-Words, the Word Clouds helped me understand what a Reviewer looks for first and as a whole while tasting the wine.

• The words such as 'Palate' and 'Aroma' having high frequency implies that a Reviewer is looking for taste and smell respectively.

Also, certain words such as 'acidity', 'citrus' and 'elegant' do talk about their primary sense of consumption, what does drinking this wine make them 'feel.

SO yes, hence, The Reviews do contain the information that relays what a Reviewer looks for first and how one consumes the Wine and what does the Wine make them feel. I do think We can do the Prediction of A Reviewer based on Reviews.

Now Let's look at the Number of Reviews each Reviewer has written.

ds = pd.Series(df_raw['taster_name'])
ax=sns.countplot(df_raw['taster_name'],data=ds)
ax.set_xticklabels(ax.get_xticklabels(), rotation=40, ha="right")
plt.tight_layout()
plt.show()

After normalizing the counts of Reviews by a Reviewer

df_raw['taster_name'].value_counts(normalize=True).plot(kind='bar')

df_raw_2 = df_raw.groupby(['taster_name']).size().reset_index(name='count')

df_raw_2

Obersvation:

• Wow, the Reviews of Fiona Adams and Christina Picarkd almost surely dissapear. This could be a problem to do justice to them.
• Cristina only has written 6 reviews, either that's the truth or the NaNs might have her information (might not be case as when I did the prediction with Nans as Unknown_Reviewers they did seem to be classified with quite good recall.), either way, for now I chose not to focus on her.
• Fiona on the other hand can be saved to an extent using SMOTE analysis.

The Prediction

-Machine Learning

Reviewers = df_raw['taster_name'].fillna('Unknown_Reviewer')

Reviews = df_raw['description'].dropna()

Reviewers.shape

Splitting the Train and Test sets, with 80-20.

X_train, X_test, Y_train, Y_test = train_test_split(Reviews,Reviewers,stratify=Reviewers,test_size=0.2,random_state=100)

tv = TfidfVectorizer(min_df = 0., max_df = 1., norm = 'l2', use_idf = True, smooth_idf = True)

train_tfidf = tv.fit_transform(X_train)
test_tfidf = tv.transform(X_test)

print(train_tfidf.shape)
print(test_tfidf.shape)

Logistic Regression.

My primary intuition to start with this was because we could consider the high frequent words to be the dimensions of the Manifold the documents create and the Reviewers to be the data points in that Manifold.

And 'curve fitting' would do a tremendous job. As seen, it definietly did.

Altough it didn't have high F1-Score for Reviewers with lesser Reviewers than 100.

log_model = LogisticRegression(penalty = 'l2', solver = 'lbfgs', multi_class = 'ovr', max_iter = 1000, C = 1, random_state = 100)
log_model.fit(train_tfidf, Y_train)

log_predictions = log_model.predict(test_tfidf)

print(classification_report(Y_test,log_predictions))

SMOTE

SMOTE first selects a minority class instance a at random and finds its k nearest minority class neighbors. The synthetic instance is then created by choosing one of the k nearest neighbors b at random and connecting a and b to form a line segment in the feature space. The synthetic instances are generated as a convex combination of the two chosen instances a and b.

I chose 4 k nearest neighbhors since Fiona Adams, when sampled during the split only had 5 representations of her Reviews.

sm = SMOTE(k_neighbors=4)
X_sm, Y_sm = sm.fit_resample( train_tfidf, Y_train)

log_model = LogisticRegression(penalty = 'l2', solver = 'lbfgs', multi_class = 'ovr', max_iter = 1000, C = 1, random_state = 100)
log_model.fit(X_sm, Y_sm)

log_sm_predictions = log_model.predict(test_tfidf)
print(classification_report(Y_test,log_sm_predictions))

Observation:

• With the SMOTE analysis Fiona Adams did get some justice, Precision as expected was high, Recall was low mostly because of the less number of her Rewviews written
• The Precision and Recall did increase although Precision by not so much, still the Recall's case is quite significant I would say

Without Stop Words

Reviews_No_Stop_Words=df_raw['description'].dropna().apply(lambda x: ' '.join([w for w in nltk.word_tokenize(x) if w.isalpha() == True and len(w)>3 and not w in stop_words]))

X_train, X_test, Y_train, Y_test = train_test_split(Reviews_No_Stop_Words,Reviewers,stratify=Reviewers,test_size=0.2,random_state=100)

tv = TfidfVectorizer(min_df = 0., max_df = 1., norm = 'l2', use_idf = True, smooth_idf = True)

train_tfidf = tv.fit_transform(X_train)
test_tfidf = tv.transform(X_test)

print(train_tfidf.shape)
print(test_tfidf.shape)

SMOTE Analysis

Chose to stick with SMOTE analysis in order to Include Fiona Adams.

sm = SMOTE(k_neighbors=4)
X_sm, Y_sm = sm.fit_resample( train_tfidf, Y_train)

log_model = LogisticRegression(penalty = 'l2', solver = 'lbfgs', multi_class = 'ovr', max_iter = 1000, C = 1, random_state = 100)
log_model.fit(X_sm, Y_sm)

log_sm_predictions = log_model.predict(test_tfidf)
print(classification_report(Y_test,log_sm_predictions))

Observations:

• Not much of a difference on average, but definietly less without Stop_Words.
• The Precisions of Reviewers definietly did drop especially the ones with less number of Reviews.
  • I suspect that stop_words usuage might be related to the slang of a Reviewer

  Different solver - LibLinear

  It uses an estimation to the inverse Hessian Matrix, which is approximated, what is means is that it saves a lot of memory, although it might have the risk of not converging to anything.

  I believe although the drawback exists, which the dataset being small it surely would converge. Which it did.

  log_model_2 = LogisticRegression(penalty = 'l2', solver = 'liblinear', multi_class = 'ovr', max_iter = 1000, C = 1, random_state = 100)
  log_model_2.fit(X_sm, Y_sm)

  log_sm_predictions = log_model_2.predict(test_tfidf)
  print(classification_report(Y_test,log_sm_predictions))

  Different Solver - SAG

  This SAG-Stochstic Average Gradient, incorporates a memory of previous gradient values, leading to a much faster convergence rate.
  

  log_model_3 = LogisticRegression(penalty = 'l2', solver = 'sag', multi_class = 'ovr', max_iter = 1000, C = 1, random_state = 100)
  log_model_3.fit(X_sm, Y_sm)

  log_sm_predictions = log_model_3.predict(test_tfidf)
  print(classification_report(Y_test,log_sm_predictions))

  Observation:

  • Logistic Regression Classification is a significant hit and friendly towards the Reviewers with high number of Reviews as expected.
  • Although, LRC has a good recall for minor classes too, so the wrong classification of Reviews of major class to the minor ones, will not happen so frequently.
  • This Is a sufficiently good model with no over fitting.

  What about Predicting a Review?

  newb=df_raw[['description','taster_name']][df_raw['taster_name']=='nan']

  test_data=df_raw[['description']][df_raw['taster_name'].isnull()]

  test_one = [test_data.at[33,'description']]

  test_data_see=test_data['description'].apply(lambda x: ' '.join([w for w in nltk.word_tokenize(x) if w.isalpha() == True and len(w)>3 and not w in stop_words]))

  test_data_see.at[33]

  test_one_review = tv.transform([test_data_see.at[33]])

  log_model_2.predict(test_one_review)

  Observation

  Works good, since Unknown_Reviewer the replacement of NaNs is as labeled, Unknown and mysterious

  I suspect this is so because they Reviews were taken from a Reviewer with an account at some place with no details.

  And, Unknown_Reviewer having quite a High Recall and Precision does say it's the same Reviewer, quite weird to be honest.

  Testing out Bunch of Different ML - Algorithms

  Random Forest

  Because limitation of my Laptop and my Time constraints I didn't use deeper trees. Although, yes, I did test this on Colab, but still it didn't generate a better performance than Logistic Regression
  
  But, yes the Minor Classes's Precision and Recall were a tad bit better than the Logistic. 
  
  I used Gini, I suspect had I used Information Grain although the time taken would be high ( only CPU was available during this), the precisson and Recall would be better.
  

  from sklearn.ensemble import RandomForestClassifier
  model = RandomForestClassifier(n_estimators = 100,criterion = 'gini', max_depth=15)
  model.fit(X_sm, Y_sm)

  Y_predicted = model.predict(test_tfidf)
  
  print(classification_report(Y_test,Y_predicted))

  K-NN

  I used the Cosine Similarity suspecting that the similarity based on angles between data points (Reviewer) over the manifold of dimensions of Words would be a better measure.
  

  from sklearn.neighbors import KNeighborsClassifier
  
  model_2 = KNeighborsClassifier(n_neighbors=30,metric='cosine')
  model_2.fit(X_sm,Y_sm)

  Y_predicted = model_2.predict(test_tfidf)
  
  print(classification_report(Y_test,Y_predicted))

  SVM - The Magician

  I always call SVMs the magicians because the very core of construction of this algorithm is such that the bottom line of a Loss function is a Dual Optimization problem of Paraboloid which has a high probability of going to the Global Optimum, since we only consider the dot product of Two points over the manifold for the construction of the hyperplane.
  

  fo

  model_3 = LinearSVC(multi_class='ovr',dual=True)
  model_3.fit(X_sm,Y_sm)

  Y_predicted = model_3.predict(test_tfidf)
  
  print(classification_report(Y_test,Y_predicted))

  Observation:

  • As assumed, SVMs do a tremendous job in classifying the Reviewers based on Reviews, in less time surely than the Logistic Regression.
  • Even the Reviewers with less number of Reviewes than 100, did perform better, unfortunately for Fiona and Carrie it wasn't the case
    • Carrie's precision was significantly better than logistic regression.

  End of Day...?

  I would say I would go with SVMs.

  Try to get more Reviews written by Christina, Fiona and Carrie. Didn't want to Impute the data more with the worry that I would just be ending up generating my data and won't be the true distribution of data_set.

  I wouldn't recommend going to Deep Learning, since ML algorithms were pretty good models.

  I would further might also try to increase the weightage of the minor classes (Reviewers with less Reviews)