sentiment_analysis_tutorial

ipynb24 kBPython · 41 cellsExtra material of chapter 5

Sentiment Analysis Tutorial

This tutorial covers both rule-based and machine learning approaches to sentiment analysis.

Table of Contents

  1. Rule-Based Approaches
    • VADER
    • TextBlob
  2. Machine Learning Approaches
    • Naive Bayes with Count Vectorizer
    • Naive Bayes with TF-IDF Vectorizer
    • Logistic Regression with Count Vectorizer
    • Logistic Regression with TF-IDF Vectorizer
  3. Model Comparison

Installation

First, let's install the required libraries:

In [ ]
!pip install vaderSentiment textblob scikit-learn pandas numpy matplotlib seaborn nltk

Import Libraries

In [ ]
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from textblob import TextBlob
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
import warnings
warnings.filterwarnings('ignore')

# Set style for better visualizations
sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (10, 6)

Part 1: Rule-Based Approaches

Rule-based sentiment analysis uses predefined lexicons and linguistic rules to determine sentiment.

Create Sample Dataset

Let's create a small dataset of 10 sentences with varying sentiments:

In [ ]
# Sample sentences with different sentiments
sample_data = {
    'text': [
        "I absolutely love this product! It's amazing!",
        "This is the worst experience I've ever had.",
        "The movie was okay, nothing special.",
        "I'm so happy with my purchase! Highly recommend!",
        "Terrible service, very disappointed.",
        "It's decent, does what it's supposed to do.",
        "Outstanding quality and great customer support!",
        "Not bad, but could be better.",
        "I hate waiting in long lines. This is frustrating!",
        "The weather is nice today."
    ],
    'expected_sentiment': [
        'Positive',
        'Negative',
        'Neutral',
        'Positive',
        'Negative',
        'Neutral',
        'Positive',
        'Neutral',
        'Negative',
        'Neutral'
    ]
}

df_sample = pd.DataFrame(sample_data)
print("Sample Dataset:")
print(df_sample)

VADER Sentiment Analysis

VADER (Valence Aware Dictionary and sEntiment Reasoner) is specifically attuned to sentiments expressed in social media.

In [ ]
# Initialize VADER
vader_analyzer = SentimentIntensityAnalyzer()

def get_vader_sentiment(text):
    """Get VADER sentiment scores and label"""
    scores = vader_analyzer.polarity_scores(text)
    compound = scores['compound']
    
    # Classify based on compound score
    if compound >= 0.05:
        return 'Positive', compound
    elif compound <= -0.05:
        return 'Negative', compound
    else:
        return 'Neutral', compound

# Apply VADER to each sentence
vader_results = df_sample['text'].apply(get_vader_sentiment)
df_sample['vader_sentiment'] = [x[0] for x in vader_results]
df_sample['vader_score'] = [x[1] for x in vader_results]

print("\nVADER Results:")
print(df_sample[['text', 'expected_sentiment', 'vader_sentiment', 'vader_score']])

TextBlob Sentiment Analysis

TextBlob provides a simple API for common NLP tasks, including sentiment analysis.

In [ ]
def get_textblob_sentiment(text):
    """Get TextBlob sentiment polarity and label"""
    blob = TextBlob(text)
    polarity = blob.sentiment.polarity
    
    # Classify based on polarity
    if polarity > 0.1:
        return 'Positive', polarity
    elif polarity < -0.1:
        return 'Negative', polarity
    else:
        return 'Neutral', polarity

# Apply TextBlob to each sentence
textblob_results = df_sample['text'].apply(get_textblob_sentiment)
df_sample['textblob_sentiment'] = [x[0] for x in textblob_results]
df_sample['textblob_score'] = [x[1] for x in textblob_results]

print("\nTextBlob Results:")
print(df_sample[['text', 'expected_sentiment', 'textblob_sentiment', 'textblob_score']])

Compare VADER vs TextBlob

In [ ]
# Calculate accuracy for both methods
vader_accuracy = (df_sample['vader_sentiment'] == df_sample['expected_sentiment']).mean()
textblob_accuracy = (df_sample['textblob_sentiment'] == df_sample['expected_sentiment']).mean()

print(f"\nVADER Accuracy: {vader_accuracy:.2%}")
print(f"TextBlob Accuracy: {textblob_accuracy:.2%}")

# Display comparison table
comparison_df = df_sample[['text', 'expected_sentiment', 'vader_sentiment', 'vader_score', 
                           'textblob_sentiment', 'textblob_score']]
print("\nDetailed Comparison:")
print(comparison_df.to_string())
In [ ]
# Visualize sentiment scores
fig, axes = plt.subplots(1, 2, figsize=(15, 5))

# VADER scores
axes[0].barh(range(len(df_sample)), df_sample['vader_score'], 
             color=['green' if x > 0 else 'red' if x < 0 else 'gray' for x in df_sample['vader_score']])
axes[0].set_yticks(range(len(df_sample)))
axes[0].set_yticklabels([f"Sentence {i+1}" for i in range(len(df_sample))])
axes[0].set_xlabel('Compound Score')
axes[0].set_title('VADER Sentiment Scores')
axes[0].axvline(x=0, color='black', linestyle='--', linewidth=0.8)

# TextBlob scores
axes[1].barh(range(len(df_sample)), df_sample['textblob_score'],
             color=['green' if x > 0 else 'red' if x < 0 else 'gray' for x in df_sample['textblob_score']])
axes[1].set_yticks(range(len(df_sample)))
axes[1].set_yticklabels([f"Sentence {i+1}" for i in range(len(df_sample))])
axes[1].set_xlabel('Polarity Score')
axes[1].set_title('TextBlob Sentiment Scores')
axes[1].axvline(x=0, color='black', linestyle='--', linewidth=0.8)

plt.tight_layout()
plt.show()

Part 2: Machine Learning Approaches

Now we'll use the IMDB Reviews Dataset to train and compare different machine learning models.

Load IMDB Dataset

In [ ]
import pandas as pd

# Load the dataset (ensure the CSV path is correct)
df_imdb = pd.read_csv('IMDB Dataset.csv')

# View the first few rows (typically 'review' and 'sentiment' columns)
print(df_imdb.head())

Prepare Data for Training

In [ ]
# Split data into training and testing sets
X = df_imdb['review']
y = df_imdb['sentiment']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

print(f"Training set size: {len(X_train)}")
print(f"Test set size: {len(X_test)}")
print(f"\nTraining set sentiment distribution:")
print(y_train.value_counts())

Model 1: Naive Bayes with Count Vectorizer

In [ ]
# Initialize Count Vectorizer
count_vectorizer = CountVectorizer(max_features=5000, stop_words='english')

# Transform training and test data
X_train_count = count_vectorizer.fit_transform(X_train)
X_test_count = count_vectorizer.transform(X_test)

# Train Naive Bayes classifier
nb_count = MultinomialNB()
nb_count.fit(X_train_count, y_train)

# Make predictions
y_pred_nb_count = nb_count.predict(X_test_count)

# Calculate accuracy
accuracy_nb_count = accuracy_score(y_test, y_pred_nb_count)
print(f"Naive Bayes with Count Vectorizer - Accuracy: {accuracy_nb_count:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_nb_count, target_names=['Negative', 'Positive']))

Model 2: Naive Bayes with TF-IDF Vectorizer

In [ ]
# Initialize TF-IDF Vectorizer
tfidf_vectorizer = TfidfVectorizer(max_features=5000, stop_words='english')

# Transform training and test data
X_train_tfidf = tfidf_vectorizer.fit_transform(X_train)
X_test_tfidf = tfidf_vectorizer.transform(X_test)

# Train Naive Bayes classifier
nb_tfidf = MultinomialNB()
nb_tfidf.fit(X_train_tfidf, y_train)

# Make predictions
y_pred_nb_tfidf = nb_tfidf.predict(X_test_tfidf)

# Calculate accuracy
accuracy_nb_tfidf = accuracy_score(y_test, y_pred_nb_tfidf)
print(f"Naive Bayes with TF-IDF Vectorizer - Accuracy: {accuracy_nb_tfidf:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_nb_tfidf, target_names=['Negative', 'Positive']))

Model 3: Logistic Regression with Count Vectorizer

In [ ]
# Train Logistic Regression classifier with Count Vectorizer
lr_count = LogisticRegression(max_iter=1000, random_state=42)
lr_count.fit(X_train_count, y_train)

# Make predictions
y_pred_lr_count = lr_count.predict(X_test_count)

# Calculate accuracy
accuracy_lr_count = accuracy_score(y_test, y_pred_lr_count)
print(f"Logistic Regression with Count Vectorizer - Accuracy: {accuracy_lr_count:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_lr_count, target_names=['Negative', 'Positive']))

Model 4: Logistic Regression with TF-IDF Vectorizer

In [ ]
# Train Logistic Regression classifier with TF-IDF Vectorizer
lr_tfidf = LogisticRegression(max_iter=1000, random_state=42)
lr_tfidf.fit(X_train_tfidf, y_train)

# Make predictions
y_pred_lr_tfidf = lr_tfidf.predict(X_test_tfidf)

# Calculate accuracy
accuracy_lr_tfidf = accuracy_score(y_test, y_pred_lr_tfidf)
print(f"Logistic Regression with TF-IDF Vectorizer - Accuracy: {accuracy_lr_tfidf:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_lr_tfidf, target_names=['Negative', 'Positive']))

Part 3: Model Comparison

Let's compare all four machine learning models using tables and visualizations.

Create Comparison Table

In [ ]
# Collect all metrics
from sklearn.metrics import precision_score, recall_score, f1_score

models = [
    'Naive Bayes + Count Vectorizer',
    'Naive Bayes + TF-IDF',
    'Logistic Regression + Count Vectorizer',
    'Logistic Regression + TF-IDF'
]

predictions = [
    y_pred_nb_count,
    y_pred_nb_tfidf,
    y_pred_lr_count,
    y_pred_lr_tfidf
]

# Calculate metrics for each model
results = []
for model_name, y_pred in zip(models, predictions):
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, pos_label='positive')
    recall = recall_score(y_test, y_pred, pos_label='positive')
    f1 = f1_score(y_test, y_pred, pos_label='positive')
    
    results.append({
        'Model': model_name,
        'Accuracy': accuracy,
        'Precision': precision,
        'Recall': recall,
        'F1-Score': f1
    })

# Create comparison DataFrame
df_results = pd.DataFrame(results)

print("\n" + "="*80)
print("MODEL COMPARISON RESULTS")
print("="*80)
print(df_results.to_string(index=False))
print("="*80)

# Find best model
best_model_idx = df_results['Accuracy'].idxmax()
best_model = df_results.loc[best_model_idx, 'Model']
best_accuracy = df_results.loc[best_model_idx, 'Accuracy']
print(f"\nBest Model: {best_model} with Accuracy: {best_accuracy:.4f}")

Visualize Model Comparison

In [ ]
# Create bar chart comparing all metrics
fig, ax = plt.subplots(figsize=(14, 6))

x = np.arange(len(models))
width = 0.2

metrics = ['Accuracy', 'Precision', 'Recall', 'F1-Score']
colors = ['#3498db', '#2ecc71', '#e74c3c', '#f39c12']

for i, metric in enumerate(metrics):
    values = df_results[metric].values
    ax.bar(x + i*width, values, width, label=metric, color=colors[i], alpha=0.8)

ax.set_xlabel('Models', fontsize=12, fontweight='bold')
ax.set_ylabel('Score', fontsize=12, fontweight='bold')
ax.set_title('Model Performance Comparison', fontsize=14, fontweight='bold')
ax.set_xticks(x + width * 1.5)
ax.set_xticklabels(models, rotation=15, ha='right')
ax.legend(loc='lower right')
ax.set_ylim([0, 1.1])
ax.grid(axis='y', alpha=0.3)

plt.tight_layout()
plt.show()
In [ ]
# Create individual metric comparisons
fig, axes = plt.subplots(2, 2, figsize=(15, 10))
axes = axes.ravel()

for i, metric in enumerate(metrics):
    values = df_results[metric].values
    bars = axes[i].barh(models, values, color=colors[i], alpha=0.7)
    
    # Add value labels on bars
    for bar in bars:
        width = bar.get_width()
        axes[i].text(width, bar.get_y() + bar.get_height()/2, 
                    f'{width:.4f}', ha='left', va='center', fontweight='bold')
    
    axes[i].set_xlabel('Score', fontweight='bold')
    axes[i].set_title(f'{metric} Comparison', fontweight='bold', fontsize=12)
    axes[i].set_xlim([0, 1.1])
    axes[i].grid(axis='x', alpha=0.3)

plt.tight_layout()
plt.show()

Confusion Matrices

In [ ]
# Plot confusion matrices for all models
fig, axes = plt.subplots(2, 2, figsize=(14, 12))
axes = axes.ravel()

for i, (model_name, y_pred) in enumerate(zip(models, predictions)):
    cm = confusion_matrix(y_test, y_pred)
    
    sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', 
                xticklabels=['Negative', 'Positive'],
                yticklabels=['Negative', 'Positive'],
                ax=axes[i], cbar=False)
    
    axes[i].set_title(f'{model_name}\nAccuracy: {results[i]["Accuracy"]:.4f}', 
                     fontweight='bold')
    axes[i].set_ylabel('True Label', fontweight='bold')
    axes[i].set_xlabel('Predicted Label', fontweight='bold')

plt.tight_layout()
plt.show()

Summary and Insights

In [ ]
print("\n" + "="*80)
print("SUMMARY AND INSIGHTS")
print("="*80)

print("\n1. RULE-BASED APPROACHES:")
print(f"   - VADER Accuracy: {vader_accuracy:.2%}")
print(f"   - TextBlob Accuracy: {textblob_accuracy:.2%}")
print("   - Best for: Quick analysis, social media text, no training data needed")

print("\n2. MACHINE LEARNING APPROACHES:")
for idx, row in df_results.iterrows():
    print(f"   - {row['Model']}: {row['Accuracy']:.4f}")

print(f"\n3. BEST PERFORMING MODEL:")
print(f"   {best_model}")
print(f"   Accuracy: {best_accuracy:.4f}")

print("\n4. KEY OBSERVATIONS:")
print("   - Logistic Regression generally performs better than Naive Bayes")
print("   - TF-IDF often provides better feature representation than Count Vectorizer")
print("   - ML models typically outperform rule-based approaches when trained on sufficient data")
print("   - Rule-based methods are faster and don't require training")

print("\n" + "="*80)

Test the Best Model on New Data

In [ ]
# Let's test the best model on some new examples
new_reviews = [
    "This movie exceeded all my expectations!",
    "Boring and predictable. Would not watch again.",
    "An absolute masterpiece of cinema!",
    "Waste of time and money. Very disappointing.",
    "Great if you just want to waste time."
]

# Use the best model (you can modify this based on results)
# For demonstration, we'll use Logistic Regression with TF-IDF
new_reviews_vectorized = tfidf_vectorizer.transform(new_reviews)
predictions = lr_tfidf.predict(new_reviews_vectorized)
probabilities = lr_tfidf.predict_proba(new_reviews_vectorized)

print("\nPredictions on New Reviews:")
print("="*80)
for review, pred, prob in zip(new_reviews, predictions, probabilities):
    index = 1 if pred == "positive" else 0
    confidence = prob[index] * 100
    print(f"Review: {review}")
    print(f"Prediction: {pred} (Confidence: {confidence:.2f}%)")
    print("-" * 80)

Conclusion

In this tutorial, we explored:

  1. Rule-Based Approaches: VADER and TextBlob provide quick sentiment analysis without training
  2. Machine Learning Approaches: Four different combinations of classifiers and vectorizers
  3. Comparison: Comprehensive evaluation using multiple metrics and visualizations

Key Takeaways:

  • ML models generally outperform rule-based methods with sufficient training data
  • TF-IDF often provides better feature representation than simple count vectors
  • Logistic Regression tends to perform better than Naive Bayes for sentiment analysis
  • Always evaluate models using multiple metrics (accuracy, precision, recall, F1-score)