sentiment_analysis_tutorial
ipynb24 kBPython · 41 cellsExtra material of chapter 5
Sentiment Analysis Tutorial
This tutorial covers both rule-based and machine learning approaches to sentiment analysis.
Table of Contents
- Rule-Based Approaches
- VADER
- TextBlob
- Machine Learning Approaches
- Naive Bayes with Count Vectorizer
- Naive Bayes with TF-IDF Vectorizer
- Logistic Regression with Count Vectorizer
- Logistic Regression with TF-IDF Vectorizer
- Model Comparison
Installation
First, let's install the required libraries:
!pip install vaderSentiment textblob scikit-learn pandas numpy matplotlib seaborn nltkImport Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from textblob import TextBlob
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
import warnings
warnings.filterwarnings('ignore')
# Set style for better visualizations
sns.set_style('whitegrid')
plt.rcParams['figure.figsize'] = (10, 6)Part 1: Rule-Based Approaches
Rule-based sentiment analysis uses predefined lexicons and linguistic rules to determine sentiment.
Create Sample Dataset
Let's create a small dataset of 10 sentences with varying sentiments:
# Sample sentences with different sentiments
sample_data = {
'text': [
"I absolutely love this product! It's amazing!",
"This is the worst experience I've ever had.",
"The movie was okay, nothing special.",
"I'm so happy with my purchase! Highly recommend!",
"Terrible service, very disappointed.",
"It's decent, does what it's supposed to do.",
"Outstanding quality and great customer support!",
"Not bad, but could be better.",
"I hate waiting in long lines. This is frustrating!",
"The weather is nice today."
],
'expected_sentiment': [
'Positive',
'Negative',
'Neutral',
'Positive',
'Negative',
'Neutral',
'Positive',
'Neutral',
'Negative',
'Neutral'
]
}
df_sample = pd.DataFrame(sample_data)
print("Sample Dataset:")
print(df_sample)VADER Sentiment Analysis
VADER (Valence Aware Dictionary and sEntiment Reasoner) is specifically attuned to sentiments expressed in social media.
# Initialize VADER
vader_analyzer = SentimentIntensityAnalyzer()
def get_vader_sentiment(text):
"""Get VADER sentiment scores and label"""
scores = vader_analyzer.polarity_scores(text)
compound = scores['compound']
# Classify based on compound score
if compound >= 0.05:
return 'Positive', compound
elif compound <= -0.05:
return 'Negative', compound
else:
return 'Neutral', compound
# Apply VADER to each sentence
vader_results = df_sample['text'].apply(get_vader_sentiment)
df_sample['vader_sentiment'] = [x[0] for x in vader_results]
df_sample['vader_score'] = [x[1] for x in vader_results]
print("\nVADER Results:")
print(df_sample[['text', 'expected_sentiment', 'vader_sentiment', 'vader_score']])TextBlob Sentiment Analysis
TextBlob provides a simple API for common NLP tasks, including sentiment analysis.
def get_textblob_sentiment(text):
"""Get TextBlob sentiment polarity and label"""
blob = TextBlob(text)
polarity = blob.sentiment.polarity
# Classify based on polarity
if polarity > 0.1:
return 'Positive', polarity
elif polarity < -0.1:
return 'Negative', polarity
else:
return 'Neutral', polarity
# Apply TextBlob to each sentence
textblob_results = df_sample['text'].apply(get_textblob_sentiment)
df_sample['textblob_sentiment'] = [x[0] for x in textblob_results]
df_sample['textblob_score'] = [x[1] for x in textblob_results]
print("\nTextBlob Results:")
print(df_sample[['text', 'expected_sentiment', 'textblob_sentiment', 'textblob_score']])Compare VADER vs TextBlob
# Calculate accuracy for both methods
vader_accuracy = (df_sample['vader_sentiment'] == df_sample['expected_sentiment']).mean()
textblob_accuracy = (df_sample['textblob_sentiment'] == df_sample['expected_sentiment']).mean()
print(f"\nVADER Accuracy: {vader_accuracy:.2%}")
print(f"TextBlob Accuracy: {textblob_accuracy:.2%}")
# Display comparison table
comparison_df = df_sample[['text', 'expected_sentiment', 'vader_sentiment', 'vader_score',
'textblob_sentiment', 'textblob_score']]
print("\nDetailed Comparison:")
print(comparison_df.to_string())# Visualize sentiment scores
fig, axes = plt.subplots(1, 2, figsize=(15, 5))
# VADER scores
axes[0].barh(range(len(df_sample)), df_sample['vader_score'],
color=['green' if x > 0 else 'red' if x < 0 else 'gray' for x in df_sample['vader_score']])
axes[0].set_yticks(range(len(df_sample)))
axes[0].set_yticklabels([f"Sentence {i+1}" for i in range(len(df_sample))])
axes[0].set_xlabel('Compound Score')
axes[0].set_title('VADER Sentiment Scores')
axes[0].axvline(x=0, color='black', linestyle='--', linewidth=0.8)
# TextBlob scores
axes[1].barh(range(len(df_sample)), df_sample['textblob_score'],
color=['green' if x > 0 else 'red' if x < 0 else 'gray' for x in df_sample['textblob_score']])
axes[1].set_yticks(range(len(df_sample)))
axes[1].set_yticklabels([f"Sentence {i+1}" for i in range(len(df_sample))])
axes[1].set_xlabel('Polarity Score')
axes[1].set_title('TextBlob Sentiment Scores')
axes[1].axvline(x=0, color='black', linestyle='--', linewidth=0.8)
plt.tight_layout()
plt.show()Part 2: Machine Learning Approaches
Now we'll use the IMDB Reviews Dataset to train and compare different machine learning models.
Load IMDB Dataset
import pandas as pd
# Load the dataset (ensure the CSV path is correct)
df_imdb = pd.read_csv('IMDB Dataset.csv')
# View the first few rows (typically 'review' and 'sentiment' columns)
print(df_imdb.head())Prepare Data for Training
# Split data into training and testing sets
X = df_imdb['review']
y = df_imdb['sentiment']
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Training set size: {len(X_train)}")
print(f"Test set size: {len(X_test)}")
print(f"\nTraining set sentiment distribution:")
print(y_train.value_counts())Model 1: Naive Bayes with Count Vectorizer
# Initialize Count Vectorizer
count_vectorizer = CountVectorizer(max_features=5000, stop_words='english')
# Transform training and test data
X_train_count = count_vectorizer.fit_transform(X_train)
X_test_count = count_vectorizer.transform(X_test)
# Train Naive Bayes classifier
nb_count = MultinomialNB()
nb_count.fit(X_train_count, y_train)
# Make predictions
y_pred_nb_count = nb_count.predict(X_test_count)
# Calculate accuracy
accuracy_nb_count = accuracy_score(y_test, y_pred_nb_count)
print(f"Naive Bayes with Count Vectorizer - Accuracy: {accuracy_nb_count:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_nb_count, target_names=['Negative', 'Positive']))Model 2: Naive Bayes with TF-IDF Vectorizer
# Initialize TF-IDF Vectorizer
tfidf_vectorizer = TfidfVectorizer(max_features=5000, stop_words='english')
# Transform training and test data
X_train_tfidf = tfidf_vectorizer.fit_transform(X_train)
X_test_tfidf = tfidf_vectorizer.transform(X_test)
# Train Naive Bayes classifier
nb_tfidf = MultinomialNB()
nb_tfidf.fit(X_train_tfidf, y_train)
# Make predictions
y_pred_nb_tfidf = nb_tfidf.predict(X_test_tfidf)
# Calculate accuracy
accuracy_nb_tfidf = accuracy_score(y_test, y_pred_nb_tfidf)
print(f"Naive Bayes with TF-IDF Vectorizer - Accuracy: {accuracy_nb_tfidf:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_nb_tfidf, target_names=['Negative', 'Positive']))Model 3: Logistic Regression with Count Vectorizer
# Train Logistic Regression classifier with Count Vectorizer
lr_count = LogisticRegression(max_iter=1000, random_state=42)
lr_count.fit(X_train_count, y_train)
# Make predictions
y_pred_lr_count = lr_count.predict(X_test_count)
# Calculate accuracy
accuracy_lr_count = accuracy_score(y_test, y_pred_lr_count)
print(f"Logistic Regression with Count Vectorizer - Accuracy: {accuracy_lr_count:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_lr_count, target_names=['Negative', 'Positive']))Model 4: Logistic Regression with TF-IDF Vectorizer
# Train Logistic Regression classifier with TF-IDF Vectorizer
lr_tfidf = LogisticRegression(max_iter=1000, random_state=42)
lr_tfidf.fit(X_train_tfidf, y_train)
# Make predictions
y_pred_lr_tfidf = lr_tfidf.predict(X_test_tfidf)
# Calculate accuracy
accuracy_lr_tfidf = accuracy_score(y_test, y_pred_lr_tfidf)
print(f"Logistic Regression with TF-IDF Vectorizer - Accuracy: {accuracy_lr_tfidf:.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_lr_tfidf, target_names=['Negative', 'Positive']))Part 3: Model Comparison
Let's compare all four machine learning models using tables and visualizations.
Create Comparison Table
# Collect all metrics
from sklearn.metrics import precision_score, recall_score, f1_score
models = [
'Naive Bayes + Count Vectorizer',
'Naive Bayes + TF-IDF',
'Logistic Regression + Count Vectorizer',
'Logistic Regression + TF-IDF'
]
predictions = [
y_pred_nb_count,
y_pred_nb_tfidf,
y_pred_lr_count,
y_pred_lr_tfidf
]
# Calculate metrics for each model
results = []
for model_name, y_pred in zip(models, predictions):
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred, pos_label='positive')
recall = recall_score(y_test, y_pred, pos_label='positive')
f1 = f1_score(y_test, y_pred, pos_label='positive')
results.append({
'Model': model_name,
'Accuracy': accuracy,
'Precision': precision,
'Recall': recall,
'F1-Score': f1
})
# Create comparison DataFrame
df_results = pd.DataFrame(results)
print("\n" + "="*80)
print("MODEL COMPARISON RESULTS")
print("="*80)
print(df_results.to_string(index=False))
print("="*80)
# Find best model
best_model_idx = df_results['Accuracy'].idxmax()
best_model = df_results.loc[best_model_idx, 'Model']
best_accuracy = df_results.loc[best_model_idx, 'Accuracy']
print(f"\nBest Model: {best_model} with Accuracy: {best_accuracy:.4f}")Visualize Model Comparison
# Create bar chart comparing all metrics
fig, ax = plt.subplots(figsize=(14, 6))
x = np.arange(len(models))
width = 0.2
metrics = ['Accuracy', 'Precision', 'Recall', 'F1-Score']
colors = ['#3498db', '#2ecc71', '#e74c3c', '#f39c12']
for i, metric in enumerate(metrics):
values = df_results[metric].values
ax.bar(x + i*width, values, width, label=metric, color=colors[i], alpha=0.8)
ax.set_xlabel('Models', fontsize=12, fontweight='bold')
ax.set_ylabel('Score', fontsize=12, fontweight='bold')
ax.set_title('Model Performance Comparison', fontsize=14, fontweight='bold')
ax.set_xticks(x + width * 1.5)
ax.set_xticklabels(models, rotation=15, ha='right')
ax.legend(loc='lower right')
ax.set_ylim([0, 1.1])
ax.grid(axis='y', alpha=0.3)
plt.tight_layout()
plt.show()# Create individual metric comparisons
fig, axes = plt.subplots(2, 2, figsize=(15, 10))
axes = axes.ravel()
for i, metric in enumerate(metrics):
values = df_results[metric].values
bars = axes[i].barh(models, values, color=colors[i], alpha=0.7)
# Add value labels on bars
for bar in bars:
width = bar.get_width()
axes[i].text(width, bar.get_y() + bar.get_height()/2,
f'{width:.4f}', ha='left', va='center', fontweight='bold')
axes[i].set_xlabel('Score', fontweight='bold')
axes[i].set_title(f'{metric} Comparison', fontweight='bold', fontsize=12)
axes[i].set_xlim([0, 1.1])
axes[i].grid(axis='x', alpha=0.3)
plt.tight_layout()
plt.show()Confusion Matrices
# Plot confusion matrices for all models
fig, axes = plt.subplots(2, 2, figsize=(14, 12))
axes = axes.ravel()
for i, (model_name, y_pred) in enumerate(zip(models, predictions)):
cm = confusion_matrix(y_test, y_pred)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
xticklabels=['Negative', 'Positive'],
yticklabels=['Negative', 'Positive'],
ax=axes[i], cbar=False)
axes[i].set_title(f'{model_name}\nAccuracy: {results[i]["Accuracy"]:.4f}',
fontweight='bold')
axes[i].set_ylabel('True Label', fontweight='bold')
axes[i].set_xlabel('Predicted Label', fontweight='bold')
plt.tight_layout()
plt.show()Summary and Insights
print("\n" + "="*80)
print("SUMMARY AND INSIGHTS")
print("="*80)
print("\n1. RULE-BASED APPROACHES:")
print(f" - VADER Accuracy: {vader_accuracy:.2%}")
print(f" - TextBlob Accuracy: {textblob_accuracy:.2%}")
print(" - Best for: Quick analysis, social media text, no training data needed")
print("\n2. MACHINE LEARNING APPROACHES:")
for idx, row in df_results.iterrows():
print(f" - {row['Model']}: {row['Accuracy']:.4f}")
print(f"\n3. BEST PERFORMING MODEL:")
print(f" {best_model}")
print(f" Accuracy: {best_accuracy:.4f}")
print("\n4. KEY OBSERVATIONS:")
print(" - Logistic Regression generally performs better than Naive Bayes")
print(" - TF-IDF often provides better feature representation than Count Vectorizer")
print(" - ML models typically outperform rule-based approaches when trained on sufficient data")
print(" - Rule-based methods are faster and don't require training")
print("\n" + "="*80)Test the Best Model on New Data
# Let's test the best model on some new examples
new_reviews = [
"This movie exceeded all my expectations!",
"Boring and predictable. Would not watch again.",
"An absolute masterpiece of cinema!",
"Waste of time and money. Very disappointing.",
"Great if you just want to waste time."
]
# Use the best model (you can modify this based on results)
# For demonstration, we'll use Logistic Regression with TF-IDF
new_reviews_vectorized = tfidf_vectorizer.transform(new_reviews)
predictions = lr_tfidf.predict(new_reviews_vectorized)
probabilities = lr_tfidf.predict_proba(new_reviews_vectorized)
print("\nPredictions on New Reviews:")
print("="*80)
for review, pred, prob in zip(new_reviews, predictions, probabilities):
index = 1 if pred == "positive" else 0
confidence = prob[index] * 100
print(f"Review: {review}")
print(f"Prediction: {pred} (Confidence: {confidence:.2f}%)")
print("-" * 80)Conclusion
In this tutorial, we explored:
- Rule-Based Approaches: VADER and TextBlob provide quick sentiment analysis without training
- Machine Learning Approaches: Four different combinations of classifiers and vectorizers
- Comparison: Comprehensive evaluation using multiple metrics and visualizations
Key Takeaways:
- ML models generally outperform rule-based methods with sufficient training data
- TF-IDF often provides better feature representation than simple count vectors
- Logistic Regression tends to perform better than Naive Bayes for sentiment analysis
- Always evaluate models using multiple metrics (accuracy, precision, recall, F1-score)