SentimentScope: PyTorch Transformer for NLP

Project Overview

SentimentScope is an end-to-end Natural Language Processing (NLP) system engineered as the capstone project for the AWS AI & ML Scholarship / Udacity AI Programming with Python program. The core engine is a custom Transformer implemented from scratch in PyTorch (without high-level wrappers such as BertForSequenceClassification), trained on the IMDB Movie Reviews dataset (50,000 balanced reviews).

The architecture underwent rigorous official evaluation by Udacity reviewers, passing on Attempt 1 with 100% rubric compliance and achieving 76.88% test accuracy across 25,000 unseen test samples.

---

Technical Architecture & Self-Attention Mechanisms

• From-Scratch Multi-Head Attention (MultiHeadAttention): Explicit implementation of affine projection matrices for Query ($Q$), Key ($K$), and Value ($V$) tensors, scaled dot-product division by $\sqrt{d_k}$, configurable causal/bidirectional masks, and head concatenation ($d_{\text{embed}} = 128$, 4 heads of dimension 32).
• Transformer Blocks & Normalization: Stack of 4 encoder blocks featuring residual skip connections, LayerNorm, Gaussian Error Linear Unit (GELU) feed-forward networks, and dropout regularization ($p = 0.1$).
• Sequence Classification via Mean Pooling: Rather than relying on an artificial [CLS] token, classification is driven by global sequence mean pooling (dim=1), mapping the aggregated contextual representations directly into a linear binary decision projection head ($128 \to 2$).
• Subword Tokenization Pipeline: Integrated Hugging Face’s AutoTokenizer (bert-base-uncased, vocabulary size: 30,522) with padding and truncation bounded to a 128-token context window to constrain quadratic self-attention complexity $\mathcal{O}(N^2)$ on GPU memory.
• Explainable AI (XAI) & Attention Heatmaps: Real-time extraction and softmax normalization of attention weight tensors to generate token-level importance heatmaps and 2D query-key alignment matrices during interactive inference.
• Software Ecosystem & Production Readiness: Features an interactive Streamlit dashboard (app.py), modular Python package architecture under src/, automated unit testing suite (pytest, 6/6 passed), and continuous integration powered by GitHub Actions CI.