How It Was Built

The
Playbook.

14 years of listening history. 12 JSON files. A DuckDB pipeline, 9 analysis scripts, and a single-page site — built from scratch with no frameworks. Here's every step.

9
Analysis scripts
29
Static charts
11
Interactive charts
80
Artist photos
147k
Streams analyzed
Stack & Process

From Export to Deploy

The full pipeline — raw JSON from Spotify to an interactive deployed site. Everything is re-runnable: drop new data files in data/raw/ and the whole analysis rebuilds.

01
Data Export
Requested Spotify Extended History via Settings → Privacy → Request Data. The extended format includes every stream since account creation — not just the last year. Wait: 2–4 weeks. Result: 12 JSON files, 147,594 records spanning 2012–2026.
02
Ingestion Pipeline
Python script parses all JSON files into a DuckDB database at data/spotify.duckdb. Deduplicates on (ts, uri, ms_played). Filters to music-only streams ≥ 30 seconds. Exports a clean Parquet for fast re-analysis.
Python 3.13 DuckDB 1.4.4 pandas Parquet
03
Categorization
SQL CASE statements classify every stream into three categories: My Music, Kids Content, and Jeannie/Taylor Swift. Artist-level exclusion lists handle edge cases. Categories are recomputed on every pipeline run — adding new raw files rebuilds everything.
04
Analysis Modules
9 analysis scripts, each focused on one domain: personality scoring (Big Five inference from behavioral data), life chapter detection (ruptures changepoint algorithm on listening features), proxy valence (artist-level mood estimation without Spotify API), genre tagging, session modeling, skip rates, daily clock patterns, and artist loyalty scoring.
ruptures scikit-learn numpy matplotlib seaborn
05
Static Chart Generation
matplotlib and seaborn render 29 PNG charts to output/charts/, then copied to site/public/charts/. Charts use a consistent dark theme: #121212 background, #1DB954 Spotify green. All re-renderable from a single pipeline run.
06
Interactive Visualizations
Chart.js 4.4 powers 11 canvas-based charts — loaded lazily via IntersectionObserver. Key charts (valence timeline, phase artists, genre evolution) replaced static PNGs for hover/touch interactivity. The year slider (Taste Evolution), Then vs Now comparison, and artist rank bump chart are fully client-side.
Chart.js 4.4 chartjs-plugin-annotation IntersectionObserver
07
Artist Imagery
Deezer API batch-fetched 80 artist photos and 19 album covers by artist name search. Images stored at site/public/artists/ with a manifest.json for slug-to-filename lookup. Custom artistPhotoPlugin draws circular photos at bar ends in Chart.js.
Deezer API 80 artist photos 19 album covers
08
Taste Galaxy — Network Clustering
Session co-occurrence graph: every pair of artists heard in the same listening session gets an edge, weighted by frequency. Louvain community detection groups 1,301 artists into 25 taste clusters — no genre database, no external labels. D3 force simulation renders the network as an interactive galaxy: drag nodes, zoom in, click any artist to see co-listening connections and unexplored recommendations. UMAP + HDBSCAN provide a secondary behavioral embedding for validation.
NetworkX Louvain UMAP HDBSCAN D3.js v7
09
Site Design
Single-page dark narrative with 16 sections, section-level color theming, smooth scroll navigation, and a right-side dot nav that tracks your position. Fonts: DM Sans, Barlow Condensed, Bebas Neue. No frameworks — vanilla HTML, CSS, and ES modules.
Vanilla JS (ES modules) DM Sans Barlow Condensed Bebas Neue
10
Deployment
Vercel CLI deploys from site/ with a single vercel deploy --prod. SPA rewrite config routes all paths to index.html. Built with Claude Code — multiple parallel agents for analysis, visualization research, and storytelling.
Vercel Claude Code SPA routing
🔧

Total pipeline: 10 steps · 9 analysis scripts · 29 static PNGs · 11 interactive Chart.js charts · D3 force galaxy with 1,301 artists · 80 artist photos · 147,594 streams across 15,748 artists · deployed on Vercel.

Source: Personal Spotify Extended History export, 2012–2026. GitHub: WhatWouldDimaDo/spotify-history-analysis