Datapulse

Every team that's piped a spreadsheet into production has lived the same nightmare: a CSV endpoint quietly changes shape — a column renamed, three thousand rows vanish, a currency field returns strings instead of floats — and nobody notices until a downstream dashboard is already wrong. CSV-backed APIs are the duct tape of the data world: vendor feeds, internal exports, government open-data, pricing sheets, almost none with change notifications. You either poll blindly and hope, or write a one-off script nobody maintains after its author leaves.

DataPulse turns blind polling into something observable. Every endpoint gets a SHA-256 fingerprint on each fetch; if the hash moves, it renders exactly what changed — rows added, rows removed, columns added, columns dropped — as a human-readable diff in a changelog you can scroll back through. Each change is an entry in an append-only ledger carrying the before/after hash, row deltas and structured column-level diffs plus a generated summary. Every fetch is a fingerprint in an audit trail, not a throwaway request.

The single biggest design decision isn't a color or font — it's that every tracked endpoint lives inside a collapsible 'cell' styled after the notebook interfaces data scientists already trust. That's deliberate psychology: familiarity lowers resistance, so anyone who has scrolled an .ipynb has the right mental model instantly, with no onboarding tax. Collapsed-by-default reduces anxiety — the eye rests on status first (a colored dot, a name, a stat line) and only commits to a full data table by choice, instead of drowning in expanded tables every session.

This is a tool built around triage: most of the screen, most of the time, should be calm — only real signal gets motion, color or emphasis. The left-edge accent line is a peripheral-vision signal (green: don't look; amber: worth a glance; red: broken), so you can scroll a notebook of thirty endpoints and triage in seconds without reading a word. The pulsing amber badge on cells with pending updates exploits the same attention mechanism as a notification dot — reserved only for things that need a decision. Even the login's time-of-day greeting signals a human is on the other end, and handing auth entirely to Google is itself a trust signal: no password to leak.

DataPulse commits fully to a dark glassmorphic language — translucent blurred cards over a near-black base (#04040c), electric violet primary and luminous teal secondary, with status semantics on a separate consistent palette (emerald healthy, amber attention, rose broken). Every color, radius, shadow and timing lives in :root as a true design token, so the whole identity re-themes from one block of globals.css. Backdrop blur is used sparingly against low-opacity borders so surfaces float rather than paste on top, and micro-animations map to real state changes — the animation vocabulary is a second channel of information parallel to color.

Built on Next.js App Router with an intentionally small stack: NextAuth with Google as the sole provider (no password storage, no reset-flow surface, the whole auth attack surface delegated to Google), and MongoDB via Mongoose across three collections — users (the Google identity), endpoints (URL, custom headers, tags, interval, flags and the last known hash/row-count/column-list baseline) and changelogs (the append-only diff ledger). PapaParse handles CSV-to-JSON for the sortable, filterable inline previews, and UI state stays deliberately simple — component state plus a couple of drawer/modal portals, no global store solving a problem this app doesn't have.

The core trick is cheap and effective: hash the raw CSV body on every fetch, compare against the stored lastHash, and only do the expensive row/column diffing when the hash actually differs. That single decision is what lets DataPulse poll many endpoints on a schedule without turning into a database-thrashing background job. The fetch-and-diff cycle runs server-side under app/api/endpoints/, which also means user-supplied custom headers — bearer tokens for private CSV feeds — never touch the browser, only the server making the outbound request.

What makes DataPulse cohere isn't any single feature — it's that the psychology, the visual design and the technical model all tell the same story. The interface stays quiet until there's a reason to be loud because the architecture is built to only do work when something's actually different. The notebook metaphor makes it familiar to the data-literate people who'll stare at it daily; the dark glass signals 'serious infrastructure tool'; and the schema treats every fetch as a fingerprint in an audit trail. It's a small app on purpose — every piece earns its place in one sentence: tell me, clearly and calmly, the moment my data stops being what I thought it was.