Tested before it is published: this site’s nightly data pipeline

The problem

The home page of this site lists the systems I built and run, each with a date from data beside it. That needs numbers: how many people and machines read the site, how it gets built, and whether the pipeline that produces those figures ran last night. The site is static HTML on Cloudflare with no analytics script, so the only source is Cloudflare’s own request counts, and Cloudflare keeps them for 31 days. A day not copied within that window is gone.

The second half of the problem is the privacy page. It promises that the site keeps only daily totals: no IP address, no full user agent, no referrer, no query string. It promises that a page is named only once it had at least 5 views from browsers in a month, that a country is named only with at least 10 views on 3 different days, that pages and countries appear for whole months only, and that the totals start on the day the site launched. A number that breaks one of those promises must not reach the page, however the data arrived.

And the whole thing runs unattended on GitHub Actions, in a repository where a workflow on any branch can read the secrets. The job that holds the Cloudflare token had to be as small as possible, and the job that writes to the repository had to accept as little as possible.

The architecture

Each night at 09:23 UTC a GitHub Actions workflow runs the pipeline on the main branch, in three jobs with different rights. The extract job holds a read-only Cloudflare token and runs one Python script that uses the standard library only, on a machine where nothing has been installed. The build job installs dbt from hash-pinned wheels and the site’s packages, and holds no secret. The commit job holds the write token, runs no third-party code, and accepts only JSON files under two folders. Every action is GitHub’s own, pinned to a full commit, with no cache anywhere.

The collector reads two sources for this site. From Cloudflare’s GraphQL analytics API it reads each day’s totals: page requests by page and client class, countries by client class, responses by status and content type, downloads of the site’s font, requests for software the site does not run (WordPress, PHP, admin pages), and the day’s rollup. Every query groups on Cloudflare’s side, and the client reduces each group to a seed label before anything is written: a user agent becomes a class and a name from user_agent_rules.csv, and a path outside the sitemap becomes other. It never selects an IP address, a query string or a referrer. From GitHub’s API it reads the repository’s commits, pull requests and CI runs, keeping the fields the models need and dropping messages, names and branch names. The same collector reads sleepanchormusic.com’s page views and link clicks for Sleep Anchor’s website in numbers, behind that site’s own privacy page.

Each day is read on three nights in a row, because Cloudflare fills in late data. A night with a backlog reads the oldest missing day first, up to a budget of GraphQL requests, so a gap clears over a few nights. Seven stages take a day from Cloudflare to the page.

The tests are the privacy page, written in SQL. assert_publish_thresholds recomputes the page and country thresholds from staging, independently of the marts, and counts only the views Cloudflare certainly saw. Cloudflare samples requests and scales each group’s count up by its sample interval, as much as 7, so a row of n requests stands for at least n divided by that interval real ones, and that is the number the thresholds apply to. assert_months_closed allows pages and countries only for months whose last day is 3 days past, once the collector has re-read them. strings_safe runs on every published text column and allows 80 characters of plain text, so a raw user agent or a piece of markup can never reach the page. Unit tests feed both rules made-up rows and check what they let through. Warnings about Cloudflare’s data are rows of a mart of their own, shown on the page: whether every day since launch was collected, whether a query hit Cloudflare’s row limit, whether each day’s totals agree with Cloudflare’s daily rollup, whether counts were sampled, and how fresh the sources are.

Every window counts from the last collected day, never from the clock, so a rebuild of the same files gives the same numbers. The home page’s index reads the pipeline’s own figures for its line on this project, and judges freshness by this site’s own data: it reads “last run” with the night’s date while the data runs through one of the last three days before the site was built, and “Paused since” with the day the data runs through once this site’s data stops arriving. A night that failed on another source alone is not a pause.

Decisions and trade-offs

Raw files in git are the permanent record

Chose: one JSON file per source, table and day, committed to the repository, and every model built from those files alone.

Cloudflare keeps 31 days of request data, so the pipeline had to keep its own copy, and the repository was the obvious place: the data is small, every change is a commit, and the history shows exactly when each day was read. dbt reads the files straight into DuckDB with its built-in JSON reader and downloads no extension. Because every model counts from the last collected day, running the build against the same files on any machine gives the same numbers as the night did. The cost is that the record is only as complete as the nights: a day not collected within Cloudflare’s window is gone for good, which is why a night reads its backlog oldest first.

The privacy page is a gate, not a description

Chose: the collector reads no visitor data until the live privacy page shows an effective date on or after the one in its own code, and a new table or field means updating that page first.

The privacy page says what is collected. Writing it after the fact would make it a description, and descriptions drift. With the date in the collector, a change to what is collected has to arrive as a pair: the page, deployed, and then the code. The gate fails closed: if the page cannot be read or shows no date, no visitor data is read that night. The cost is that the gate checks a date, not the words, so an edit that dropped the visit-counts section would also have to stop the pipeline, through the repository variable that turns it on.

The publishing rules are tests

Chose: each promise on the privacy page is a dbt test that fails the build, and the nightly commits new numbers only when every test passed.

The thresholds, the closed-month rule and the safe-strings rule could have been filters inside the models. As tests, they are checked against the finished marts, recomputed from staging by a different path, and they fail loudly: the export refuses to write, the commit job commits the collected files but no numbers, and the page keeps the last set that passed. The sampling rule came from reading Cloudflare’s data, not from a plan. Cloudflare scales sampled counts up, so a threshold applied to the scaled count could name a page on fewer real views than the page promises; the tests apply it to the fewest real views the count can stand for. The cost is that a rule lives in three places, the page, the test and sometimes a schema, and a bad day from Cloudflare stops the update until it is understood.

An allowlist, from the exposure to the page

Chose: the export writes exactly the marts a dbt exposure depends on, the pipeline file from an explicit list of fields, and the site parses every file with a strict schema.

The alternative was to export whatever dbt built and let the page pick. An allowlist means a new mart or column reaches the page only on purpose: it needs a contract with a description, a place in the exposure, a schema entry, and a regenerated export, in one commit. Sleep Anchor’s website numbers have an exposure and a page of their own with their own rules, and the file the site’s MCP server answers from is built from the first exposure only, so those numbers appear on one page and nowhere else. The cost is ceremony: a column cannot be added quickly.

Three jobs, three sets of rights

Chose: the token that reads Cloudflare, the code that models the data and the token that writes to the repository never sit in the same job.

The extract job runs standard-library Python with the interpreter’s isolated flags, so nothing installed can see the token. The build job runs dbt and the site build, which install third-party code, with no secret at all; what it collected is fetched by the artifact id the extract job reported, so a replaced artifact fails closed. The commit job runs no third-party code, checks every file it accepts is JSON under one of two folders, and refuses on a conflict with a newer main rather than put older files back. Only I push to the repository, and the workflow runs on main alone. The cost is choreography: a partly failed night has to flow through all three jobs so that a source that worked keeps its days, but never after a cancel.

Deploy what CI passed, and prove it

Chose: the deploy workflow puts main live only when CI passed on that commit, or on an earlier one with nothing but data changed since, and fails unless the live pages match the build byte for byte.

The nightly’s own build job has already built the site with the night’s numbers and tested the MCP bundle, so a data-only commit needs no second CI run. A deploy token with rights over this one Worker uploads a version tagged with the commit, sends all traffic to it, and then fetches the home page, the two data pages, the privacy and résumé pages and the MCP data file, and compares each with the build. A merge whose CI is still running waits for that run, and a deploy skips if main moves while it runs. The cost is a few minutes between a merge and the live site, which a one-person site can afford.

What the AI did, what I decided, what caught it

An AI coding assistant, Claude Code, wrote the code to my design, and the data page counts how many commits on main carry its co-author line.

• I decided the shape: raw files in git, the privacy page as a gate, the publishing rules as tests, an allowlist from the exposure to the page, and three jobs with separate rights. Claude Code wrote the collector, the models and their tests, the export, the workflows and the page.
• The collector has unit tests of its own, run in CI with the standard library only: that no raw string reaches disk, that the gate fails closed, that a failure part way keeps the days already read, and that a late or empty day is handled as the rules say.
• dbt’s unit tests feed the threshold rules made-up rows; the strict schemas on the site side catch a drift in any exported file; the export and the commit job refuse anything outside their lists.
• A code review pass before merging found problems in two of the pipeline’s pull requests, and each was fixed in the same pull request before it reached main.
• The sampling rule was a reading of Cloudflare’s data, not a design decision made up front: once the data showed scaled counts, the thresholds were moved to the views Cloudflare certainly saw, and the page says so.

Where it breaks

• A failed source keeps the others’ days. A night that fails on one source still commits what the others read, and GitHub’s failure email links to the run, where an annotation says which source failed and why. To retry, I re-run all jobs: the extract job checks out the newest main, so it sees the days an earlier run committed.
• A day read late. Sleep Anchor’s web analytics re-samples a day after about 4 days, so a day the nightly first reads more than 3 days late is saved as late and fails the night, and the day is replaced from my local report, which reads each day the next morning.
• 19 October 2026. The night failed on both Sleep Anchor sources: sleepanchormusic.com had gained a Downloads page and per-track Ko-fi links on 17 and 18 October, 48 link slugs the seed did not know, and Cloudflare’s web analytics returned no page views at all for 18 October, so the collector wrote nothing for that day. This site’s own days were read and committed as usual. The repair was one pull request the same day: the 48 slugs joined the seed, and the local report filled 18 October, as the runbook says to do after a gap.
• Cloudflare’s window. A day not collected within 31 days is gone, and the budget of GraphQL requests means a long backlog clears over several nights, oldest first.
• Sampling. Cloudflare samples requests and scales the counts up, so small numbers can move a little between nights. The thresholds count only the views it certainly saw, and a check row on the page says when counts were sampled.
• The gate checks a date. An edit to the privacy page that dropped the visit-counts section would also have to stop the pipeline, and that step is mine to take by hand.

What changed

Before the collector’s first run, the only record of this site’s traffic was Cloudflare’s dashboard, which keeps 31 days. The figures below are as of 19 October 2026; the data page states the current ones, with the day they run through.

• The record holds every day since the site launched on 10 October 2026, and the published data runs through 18 October 2026.
• 152 of 152 tests passed on that data, 3 of them unit tests of the publishing rules, and twelve tables are published through the exposure, each with its contract.
• The collector has run 4 times since 17 October 2026 and read every source on 3 of them; the fourth was the 19 October night above, which still committed this site’s own days.
• The home page states this project’s outcome from the pipeline itself, never typed by hand: the tests passed and the day the data runs through, as of the last run.
• A wrong number cannot be published. A failed test keeps the last numbers that passed, a failed source keeps the others’ days, and every deploy proves the live pages are the build.
• Nothing about a visitor is kept beyond daily totals. The collector never selects an IP address, a query string or a referrer, and the commit history shows exactly what was written each night.

What I would carry into any data platform

• Keep the raw extract, unchanged, and rebuild everything from it.
• Make the policy a gate the code checks, not a document the code is supposed to follow.
• Write the publishing rules as tests that fail the build, and apply them to what the data can prove, not to an estimate.
• Publish through an allowlist with a contract on every table.
• Give each job only the rights its step needs, and let a partial failure keep what worked.