A sample audit plan: a small site’s analytics with promises to keep

The architecture and automation audit ends in a written plan: what to build, in what order, and which work to automate first. This is that document, written for real. The system audited is the pipeline behind this site’s own data, and every finding below comes from what the site already publishes about it. A client’s plan reads the same way, about their systems, with their numbers in place of mine.

One honest note first. I designed and run the system under audit, so this is a review of my own work; the client’s version is a review of yours by someone who did not build it, which is the point of asking for one.

What was audited, and what the audit asked

The system is small and completely described in public. Each night a Python script copies the previous day’s request totals for this site from Cloudflare into the site’s code repository. dbt models those files in DuckDB, a set of tests checks every number against the promises on the privacy page, and the numbers that pass are published on the data page and rebuilt into the site. The case study describes the architecture; this plan takes it as read and asks three questions of it.

Can a number reach the page that breaks a promise the privacy page makes?
Can the record be lost, and would anyone know?
Where does a person still do by hand what a machine should do?

The sources were the data page as of 19 October 2026 (data through 18 October), the privacy page, the case study, the collector and model code, the workflow definitions and the project’s own working notes. The client’s version names the people interviewed and the systems opened; here the sources were the published pages and the code, with nobody to interview but me.

What the audit found sound is listed under what to leave alone. Five things need work, in the order given further down.

Findings

Each finding states the problem, the evidence for it, and the risk if nothing is done.

Nobody is told when the data stops arriving

Problem. The nightly collector runs only while a switch in the repository is on, and a scheduled job can also be switched off by GitHub itself in a repository it judges inactive for sixty days. When a run fails, GitHub sends an email. When no run happens at all, nothing happens: no email, no failed test, no red mark.

Evidence. The data page’s own freshness check, the row that reads “Data through 18 October 2026, collected 19 October 2026”, is a warning row, not a test that fails the night. The home page reads “Paused since” once this site’s data is more than three days old, which tells whoever happens to look. Cloudflare keeps request data for 31 days, so a quiet stop longer than that loses days for good, and the case study lists that window as the one loss the design cannot undo.

Risk. Low likelihood, permanent consequence. Everything else in this plan can be fixed late without losing anything; this one cannot.

The privacy gate checks a date, not the words

Problem. Before it reads any visitor data, the collector fetches the live privacy page and refuses to continue unless the page carries an effective date on or after the one written in its own code. That is a good gate, and it fails closed. But it checks only the date. An edit that dropped or softened the visit-counts section, and kept the date, would pass.

Evidence. The gate’s code compares two dates and nothing else. The case study says so in its own list of where the system breaks, and adds that stopping the pipeline after such an edit is a step the owner takes by hand.

Risk. Low likelihood with one careful editor; high consequence, because the site would then be collecting what its privacy page no longer says it collects.

The same rule is written in three places

Problem. The privacy page promises that a page is named only with at least 5 views from browsers in a month, and a country only with at least 10 views on at least 3 different days. Those numbers appear in the privacy page’s text, in the dbt test that enforces them, and in the captions the data page shows under its tables. Nothing checks that the three agree.

Evidence. Three files hold the literals 5, 10 and 3, and a change to one does not fail the build when the others stay. The case study names this cost itself: a rule lives in the page, the test and sometimes a schema.

Risk. Drift. The page could promise one threshold while the test enforces another, and the site would be wrong in whichever direction nobody noticed. The first month of page and country tables publishes on 4 November 2026, which is the first day the thresholds decide what a reader sees.

The headline count of readers is an upper bound, and the launch weeks include the builder

Problem. The data page leads with browser page views: 2,359 since launch as of 19 October. It also says, in the caption, that some automated tools pose as browsers, so the figure is an upper bound. The better count of people is further down: 140 browsers downloaded the site’s font in the same period, which a browser does about once a month. The gap between those two numbers is where the uncertainty lives, and the headline sits on the loose side of it.

Evidence. Both figures and both captions are on the reading section of the data page. In the table of who else reads the site, the largest single row is clients that do not say what they are. And the site was built in the open through these days, as the now page says; Cloudflare gives the collector no way to tell its owner’s browser from a reader’s, so nothing does.

Risk. A reader, or the owner, takes the headline as people. Decisions about what to write next would then rest on a number that is partly tools and partly the builder checking his own work.

The repository’s secrets are readable from any branch

Problem. On GitHub’s free plan, which this repository is on, a workflow on any branch can read the repository’s secrets, and anyone who can push can deploy. The design already minimises what that is worth: the Cloudflare token can only read analytics, the deploy token can change one Worker and nothing else, and the three nightly jobs never hold a token and third-party code at the same time.

Evidence. The case study and the project’s notes state the residual risk in those words, and the workflow files show the three-job split.

Risk. Low while one person pushes, which is the case today. It becomes the first thing to fix the day a second person gets push rights, and that day is worth writing down now so the fix is not forgotten then.

The recommended order of work

The order follows one rule: protect what cannot be recovered first, then protect the promise, then improve the numbers, then tidy. Each piece is sized in days of work; the client’s version carries a fixed price beside each size.

An alarm for silence. Half a day. First because the only permanent loss in the system is a day not collected within Cloudflare’s window. The work is in the next section.
A gate that checks the words. Half a day. Store, beside the required date in the collector, a fingerprint of the visit-counts section’s text as published, and refuse to collect when the live section differs. A rewording then stops collection until the code is updated knowingly, in the same pull request as the page.
One place for the thresholds. One day. Put the three numbers in one file that the dbt test, the data page’s captions and the privacy page all read, so a change is one edit and the build fails if any reader of it is left behind. Do this before 4 November, when the thresholds first decide what is published.
A people figure the owner can trust. One day, after the thresholds move, because the caption text moves with them. Lead the reading section with the font-download count, which the page already calls the better count of people, and keep browser page views as the second line. Add one sentence that the launch weeks include the owner’s own visits. Add a check row that warns when views per new browser jump, which is what a tool posing as a browser looks like.
Secrets limited to one branch. No work now. Write a one-line trigger in the project’s notes: when a second person gets push rights, move the deploy token behind a deployment environment limited to the main branch, which GitHub offers on its paid plans for a private repository, or keep the number of people who can push at one.

Pieces 1 to 3 have no dependency on each other and could run in one week. Piece 4 follows piece 3. Piece 5 waits on an event, not on time.

What to automate first, and why

The one task still done by a person, every day, is looking. The owner opens the data page or the home page and reads the date beside “last run”. The failure email covers a night that ran and failed; nothing covers a night that did not run.

Automate that look first, and automate it outside the system it watches. A monitor that lives inside the nightly cannot report that the nightly did not start. The smallest version is a free external uptime monitor that fetches the data page once a day and alerts when the page reads “Paused since” or when the data-through date is more than three days old. It needs no code in the repository, no new scheduled job on GitHub, and no new secret. It turns the one permanent loss in the system into an email within a day of the stop, with 27 days of Cloudflare’s window still to spare.

Two things look like candidates and are not. Retrying a failed night is already automatic for this site: the next night reads whatever is missing, oldest day first, and the case study’s record shows a failed night on 19 October that still committed this site’s days. And the pairing of a privacy-page edit with a pipeline change is a decision, not a chore; piece 2 makes the machine refuse to proceed without it, which is the right amount of automation for a judgement call.

What to leave alone

An audit should also say what is sound, so that nobody spends a week improving it.

Raw files in git as the permanent record. Small, versioned, rebuilt from scratch on any machine with the same result. Do not move this into a database.
The three jobs with three sets of rights. The job with the Cloudflare token runs standard-library Python and installs nothing; the job that models the data holds no secret; the job that writes to the repository runs no third-party code and accepts only JSON files. It is more choreography than a one-person site strictly needs, and it is the reason the secrets finding above is low risk. Keep it.
The publishing rules as tests. 152 tests passed on the current data, 3 of them unit tests of the rules themselves, and a failed test keeps the last numbers that passed. This is the part of the system a client should copy.
Cloudflare’s sampling. The data page reports that Cloudflare sampled requests on 9 of 9 days and scaled the counts up, and that every day’s totals came within 20 percent of Cloudflare’s own daily rollup. The thresholds already count only the views Cloudflare certainly saw. Do not buy a paid analytics plan to remove sampling from a site this size.
The scans. Thousands of requests a week ask for WordPress, PHP, hidden files and admin pages that the site does not have. Against static HTML there is nothing to break into. The one rate-limiting rule the site has guards the MCP server, which is the one path that runs code. Add no firewall rules for the scans; keep the site static.
The empty monthly tables. The page and country tables show nothing until October closes for counting on 4 November. That is the rule working as the privacy page promises, not a gap.
The build’s own time. The accessibility audit is the longest step in a run of about two and a half minutes. Nothing here is worth an hour of tuning.

What one to two weeks covers, and what it does not

Covered. Reading every published page about the system and every line of its collector, models, tests and workflows. Rebuilding the models locally from the raw files, which this system makes possible by design, and confirming the numbers match the page. Checking each promise on the privacy page against the test that enforces it. The findings, the order of work, the sizes in days, and a walk-through of this document with the owner.

Not covered. A security review of the Cloudflare and GitHub accounts beyond what the repository shows. Any check on the accuracy of Cloudflare’s own counts against a second analytics tool. Load or cost testing. The second website whose numbers the same collector reads, which has its own privacy page and would be its own audit. And any building: the audit sizes the work and leaves the owner free to do it with me or with anyone else.

In the client’s version, this last section also lists the hours spent and the people and systems the time went to, so the next audit can start from it.