How to build a per-Worker spending limit on Cloudflare
Read usage from GraphQL, price it, and deploy a stub version when one script goes over. The rough design, with the parts that bit me.
If you want to roll your own spending limit, here's roughly how I'd do it (because it's how I did it).
- Run a Worker on a cron, every 5 minutes.
- Query the GraphQL analytics API for usage since the start of the UTC day, or the last hour if that's earlier (otherwise your hourly window breaks right after midnight). Use 5-minute buckets. One query can cover Workers, Durable Objects, D1, KV, R2, Queues and more.
- Price it. I use list prices and ignore the included allowances on purpose, so the estimate runs high and a runaway trips early.
- Map each resource to the scripts that use it. This is the annoying part. A Durable Object namespace belongs to the script that defines the class. A D1 database or KV namespace gets charged to every script with a binding to it.
- If a script is over the hourly or daily limit, upload a stub version of it and deploy that. HTTP handlers return 503, cron and alarm handlers do nothing, and queue handlers call
retryAll(). An empty queue handler acks the batch and you lose the messages. - If the script has Workflows, terminate its queued, running and waiting instances too. A deploy doesn't stop them. workerfuse does up to 100 per status per run, and later runs pick up the rest while the script stays over its limit.
- Email yourself the rollback command.
Things that bit me
- New scripts and namespaces show up as unknown in analytics for about 20 to 25 minutes. During that window you can alert, but you can't tell who to pause.
- The stub has to export the same Durable Object classes, or the deploy fails. Exporting them keeps the stored data too.
- Only count usage after the script's latest deploy. Otherwise you fix the bug, roll back, and the guard pauses you again for the old usage. Ask me how I know.
- The guard runs on Cloudflare. If Cloudflare is down, so is the guard. Not much to do about that one.
Most of the guard logic is one TypeScript file. If you'd rather not maintain it, workerfuse is this design, and you can self-host it or let us run it.