Skip to content
AdStack Engineering Docs
Esc
navigateopen⌘Jpreview
On this page

Workers

The agents Worker that owns every LLM call, and the pipeline Worker that owns every queue, cron, and media render.

Two Workers sit behind the app. Neither auto-deploys on merge. Both are deployed by hand, which is the single most common cause of “the code says X but production does Y”.

Agents Worker

cloudflare-agents/, deployed as adstack-agents. Every LLM call in the product goes through it. The app never calls Anthropic directly.

cd cloudflare-agents && bunx wrangler deploy --config wrangler.jsonc

The --config flag is required. Without it, wrangler picks up a root deploy config and targets the wrong worker.

Endpoints

Endpoint Purpose
POST /run Non-streaming completion
POST /stream Streaming completion, used by the chat loop and agents
POST /agents/offer-intelligence The offer analysis agent, scraping included
POST /tools/scrape Firecrawl scrape
POST /tools/screenshot Page screenshot via Browser Rendering
POST /tools/market-research Multi-source research scraping
POST /tools/page-structure Page structure extraction

/run and /stream accept both the legacy shape (system_prompt plus user_prompt) and a messages array. Prompt caching is applied with cache_control: ephemeral on the system message and the last tool definition.

Scrapers

cloudflare-agents/src/scrapers/ has purpose-built extractors for Amazon, Reddit, YouTube, Trustpilot, and Facebook Ads, plus a stealth helper and an accessibility-tree reader. These feed market research with real customer language rather than model assumptions.

The BYOK alias

The worker sends a cf-aig-byok-alias header naming the Anthropic provider key stored in the AI Gateway. The value is tryadstack-agents, defined once in lib/registry.ts as AI_GATEWAY_BYOK_ALIAS.

Pipeline Worker

workers/, deployed as adstack-pipeline (staging) and adstack-pipeline-prod (production). A Hono app plus queue consumers plus crons.

cd workers && bunx wrangler deploy

Queues

Six queues per environment, plus a dead letter queue. A Cloudflare queue allows only one consumer, so each environment owns a full set. The worker dispatches on the name suffix after stripping the adstack- or adstack-prod- prefix, which is why one codebase serves both.

Queue Job
image-submit Send an image generation task to KIE.ai
video-submit Send a text-to-video or reference video task
i2v-submit Dead. Image-to-video: the queue, its consumer, the /api/i2v/create route and i2v_generations all exist and have zero app-side callers
image-store Download the finished asset, put it in R2, update the row
video-store Same for video
offer-process Scrape an offer, then auto-chain research and persona
dlq Dead letter

Crons

Schedule Job
every minute Poll KIE.ai for pending tasks
every 5 minutes Clean up stuck videos and stuck background jobs, then reconcile unstored media
daily at 03 Delete unsaved images

The poll cron matters: KIE callbacks are not guaranteed, so polling is the reliable completion path and the callback is the fast path.

Why the storage reconcile exists

Poll marks a row Completed the moment KIE reports the asset, pointing at KIE’s URL, and only then enqueues the store that copies the bytes into R2. That ordering is deliberate, so a user sees their image the second it exists rather than after a download. It also makes a failed store invisible: the row says Completed, the asset opens fine today, and the provider URL expires days later with nothing logged.

Retries and the dead letter queue do not solve this. They stop the message being lost; they do nothing about a row that is already lying.

So crons/reconcile-storage.ts reconciles from the database rather than the queue: any Completed row whose URL is not ours gets re-queued. It is idempotent, because the store consumer no-ops on a URL already in R2. It is bounded at both ends, and the second bound is the one that is easy to forget:

  • 15-minute grace, so a store still working is never re-enqueued under itself
  • 7-day abandon, or a URL that can never be stored would be re-queued every five minutes forever, writing an admin log line each time
  • 25 rows per run, so a systemic failure re-queues steadily instead of in one flood

It applies to images as well as videos. The bug was originally filed as video-only; the two paths are identical in crons/poll.ts.

Bindings

R2 (adstack-media), Hyperdrive to the matching Postgres, and the six queue producer bindings. KIE_API_KEY lives on this worker, not the app worker. When a user brings their own key, it is forwarded through the queue message instead.

Deploy discipline

The app deploys itself

Push to staging and Cloudflare Workers Builds rebuilds adstack-staging. GitHub Actions is not the deploy path and never was.

The other two do not

Agents and pipeline Workers deploy only when you run wrangler. A merged PR that changes cloudflare-agents/ or workers/ has changed nothing in production until you deploy it.

Verify against live infra

A rename is done when the deployed infrastructure answers to the new name, not when the code compiles. Never park a code-versus-infra mismatch as a note.

Last updated on August 6, 2026

Was this page helpful?