reportgen
Back to all posts

No Report Left Behind: How reportgen.io Guarantees Reliable PDF Delivery (Part 3)

July 19, 2026

This is Part 3 of a 3-part series on how reportgen.io works under the hood.

We started reportgen.io because a vendor silently failed 3,000 of our invoices and we found out from customers, not from the vendor. That origin story shaped one non-negotiable design rule:

You always find out what happened to your report.

Not "usually." Not "unless a worker crashed at a bad moment." Always. This post is about how that rule is actually engineered.

TL;DR

  • Every PDF generation runs as a durable Temporal workflow — a crashed worker or a mid-generation deploy resumes work instead of losing it.
  • Retries are calibrated per failure type: transient failures back off exponentially and retry; deterministic failures (template bugs) fail fast with the real error message.
  • A terminal-state guarantee ensures every report ends completed or error — the status update runs on every exit path, including validation failures, with its own extra-persistent retry policy.
  • Finished PDFs live in S3-compatible object storage and are delivered through short-lived signed URLs — storage is private by default.
  • Billing rides the same rails: a post-generation workflow counts usage and records Stripe billing meters with the same retry machinery — and only successful PDFs consume your free tier.
  • Reads are served through a Redis cache wrapping PostgreSQL repositories, so dashboard traffic never competes with the pipeline.

Durable Execution: The Foundation

As covered in Part 1, every generation is a Temporal workflow. Temporal persists workflow state at each step boundary, which buys a property most job-queue systems only approximate:

Progress survives infrastructure. If a worker pod dies after your HTML is rendered but before conversion, another worker picks up the workflow exactly where it left off. The render isn't redone; the report isn't lost; nobody writes a reconciliation cron. Deploys, node reboots, and pod rescheduling all become non-events for in-flight reports.

This is also why we can deploy in the middle of the day without a maintenance window: workflows in flight just... continue, on whichever worker is up.

Retries That Match Reality

Blanket retries are a beginner's reliability strategy — retrying a template syntax error five times just delays the bad news. Our retry policy is calibrated to the cause of each failure:

Transient failures retry with exponential backoff. Storage hiccups, network errors, browser flakes: first retry after a few seconds, doubling up to a capped interval, with a bounded number of attempts. In practice, most of these succeed on the second try and you never know it happened.

Deterministic failures never retry. Render errors (your template referenced a missing variable) and size-limit violations are marked non-retryable at the source. The workflow fails immediately, and the report's error message carries the actual underlying cause — the templating engine's own error text, not a generic "generation failed."

That last detail is a deliberate piece of engineering. Error messages travel from the failing activity through the workflow into the report record with the root cause preserved at every hop, because the human-readable message is the product when something goes wrong. We wrote a whole post on debugging PDF generation issues — being able to show you the real error is what makes that post short.

The Terminal-State Guarantee

Here's our favorite pattern in the codebase, and the direct answer to the silent-failure trauma that started the company.

The naive workflow shape looks like this: do the steps, and if all succeed, mark the report completed; in error handlers, mark it error. The bug in that shape is subtle: any exit path you forgot — an early return on a validation failure, a new error branch added a year later — leaves the report pending forever. Your customer sees a spinner that never resolves. That's the silent failure, reborn inside your own system.

Our workflow inverts the structure:

  1. The finalization step is registered before any other logic runs — before validation, before the first activity. It's a deferred step that executes on every exit path, success or failure. A code path that skips it is structurally impossible, not just unlikely.
  2. Finalization gets its own, more persistent retry policy. It's the last line of defense, so it retries harder and longer than ordinary steps. A transient database blip at the exact moment of finalization doesn't strand the report.
  3. Finalization decides the terminal state from the workflow's outcome: completed with a success message, or error with the preserved root cause.

The property that falls out: a report can be pending only while work is genuinely in progress. Every ending — success, template bug, invalid input, exhausted retries — converges to a terminal status you can see in the dashboard or query via the API. Combined with webhooks and async workflows, your system can react to every outcome instead of polling and praying.

Across 300,000+ PDFs in production, this pattern — durable execution plus calibrated retries plus guaranteed finalization — is what our 99.999% delivery success rate is made of. The remaining fraction isn't silent either: it's an error status with a reason attached.

Storage and Delivery

Finished PDFs are uploaded to S3-compatible object storage, keyed by report ID. Two decisions matter here:

Storage is private by default. There are no public bucket URLs. When you download a report — from the dashboard or via the API's download endpoint — we mint a short-lived signed URL scoped to exactly that file. Links expire; your documents don't float around the internet indefinitely because someone pasted a URL into Slack.

The API stays out of the data path. PDFs can be large, and streaming them through the API server would couple its memory and bandwidth to your document sizes. Signed URLs mean the bytes flow directly from object storage to you; the API only ever handles metadata.

Billing on the Same Rails

A reliability post that skips billing is hiding something, because billing is where inconsistency actually costs you money. Our answer: billing isn't a separate system bolted onto the pipeline — it's another workflow on the same rails.

When a generation workflow finalizes, it schedules a post-generation workflow that:

  1. Increments your usage counters (successful and failed generations are tracked separately).
  2. Checks your position against the free tier — your first 150 successful PDFs each month are free.
  3. Only past that threshold, records a usage event on a Stripe billing meter at $0.0025 per PDF.

Because it's a Temporal workflow with its own retry policy, a Stripe API blip means the meter event retries with backoff — it doesn't get lost, and it doesn't get double-recorded. And because the same workflow knows exactly how each report ended, usage counting stays honest: only successful PDFs consume your free tier. A report that errors is a support signal, not a line item.

The Supporting Cast

Redis caching. Dashboard and API reads (report lists, usage counters) are served through a caching layer that wraps the PostgreSQL repositories — same interface, cache-aside transparently, TTLs per data type. Hot-path reads don't contend with the write path of active generation.

Sentry everywhere. Both the Go services and the Node worker report to Sentry, with errors tagged by workflow ID. A weird conversion failure can be traced from the alert to the exact workflow execution — and Temporal's full event history for that execution — in one hop.

Workflow history as a debugging tool. Every generation's complete event history is inspectable: which activities ran, on which queue, how many attempts, what failed and why. "What happened to report X?" is a lookup, not an investigation.

What We'd Tell You to Steal

If you're building any multi-step pipeline — not just PDFs — the five ideas from this series we'd most recommend taking:

  1. Use durable execution for multi-step work. The orchestrator should remember progress so your workers don't have to.
  2. Calibrate retries to failure type. Deterministic failures fail fast with real messages; transient failures back off and retry silently.
  3. Register finalization before logic. Terminal state should be structurally guaranteed, not maintained by discipline across every error branch.
  4. Pass references, not files. Workflow history carries pointers; object storage carries artifacts.
  5. Put billing on the same reliability rails as the product. If your pipeline retries but your metering doesn't, you'll eventually have a very awkward customer email to write.

Frequently Asked Questions

What happens if a server crashes while my PDF is generating?

The workflow resumes on another worker from the last completed step. Durable execution means progress is persisted at every step boundary — a crash costs seconds of latency, not your report.

Can a reportgen.io report get stuck in "pending"?

No. Finalization runs on every workflow exit path — including validation failures and exhausted retries — so every report reaches completed or error. pending always means work is genuinely in progress.

Does reportgen.io charge for failed PDF generations?

Billing is based on successful generations: failed reports don't consume your free 150/month, and a report that errors shows up as an error status with a reason — not as a charge. If anything on your bill ever looks off, contact us and we'll make it right.

How long are generated PDFs accessible?

PDFs are stored privately in object storage and accessed through short-lived signed download links you can mint any time from the dashboard or API. The file persists; each link expires. You can also delete reports (and their files) whenever you like.

What's reportgen.io's actual delivery success rate?

Across 300,000+ production PDFs: 99.999% delivered successfully — and the exceptions end as explicit error statuses with the underlying reason attached, never as silence.

That's the series: the architecture, the rendering pipeline, and the reliability layer that ties it together. If it sounds like infrastructure you'd rather rent than build — get a free API key. 150 PDFs a month, no credit card, no silent failures.