Invental/Review lessons/Background jobs when something fails

Background jobs when something fails

Self-requeueing jobs need their own limit

A job that re-enqueues itself needs its own attempt counter in the payload: a new job starts with fresh retries, so the queue's retry limit no longer protects you from a batch that always fails.

01 · Before and afterThe pattern

TypeScript
// before: on a stall, try again later from the same cursor, forever
await enqueue({ resumeFrom: cursor }, { delayMs: 120_000 });
// after: count stalls across runs, give up loudly
const stalled = madeNoProgress ? (payload.stalledRunCount ?? 0) + 1 : 0;
if (stalled >= MAX_STALLED_RUNS) throw new Error(`Backfill stuck at ${JSON.stringify(cursor)}`);
await enqueue({ resumeFrom: cursor, stalledRunCount: stalled }, { delayMs: 120_000 });

02 · Run it on your appThe check

Do any of your background jobs schedule a follow-up job of themselves? What stops that chain if the same step keeps failing?

If the honest answer is no or not sure, this is worth a look before it reaches production.

Want this checked on your code?

Tell us what you are building and what worries you. We match a senior reviewer to your code and confirm scope before anything starts.

— Get in touch
hi@invental.co ↗
— Or
— What to include

A link or short description of the code or system, what you want checked, your stack, and when you need the answer. No repository access is needed until scope is agreed.

— Or leave a note
We reply within one business day.