When you convert a file on BookConv, the upload finishes in a second or two — long before your book is actually ready. Here's what happens behind that live progress bar, and why a background worker is the reason big conversions never hold up your browser.
Why BookConv doesn't convert inside your upload
When you convert a file on BookConv, the bytes you send are validated and handed off almost immediately. The noticeable wait isn't in the upload — it's in the conversion, and BookConv deliberately runs that part somewhere else.
A small EPUB might finish in a blink, but a 10 MB scanned PDF can take a minute or more. Trying to do that inside the same request that uploaded the file would leave your tab spinning on an open connection, and it would let fifty simultaneous uploads fight over the same CPU until everything crawled.
So BookConv checks your upload, blocks DRM-protected files on the spot, and drops the real work onto a background queue. You get a job ID right away, and the heavy lifting happens off to the side, on its own schedule.
The handoff in one line
Upload, validate, enqueue, respond with a job ID. Everything expensive happens after the response has already gone out.
Inside the BookConv queue: Redis and BullMQ
The queue is BullMQ, a job queue that keeps its state in Redis. BookConv runs a single queue for every conversion. When you submit a file, one job is created carrying the source format, the target format, and a job ID you'll use to check progress.
Redis does double duty. Besides holding the queue, it backs BookConv's rate limiter. On the hosted service the free tier allows 10 MB per file and 5 conversions per hour, which stops one script from flooding the system with jobs. If Redis is briefly unreachable at submit time, the API still returns a job ID instead of failing — so you always get something you can poll. Both the Redis connection and the rate limits are set through environment variables, covered in the environment variables setup guide.
The worker: where Calibre actually runs
A queue with no consumer is just a list. BookConv's worker is the consumer — a loop that pulls jobs and runs the conversion with the server-side Calibre engine.
The worker is throttled on purpose. It runs only a handful of jobs at once, because Calibre is CPU-hungry and a few conversions done well beat twenty done badly. Each job gets its own scratch folder, which the worker deletes whether the job succeeded or failed, so a crash mid-conversion doesn't slowly fill the disk.
Rejecting bad files early
Before Calibre starts, the worker checks the file's magic bytes. An EPUB has to look like a real ZIP; a PDF must start with the PDF marker. Files that fail are rejected in milliseconds instead of wasting two minutes inside Calibre. DRMed uploads never even reach the worker — BookConv refuses them at upload time.
Watching progress on BookConv's live bar
Because the conversion finishes after your request returns, the browser needs a way to check in. That's the live progress bar you see on BookConv: it polls a status endpoint for your job and shows where things stand.
The status tells BookConv — and you — what it needs to know:
- status — queued, active, or completed
- progress — a percentage the worker updates as it works
- attempt — which retry you're on, if any
- eta — a rough estimate of seconds left
When the bar fills, your download link appears. Those links are temporary — BookConv deletes the file after a set period, so grab it as soon as it's ready. If a link expires before you do, the download troubleshooting guide walks through the usual causes. Lighter pairs like EPUB to MOBI finish fast; heavier ones like PDF to EPUB take longer because they have to rebuild structure the source never stored.
Retries, timeouts, and failure on BookConv
Some failures are just bad luck — a disk hiccup, a brief load spike. BookConv retries a failed job up to three times with exponential backoff, so the gap grows each round and gives a struggling system room to recover. The retry counter lives in the queue, so the number you see in the status is the real one.
Not every error deserves a retry. A corrupted file will be just as broken on attempt three, so BookConv maps those to a clear message — you get "this file appears to be damaged" instead of a stack trace. A hard two-minute timeout also caps each attempt, so a pathological file can't hold a worker slot hostage forever.
When a job is truly stuck, BookConv can fire an HMAC-signed webhook so your own system learns the result the moment a conversion finishes or fails.
Key Takeaways
- Asynchronous by design. Uploads return a job ID right away; conversion runs on a worker, so slow files never block your request.
- Redis and BullMQ coordinate it. One queue holds every job, and the same Redis backs the rate limiter that enforces BookConv's free-tier limits.
- Throttling keeps things fast. Limited concurrency means Calibre conversions stay predictable instead of starving each other.
- Links are temporary. BookConv deletes finished files after a period, so download as soon as the bar completes.
- Retries belong to the queue. Three attempts with backoff and clear error messages when a file is genuinely unrecoverable.