⚡ Bolt: Offload blocking I/O to threads in FastAPI#367
Conversation
Replaces blocking synchronous operations (PDF generation, file system writes/reads, synchronous API requests) inside async endpoints with anyio.to_thread.run_sync() to prevent event loop starvation and improve API concurrency. Co-authored-by: anchapin <[email protected]>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
1 participant
💡 What: Replaced direct blocking function calls in
api/main.py(likegenerator.generate, file reads/writes, andanthropicsync client calls) withanyio.to_thread.run_sync(), usingfunctools.partialfor kwargs.🎯 Why: FastAPI runs
async defendpoints directly on the main async event loop. Since operations like PDF compilation via LaTeX and reading files are synchronous and heavily blocking, they starve the event loop, effectively reducing the API's concurrency to 1 request at a time.📊 Impact: Significantly increases throughput under load by freeing up the main event loop to handle concurrent API requests while worker threads process blocking jobs.
🔬 Measurement: Verify by running concurrent requests against the API endpoints; the server should now process incoming requests without timing out during heavy PDF generation operations. Confirm via
pytest tests/test_api_endpoints.py.PR created automatically by Jules for task 2849466650206990477 started by @anchapin