ChiPy: Chicago's Official Python User Group

ChiPy July Main Meeting - July 9, 2026, 6pm.
We will have food and beverages at 6pm, with talks starting at 6:30pm.
Hosted by: mHUB

Inference at Scale: Transcribing Millions of Insurance Calls with Whisper and Azure ML
By: Jimmy Scray
Experience Level: Intermediate
Length: 25 Minutes
Description:
Transcribing a few audio files with Whisper is easy; doing it for millions of recordings reliably and cost-effectively is another problem entirely. This talk digs into the Python code and infrastructure behind a large-scale speech-transcription platform built for the insurance industry, tracing how a notebook prototype grew into a distributed inference pipeline running across thousands of GPU workers. The focus is inference engineering rather than ML theory: benchmarking CPU vs. GPU workloads, maximizing throughput, orchestrating jobs with Azure ML, surviving spot-instance interruptions, and writing resilient code that recovers from failures and resumes automatically. Expect real benchmark results, architecture decisions, code examples, and lessons learned from production.