From Notebooks to Production in 15 Minutes

An AI-powered sales-call analytics company already had the essential pieces of a valuable product. Its application accepted call recordings, sent them to an external transcription service, classified the resulting text, analyzed its sentiment and other attributes, and delivered reports through customer-facing dashboards and CRM workflows.

Its data science team had also developed several promising models. The problem was that these models still lived primarily in Jupyter notebooks, local files, and occasionally Google Colab. A data scientist could train a model and plot its results, but turning that experiment into an unattended, repeatable production process was a different discipline—one the young team had never been taught.

The assignment was therefore broader than deploying a model. We needed to design the processing architecture, introduce a safe delivery workflow, monitor both infrastructure and prediction quality, and leave the team able to evolve the system without depending on an external consultant.

Start with the latency the business actually needs

Before choosing infrastructure, I worked with the client to define the non-functional requirements. Did every model need to respond online? Could analyses run in parallel? Did later models depend on features created by earlier ones? How soon did a customer really need the result?

The answer changed the architecture. Customers mostly consumed aggregated statistics and reports. They wanted new calls processed within minutes—not milliseconds—and complete recordings arrived from the telephony system in batches. A fully online serving platform would have added cost and operational complexity without improving the customer experience.

The right MLOps architecture starts with the acceptable business delay, not with the most sophisticated infrastructure available.

We selected a hybrid between streaming and batch processing: frequent, event-like processing with minute-level latency and native support for batches. The first usable MVP components were available within the first month.

Object storage as the source of truth

S3-compatible object storage became the master store for audio recordings, transcripts, and derived artifacts. The rest of the system passed references to those objects and downloaded large files only when a processing step required them. PostgreSQL stored call metadata, processing state, and links between the original recording and its outputs.

We considered a dedicated message bus such as RabbitMQ or Kafka. Either could have provided a classical event-driven design, but would also have introduced another stateful component to deploy, monitor, scale, and support. A managed cloud Pub/Sub service reduced that operational burden but created tighter vendor lock-in.

Instead, frequently scheduled managed Airflow DAGs detected new object keys and processed the available calls in batches. S3-compatible storage kept the data layer portable, while managed Airflow handled orchestration and scaling without requiring the client to operate a separate messaging cluster.

One pipeline, two model-serving patterns

The pipeline first requested transcription, saved the text and metadata, and then passed the result through approximately five models responsible for different annotations and classifications.

Lightweight models ran in short-lived Docker containers launched by Airflow on Kubernetes. Heavier BERT-based models took too long to load for every call, so they remained available as persistent services. DAG tasks sent transcripts to those workers through REST APIs and used their responses in subsequent stages.

Keeping orchestration separate from model code made the system flexible. The Airflow DAG described dependencies, parallel branches, and customer-specific processing paths. Each model remained a reusable Python component that could run in a notebook, a script, a containerized task, or a service.

Design failure recovery before the first failure

Network timeouts and transient service failures were retried automatically. Calls that repeatedly failed were moved into a separate error area and surfaced through monitoring, giving data scientists the exact production examples that broke their code. Once the cause was corrected, those calls could be processed again.

Every stage persisted enough state to resume a call from that point. Components could therefore be updated independently: work accumulated before the paused stage, and the restarted component processed the backlog without losing calls already moving through other parts of the system.

Turn notebook code into deployable software

The most difficult change was not technical. It was helping the data science team adopt a different development model. Through short project-specific workshops, repository examples, and reusable templates, I showed how to move model logic into Python packages while continuing to call that same code from notebooks.

Code and notebooks moved into version control. Docker Compose templates let data scientists reproduce the relevant Airflow, object-storage, and integration environment locally. Creating a merge request deployed the change to staging, where the team could run tests and representative examples. Merging into the main branch built and deployed the production artifacts.

Once a model had been validated in a notebook and its reusable code was ready, it could reach staging in approximately 10–15 minutes. Production deployment took a similar amount of time after QA. Access controls required both staging and production deployments to go through CI/CD, eliminating manual execution against production data.

Monitor model quality, not only service uptime

Airflow provided operational status and alerts, which were routed into a shared Telegram channel for a simple, immediate view of processing failures. Grafana dashboards covered the other half of production monitoring: whether the models' predictions were still useful.

The client already sent a sample of calls through manual quality control. Scheduled DAGs compared those human labels with model predictions and refreshed quality metrics approximately once per day. When a metric dropped beyond its threshold, the team received an alert and could investigate the input data, code, or model. Fresh labels also fed regular retraining workflows.

Before this feedback loop, a weak production model could potentially remain unnoticed for weeks or even a month. The new process exposed degradation with a delay of only one or two days. The project did not claim a specific accuracy increase; it gave the team the much more fundamental ability to see, diagnose, and correct quality changes quickly.

A production platform the internal team could own

The complete engagement took under three months. The production workload was several thousand calls per day, and load tests demonstrated approximately ten times that volume without additional scaling. If demand increased further, the persistent model services could be replicated on Kubernetes and scaled elastically.

More importantly, the client's data scientists could now add new processing components themselves. They extended the DAG into parallel and conditional paths, enabling different customers to receive different analyses while keeping model code modular and independently testable.

Documentation was treated as a deliverable and tested on new team members. A recently hired data scientist could read how the infrastructure worked, gain access, use the templates, and begin integrating a model without relying on the person who had originally built the platform.

The strongest proof of successful knowledge transfer is not that the original team understands the system. It is that the next person can use it.