Deepgram and AWS’s Forward Deployed Engineering team ported Deepgram’s Nova-3 speech-to-text model to AWS Trainium2 for offline batch applications, according to an AWS blog post. The cited use cases include contact-center analytics and medical transcription.

The teams began with standard PyTorch code and used the AWS Neuron software stack to compile the model for Trainium2. AWS says the compiled path ran the model’s operations on Trainium2 without detected CPU fallbacks. The encoder and decoder were compiled as separate static-shape graphs, while numerical accuracy was checked against CPU outputs and Deepgram’s reference Word Error Rate results.

The optimization work included pre-built and custom Neuron Kernel Interface kernels for operations including feed-forward layers and convolutions. Deepgram also developed a Rust runtime that calls ahead-of-time-compiled model files through the Neuron Runtime API. The final configuration used eight model replicas on one Amazon EC2 trn2.3xlarge instance.

For Deepgram’s offline, throughput-oriented batch workloads, the company and AWS report that this setup processed 11% to 70% more audio per dollar than the compute options Deepgram had been using. The measurement was based on instance pricing at the time of evaluation. The post reports a 7.5% throughput gain over a multiprocessing Python baseline at the throughput-maximizing batch size.

The results described apply to batch transcription workloads and do not establish performance across other deployment scenarios.