Text of docs.pytorch.org › audio › 2.1 › tutorials › asr_inference_with_cuda_ctc_decoder_tuto… as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

ASR Inference with CUDA CTC Decoder — Torchaudio 2.1.2 documentation

Get Started Ecosystem Mobile Blog Tutorials Docs PyTorch torchaudio torchtext torchvision torcharrow TorchData TorchRec TorchServe TorchX PyTorch on XLA Devices Resources About Learn about PyTorch’s features and capabilities PyTorch Foundation Learn about the PyTorch foundation Community Join the PyTorch developer community to contribute, learn, and get your questions answered. Community Stories Learn how our community solves real, everyday machine learning problems with PyTorch.

Developer Resources Find resources and get questions answered Events Find events, webinars, and podcasts Forums A place to discuss PyTorch code, issues, install, research Models (Beta) Discover, publish, and reuse pre-trained models GitHub Table of Contents Docs > ASR Inference with CUDA CTC Decoder > Old version (stable) Shortcuts tutorials/asr_inference_with_cuda_ctc_decoder_tutorial Run in Google Colab Colab Download Notebook Notebook View on GitHub GitHub Note Click here to download the full example code ASR Inference with CUDA CTC Decoder ¶ Author : Yuekai Zhang This tutorial shows how to perform speech recognition inference using a CUDA-based CTC beam search decoder.

We demonstrate this on a pretrained Zipformer model from Next-gen Kaldi project. Overview ¶ Beam search decoding works by iteratively expanding text hypotheses (beams) with next possible characters, and maintaining only the hypotheses with the highest scores at each time step. The underlying implementation uses cuda to acclerate the whole decoding process A mathematical formula for the decoder can be found in the paper , and a more detailed algorithm can be found in this blog .

Running ASR inference using a CUDA CTC Beam Search decoder requires the following components Acoustic Model: model predicting modeling units (BPE in this tutorial) from acoustic features BPE Model: the byte-pair encoding (BPE) tokenizer file Acoustic Model and Set Up ¶ First we import the necessary utilities and fetch the data that we are working with import torch import torchaudio print ( torch . __version__ ) print ( torchaudio . __version__ ) 2.1.1 2.1.2 import time from pathlib import Path import IPython import sentencepiece as spm from torchaudio.models.decoder import cuda_ctc_decoder from torchaudio.utils import download_asset We use the pretrained Zipformer model that is trained on the LibriSpeech dataset .

The model is jointly trained with CTC and Transducer loss functions. In this tutorial, we only use CTC head of the model. def download_asset_external ( url , key ): path = Path ( torch . hub . get_dir ()) / "torchaudio" / Path ( key ) if not path . exists (): path . parent . mkdir ( parents = True , exist_ok = True ) torch . hub . download_url_to_file ( url , path ) return str ( path ) url_prefix = "https://huggingface.co/Zengwei/icefall-asr-librispeech-pruned-transducer-stateless7-ctc-2022-12-01" model_link = f " { url_prefix } /resolve/main/exp/cpu_jit.pt" model_path = download_asset_external ( model_link , "cuda_ctc_decoder/cpu_jit.pt" ) 0%| | 0.00/269M [00:00<?, ?B/s] 12%|#2 | 32.7M/269M [00:00<00:00, 343MB/s] 24%|##4 | 65.4M/269M [00:00<00:00, 331MB/s] 39%|###9 | 106M/269M [00:00<00:00, 372MB/s] 54%|#####4 | 146M/269M [00:00<00:00, 391MB/s] 68%|######8 | 183M/269M [00:00<00:00, 388MB/s] 82%|########1 | 220M/269M [00:00<00:00, 310MB/s] 94%|#########3| 252M/269M [00:00<00:00, 299MB/s] 100%|##########| 269M/269M [00:00<00:00, 324MB/s] We will load a sample from the LibriSpeech test-other dataset. speech_file = download_asset ( "tutorial-assets/ctc-decoding/1688-142285-0007.wav" ) waveform , sample_rate = torchaudio . load ( speech_file ) assert sample_rate == 16000 IPython . display .

Audio ( speech_file ) 0%| | 0.00/441k [00:00<?, ?B/s] 100%|##########| 441k/441k [00:00<00:00, 13.6MB/s] Your browser does not support the audio element. The transcript corresponding to this audio file is i really was very much afraid of showing him how much shocked i was at some parts of what he said Files and Data for Decoder ¶ Next, we load in our token from BPE model, which is the tokenizer for decoding. Tokens ¶ The tokens are the possible symbols that the acoustic model can predict, including the blank symbol in CTC.

In this tutorial, it includes 500 BPE tokens. It can either be passed in as a file, where each line consists of the tokens corresponding to the same index, or as a list of tokens, each mapping to a unique index. # tokens < blk > < sos / eos > < unk > S _THE _A T _AND ... bpe_link = f " { url_prefix } /resolve/main/data/lang_bpe_500/bpe.model" bpe_path = download_asset_external ( bpe_link , "cuda_ctc_decoder/bpe.model" ) bpe_model = spm .

SentencePieceProcessor () bpe_model . load ( bpe_path ) tokens = [ bpe_model . id_to_piece ( id ) for id in range ( bpe_model . get_piece_size ())] print ( tokens ) 0%| | 0.00/239k [00:00<?, ?B/s] 100%|##########| 239k/239k [00:00<00:00, 136MB/s] ['<blk>', '<sos/eos>', '<unk>', 'S', '▁THE', '▁A', 'T', '▁AND', 'ED', '▁OF', '▁TO', 'E', 'D', 'N', 'ING', '▁IN', 'Y', 'M', 'C', '▁I', 'A', 'P', '▁HE', 'R', 'O', 'L', 'RE', 'I', 'U', 'ER', '▁IT', 'LY', '▁THAT', '▁WAS', '▁', '▁S', 'AR', '▁BE', 'F', '▁C', 'IN', 'B', '▁FOR', 'OR', 'LE', "'", '▁HIS', '▁YOU', 'AL', '▁RE', 'V', '▁B', 'G', 'RI', '▁E', '▁WITH', '▁T', '▁AS', 'LL', '▁P', '▁HER', 'ST', '▁HAD', '▁SO', '▁F', 'W', 'CE', '▁IS', 'ND', '▁NOT', 'TH', '▁BUT', 'EN', '▁SHE', '▁ON', 'VE', 'ON', 'SE', '▁DE', 'UR', '▁G', 'CH', 'K', 'TER', '▁AT', 'IT', '▁ME', 'RO', 'NE', 'RA', 'ES', 'IL', 'NG', 'IC', '▁NO', '▁HIM', 'ENT', 'IR', '▁WE', 'H', '▁DO', '▁ALL', '▁HAVE', 'LO', '▁BY', '▁MY', '▁MO', '▁THIS', 'LA', '▁ST', '▁WHICH', '▁CON', '▁THEY', 'CK', 'TE', '▁SAID', '▁FROM', '▁GO', '▁WHO', '▁TH', '▁OR', '▁D', '▁W', 'VER', 'LI', '▁SE', '▁ONE', '▁CA', '▁AN', '▁LA', '▁WERE', 'EL', '▁HA', '▁MAN', '▁FA', '▁EX', 'AD', '▁SU', 'RY', '▁MI', 'AT', '▁BO', '▁WHEN', 'AN', 'THER', 'PP', 'ATION', '▁FI', '▁WOULD', '▁PRO', 'OW', 'ET', '▁O', '▁THERE', '▁HO', 'ION', '▁WHAT', '▁FE', '▁PA', 'US', 'MENT', '▁MA', 'UT', '▁OUT', '▁THEIR', '▁IF', '▁LI', '▁K', '▁WILL', '▁ARE', 'ID', '▁RO', 'DE', 'TION', '▁WA', 'PE', '▁UP', '▁SP', '▁PO', 'IGHT', '▁UN', 'RU', '▁LO', 'AS', 'OL', '▁LE', '▁BEEN', '▁SH', '▁RA', '▁SEE', 'KE', 'UL', 'TED', '▁SA', 'UN', 'UND', 'ANT', '▁NE', 'IS', '▁THEM', 'CI', 'GE', '▁COULD', '▁DIS', 'OM', 'ISH', 'HE', 'EST', '▁SOME', 'ENCE', 'ITY', 'IVE', '▁US', '▁MORE', '▁EN', 'ARD', 'ATE', '▁YOUR', '▁INTO', '▁KNOW', '▁CO', 'ANCE', '▁TIME', '▁WI', '▁YE', 'AGE', '▁NOW', 'TI', 'FF', 'ABLE', '▁VERY', '▁LIKE', 'AM', 'HI', 'Z', '▁OTHER', '▁THAN', '▁LITTLE', '▁DID', '▁LOOK', 'TY', 'ERS', '▁CAN', '▁CHA', '▁AR', 'X', 'FUL', 'UGH', '▁BA', '▁DAY', '▁ABOUT', 'TEN', 'IM', '▁ANY', '▁PRE', '▁OVER', 'IES', 'NESS', 'ME', 'BLE', '▁M', 'ROW', '▁HAS', '▁GREAT', '▁VI', 'TA', '▁AFTER', 'PER', '▁AGAIN', 'HO', 'SH', '▁UPON', '▁DI', '▁HAND', '▁COM', 'IST', 'TURE', '▁STA', '▁THEN', '▁SHOULD', '▁GA', 'OUS', 'OUR', '▁WELL', '▁ONLY', 'MAN', '▁GOOD', '▁TWO', '▁MAR', '▁SAY', '▁HU', 'TING', '▁OUR', 'RESS', '▁DOWN', 'IOUS', '▁BEFORE', '▁DA', '▁NA', 'QUI', '▁MADE', '▁EVERY', '▁OLD', '▁EVEN', 'IG', '▁COME', '▁GRA', '▁RI', '▁LONG', 'OT', 'SIDE', 'WARD', '▁FO', '▁WHERE', 'MO', 'LESS', '▁SC', '▁MUST', '▁NEVER', '▁HOW', '▁CAME', '▁SUCH', '▁RU', '▁TAKE', '▁WO', '▁CAR', 'UM', 'AK', '▁THINK', '▁MUCH', '▁MISTER', '▁MAY', '▁JO', '▁WAY', '▁COMP', '▁THOUGHT', '▁STO', '▁MEN', '▁BACK', '▁DON', 'J', '▁LET', '▁TRA', '▁FIRST', '▁JUST', '▁VA', '▁OWN', '▁PLA', '▁MAKE', 'ATED', '▁HIMSELF', '▁WENT', '▁PI', 'GG', 'RING', '▁DU', '▁MIGHT', '▁PART', '▁GIVE', '▁IMP', '▁BU', '▁PER', '▁PLACE', '▁HOUSE', '▁TH

The rest of this page is on the live site.