Narrator in a Box: Read Any Article Aloud

Build a text-to-speech app that narrates a whole article — split sentences, synthesize each with SpeechT5, stitch the clips, and wrap it in a Gradio UI.

Series: Practical PyTorch: Running Models — Audio — Part 4 of 8

Back in Part 2 you made SpeechT5 say a sentence, and I left you with a warning: hand it a paragraph and it truncates or slurs. That ceiling is the gap between a demo and a tool. This post closes it. By the end you’ll paste a whole article into a text box and get back a single audio clip that reads the thing top to bottom — a narrator you could hand to someone who has never opened a notebook in their life.

Open the companion notebook in Colab

The trick isn’t a bigger model. It’s a small, honest piece of plumbing around the model you already have: split the article into sentences, synthesize each one on its own, drop a beat of silence between them, and concatenate the lot into one waveform. That’s the whole idea. Let’s build it.

Load SpeechT5 once

Same installs as Part 2, plus gradio for the UI we’ll bolt on at the end:

!pip install -q transformers datasets soundfile gradio sentencepiece

We’ll skip the pipeline this time and load the three parts by hand — processor, model, vocoder — because the narrator calls the model in a loop, and the by-hand generate_speech is the cleanest way to do that. Load them once, up top, and reuse them for every sentence:

import torch, numpy as np
from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech, SpeechT5HifiGan
from datasets import load_dataset

processor = SpeechT5Processor.from_pretrained("microsoft/speecht5_tts")
model = SpeechT5ForTextToSpeech.from_pretrained("microsoft/speecht5_tts")
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")

embeddings = load_dataset("Matthijs/cmu-arctic-xvectors", split="validation")
speaker = torch.tensor(embeddings[7306]["xvector"]).unsqueeze(0)

If any of that looks unfamiliar, Part 2 walks each line — the three objects are the text encoder, the spectrogram step, and the vocoder, and speaker is the 256-number voice fingerprint the model conditions on. The important thing here is that we load it once. Downloading weights and parsing a dataset on every sentence would make a long article unbearable; loading up front means each sentence is just a forward pass.

The narrator: split, synthesize, stitch

Here’s the entire tool — one function:

import re

def narrate(article: str):
    sentences = re.split(r"(?<=[.!?])\s+", article.strip())
    clips = []
    for s in sentences:
        if not s:
            continue
        inputs = processor(text=s, return_tensors="pt")
        clip = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
        clips.append(clip.numpy())
        clips.append(np.zeros(int(0.3 * 16000)))   # 0.3 s pause between sentences
    return 16000, np.concatenate(clips)

Three moving parts, each earning its place:

  • The split. re.split(r"(?<=[.!?])\s+", ...) cuts the text wherever a sentence-ending punctuation mark is followed by whitespace. The (?<=...) is a lookbehind — it matches the gap after a ., !, or ? without eating the punctuation, so "Hello. World." becomes ["Hello.", "World."] with the periods intact. We split because SpeechT5 has a per-call length limit (the ceiling from Part 2); feed it one sentence at a time and every clip comes out clean.
  • The synth loop. For each sentence we run the exact processor → generate_speech pair from Part 2 and keep the resulting waveform. clip.numpy() turns the PyTorch tensor into a plain array so we can concatenate it later. The if not s: continue skips empty strings, which the split can leave behind on trailing whitespace.
  • The pause. np.zeros(int(0.3 * 16000)) is 4,800 samples of pure silence: 0.3 seconds at SpeechT5’s 16 kHz rate. We append one after every sentence. Without it the clips butt straight up against each other and the narration sounds rushed and breathless; the pause is what makes it read like a person who stops between thoughts. That sample count must be computed from the model’s 16,000 Hz rate, not from anything about the article — silence is measured in samples per second, same as the speech.

At the end, np.concatenate(clips) glues every speech clip and every silence gap into one long array, and we return (16000, that_array). Hold onto that return shape — (sample_rate, samples) — because in a minute it’s going to do double duty.

Try it

sr, audio = narrate(
    "Audio is just a tensor. A model turns text into a spectrogram. "
    "A vocoder turns that spectrogram into sound."
)
from IPython.display import Audio
Audio(audio, rate=sr)

Three sentences in, one clip out, with a small breath between each. Notice the input is one Python string — the two source lines are just adjacent string literals Python joins for you, not three separate calls. narrate did the splitting. Feed it a three-paragraph blog post and the same call hands back a clip a couple of minutes long; the function doesn’t care how many sentences there are.

Wrap it in a UI

A function in a notebook is a tool for you. A text box and a play button is a tool for anyone. Gradio turns the first into the second in five lines — you describe the inputs and outputs, hand it your function, and it builds the web UI and serves it:

import gradio as gr

gr.Interface(
    fn=narrate,
    inputs=gr.Textbox(lines=8, label="Paste an article"),
    outputs=gr.Audio(label="Narration"),
    title="Narrator in a Box",
).launch()

fn=narrate is the whole connection: Gradio calls your function with whatever’s in the textbox and renders whatever it returns. The textbox gives you an eight-line input area, and gr.Audio renders a player with a built-in download button.

And here’s why we were careful about that return value. gr.Audio expects exactly a (sample_rate, samples) tuple — which is precisely what narrate already returns. No adapter, no reshaping; the function we wrote for the notebook drops straight into the app because we picked its output shape to match. .launch() starts a local server and, in Colab, prints a public share link you can open on your phone or send to a friend. Paste an article, hit submit, get a narration. That’s the tool.

Gotchas

  • The split is the point — don’t skip it. Feeding the whole article to generate_speech in one call hits SpeechT5’s per-call ceiling and the output truncates or degrades. The sentence-at-a-time loop is what keeps every clip clean; it’s not premature optimization, it’s the thing that makes long text work at all.
  • Silence is samples, not seconds. np.zeros(int(0.3 * 16000)) uses the model’s 16 kHz rate. Compute the pause length from anything else — the text length, a hard-coded number you guessed — and your pauses come out the wrong duration or, worse, you concatenate arrays that sound sped-up or dragging.
  • CPU is slow on long input. Every sentence is a separate forward pass, so a long article on Colab’s CPU can take a real minute or two. Attach a GPU (Runtime → Change runtime type → GPU) or test with shorter text first; the math is unavoidable — n sentences means n passes.
  • Gradio is fussy about the audio format. gr.Audio wants (sr, samples) with the samples either float in [-1, 1] or int16. SpeechT5 already returns floats in range, so we’re fine — but if you ever stitch in audio from elsewhere, match that contract or the player will refuse it or play noise.

Final thoughts

This is the same arc the Running Models capstones walked, just in a new medium. There, three lines of pipeline("sentiment-analysis") became a feedback sorter you could hand to a coworker. Here, a research model that says one robotic sentence becomes an app that reads an article aloud — and the distance between those two things was never a better model. It was a sentence splitter, a few thousand zeros, and a text box. Most of the work of turning a model into a product is plumbing exactly this unglamorous, and that’s good news: you can already write it.

That’s the TTS half of the series put to bed. The voice is sorted. Next we change the question entirely — instead of turning text into speech, we’ll turn text into music. And to get there we first have to understand how a model represents sound it’s going to generate from scratch, which means cracking open the neural codec that the music models are built on.

Next: Part 5 — How Audio Becomes Tokens: Neural Codecs and Encodec — the trick that lets a language-model-shaped network generate sound, one token at a time.

Target keyword(s): text-to-speech app, gradio audio app, speecht5 narrate article.

Comments