Steering the Band: Length, Melody, and Sampling

Take control of MusicGen in PyTorch — set clip length with max_new_tokens, tune do_sample, temperature, and guidance_scale, batch multiple prompts, and condition on a melody with musicgen-melody.

Series: Practical PyTorch: Running Models — Audio — Part 7 of 8

Last post you typed a vibe and got a track. That first run uses defaults for everything — how long the clip is, how closely it hugs your prompt, whether two runs of the same prompt sound the same. Defaults are fine for a demo. But the moment you want a specific clip — exactly ten seconds, a little wilder, a jazz cover of a tune you already have — you reach past the defaults and start turning knobs. MusicGen has four worth knowing, and this post is all four.

Open the companion notebook in Colab

Reload the band

We’re picking up where Part 6 left off, so let’s get processor and model back in memory. Same small checkpoint, same two lines:

from transformers import AutoProcessor, MusicgenForConditionalGeneration
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small").to(device)
sr = model.config.audio_encoder.sampling_rate

inputs = processor(text=["a warm acoustic guitar loop"], padding=True, return_tensors="pt").to(device)

This is GPU-leaning territory. Generation is the slow part, and a longer clip or a batch of prompts magnifies that. If you’re on a free Colab CPU, everything below still runs, it just makes you wait. Flip on a GPU (Runtime → Change runtime type → GPU) before you start cranking the knobs.

Lever 1 — length, in tokens

MusicGen doesn’t think in seconds; it thinks in audio tokens. It emits them at a fixed rate, roughly 50 tokens per second for this model, so clip length is just a token budget you set with max_new_tokens:

audio = model.generate(**inputs, max_new_tokens=512)   # ~10 s at ~50 tokens/sec

Want five seconds, set it to 256. Want fifteen, set it to 768. The arithmetic is honest: seconds × 50. There’s no length_seconds argument — max_new_tokens is the dial, and once you’ve internalized the 50-per-second rate you can size a clip in your head.

The catch is at the long end. MusicGen-small was trained on short clips, and asking for thirty seconds tends to drift: the groove wanders, the structure dissolves. Treat anything past ~15 seconds as a loop you’ll repeat rather than a through-composed piece.

Lever 2 — how it picks each token

This is the lever I most want you to recognize, because you’ve already met it. Back in the decoding and sampling post, a text model handed you a probability for every possible next token and you chose how to pick one: take the highest every time (greedy), or roll the dice weighted by those probabilities (sampling), with temperature controlling how loaded the dice are. MusicGen generates the exact same way, one token at a time — the tokens are just audio instead of words. The knobs are identical:

audio = model.generate(
    **inputs,
    do_sample=True,        # sample tokens instead of greedy argmax -> more variety
    temperature=1.0,       # higher = wilder
    guidance_scale=3.0,    # classifier-free guidance: how strongly to follow the prompt
    max_new_tokens=256,
)

Two of those three should feel like reruns:

  • do_sample=True swaps greedy argmax for weighted sampling. Greedy gives you the model’s single most-likely continuation every time — safe, repetitive, a little flat. Sampling introduces variety, which for music is usually what you want; a “safest possible note” generator makes dull music.
  • temperature scales how adventurous that sampling is. Around 1.0 is a sensible default. Push it up (1.2, 1.5) and the model takes bigger risks: more surprising, more likely to go off the rails. Pull it down toward 0.7 and it plays it safe.

The third knob, guidance_scale, is the new one and it’s worth a section of its own.

Lever 3 — guidance: how hard to obey the prompt

guidance_scale is classifier-free guidance — a single number for “how literally do you want me to follow the text.” Internally the model generates two ways at once, one paying attention to your prompt and one ignoring it, then leans toward the prompt by this factor. You don’t have to track the mechanics; track the feel:

  • High (say 7.0) — the model obeys your words hard. Every adjective in the prompt gets honored, but the result often comes out stiff, over-literal, less musical. It’s following instructions instead of making music.
  • Low (say 1.5) — the model treats your prompt as a loose suggestion and wanders toward whatever sounds good to it. Looser, often more musical, sometimes ignoring half of what you asked for.
  • The sweet spot is around 3.0, which is why it’s the default in every example here. Enough pull to stay on-topic, enough slack to stay musical.

If a clip sounds technically correct but lifeless, your first move is to lower guidance, not raise it. The instinct to crank it up so the model “listens better” is the classic mistake — past the sweet spot you get a more obedient, less listenable track.

Reproducing a take

Sampling is, by design, nondeterministic: run the same prompt twice and you get two different clips. That’s the point most of the time. But when you land on a take you love and want it back exactly, seed the random number generator first:

torch.manual_seed(0)
audio = model.generate(**inputs, do_sample=True, guidance_scale=3.0, max_new_tokens=256)

Same seed, same prompt, same knobs → the same clip, every time. Change the seed and you get a fresh roll of the dice. This is how you audition: fix the seed, sweep one knob, and you’re comparing apples to apples instead of chasing a moving target.

Lever 4 — batching several prompts

Generation is slow, and calling generate four times for four prompts pays that startup cost four times over. Hand the processor a list of prompts instead and the model generates them together, in parallel, in one call:

inputs = processor(
    text=["a tense cinematic build-up", "a calm acoustic guitar loop"],
    padding=True,
    return_tensors="pt",
).to(device)

audios = model.generate(**inputs, do_sample=True, guidance_scale=3.0, max_new_tokens=256)
# audios[0] and audios[1] are the two clips

The output is stacked: audios[0] is the first prompt’s clip, audios[1] the second, in the order you listed them. On a GPU this is meaningfully faster than a loop, because you’re filling the device with work instead of starting and stopping it.

The one thing that will bite you here is padding=True. Your two prompts tokenize to different lengths, and the model needs them squared off into one rectangular tensor; padding does that. Drop it and a batch of uneven prompts throws a shape error. Keep it on for every batch and you’ll never think about it again.

Lever 5 — melody conditioning (a different, heavier model)

So far we’ve steered with words and dice. The last lever steers with audio: hand the model a melody — something you hummed, played, or pulled off disk — and ask it to restyle that tune. “Here’s this folk melody, give it to me as jazz piano.” That’s style transfer for music.

The important caveat first: this is a different checkpoint. musicgen-small from Part 6 doesn’t accept audio. Melody conditioning lives in musicgen-melody, a separate, heavier model with its own processor and its own generation class. You’re loading new weights, not passing a new argument to the old model.

from transformers import AutoProcessor, MusicgenMelodyForConditionalGeneration
import torchaudio, torch

processor = AutoProcessor.from_pretrained("facebook/musicgen-melody")
model = MusicgenMelodyForConditionalGeneration.from_pretrained("facebook/musicgen-melody").to(device)

melody, msr = torchaudio.load(
    torchaudio.utils.download_asset("tutorial-assets/Lab41-SRI-VOiCES-src-sp0307-ch127535-sg0042.wav")
)

inputs = processor(
    audio=melody[0].numpy(),
    sampling_rate=msr,
    text=["jazzy piano version"],
    padding=True,
    return_tensors="pt",
).to(device)

audio = model.generate(**inputs, max_new_tokens=256)

Two new things go into the processor: audio= (the waveform to follow) and sampling_rate= (so it knows the source’s timeline — the same sample-rate bookkeeping from Part 1 of this series). The text= prompt now describes the style you want applied, not the whole piece. The model keeps the melodic contour and repaints everything around it.

One honesty note, and it’s the thing I’m least certain about in this whole post: the melody API is version-sensitive. The exact argument names and the melody class have shifted across transformers releases, and HuggingFace has reorganized these checkpoints more than once. If the import or the processor(...) call errors on your version, that’s the first place to look — check the model card and the transformers docs for your installed version rather than assuming the snippet above is eternal. The idea (load musicgen-melody, pass audio= plus a style text=) is stable; the precise spelling is not.

Gotchas

  • Guidance has a ceiling of usefulness. Cranking guidance_scale makes the model follow your words more literally but usually less musically — there’s a sweet spot around 3.0, and “it sounds stiff” means go lower, not higher.
  • Batching needs padding=True. Prompts of different lengths can’t pack into one tensor without padding; leave it off and an uneven batch throws a shape error.
  • musicgen-melody is not musicgen-small. It’s a separate, heavier checkpoint with its own class. Don’t expect the Part 6 model to take an audio= argument; it won’t.
  • Sampling is nondeterministic. Two runs of the same prompt differ on purpose. If you need a specific clip back, call torch.manual_seed(...) before generate.
  • max_new_tokens is seconds × ~50. There’s no length-in-seconds argument; do the multiplication, and don’t push much past ~15 seconds with the small model before the structure drifts.

Final thoughts

The mental shift here is small but real: MusicGen isn’t a slot machine you pull and accept. It’s an instrument with a handful of controls, and three of them — sampling, temperature, the whole greedy-versus-random business — are the same controls you learned for text generation, because under the hood it’s the same next-token loop wearing different clothes. Guidance is the one genuinely audio-flavored knob, and the counterintuitive lesson is that turning it down is usually what rescues a stiff clip. Learn the sweet spots once and you stop generating until you get lucky; you start generating on purpose.

Next: Part 8 — Score Your Own Scene: A Prompt-to-Soundtrack App — wrap all of this into a small tool that turns a scene description into a soundtrack.

Target keyword(s): musicgen guidance_scale, condition musicgen melody.

Comments