Build a Chatbot You Can Talk To

Series: Practical PyTorch: Running Models — LLMs — Part 6 of 6

This is it — the finale the whole series has been walking toward. You’ve built a digit reader, shipped an image classifier, and stood up a semantic search engine. Now you’ll build the one everyone pictures when they hear “AI app”: a chatbot you can actually talk to, with a real chat window, a personality you set, memory across turns, and a link you can hand to a friend. It’s a few lines on top of everything you already know.

▶ Open the companion notebook in Colab

The whole bot is one function

By now the pattern is muscle memory: a model is a function, and an app is that function with a UI. For chat, the function takes the new message plus the conversation so far and returns a reply. We’ll use the instruct model and roles from Part 5.

from transformers import pipeline

chat = pipeline("text-generation", model="Qwen/Qwen2.5-0.5B-Instruct", device_map="auto")
SYSTEM = "You are a friendly, concise assistant."

def respond(message, history):
    messages = [{"role": "system", "content": SYSTEM}]
    messages += history                              # past turns, as role/content dicts
    messages.append({"role": "user", "content": message})
    out = chat(messages, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.9)
    return out[0]["generated_text"][-1]["content"]

Read it and every line is something you’ve already met: the system message sets behavior, history is the memory (Part 5), the user’s new turn goes on the end, and the sampling knobs are Part 4’s. The function builds the conversation, runs the model, returns the last message’s text.

A real chat window, free

Gradio has a component built exactly for this — ChatInterface — that gives you the whole messaging UI: scrollback, the text box, send button, even a typing indicator. You hand it your function; it builds the chat.

import gradio as gr

gr.ChatInterface(
    respond,
    type="messages",
    title="Chat with a 0.5B model",
    description="A tiny instruction-tuned model, running in your notebook.",
).launch(share=True)

Run that cell and a chat window appears in the notebook. Type a message, get a reply, keep the thread going — ChatInterface passes the growing history back to your respond function on every turn, which is how the bot “remembers.” And share=True, one last time, hands you a public link. You built a chatbot. Send it to someone.

That’s the series, closed: from not knowing what a tensor was to a working chatbot with a URL.

Make it yours

Two one-line changes turn the demo into your demo:

• Rewrite SYSTEM. “Reply only in haiku.” “You are a patient SQL tutor.” “Answer as a grumpy pirate.” The system prompt is the personality dial, and it’s free.
• Turn the knobs. Drop temperature toward 0.3 for focused, factual answers; push it toward 1.0 for playful ones. Same generation controls from Part 4.

Gotchas

• type="messages" matters. It tells ChatInterface to pass history as the list of {"role", "content"} dicts your function expects. (Older Gradio defaulted to (user, bot) tuples — if messages += history errors, that’s the version mismatch, and these samples are written against the documented API, not run at publish time.)
• Small model, small expectations. A 0.5B model is delightful for a demo and will still flub facts and lose hard threads. For better answers, swap in a larger instruct model — and lean on Part 1’s quantization to fit it on the free GPU.
• Long chats fill the context. Every turn resends the whole history; eventually it gets long and slow. For a demo it’s fine; production apps trim or summarize old turns.
• It can be confidently wrong. Same caveat as every generative model — fluent is not the same as correct. Don’t ship it as a source of truth.
• The share link is temporary. Great for “try my bot”; for something permanent, host it on Hugging Face Spaces (the Gotchas in the model-app capstone still apply).

What’s next — the real wrap

Look back at the ground you covered. You started not knowing what a tensor was, and you’re ending with four working apps and the skill behind them:

• A digit reader — your hand-built MLP, given trained weights and a face.
• An image classifier — a real CNN, wrapped and shareable.
• A semantic search engine — embeddings finding meaning, not keywords.
• A chatbot — an instruction-tuned model holding a conversation.

Along the way you learned to read a model like a spec sheet, run it end to end, feed it the right inputs, read its outputs, generate text, and put a UI on any of it — across vision, encoders, and decoders, and you got there without deriving a single gradient.

So where to from here? The Running Models series was about driving — running what already exists. The Training Models series lifts the hood: why a model does what it does, and how to fine-tune one on your own data so it learns your task. That’s where the calculus we cheerfully ignored — and the training we kept deferring — finally earns its keep. You’ll meet it from a position of strength, because you already know what a model is and what it feels like to ship one.

For now, though: open the notebook, build the bot, grab the link, and send it to someone. You earned the demo.

---

Target keyword(s): build a chatbot python, gradio chatinterface, qwen chatbot app, llm chat app beginner.