Deployments: From .serve() to Production

A flow you can run is a script. A deployment is the server-side object that turns it into something the platform schedules and triggers for you.

Everything so far has been a script you run — call the flow, watch it execute, read the logs. That’s the right way to develop and exactly the wrong way to operate. Production wants a flow that runs on a schedule you don’t babysit, that someone can trigger from a UI or an API without your laptop being open, that has a stable identity the platform can attach runs and history to. In Prefect that identity is a deployment, and creating one is the line between “a thing I run” and “a workflow the platform runs.”

Airflow schedules everything; Prefect makes you ask

This is a real reversal worth sitting with. Drop a DAG file in Airflow’s dags/ folder and the scheduler finds it, parses it, and — subject to the schedule you set — starts scheduling it. Scheduling is the default; the folder is the registration. A @flow-decorated function has no such ambient home. Import it, call it, and it runs once, right there, and stops. Nothing is watching for the next 6 a.m. Prefect never scans a directory and never assumes your flow wants a schedule.

You make it schedulable by deploying it, explicitly. The upside of the extra step is that there’s no accidental scheduling, no half-finished flow going live because it landed in the wrong folder, and no scheduler that has to stay alive parsing every file to keep the lights on. The cost is one deliberate action per flow you want the platform to own.

The one-liner: serve()

The fastest way to a scheduled flow is serve():

from flows import daily_load

if __name__ == "__main__":
    daily_load.serve(name="bookshop-daily", cron="0 6 * * *")

Run that and two things happen. First, a deployment named bookshop-daily is registered on the server — it shows up in the UI, it has a schedule, and anyone can trigger a run of it from there or via the API. Second, this process becomes a runner: it stays up, polls for runs the schedule creates, and executes them right here, in this process. Leave it running — under systemd, in a container, wherever — and the bookshop load fires every morning at six without you touching it.

That’s a genuinely complete first deployment. No work pool, no image build, no extra infrastructure — the schedule lives on the server, the execution lives in the process you launched. serve() executes runs in the process you started: that’s its strength, dead simple, and its ceiling. The flow runs where you launched it, with that machine’s Python environment. If the box dies, the runner dies with it; if two flows need conflicting dependencies, or a run needs more memory than the host has, serve() has no answer. It’s the right tool for a handful of flows on a stable box, and the wrong one for a fleet.

Three ways to deploy, one deployment

When you outgrow a single process, the schedule and the execution have to come apart. You deploy the flow to a work pool, and separate workers pull runs and provision fresh infrastructure per run — the whole subject of the next post. What matters here is the choice you make when you create the deployment: how does the worker get your code? There are three answers, and they’re the real fork in the road.

The first is serve(), above — the worker is your process, the code is already imported, nothing is fetched. Single-host, tied to that process’s lifetime.

The second is bake the code into an image. deploy() builds a Docker image with your flow inside it, pushes it to a registry, and the deployment records the image tag. Every run pulls that image and runs the pinned code:

from flows import daily_load

if __name__ == "__main__":
    daily_load.deploy(
        name="bookshop-daily",
        work_pool_name="k8s-pool",
        image="registry.example.com/bookshop:1.4.0",
    )

The code and its dependencies travel together, immutably, which is exactly what you want for a Kubernetes pool where each run is a fresh pod. The cost is a build-and-push on every change.

The third — the one this chapter leans on — is pull the code from source at run time, no image bake. flow.from_source() points the deployment at a git repository and an entrypoint; the worker clones the repo when a run starts and imports the flow from it:

from prefect import flow

if __name__ == "__main__":
    flow.from_source(
        source="https://github.com/bookshop/pipelines.git",
        entrypoint="flows/daily_load.py:daily_load",
    ).deploy(
        name="bookshop-daily",
        work_pool_name="process-pool",
    )

Note what’s not there: no image. from_source can take a git URL, a Prefect storage block, or a local path; entrypoint is path/to/file.py:flow_function relative to the repo root. At deploy time Prefect stores the source location and entrypoint on the deployment — it does not fetch anything now. When a run comes due, the worker clones (or pulls) the repo, imports the flow, and runs the current code on the referenced branch. Push a fix to main, and the next scheduled run picks it up with no rebuild, no re-deploy. That is the appeal, and — as we’ll see — the trap, of pull-from-source.

Three transports, one concept. The deployment is always the same thing: a schedulable, triggerable, server-side identity for your flow. What differs is where the Python lives when a run fires — in the runner’s own process, in a pinned image, or in a repo the worker clones on demand.

Parameters: deployments are callable, with defaults

A deployment isn’t just a schedule attached to a function — it’s a bound call. Your flow signature has parameters; the deployment can pin default values for them, and any run can override them.

from prefect import flow

@flow
def daily_load(source: str = "s3://raw/bookshop", full_refresh: bool = False):
    ...

if __name__ == "__main__":
    daily_load.serve(
        name="bookshop-daily",
        cron="0 6 * * *",
        parameters={"source": "s3://raw/bookshop", "full_refresh": False},
    )

The parameters= dict sets the defaults this deployment runs with. Scheduled runs use them as-is. A manual run from the UI or CLI can override any of them for that run only — prefect deployment run 'daily_load/bookshop-daily' -p full_refresh=true fires a one-off full refresh without touching the schedule’s defaults. This is the clean version of the thing Airflow does with params and dag_run.conf: same flow, different inputs per run, without editing the definition.

Because your flow is type-annotated, Prefect builds a JSON-schema for its parameters and, with enforce_parameter_schema=True (the default for deployments created via deploy/prefect.yaml), validates every run’s inputs against it before the run starts. Pass a string where the flow wants an int, or omit a required argument, and the run is rejected up front with a clear error rather than blowing up three tasks deep. The UI reads the same schema to render a typed form for manual runs. If you have a parameter Prefect can’t introspect — an exotic type, a positional-only quirk — set enforce_parameter_schema=False to opt that deployment out.

To trigger a deployment from another flow — the orchestrator-of-orchestrators pattern — use run_deployment:

from prefect import flow
from prefect.deployments import run_deployment

@flow
def nightly_orchestrator():
    load = run_deployment(
        name="daily_load/bookshop-daily",
        parameters={"full_refresh": True},
    )
    run_deployment(
        name="dbt_build/bookshop-transform",
        parameters={"upstream_run_id": str(load.id)},
    )

run_deployment creates a flow run of the target deployment — scheduled onto its work pool, run by its workers, with the parameters you pass — and by default waits for it to finish, returning the FlowRun. It’s how one deployment invokes another as a subroutine while each keeps its own infrastructure, and it’s the Prefect equivalent of Airflow’s TriggerDagRunOperator, minus the cross-DAG dependency bookkeeping.

Schedules, past the one cron string

cron="0 6 * * *" is the training-wheels form. The real object is a schedule, and a deployment can carry a list of them.

from prefect.schedules import Cron, Interval, RRule
from datetime import timedelta, datetime

daily_load.serve(
    name="bookshop-daily",
    schedules=[
        Cron("0 6 * * *", timezone="America/New_York"),
        Cron("0 18 * * 1-5", timezone="America/New_York"),
    ],
)

Three schedule types cover the field. Cron is wall-clock times with an optional timezone — and the timezone matters, because a bare cron string runs in UTC and will drift an hour off your business day twice a year. Interval is a fixed cadence — Interval(timedelta(hours=6), anchor_date=datetime(2026, 1, 1, 0, 0), timezone="UTC") — where anchor_date fixes the phase so “every 6 hours” lands at 00:00/06:00/12:00/18:00 rather than at whatever minute you happened to deploy. RRule is the iCalendar recurrence rule for calendar logic cron can’t state — last business day of the month, third Thursday, every other week — e.g. RRule("FREQ=MONTHLY;BYDAY=-1MO").

Multiple schedules on one deployment is the feature Airflow made you fake with a second DAG or a branching timetable. Here, a “6 a.m. plus a weekday 6 p.m. top-up” pipeline is two Cron entries on the same deployment, so both fire runs of the same flow with the same parameters and share one run history. Each schedule can be independently active or paused — prefect deployment schedule pause / resume, or active: false in YAML — so you can silence the evening run during a freeze without deleting it. Pausing a schedule stops new runs from being created; runs already scheduled or in flight are unaffected.

Event-driven runs: deployment triggers

Schedules answer “run at these times.” Some pipelines want “run when this happens” — a file lands, an upstream deployment succeeds, an external system emits an event. Prefect’s answer is a deployment trigger, declared right on the deployment in prefect.yaml:

deployments:
  - name: bookshop-transform
    entrypoint: flows/dbt_build.py:dbt_build
    work_pool:
      name: process-pool
    triggers:
      - type: event
        enabled: true
        match:
          prefect.resource.id: "prefect.flow-run.*"
        expect:
          - prefect.flow-run.Completed
        match_related:
          prefect.resource.id: "prefect.deployment.<daily-load-deployment-id>"
        parameters:
          triggered_by: "{{ event.resource.id }}"

That trigger says: when a flow run of the daily_load deployment reaches Completed, start a run of bookshop-transform. It’s an automation scoped to this deployment and defined alongside it, so the “load, then transform” chain is expressed as a reaction to an event rather than as a wired dependency in a single DAG. Triggers can match on any event in Prefect’s event stream — including custom events you emit and events from integrations — and pass values from the event into the run’s parameters via {{ event.* }} templating. This is the event-driven half of orchestration that Airflow only grew recently with Assets; in Prefect it’s a first-class deployment field.

The distinction from run_deployment is worth drawing. run_deployment is a push — the upstream flow explicitly calls the downstream one, so the dependency is written into the caller’s code and the caller waits. A trigger is a pull — the downstream deployment declares what it reacts to, and the upstream flow neither knows nor cares that anything watches its completion. Push when the orchestration is one flow’s job and you want it to block on the result; use a trigger when the coupling should live with the consumer and the producer should stay oblivious. Real pipelines mix both: a nightly flow that pushes its immediate children with run_deployment, and a reporting deployment that quietly triggers itself off the whole batch reaching Completed.

Serving several flows from one process

serve() on a single flow is the common case, but a small project often has a fistful of flows that don’t each deserve their own runner host. to_deployment() plus the top-level serve() runs several from one process:

from prefect import serve
from flows import daily_load, hourly_sync, weekly_report

if __name__ == "__main__":
    serve(
        daily_load.to_deployment(name="bookshop-daily", cron="0 6 * * *"),
        hourly_sync.to_deployment(name="bookshop-sync", interval=3600),
        weekly_report.to_deployment(
            name="bookshop-report", cron="0 8 * * 1", parameters={"format": "pdf"}
        ),
    )

flow.to_deployment(...) builds a deployment object without serving it — same arguments as serve(), but deferred. The module-level serve(d1, d2, d3) then registers all of them and runs one poller that executes runs for any of the three. One process, one thing to keep alive, three independently scheduled deployments. It’s the sweet spot between “a serve() per flow, each its own process to babysit” and “stand up a work pool and workers” — ideal for a handful of related flows on one trusted box.

prefect.yaml, in full

serve() and deploy() are Python. For anything that lives in CI, you want the declarative form: a prefect.yaml at the repo root, deployed with prefect deploy. Here is the whole anatomy, not a fragment:

# prefect.yaml
name: bookshop
prefect-version: 3.4.0

# Build a Docker image (omit entirely for pull-from-source deployments)
build:
  - prefect_docker.deployments.steps.build_docker_image:
      id: build-image
      requires: prefect-docker
      image_name: registry.example.com/bookshop
      tag: "{{ get-commit-hash.stdout }}"
      dockerfile: Dockerfile

# Push that image to the registry
push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker
      image_name: "{{ build-image.image_name }}"
      tag: "{{ build-image.tag }}"

# How a worker gets the code at run time
pull:
  - prefect.deployments.steps.git_clone:
      repository: https://github.com/bookshop/pipelines.git
      branch: main

deployments:
  - name: bookshop-daily
    entrypoint: flows/daily_load.py:daily_load
    work_pool:
      name: k8s-pool
      job_variables:
        image: "{{ build-image.image }}"
        env:
          BOOKSHOP_ENV: production
    parameters:
      full_refresh: false
    schedules:
      - cron: "0 6 * * *"
        timezone: America/New_York
    tags: ["bookshop", "etl"]
    version: "{{ get-commit-hash.stdout }}"
    concurrency_limit: 1

Three step lists run in order at deploy time. build produces artifacts — most often a Docker image; each step can expose outputs (here build-image.image) that later steps and deployments reference. push ships those artifacts somewhere workers can reach — a registry, object storage. pull is the one that runs at the start of every flow run, on the worker: it’s the recipe for getting your code in place, most commonly git_clone (pull-from-source) or set_working_directory (code already baked into the image). If you’re deploying pull-from-source with no image, you delete build and push entirely and keep only pull.

The deployments list is where each deployment’s identity lives: entrypoint, work_pool (with per-deployment job_variables that override the pool’s base job template — image, env vars, CPU/memory, whatever the pool type exposes), parameters, schedules, triggers, tags, version, and concurrency_limit. The {{ }} templating pulls from three sources: step outputs ({{ build-image.tag }}), Prefect blocks and variables ({{ prefect.blocks.secret.my-token }}), and environment variables — so one YAML file parameterizes cleanly across environments. A sibling .prefectignore (same syntax as .gitignore) controls what’s excluded when code is uploaded to storage, keeping .venv/, data files, and secrets out of what ships.

job_variables is the seam most people underuse. A work pool ships a base job template — the full spec for how that pool’s runs are configured, with defaults for image, environment, resource requests, service accounts, and so on. job_variables on a deployment overrides just the keys you name, per deployment, without editing the pool. That’s how one Kubernetes pool serves a memory-hungry backfill deployment (job_variables: {memory: "8Gi"}) and a lightweight sync (the pool default) side by side, and how the same YAML targets staging versus production by resolving {{ }} env vars into different image tags and BOOKSHOP_ENV values. The pool defines the shape; each deployment perturbs only what it must. Reach for job_variables before you consider a second work pool — most of the time “these runs need more memory” or “these runs need this env var” is a job-variable override, not new infrastructure.

Deploy from the file with the CLI:

prefect deploy --all              # every deployment in the file
prefect deploy -n bookshop-daily  # just this one, by name

Wiring it into CI

Because prefect deploy is declarative and non-interactive when the YAML is complete, promotion becomes a CI job. A GitHub Actions workflow that re-deploys on every push to main:

# .github/workflows/deploy.yml
name: Deploy Prefect flows
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt prefect-docker
      - name: Authenticate to Prefect Cloud
        env:
          PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }}
          PREFECT_API_URL: ${{ secrets.PREFECT_API_URL }}
        run: prefect deploy --all

The job checks out the repo, installs dependencies, authenticates to Prefect Cloud (or your self-hosted server) with an API key and URL from GitHub secrets, and runs prefect deploy --all. Every merge re-registers the deployments with a fresh version — commonly the commit SHA via a get-commit-hash build step — so the UI shows exactly which commit each deployment is running, and a bad deploy is a revert-and-merge away. This is the payoff of the declarative form: deployments are code, reviewed in PRs and shipped by the same pipeline as everything else.

The decision that actually matters: image vs. source

Everything above circles one operational choice, so state it plainly. Bake code into an image and every run is immutable and reproducible: the deployment pins a tag, the tag pins the code and its dependencies, and a run from six months ago can be reproduced exactly. The cost is a build-and-push in the loop — every change is a new image before it’s a new run — and a registry to feed. This is the right default for Kubernetes and Docker pools, anything multi-tenant, and anywhere “what code ran?” must have a bit-exact answer.

Pull from source and there’s no build step at all: push to the branch, and the next run clones the current code. Iteration is instant and the registry disappears. The cost is that “what code ran?” now means “whatever was on main at run time” — a merge changes production with no deploy gesture, dependencies aren’t pinned by the transport (the worker’s environment has to already satisfy them), and reproducing an old run means checking out an old commit. This is the right default for a process pool on a stable host, for fast internal iteration, and for teams that pin the branch and treat the repo as the source of truth.

The two aren’t mutually exclusive — pin dependencies in an image, pull code from git on top of it, and you get reproducible dependencies with instant code changes. But most teams pick a lane per work pool, and picking it deliberately — rather than discovering it when a run can’t be reproduced — is most of what “productionizing Prefect” means.

The prefect deployment surface

Once deployments exist, the CLI manages them without touching code:

prefect deployment ls                              # list all deployments
prefect deployment inspect 'daily_load/bookshop-daily'
prefect deployment run 'daily_load/bookshop-daily' -p full_refresh=true
prefect deployment schedule pause 'daily_load/bookshop-daily'
prefect deployment schedule resume 'daily_load/bookshop-daily'

Deployments are addressed as flow-name/deployment-name. run fires an ad-hoc run with optional -p parameter overrides; schedule pause/resume toggle schedules without deleting them; inspect dumps the full server-side object. Alongside these, three deployment fields carry weight in production: version stamps each deploy so the UI and history record which code is live (set it to the commit SHA); tags group and filter deployments across flows — a bookshop or etl tag lets you find, and set concurrency across, everything in a domain; and concurrency_limit caps how many runs of this deployment execute at once, so a slow backfill can’t stack a dozen overlapping runs on top of the morning load. Set the limit to 1 on anything that mustn’t overlap itself.

Final thoughts

The deployment is the smallest unit of “production” in Prefect — the object that lets the platform, rather than you, decide a flow should run now, and with what inputs, on what schedule, in reaction to what event. Airflow folded that registration into the filesystem, which is convenient right up until a stray file is live in production and you’re explaining how. Prefect’s insistence on an explicit deploy is a little more typing and a lot less ambient behavior: a flow is a script until you say otherwise, and when you say otherwise you make three real choices — how the code travels (process, image, or source), how it’s scheduled (cron, interval, rrule, or an event trigger), and what infrastructure the promotion is worth. serve() gets you scheduled today; to_deployment() fans a few flows out of one process; a prefect.yaml in CI turns deployments into reviewed, versioned code. What none of these decided is where a scheduled run actually executes when no runner of yours is up — that’s the work pool, and it’s next.

Next: Work Pools and Workers

Comments