Running Prefect: Your First Flow Run

Install Prefect, write a bookshop flow, and run it by calling the function — then bolt on the UI, profiles, and Cloud when you want them.

Getting Airflow running locally is a project of its own — a scheduler, a metadata database, a webserver, and usually the Astro CLI to hold them together before a single DAG runs. Prefect’s first run is one virtual environment and a function call. The gap between those two experiences is the whole point, so let’s feel it, and then let’s understand exactly what is and isn’t running underneath.

Install and run

Prefect needs Python 3.10 or newer. Make a project directory, create a fresh virtualenv with uv, and install into it:

mkdir bookshop && cd bookshop
uv venv --python 3.12
source .venv/bin/activate
uv pip install prefect

uv venv --python 3.12 pins the interpreter for this project so you’re not at the mercy of whatever python3 happens to be on the machine. If you’d rather manage the whole project through uv’s lockfile — which is what you’ll want the day this pipeline runs somewhere other than your laptop — uv add prefect writes prefect into a pyproject.toml and pins the exact resolved version in uv.lock. Either way, confirm what you got:

prefect version
Version:             3.4.0
API version:         0.8.4
Python version:      3.12.3
Git commit:          abcdef12
Built:               ...
Server type:         ephemeral
Profile:             default

That Server type: ephemeral line and the Profile: default line are the two facts this whole post is about — hold onto them. Now write the bookshop’s daily load in bookshop.py:

from prefect import flow, task

@task
def extract_orders() -> list[dict]:
    return [
        {"title": "Dune", "qty": 2, "price": 18},
        {"title": "Piranesi", "qty": 1, "price": 15},
    ]

@task
def transform(orders: list[dict]) -> dict:
    return {
        "count": len(orders),
        "revenue": sum(o["qty"] * o["price"] for o in orders),
    }

@flow(log_prints=True)
def daily_load():
    orders = extract_orders()
    summary = transform(orders)
    print(f"{summary['count']} orders, revenue {summary['revenue']}")

if __name__ == "__main__":
    daily_load()

Run it like any script:

python bookshop.py

Here’s the part that stops Airflow people short: it just ran. No dags/ folder, no scheduler process waiting to pick it up, no toggling the flow on in a UI first. You called daily_load() and Python executed it — top to bottom, extract_orders before transform because that’s the order you wrote. There is no deployment to register and no scheduler to be alive. The terminal shows Prefect’s log lines wrapping your print, because log_prints=True routes print output into the flow’s logs:

Beginning flow run 'daily_load'
Created task run 'extract_orders'
Created task run 'transform'
2 orders, revenue 51
Finished in state Completed()

That last line is Prefect telling you the flow run ended in the Completed state. It tracked states, task runs, and logs for a script you ran by hand — no server involved yet. So where did that record go?

The ephemeral API: a flow run with no server

You didn’t start a server, but Prefect still needed an API to talk to — states have to be written somewhere, or the Finished in state Completed() line couldn’t exist. When no server is configured, Prefect spins up an ephemeral API inside your process for the duration of the run: an in-process instance of the same orchestration API a real server exposes, wired straight to a local database. It lives and dies with the Python process. That’s the Server type: ephemeral you saw in prefect version. It’s why the flow could record its own states with nothing listening on a port — the API was riding along inside python bookshop.py the whole time.

The database it writes to isn’t ephemeral, though. By default Prefect uses a SQLite file at ~/.prefect/prefect.db:

ls -la ~/.prefect/prefect.db

Every ephemeral run appends to that file, which is why your history survives from one python bookshop.py to the next even with no server. It’s also why, if you ever get the database into a weird state during experimentation, there’s a reset:

prefect server database reset

That drops and recreates the schema — a clean slate, and a genuinely useful thing to know exists the first time a migration or a corrupted local DB trips you up. (It affects whatever database your current profile points at; against a shared Postgres, treat it with the respect that implies.)

The ephemeral API is a fast local loop, not a place to keep anything. It has no long-running scheduler, so it can’t fire scheduled runs on its own, and nothing outside your process can query it. The moment you want history that outlives the process, a UI to look at it, or a scheduler that triggers runs on a cron, you graduate to a real server.

One consequence to know before it surprises you: ephemeral mode is a convenience for local development, and some environments disable it deliberately (it’s gated by the PREFECT_SERVER_ALLOW_EPHEMERAL_MODE setting). When it’s off — as it typically is on a shared or production box — a flow that finds no PREFECT_API_URL configured will error instead of quietly standing up its own API, which is the safe default there: you want production runs to fail loudly rather than record into a throwaway in-process database nobody can see. On your laptop it’s on, and the whole “just call the function” experience above depends on it.

Stand up the server when you want to watch

Running a flow doesn’t need a server, but observing your history does — the equivalent of Airflow’s webserver and its grid of colored squares, plus its scheduler. In a second terminal:

prefect server start

That launches the Prefect API and UI at http://127.0.0.1:4200, backed by that same SQLite database at ~/.prefect/prefect.db. Now it’s a real, long-lived process instead of an in-process guest — the difference is that it stays up, serves the web UI, and runs the components that fire scheduled and triggered runs. You can move it off the default port and bind:

prefect server start --host 0.0.0.0 --port 4200

--host 0.0.0.0 makes it reachable from other machines on the network rather than just localhost — the first thing you reach for when the server lives on a box and you want to browse the UI from your laptop.

There’s one wrinkle Airflow engineers hit immediately: starting the server doesn’t automatically point your client at it. python bookshop.py will keep using the ephemeral API until you tell your CLI and your flows to talk to the running server. That’s what PREFECT_API_URL is for:

prefect config set PREFECT_API_URL="http://127.0.0.1:4200/api"

Now run the flow again:

python bookshop.py

Open http://127.0.0.1:4200 and the run is there. Click into it and you’ll see the flow run with its two task runs underneath, each with a state, a duration, and its own logs — your 2 orders, revenue 51 sitting in the daily_load run’s log, exactly where you’d click when a task goes red.

Backing the server with Postgres

SQLite is perfect for one laptop. It is the wrong choice the moment more than one process writes concurrently — multiple workers, a team sharing a server, anything that isn’t just you. For that, point the server at Postgres by setting its connection URL before you start it:

prefect config set PREFECT_SERVER_DATABASE_CONNECTION_URL="postgresql+asyncpg://prefect:prefect@localhost:5432/prefect"
prefect server database reset   # initialize the schema in the new database
prefect server start

The postgresql+asyncpg:// scheme matters — Prefect talks to Postgres over the async asyncpg driver, not psycopg2, so the URL prefix isn’t optional. Everything else about the server is identical; you’ve only swapped the storage engine underneath it from a file to a database that handles concurrent writers. This is the standard self-hosted production shape, and we’ll return to it when we deploy for real.

Compare the setup to Airflow’s. Airflow’s UI is inseparable from running it: no scheduler and webserver, no observability, and standing those up locally is the whole of the run-it-locally exercise. In Prefect the flow already ran on its own; the server is a recorder-and-scheduler you attach when you want history and scheduling, not a prerequisite for anything to execute.

Profiles and settings

You just set two settings — PREFECT_API_URL and a database URL — with prefect config set. Where did they go, and how do you keep a “local SQLite” configuration and a “shared Postgres server” configuration from clobbering each other? That’s profiles.

A profile is a named bundle of settings. Prefect ships with one called default, which is the one you’ve been editing. List them, see the active one, and inspect what’s in it:

prefect profile ls
prefect config view

prefect config view shows the effective settings and — usefully — where each came from: a profile, an environment variable, or a built-in default. When a flow talks to the wrong API, this is the first command to run, because it tells you which PREFECT_API_URL actually won.

The point of profiles is switching cleanly. Create one for a local server and one for, say, Cloud, and flip between them without hand-editing anything:

prefect profile create local-server
prefect profile use local-server
prefect config set PREFECT_API_URL="http://127.0.0.1:4200/api"

Now default still has whatever it had, and local-server holds the running-server URL. prefect profile use default puts you back on the ephemeral setup with one command. The settings themselves live in a TOML file at ~/.prefect/profiles.toml, so a profile is portable and inspectable — but you rarely edit it by hand; prefect config set and prefect config unset do it for you. One caution that follows from the file being plain TOML: an API key set on a Cloud profile is stored there in the clear, so ~/.prefect/profiles.toml is not something to commit to a repo or share — treat it like any other credential on disk, and in CI prefer passing PREFECT_API_URL and PREFECT_API_KEY as environment variables (which, remember, override the profile anyway) over baking a profile into the image. Two things worth knowing: prefect config set writes to the currently active profile (check prefect config view if you’re unsure which that is), and an environment variable of the same name overrides the profile at runtime — so PREFECT_API_URL=... python bookshop.py wins over whatever the profile says, which is exactly what you want in CI.

Prefect Cloud

The self-hosted server is one option; the other is Prefect Cloud, where Prefect runs the API, database, and UI for you and you never prefect server start at all. The client-side flow is a login and a workspace selection:

prefect cloud login

That opens a browser to authenticate and hands your CLI an API key, or you can paste a key directly for headless environments:

prefect cloud login -k pnu_your_api_key_here

API keys are how anything non-interactive — a worker on a server, a CI job — authenticates to Cloud; you generate them in the Cloud UI under your account settings. Cloud organizes runs into workspaces (think of a workspace as one isolated environment — its own runs, deployments, and blocks). If you belong to more than one, pick the active one:

prefect cloud workspace ls
prefect cloud workspace set --workspace "my-org/production"

Under the hood, prefect cloud login is just setting PREFECT_API_URL (to your workspace’s Cloud endpoint) and an API key on a profile — the same machinery as pointing at a local server, aimed at Prefect’s infrastructure instead of yours. Which is why switching between “my laptop’s ephemeral DB,” “our self-hosted server,” and “Cloud production” is really just switching profiles.

Cloud vs. self-hosted, at a glance

You don’t have to decide this now, but the shape of the choice is worth having in mind:

  • Self-hosted (SQLite or Postgres) gives you full control and keeps everything on your infrastructure — the right call when data-residency or network policy rules out a SaaS control plane, or when you simply don’t want to pay for one. The cost is that you run and upgrade the server, the database, and its availability yourself.
  • Cloud removes all of that operational surface and adds things a bare self-hosted server doesn’t have out of the box — accounts and RBAC, multiple workspaces, SSO, audit logs, and managed automations. The cost is a per-seat/usage bill and your control plane living in Prefect’s cloud (your flows still run on your own infrastructure via workers — Cloud orchestrates, it doesn’t execute your code).

We’ll go deeper on the self-hosted production shape and on workers when we get to deployments and work pools; for now, the mental model is that both are “an API your client points at,” and the code you write doesn’t change between them.

A short CLI tour

The prefect CLI is how you inspect everything the API knows without opening the UI. A few commands you’ll reach for constantly:

prefect version                    # client + API version, server type, active profile
prefect config view                # effective settings and where each came from
prefect flow-run ls                # recent flow runs, with their states
prefect deployment ls              # registered deployments (empty until you make one)
prefect profile ls                 # your profiles; the active one is marked

Run prefect flow-run ls now and you’ll see the runs of daily_load you’ve fired, each with an id, name, and state — the CLI reading back the same records the UI renders. Note that prefect flow-run ls reads from whatever API your active profile points at, so if a run you expected is missing, the usual cause is that you ran the flow under the ephemeral profile and are now listing against the server (or vice versa) — prefect config view settles it. To undo a setting you no longer want on a profile, prefect config unset PREFECT_API_URL removes it and falls back to the default (the ephemeral API), which is the clean way back to a laptop-only setup.

prefect deployment ls is empty at this point, and that’s the correct answer: you haven’t created a deployment yet (that’s the deployments post), and running a flow by calling its function never makes one. It’s a good early sanity check that “I ran a flow” and “I have a deployment” are genuinely separate things in Prefect, which trips up people expecting Airflow’s “the file is the DAG registration” model. Almost every noun has a corresponding CLI group — prefect flow-run, prefect deployment, prefect work-pool, prefect block, prefect profile — and prefect <group> --help lists what it can do.

Logging: get_run_logger() vs. log_prints

You’ve already seen log_prints=True fold your print calls into the flow’s logs. That’s the quick path, and it’s genuinely useful for a print you dropped in while debugging — it shows up next to the orchestration logs instead of vanishing into stdout. But the idiomatic way to log inside a flow or task is to ask Prefect for the run’s logger:

from prefect import flow, task, get_run_logger

@task
def transform(orders: list[dict]) -> dict:
    logger = get_run_logger()
    revenue = sum(o["qty"] * o["price"] for o in orders)
    logger.info("Transformed %d orders → revenue %d", len(orders), revenue)
    return {"count": len(orders), "revenue": revenue}

get_run_logger() returns a standard Python logger that’s bound to the current flow or task run, so its output is tagged with the run and level (INFO, WARNING, ERROR) and lands in exactly the place the UI shows per-run logs. Reach for it over print + log_prints when you care about levels, structured messages, or code that also runs outside Prefect — the logger is a no-op-safe standard logger, whereas log_prints is a blunt “capture all stdout” switch. Note that get_run_logger() only works inside a running flow or task; call it at module top level and it’ll raise, because there’s no run to bind to. The two aren’t mutually exclusive — keep log_prints=True on for convenience and use get_run_logger() where you want real log lines. The dedicated logging post goes deeper on formatting, levels, and shipping logs off-box; this is the working minimum.

The states you’ll actually see

That view in the UI is built from states, and because later posts on retries, caching, and automations all act on specific state names, it’s worth naming the real set now rather than waving at “and more.” A flow run or task run is always in exactly one of these:

  • Scheduled — Prefect intends to run it, but not yet; it’s waiting for its time or a slot. Late is a Scheduled sub-state meaning its start time has passed and it still hasn’t begun — the signal that something (usually a worker) isn’t picking work up.
  • Pending — the run has been claimed and is preparing to execute.
  • Running — actively executing right now.
  • Completed — finished successfully. This is the one you saw.
  • Failed — finished by raising an error. Retries operate on transitions out of this.
  • Crashed — the run’s infrastructure died (process killed, pod evicted, OOM) rather than the code raising — Prefect distinguishes “your code failed” from “the thing running your code went away.”
  • Cached — completed by reusing a previous result instead of re-executing, because a cache key matched.
  • Cancelled — stopped deliberately, by a human or an automation.
  • Paused — suspended mid-run, waiting to be resumed (for an approval step, say).

These aren’t just UI colors; they’re a state machine, and much of what makes Prefect feel different from Airflow lives in the transitions between them. The dedicated states and hooks post leans on this list directly — Cached comes back in results and caching, the Failed → Scheduled → Running loop is retries, and Crashed vs. Failed is why Prefect’s failure handling is more precise than “the task went red.” Worth internalizing early: Airflow has task states too, but it has no first-class equivalent of Cached (its result reuse lives in XComs and custom logic) or Crashed as a distinct outcome, so this vocabulary is part of what you’re learning, not a renaming of something you already have.

The everyday loop

Working in Prefect settles into a tight cycle. Edit the Python. Run python bookshop.py. Watch the run land in the UI (or read it back with prefect flow-run ls). Because the flow is a plain function, that inner loop needs nothing running — the ephemeral API records it either way — and you only attach a server when you want durable history, a scheduler, and a shared view.

Keep the mental split clean. A flow runs whether or not a server is up: with no server it uses the ephemeral in-process API and still records to ~/.prefect/prefect.db; with a server it records to that server’s database and shows up in the UI. What changes when you attach a real server isn’t whether the code executes or even whether it’s recorded — it’s whether that record is durable, shared, and schedulable. That’s the line to keep in your head as the rest of the series turns these local runs into deployed, triggered, production workflows.

Final thoughts

The friction Airflow charges up front — a scheduler, a database, a webserver before “hello world” — buys real things at scale, but it also means you can’t casually try a pipeline. Prefect moves that cost to where you actually need it. A flow is a function you run today against an ephemeral API and a SQLite file you didn’t have to configure; the server, Postgres, profiles, and Cloud are infrastructure you add the day you want history, scheduling, and a team — and adding them changes nothing about the code, only which API your client points at. Start with the function, earn the server. The next post takes the two decorators you just used and pushes on what they’re really for.

Next: Flows, Tasks, and Subflows

Comments