Who Gets to Press the Buttons

Auth managers, FAB roles, DAG-level access control, JWT, and the honest limits of Airflow's multi-tenancy — the chapter that decides who can clear a task at 2am.

Seventeen chapters of this book have been about making Airflow do things: run a task in a pod, resolve a secret from Vault, defer a sensor, route work to a second executor, survive a rolling deploy. Every one of them assumed a person or a system was allowed to ask. This chapter is about that assumption, and it’s the one most production deployments have never actually examined.

The problem, stated plainly. Airflow’s UI has a button that clears a task and reruns it, a button that deletes a DAG run, a page listing every connection you own, and an API that will start any DAG with any conf payload you like. Somewhere in your company there’s an analyst who needs to look at a graph, an engineer who needs to clear a failed task, and a platform engineer who needs to rotate a connection — and in an alarming number of deployments all three log in as admin/admin, because that’s what the tutorial said and nobody circled back.

Worse, and more subtly: Airflow’s real privilege boundary isn’t the UI at all. It’s the dags/ folder. A DAG is Python that Airflow executes. Anyone who can merge a file into that folder can run arbitrary code inside your orchestrator — as the DAG processor, as the triggerer, as the worker. “Just give the analytics team write access to the DAGs repo, it’ll be easier” is not a convenience decision. It’s a grant of remote code execution, and you should make it deliberately or not at all.

So: who gets to press the buttons.

Airflow 3 moved the whole thing behind an auth manager

If you carry security knowledge from Airflow 2, quarantine it first. In 2.x, authentication and authorization were Flask-AppBuilder. The webserver was a Flask app, FAB was bolted into it, roles lived in FAB’s tables, and configuring auth meant editing webserver_config.py and hoping. There was no seam. Security was a property of the webserver.

Airflow 3 deleted the Flask webserver. What replaced it is a React UI served by a FastAPI api-server, and — this is the part that matters — authentication and authorization were factored out behind a pluggable interface: the auth manager. One config option selects it:

airflow config get-value core auth_manager
# airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager

The auth manager is a single component with a defined job, and knowing its job tells you exactly where to look when something is denied. It owns:

  • Authentication — the login and logout flow, and what “a user” even is in this deployment.
  • Authorization — the yes/no on “may this user read this DAG / edit this connection / see this pool,” resource by resource.
  • JWT issuance — minting and refreshing the tokens the React UI and the REST API run on.
  • User representation — serializing a user into and out of a token.

Two auth managers ship in the box, and they are not two flavors of the same thing. They are a development one and a production one, and the gap between them is the entire subject of the next two sections.

# The default — dev-oriented, users in config
[core]
auth_manager = airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager

# The real one — roles, permissions, SSO
[core]
auth_manager = airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager

A third path exists: auth managers ship as providers, so apache-airflow-providers-keycloak gives you a Keycloak-backed one, and the interface is public enough that a platform team with an unusual identity story can write their own. But for almost everyone the decision is binary: Simple for local, FAB for production.

SimpleAuthManager: exactly as much security as its name promises

SimpleAuthManager is what astro dev start gives you, and it’s what has been quietly authenticating you for seventeen chapters. It stores no users in the database. Users are a string in your config:

[core]
auth_manager = airflow.api_fastapi.auth.managers.simple.simple_auth_manager.SimpleAuthManager
simple_auth_manager_users = "priya:admin,marco:op,sam:user,analysts:viewer"

Four roles, hardcoded into the implementation, and you cannot add a fifth or change what one means:

  • viewer — read-only. DAGs, assets, pools. Looks, doesn’t touch.
  • user — viewer, plus full edit/create/delete on DAGs. This is the “clear a task, trigger a run” role.
  • op — user, plus management of pools, assets, config, connections, and variables.
  • admin — everything.

Passwords are generated, not chosen. On startup Airflow mints one per user, prints it to the api-server logs, and writes them to a file:

[core]
simple_auth_manager_passwords_file = /opt/airflow/simple_auth_manager_passwords.json.generated

Edit that file to set a password you’ll actually remember. And there’s a switch that turns the whole thing off:

[core]
simple_auth_manager_all_admins = True    # no login screen; everyone is admin

Which is genuinely useful — behind an identity-aware proxy that has already established who the caller is, having Airflow ask a second time is friction with no security value. But understand what you’ve done: you’ve delegated 100% of your access control to the thing in front of Airflow. If that proxy is misconfigured, or if anyone can reach the api-server’s port directly, you have an unauthenticated admin console on your network.

The honest summary, which Airflow’s own docs make: SimpleAuthManager is for development and testing. No custom roles, no per-DAG scoping, no SSO, no LDAP, no group mapping, no audit of who changed what. If your Airflow schedules something a customer or a finance close depends on, you’ve outgrown it.

FabAuthManager: the roles-and-permissions machine

FAB didn’t die in Airflow 3 — it was extracted into a provider. The classic RBAC model that 2.x baked into the webserver is now something you opt into, and opting in is three steps: install it, point auth_manager at it, and migrate its tables — because FAB keeps users, roles, and permissions in its own schema, with its own migration command.

pip install apache-airflow-providers-fab
[core]
auth_manager = airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
airflow fab-db migrate      # creates/upgrades the FAB tables (ab_user, ab_role, ab_permission, ...)

That last command is new in 3.x and it catches everyone. airflow db migrate migrates Airflow’s schema; the FAB provider’s tables are a separate DB manager with a separate command. If you skip it, the api-server boots and then dies on the first login with a missing-table error that reads like a broken install. (You can wire FAB into the main airflow db migrate by listing FABDBManager in [core] external_db_managers, which is the tidier thing to do in a Helm chart’s migration job — one command, all schemas.)

Now you have users:

airflow users create \
  --username priya \
  --firstname Priya \
  --lastname Raman \
  --email [email protected] \
  --role Admin \
  --password '<not-this-please>'

airflow users list
airflow users add-role --username sam --role User
airflow users remove-role --username sam --role Op
airflow users export users.json      # and `import` — how you seed a fresh environment

Note that airflow users and airflow roles are vended by the FAB provider. They don’t exist under SimpleAuthManager. If you type airflow users list and the CLI tells you there’s no such command, that’s not a broken install — that’s Airflow telling you which auth manager is loaded.

The five roles, and what they actually do

FAB ships five built-in roles, and they nest:

RoleWhat it can do
PublicNothing. This is the anonymous, not-logged-in role. Leave it empty.
ViewerRead. DAGs, DAG runs, task instances, logs, XComs. Cannot trigger, cannot clear, cannot see connections.
UserViewer, plus can_create/can_edit/can_delete on DAGs, DAG runs, task instances. This is the working data engineer: trigger, clear, mark success, delete a run.
OpUser, plus the Admin menu — full CRUD on connections, variables, pools, XComs, and access to config. This is a platform-adjacent role and it is much more powerful than it sounds.
AdminOp, plus user management, role management, audit logs, triggers, task reschedules. Total control.

The line worth tattooing on the back of your hand is between User and Op. User operates pipelines. Op can write connections — which means Op can point the bookshop_dw connection at a host they control, and the next DAG run will happily authenticate to it. Granting Op because “they needed to add a variable” has a much larger blast radius than the request implied.

(The 3.x mitigation is real: connection passwords are no longer readable in the UI, even for users who can edit connections. An Op can overwrite a credential; they can’t exfiltrate the one that’s there. It’s another argument for keeping the real credential in a secrets backend, where Airflow never holds it at all.)

The permission model underneath

Those five roles are not primitives — they’re bundles of permissions, and a permission is a (resource, action) pair. The actions are the CRUD set plus a menu one:

ACTION_CAN_READ  ACTION_CAN_CREATE  ACTION_CAN_EDIT  ACTION_CAN_DELETE  ACTION_CAN_ACCESS_MENU

The resources are Airflow’s nouns: DAGs, DAG Runs, Task Instances, Connections, Variables, Pools, XComs, Assets, Audit Logs, Configurations, Plugins, plus the per-DAG resources named DAG:<dag_id>. A permission reads DAG:bookshop_daily.can_edit or Connections.can_read. Each endpoint declares the permissions it needs, and — the rule that explains every confusing 403 — the user needs all of them, not any of them. A page listing DAG runs and their DAGs needs read on both.

Custom roles are built from exactly those pairs. If the bookshop’s analysts should be able to trigger the reporting DAGs and touch nothing else, that’s a role, not a plea for Op:

airflow roles create BookshopAnalyst

airflow roles add-perms BookshopAnalyst \
  -r "DAGs"          -a can_read \
  -r "DAG Runs"      -a can_read -a can_create \
  -r "Task Instances" -a can_read

airflow roles list-perms BookshopAnalyst
airflow roles export roles.json     # version-control this; `import` it into staging

Exporting roles to JSON and committing the file is what turns RBAC from tribal knowledge into infrastructure. Roles drift — someone grants a permission in the UI at 2am to unblock a release and nobody remembers. A roles.json in git, imported by CI into every environment, means the role is reviewed like code and identical everywhere.

SSO, because nobody should be typing an Airflow password

AUTH_DB — usernames and passwords in Airflow’s own tables — is where you start and not where you should stop. The FAB provider still generates a webserver_config.py (yes, that file survived; no, the Flask webserver did not), and that’s where you switch to OAuth, LDAP, or a header-trusting REMOTE_USER:

# webserver_config.py
from flask_appbuilder.security.manager import AUTH_OAUTH

AUTH_TYPE = AUTH_OAUTH

# Anyone who authenticates but matches no mapping gets this. Make it the safe one.
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Viewer"

# The load-bearing line: map your IdP's groups onto Airflow roles.
AUTH_ROLES_SYNC_AT_LOGIN = True
AUTH_ROLES_MAPPING = {
    "airflow-admins":   ["Admin"],
    "data-engineering": ["User"],
    "data-platform":    ["Op"],
    "analytics":        ["BookshopAnalyst"],
}

OAUTH_PROVIDERS = [
    {
        "name": "okta",
        "icon": "fa-circle-o",
        "token_key": "access_token",
        "remote_app": {
            "client_id": "<client-id>",
            "client_secret": "<client-secret>",   # from an env var, not this file
            "api_base_url": "https://bookshop.okta.com/oauth2/v1/",
            "server_metadata_url":
                "https://bookshop.okta.com/.well-known/openid-configuration",
            "client_kwargs": {"scope": "openid profile email groups"},
        },
    },
]

AUTH_ROLES_MAPPING plus AUTH_ROLES_SYNC_AT_LOGIN is the whole prize. Offboarding becomes removing someone from an IdP group — which the IdP already does on their last day, automatically, without anyone remembering that Airflow exists. If Airflow keeps its own passwords, offboarding is a checklist item, and checklist items are how a departed contractor keeps Op for eight months. Providers cover the usual suspects (Okta, Azure/Entra, Google, Keycloak, GitHub, Auth0); LDAP is there for the estates that need it.

access_control: scoping a role to specific DAGs

Roles answer “what kinds of things can this person do.” They don’t answer “which DAGs.” For that — the primary multi-tenancy lever Airflow actually offers — there’s DAG-level access control, declared on the DAG and enforced by the FAB auth manager:

from airflow.sdk import DAG

with DAG(
    dag_id="bookshop_daily",
    schedule="@daily",
    access_control={
        "BookshopAnalyst": {"can_read"},
        "BookshopEngineer": {"can_read", "can_edit", "can_delete"},
    },
) as dag:
    ...

Under the hood this creates and grants the DAG:bookshop_daily.can_read style permissions to those roles. The finer-grained form lets you scope by resource within the DAG — read the graph but also be allowed to create a run, without being allowed to edit the DAG:

with DAG(
    dag_id="bookshop_reprocess",
    schedule=None,
    access_control={
        "BookshopAnalyst": {
            "DAGs": {"can_read"},
            "DAG Runs": {"can_create"},     # may trigger it; may not change it
        },
    },
) as dag:
    ...

Three behaviors here are load-bearing, and getting them wrong is how permissions mysteriously disappear:

  • If access_control is present, it is authoritative. It doesn’t add to existing DAG-level permissions; it replaces them. Whatever someone granted in the UI is gone at the next parse.
  • access_control={} wipes all DAG-level access for that DAG. That’s the explicit “nobody but the global roles” statement.
  • Omitting access_control entirely leaves existing permissions alone. Silence is not a denial. A DAG with no access_control inherits whatever the global roles grant — which for User and above is everything.

That third point is the one that bites. If your plan is “team A can only see team A’s DAGs,” you cannot achieve it by putting access_control on team A’s DAGs. You must also ensure team A’s role has no global DAGs.can_read, or they’ll see everything anyway. Per-DAG grants are additive on top of global grants; they don’t subtract. Build the role with no global DAG permissions, then hand it DAGs one access_control block at a time.

And because access_control lives in the DAG file, it’s reviewed in the same PR as the pipeline — which is either elegant (permissions as code, diffable) or alarming (a DAG author can grant their own role access to their own DAG), depending on your posture. It’s both. Which is fine, for reasons that will become clear two sections from now.

The API is a first-class door, and JWTs are credentials

The REST API chapter showed the mechanics: POST /auth/token, get an access_token, send it as Authorization: Bearer …. What that chapter didn’t dwell on is that this is not a separate security surface — it’s the same one the UI uses. The React UI is a client of the api-server; it authenticates the same way; anything a logged-in operator can click, a token with that user’s role can call. There is no “the API is for machines and therefore less privileged” tier.

Three things follow.

First, the 2.x API auth story is gone. The experimental API was removed. The [api] auth_backends list of pluggable backends — basic auth, Kerberos, deny-all — isn’t how you configure this anymore; the auth manager is. If a runbook tells you to set auth_backends = airflow.api.auth.backend.basic_auth, it predates 3.0. If it hits /api/experimental/, it’s been broken since you upgraded.

Second, the JWT signing key is a real piece of infrastructure, and forgetting to set it produces a failure that looks like anything but a config problem:

[api_auth]
jwt_secret = <long random string, from the environment>     # AIRFLOW__API_AUTH__JWT_SECRET
jwt_expiration_time = 86400        # REST API tokens: 24h default
jwt_cli_expiration_time = 3600     # CLI tokens: 1h default
jwt_leeway = 10                    # clock-skew tolerance, in seconds

If you don’t set jwt_secret, Airflow generates a random one per process. On a laptop that’s invisible. On a real deployment with two api-servers behind a load balancer, a token minted by pod A is rejected by pod B — so every other request 401s and your users report that “Airflow logs me out randomly.” Set it explicitly, from a secret, everywhere. And synchronize your clocks: JWT validation is time-based, and skew beyond jwt_leeway produces auth failures that look exactly like a permissions bug.

Better still, go asymmetric. With a symmetric secret, every component that can validate a token can also forge one — same key, both jobs. With jwt_private_key_path, only the components that mint tokens (scheduler, api-server) hold the private key; workers validate with the public half and cannot sign anything:

[api_auth]
jwt_private_key_path = /etc/airflow/keys/jwt-signing.pem   # scheduler + api-server only
trusted_jwks_url = https://airflow.bookshop.dev/.well-known/jwks.json

Third, treat tokens like the passwords they are. A JWT that can POST /api/v2/dags/bookshop_marts/dagRuns can trigger a full refresh at quarter close. Issue a dedicated service identity per calling system — not a shared human’s login — scope its role to the minimum (a BookshopTrigger role with DAG Runs.can_create on exactly the DAGs it may start), keep expirations short, and store them where you store every other credential.

The genuine 3.x security win: workers don’t touch the database

Here’s what I’d put at the centre of any “should we upgrade to Airflow 3” conversation that touches security, and it has nothing to do with roles.

In Airflow 2, a worker held the metadata database connection string. It had to — task code called into Airflow’s ORM to read a Variable, push an XCom, update its own state. Which meant any code running on a worker — any DAG, from any author — could from airflow.settings import Session, open that session, and do whatever it liked. Read every connection row. Rewrite another team’s run history. Drop a table. There was no boundary; the worker held the keys to the kingdom because the kingdom had one door.

Airflow 3 closes it. Workers now talk to a Task Execution API — a separate surface on the api-server — over HTTP, authenticated with a short-lived JWT scoped to that task instance. State transitions, heartbeats, XCom reads and writes, connection and variable fetches: all of it goes through that API. And critically, workers are not given database credentials at all. The connection string isn’t in the worker’s environment. Task code cannot open a session to a database whose address and password it doesn’t have.

[execution_api]
jwt_expiration_time = 600                        # 10 minutes; a task's token is ephemeral
jwt_audience = urn:airflow.apache.org:task

That token carries a ti:self scope: a task can act on itself — heartbeat, report success, push its own XCom — and not on other task instances. An entire class of vulnerability, the one where a buggy or malicious DAG corrupts the orchestrator’s own state, is now structurally impossible rather than merely discouraged.

Be precise about the limits, though. The Execution API has no cross-workload isolation. Connections, variables, and XComs are shared: any task can fetch any connection id it knows the name of. The scope stops a task from impersonating another task; it does not stop a task from reading the credentials another team’s tasks use. If task A can call BaseHook.get_connection("payroll_db"), it gets payroll_db. That’s not a bug — it’s the model.

And the DAG file processor and the triggerer do still hold database credentials, running as the same Unix user as their parent process. Which is the perfect segue.

The blast radius of a DAG file

Read Airflow’s own security-model documentation and you’ll find a sentence stated without apology: DAG authors can execute arbitrary code on workers, the DAG file processor, and the triggerer.

Not “might.” Can. By design. A DAG is a Python module and the DAG processor imports it — top-level code runs, on a loop, forever, in a process holding your metadata database credentials. Then tasks run on workers. Then deferred triggers run in the triggerer. Three processes, three chances, and a subprocess.run at module scope executes in all of them.

So the trust boundary of your Airflow deployment is not the login page. It’s your repository’s branch protection rules. Airflow’s model treats DAG authors as trusted insiders — people who could abuse the position but won’t, the same way your backend engineers could ship a data-exfiltrating endpoint but don’t. And there is no per-author isolation: every DAG author functionally has access to every DAG in the deployment, because they can write a DAG that reads any connection.

The real access-control questions, then, are not “what role does Priya have in the UI”:

  • Who can merge to the branch git-sync pulls from? That’s your actual admin list. If deploying uses git-sync against main, merge rights on main are equivalent to code execution in production. Branch protection, required reviews, and CODEOWNERS on dags/ aren’t process hygiene — they are the access control.
  • What’s in each DAG bundle? Airflow 3’s DAG bundles are the delivery abstraction — a git repo, a local path, an object-storage prefix — and each bundle is a trust unit. Separate bundles let the analytics team’s DAGs live in a different repo, with different reviewers, from the platform team’s.
  • Who can push the image? If DAGs are baked into the container, the registry credentials are the grant.

So when the analytics team asks for write access to the DAGs folder “so we don’t have to bother you for a one-line change,” the answer is neither “no” nor “sure.” It’s: that’s a request for arbitrary code execution as the orchestrator’s service account, so let’s put it behind a review gate and a bundle boundary. Give them their own bundle, their own repo, their own reviewers, and a DAG-parse check in CI that gates the merge. That’s a yes with a spine.

One small, genuinely useful control: .airflowignore, which tells the processor what in a bundle not to import. In Airflow 3 the default syntax is glob, .gitignore-style, not the regex of 2.x:

# dags/.airflowignore  (glob syntax, Airflow 3 default)
scratch/
*_notebook.py
!scratch/keep_this.py

It’s not a sandbox — an ignored file isn’t contained, it’s simply not imported — but it stops a data scientist’s exploratory script from being executed on a loop by your DAG processor because it landed in the wrong folder. Which is a surprisingly common way to discover that the DAG processor runs everything.

Secrets hygiene: what leaks, and where

Chapter 13 covered where secrets live. This is about where they escape.

Airflow masks sensitive values in logs, the UI, and rendered templates — and the masker works on field names, matching a built-in keyword list: password, passwd, secret, token, access_token, api_key, apikey, authorization, passphrase, private_key, keyfile_dict, service_account. A connection’s password is always masked. A variable named orders_api_key is masked everywhere it appears. A variable named orders_credential is not, because “credential” isn’t on the list — which is why the extension knob exists:

[core]
sensitive_var_conn_names = credential,pat,session_cookie,signing_key
hide_sensitive_var_conn_fields = True     # leave this True. Always.

For values that don’t come from a connection or a variable at all — something you fetched from an internal API mid-task, say — you register them with the masker yourself, before anything logs them:

from airflow.sdk.log import mask_secret

@task
def call_partner_api():
    token = PartnerHook().mint_short_lived_token()
    mask_secret(token)          # from here on, this string is ***** in every log line
    return PartnerHook().fetch(token=token)

Now the leaks the masker cannot save you from, because they happen outside its reach:

  • Environment variables are not masked. A secret injected as an env var into a worker or a KubernetesPodOperator pod is a secret Airflow doesn’t know is a secret. It won’t show up in a task log unless your code prints it — but it will show up to anything that can read the process environment.
  • ps and the command line. The classic. A BashOperator with bash_command="psql 'postgresql://user:{{ var.value.db_password }}@host/db'" renders that password into an argv every process on the box can read in ps aux. Pass secrets through the environment or a file, never as a command-line argument.
  • Rendered templates. The UI’s “Rendered Template” tab shows the post-Jinja value of every templated field. The masker covers it — so a field rendering a masked-name variable is safe, and a field rendering an unmasked-name one is a credential on a web page.
  • XComs. An XCom is a value in the metadata database, or in your custom XCom backend — which is worse if that’s a bucket. Returning a token from a task is how tokens end up in object storage with a five-year retention policy.

And the credential everybody forgets: the Fernet key, which encrypts sensitive fields at rest in the metadata DB. Chapter 13 gave the rotation dance (new,old → airflow rotate-fernet-key → new). The point to restate: once a secrets backend holds your real credentials Fernet protects a much smaller surface — but losing it still turns whatever is in the metastore into unrecoverable ciphertext. Back it up like a database password, because it is one.

Multi-tenancy: the honest section

Every platform team eventually asks whether they can run one Airflow for the whole company with each team isolated from the others. For Airflow 3.3 the honest answer is not in the way you mean it — and the precision of why is what lets you design around it.

Airflow 3.3 ships an experimental multi-team mode. It’s real work and genuinely useful, and it’s explicitly labeled incomplete and subject to change:

[core]
multi_team = True

Teams attach to DAG bundles, and DAGs inherit their team from the bundle they came from — Task → DAG → Bundle → Team:

[dag_processor]
dag_bundle_config_list = [
    {
        "name": "analytics_dags",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {"path": "/opt/airflow/dags/analytics"},
        "team_name": "analytics"
    },
    {
        "name": "shared_dags",
        "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
        "kwargs": {"path": "/opt/airflow/dags/shared"}
    }
]
airflow teams create analytics
airflow teams list

A bundle with no team_name is global — visible to everyone. What that buys you in 3.3 is genuine scoping of variables, connections, pools, XComs, and asset events, plus per-team executors (executor = LocalExecutor;analytics=CeleryExecutor;ml=KubernetesExecutor). That’s a lot, and for the common case — several cooperating teams who want their own connections, their own worker pool, and not to see each other’s clutter — it’s a real answer.

What it is not is a boundary against a hostile tenant. Take the limits at face value:

  • All teams share one metadata database and one set of core infrastructure.
  • DAG ids, variable keys, and connection ids must be globally unique — two teams cannot each have a warehouse connection.
  • The Execution API has no cross-workload isolation: teams share the same API, the same signing keys, the same runtime surface.
  • By default a single DAG processor and triggerer serve all teams, both holding database credentials, both executing every team’s code. (Run them per-team — you should — and each still has DB access.)
  • Parts of the UI aren’t team-aware yet.

So: Airflow’s multi-tenancy is a convenience boundary, not a trust boundary. It organizes teams that trust each other. It does not contain a team that doesn’t.

When the isolation requirement is real, here is what actually works:

  • Separate deployments. One Airflow per trust boundary — separate metadata DB, separate workers, separate everything. It’s the only airtight answer, and on Kubernetes it’s a different namespace and a different Helm release, which is less scary than it sounds. When someone says “we can’t afford five Airflows,” price five Airflow deployments against one incident and the arithmetic usually resolves itself.
  • KubernetesExecutor with per-task service accounts. From the Kubernetes chapter: every task is its own pod, and a pod carries its own service account. Bind that SA to an IAM or Vault role scoped to exactly one team’s data, and a task from team A that reaches for team B’s bucket gets a 403 from the cloud, not from Airflow. This is the strongest isolation you can build inside one Airflow, precisely because the enforcement lives outside it. That’s the whole trick: don’t ask the orchestrator to be the authority — make the resource the authority.
  • Queues and pools per team, so a runaway backfill starves its own capacity and nobody else’s. Blast-radius reduction rather than access control, but it’s the failure you’ll actually hit.
  • Bundle-per-team with separate repos and CODEOWNERS, so the code-execution grant is scoped where it actually lives.

What you should not do is run one Airflow for teams who must not read each other’s credentials and tell yourself that roles and access_control have it covered. They cover the console. They do not cover a DAG file.

The hardening checklist

Everything above, as the thing you’d actually run down before a production launch:

Identity

  1. auth_manager is FabAuthManager, not SimpleAuthManager. Verify with airflow config get-value core auth_manager against the running deployment, not the repo.
  2. airflow fab-db migrate runs in your migration job (or FABDBManager is listed in [core] external_db_managers).
  3. AUTH_TYPE = AUTH_OAUTH (or LDAP) with AUTH_ROLES_MAPPING + AUTH_ROLES_SYNC_AT_LOGIN. Nobody has an Airflow-local password.
  4. AUTH_USER_REGISTRATION_ROLE = "Viewer" — the default for an unmapped user is the least-privileged role, not User.
  5. The Public role has zero permissions.
  6. Op and Admin are a countable list of humans, and you can name them.
  7. Roles are exported to a version-controlled roles.json and imported by CI, not hand-edited in the UI.

API and tokens 8. [api_auth] jwt_secret (or jwt_private_key_path) is explicitly set, from a secret, and identical across every component. Asymmetric if you can. 9. NTP is running everywhere. JWT validation is a clock. 10. Machine callers have their own service identities and minimal roles — no shared human logins in CI. 11. CORS origins are an explicit allowlist (Airflow rejects * at startup, which is a feature). 12. The api-server is behind TLS and not reachable from the open internet.

Code and secrets 13. dags/ is protected: required reviews, CODEOWNERS, and a DAG-parse test that gates the merge. 14. Merge rights on the git-sync branch are treated as production access, and audited as such. 15. Real credentials live in a secrets backend; the worker’s identity can read only its own prefix. 16. hide_sensitive_var_conn_fields = True; sensitive_var_conn_names extended for your naming conventions. 17. No secret is ever a command-line argument. Grep your DAGs for bash_command with a {{ var.value. in it. 18. The Fernet key is backed up, and you’ve rehearsed rotating it.

Isolation 19. One Airflow per trust boundary. Multi-team mode is for teams that trust each other. 20. Tasks that touch sensitive data run as pods with their own service account, and the cloud enforces the boundary.

Final thoughts

We started this book with a custom operator and we’re ending it with an argument about who’s allowed to merge a Python file. That’s not a swerve — it’s the same subject arriving at its conclusion.

Everything Airflow does, it does by running your code with some identity, against some data, on some schedule. Chapter by chapter we made that machinery better: operators that encapsulate, sensors that don’t burn a slot, mapping that fans out at runtime, Assets that let producers and consumers stop knowing about each other, executors that put the right work on the right hardware, backends that keep secrets out of the database, tuning that keeps the scheduler quick, Kubernetes that gives each task a clean room. Every one of those made Airflow more capable. This one is the only one that asks what happens when that capability gets pointed at something it shouldn’t be.

The clarifying truth is that Airflow’s most powerful interface has never been the UI. It’s the DAG file. Roles and access_control and OAuth and JWTs are all real and all worth doing — they’re what stops an analyst from deleting a run by accident and a leaked token from triggering a quarter-close refresh — but they govern the console, and the console was never where the power was. The power is in a Python module that a process imports, in a loop, forever, holding your database credentials. Airflow 3 made that meaningfully better: workers genuinely cannot reach the metadata database anymore, task tokens are short-lived and scoped to themselves, connection passwords are write-only in the UI. Take the win — it’s a big one. Then keep being honest about what’s left. The DAG processor still runs whatever you give it. Teams in one deployment still share a credential namespace. Multi-tenancy is a convenience, not a wall.

So build accordingly. Put the real boundaries where they can actually be enforced — in the identity provider, in branch protection, in a Kubernetes service account, in an IAM policy, in a separate deployment — and use Airflow’s own controls for what they’re genuinely good at: making sure the person clicking “clear” meant to, and that the person who never needed the connections page never saw it. That’s a defensible posture. “Everyone’s an admin because it was easier” is not, and the day that stops being a philosophical position and starts being an incident is not a day you get to schedule.

That’s the book. Seventeen chapters of making Airflow do more, and one of making sure it only does it for the right people. Go run something.

Examples in this chapter are docs-checked against the series’ stated Airflow 3.3.0 version, but they were not executed in this repository.

Comments