Results and Caching, or: Where Your Return Values Actually Live
If you don't know where your results land, you don't have caching — you have luck. The full cache-policy catalog, the hashing that bites, and the locking nobody configures.
The last chapter ended on a cliff: the same flow, the same retries=2, one setting apart — and one version re-ran the expensive extract three times while the other ran it once. The setting was whether the task’s return value got written down.
That’s the hinge of the whole book. Prefect’s headline features — a retry that resumes instead of restarting, a cache hit that skips the work, a transaction that rolls back a half-done load — are not three features. They are three questions asked of one store: have we already got a value for this key? Airflow cannot ask that question, not because nobody thought of it, but because Airflow has nowhere to look. A task instance’s return value goes to XCom if it’s small and nowhere if it isn’t, and XCom is a message-passing channel, not a result store you cache against and roll back through.
So results are the substrate, and caching is the first thing you build on it. But lead with the sentence I wish someone had put in front of me two years ago, because it’s the difference between the feature working and the feature appearing to work:
If you don’t know where your results land, you don’t have caching. You have luck.
And luck has a very specific expiry date. It runs out the moment your work crosses a process boundary — the moment the task that writes the result and the task that would hit the cache are two different pods, on two different nodes, and the “shared” result store is ~/.prefect/storage on a container filesystem that ceased to exist eleven seconds after the pod exited. Nothing errors. Nothing warns. Every run is a first run, forever, and the caching you configured is a decoration on a task that recomputes everything, every time, while you tell people in standup that you’ve turned on caching.
Let’s make sure that isn’t you.
What a result actually is
A result is a task’s (or flow’s) return value, as a durable artifact. That’s it. Not a dataset, not a table, not an XCom — the literal Python object your function returned, serialized and written somewhere Prefect can find it again by key.
By default, in a fresh Prefect install, that doesn’t happen. The return value lives in memory for the duration of the flow run and then it’s garbage. PREFECT_RESULTS_PERSIST_BY_DEFAULT ships as False, and the bare @task you’ve been writing all series has been quietly throwing its output away the instant the process ended.
Which raises the obvious question: then how did anything in the last chapter work at all, and how does the cache_policy=INPUTS you’ve seen in every Prefect tutorial ever manage to hit? The answer is a rule that is genuinely useful and genuinely under-documented. Prefect turns persistence on for you, implicitly, if you do any of five things. I went and read the constructor rather than guessing, and here is the actual list — if persist_result is left unset, it becomes True when any of these is true:
- you set a
cache_policythat isn’tNO_CACHE, - you set a
cache_key_fn, - you set a
result_storage_key, - you set a
result_storage, - you set a
result_serializer.
The logic is sound: every one of those settings is meaningless without a durable result, so asking for one implies the other. In practice this means you will almost never type persist_result=True — you’ll write a cache policy and persistence will follow. But it also means the inverse is the trap that bit us in the last chapter: a bare @task, with none of those five, gets the DEFAULT cache policy but does not persist. It computes a perfectly good cache key, looks in the store, finds nothing (because nothing was ever written), and runs. Forever. The key without the store is a lock with no door behind it.
Three questions decide a result’s fate: whether it’s persisted, where it’s written, and how it’s encoded.
Whether: persist_result and the global switch
Set it explicitly when you want it and mean it, with @task(persist_result=True). Or flip the process-wide default and stop thinking about it:
prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=trueFor any deployment on real infrastructure, I’d argue the global switch is the right call. The failure mode of persisting a result you didn’t need is a small file you didn’t need. The failure mode of not persisting one you did need is a flow retry that re-runs your six-minute extract, a cache that never hits, and a transaction record that never gets written — all silently. The costs are not symmetric.
persist_result=False is the explicit off switch, and it has one edge worth knowing: it wins over a cache policy, and it tells you so. Set it alongside a real cache_policy and Prefect forces the policy to NO_CACHE and logs Ignoring 'cache_policy' because 'persist_result' is False. It refuses to pretend. Use it on tasks whose return values are enormous and worthless — one that returns a 400 MB DataFrame the next task immediately reduces to a single number, where writing that object to S3 every run is pure cost.
Where: result_storage, and the moment local disk stops working
With nothing configured, results go to a LocalFileSystem rooted at ~/.prefect/storage (the PREFECT_LOCAL_STORAGE_PATH setting; I checked mine and it resolves to exactly that). Files there are named by an opaque hash — you can go look, and you’ll find a directory of UUID-ish filenames that tell you nothing.
On a laptop this is perfect. In production it is the single most common way a correct-looking Prefect setup does nothing at all, and the boundary doesn’t announce itself. Local disk works right up until your flow runs in a Kubernetes job and the next run gets a different pod with an empty volume; or you run two workers on two hosts and the result written by one is invisible to the other; or, generally, the process that writes and the process that reads are not the same process on the same machine with the same filesystem.
Cross that line and every cache lookup misses, every flow retry restarts from scratch, and every transaction record — because transaction records live in the same store — vanishes with the pod that wrote it. Your “we only load each day once” guarantee now holds once per pod. Which is to say: not at all.
The fix is to point result_storage at something every process can see:
from prefect import task
from prefect_aws import S3Bucket
@task(result_storage=S3Bucket.load("bookshop-results"))
def transform(rows: list[dict]) -> list[dict]:
return [clean(r) for r in rows]Note .load(), not a constructor. This matters, and it’s a hard error rather than a subtle one — Prefect will refuse an unsaved block outright:
TypeError: Result storage configuration must be persisted server-side. Please call
`.save()` on your block before passing it in, or use a string reference like
'local-file-system/my-storage' instead.The reason is sound: the worker on the other machine has to reconstruct that storage config from the API, and it can only do that if the block exists server-side. So either .save("bookshop-results") your block once and .load() it thereafter, or use the string-reference shorthand the error suggests — result_storage="s3-bucket/bookshop-results" — which resolves at runtime. Both work; I ran both.
Set it on the flow (@flow(result_storage=...)) to establish a default for every task inside, then override on the individual tasks that need somewhere else. That’s the shape I’d recommend: one storage decision at the top of the pipeline, inherited downward, rather than a bucket sprinkled across forty decorators.
How: result_serializer, and what’s actually in the file
The default serializer is pickle, which round-trips almost any Python object and ties the bytes to your Python environment. A pickled result written by Python 3.12 with pandas 2.2 may be unreadable to a worker on Python 3.11 with pandas 2.1 — and the failure will be an AttributeError deep inside an unpickle, at 3 a.m., which is an unpleasant way to learn about your deployment’s version drift.
json is the alternative: portable, human-readable, inspectable by anything, and constrained to JSON-friendly values.
@task(result_serializer="json", result_storage=S3Bucket.load("bookshop-results"))
def transform(rows: list[dict]) -> list[dict]:
return [clean(r) for r in rows]My rule: json for anything a human might want to look at, or that crosses a version boundary; pickle for rich in-process objects that only Prefect will ever read. If the result is list[dict] or a dict of numbers — which, in a data pipeline, it very often is — you’re paying pickle’s fragility tax for nothing.
Worth knowing what actually lands on disk, because people assume it’s their payload and it isn’t. I wrote a result with result_serializer="json" and opened the file:
{
"metadata": {
"storage_key": "orders/2026-05-30.json",
"expiration": null,
"serializer": {"type": "json", ...},
"prefect_version": "3.7.8",
"storage_block_id": "..."
},
"result": "[{\"title\": \"Dune\", \"qty\": 2}]"
}It’s an envelope: metadata plus your serialized payload as a nested string. A result file is not a data file. You can’t hand orders/2026-05-30.json to pd.read_json and expect joy, and you definitely shouldn’t point a downstream consumer at Prefect’s result store. It’s Prefect’s bookkeeping, in Prefect’s format, and Prefect reserves the right to change it.
result_storage_key, and the trap inside it
By default a result’s filename is a hash. result_storage_key lets you template it from the run context instead, so results are addressable and legible:
@task(
result_storage=S3Bucket.load("bookshop-results"),
result_storage_key="orders/{parameters[day]}.json",
result_serializer="json",
)
def extract_orders(day: str) -> list[dict]:
...A run for 2026-05-30 writes orders/2026-05-30.json. I ran it; that’s the path. The template draws from the run context and from parameters, so {parameters[day]} resolves to the day argument.
Now the trap, and it’s a good one, because it’s invisible. Setting a result_storage_key on a task with a cache policy replaces the cache key with the storage key. In the source: supply a result_storage_key with a cache policy that isn’t NO_CACHE, and Prefect sets self.cache_policy = None, then uses the formatted storage key as the task’s key instead.
Sometimes that’s exactly right — “the key for this work is the day, and the file is named after the day” is a tidy design. But it means your carefully-composed INPUTS + TASK_SOURCE policy is silently not in effect, and editing the function body no longer invalidates anything, because TASK_SOURCE isn’t in the key. There is no key. There’s a filename. Know which of the two you’re configuring, and don’t reach for both expecting them to compose.
The last knob here is cache_result_in_memory (default True), which decides whether Prefect also holds the value in memory after persisting it. For a big result you only need occasionally, cache_result_in_memory=False keeps the object out of the flow’s memory footprint and reads it back from storage when something asks. It’s the pressure valve for pipelines that pass large payloads between tasks — and the reason a flow that “shouldn’t” be using 8 GB sometimes is.
Results are not datasets
One clarification that saves an architectural argument later. A Prefect result is an orchestration artifact — the value Prefect stores so it can recover, cache, or inspect a task’s output, sitting in an envelope in the store you configured. The bookshop’s actual output table is still something a task writes on purpose, with a warehouse client, to the warehouse.
Conflate the two and you end up with people treating ~/.prefect/storage as a data lake, and then a dbt model reading from a pickle. Keep the buckets separate: results are how the framework remembers what ran; datasets are what your tasks deliberately write to the systems your consumers read. A task can and often should return a pointer — an S3 URI, a table name, a row count — rather than the payload. That’s a small result, cheap to persist, perfectly cacheable, and it doesn’t turn your orchestrator into a storage tier it was never designed to be.
Caching: the feature you used to hand-roll
Now the payoff. Re-running the bookshop transform for a day you already processed should do nothing — the inputs are identical, so the output is identical, so why recompute it.
In Airflow, “so why recompute it” is a project: a sensor to check whether the output exists, a BranchPythonOperator to skip the work if it does, a dummy task to join the branches back, and idempotency logic you wrote by hand and tested by hoping. In Prefect it’s an argument.
A cache policy describes how the cache key is computed. Prefect ships a catalog in prefect.cache_policies, and the whole vocabulary is six names:
INPUTS— key on a hash of the task’s arguments. Same inputs, same key, cache hit.TASK_SOURCE— key on the task’s source code. Edit the function body and the cache invalidates, instead of serving output produced by logic that no longer exists.RUN_ID— scope the key to the enclosing flow run, so a hit can only happen within one run. This is what makes flow-level retries reuse completed tasks without leaking results between separate runs.FLOW_PARAMETERS— key on the parameters the enclosing flow was called with. For when the question “is this stale?” is really about the run’s parameters rather than the task’s own arguments.NO_CACHE— never cache. Also the way to say “and don’t auto-persist this one either.”DEFAULT— what you get when you don’t choose. It is exactlyTASK_SOURCE + RUN_ID + INPUTS— I printed it to be sure — which means a bare task caches within a flow run and never across runs.
from datetime import timedelta
from prefect import task
from prefect.cache_policies import INPUTS
@task(cache_policy=INPUTS, cache_expiration=timedelta(hours=24))
def transform(rows: list[dict]) -> list[dict]:
return [clean(r) for r in rows]Call transform again with the same rows inside the window and it’s a cache hit: the body doesn’t run, and Prefect returns the persisted result. cache_expiration is a timedelta bounding how long a cached result stays valid; leave it off and the cache never expires on its own. There’s also refresh_cache=True, the “no, really, run it again” override that bypasses a hit and overwrites the record — the manual escape hatch you’ll want in exactly one situation, which is when you’re standing in front of a stale cache at 3 p.m. with an analyst waiting.
Composing with + and -
The catalog is small because the policies compose. + builds a compound policy that requires both dimensions in the key — DEFAULT is literally that composition made explicit.
The move you’ll reach for most is dropping RUN_ID, because dropping it is what turns a within-run optimisation into the across-run “re-running yesterday does nothing” behaviour you actually came for:
from prefect.cache_policies import INPUTS, TASK_SOURCE
@task(cache_policy=INPUTS + TASK_SOURCE, cache_expiration=timedelta(days=7))
def transform(rows: list[dict]) -> list[dict]:
return [clean(r) for r in rows]Now the cache survives across flow runs and across deployments, keyed on inputs and source together. Edit the transform and the source hash changes, forcing a recompute rather than serving output from logic you’ve since deleted. I’d keep TASK_SOURCE in nearly every cross-run policy, and the reason is in the next section.
- subtracts a named argument from the INPUTS hash — it doesn’t remove a policy, it adds to Inputs’ exclude list:
@task(cache_policy=(INPUTS - "run_note") + TASK_SOURCE)
def transform(rows: list[dict], run_note: str) -> list[dict]:
...run_note is logging metadata; changing it shouldn’t bust a cache that’s really about rows. (Because - operates on Inputs, it only means anything on a policy that contains INPUTS. TASK_SOURCE - "x" is not a thing worth typing.)
When the built-ins can’t express “the same work”
cache_key_fn lets you compute the key yourself. The signature — verified, because it’s the kind of thing people get backwards — is (context, parameters) -> str | None: context is the TaskRunContext, parameters is a dict of the task’s arguments, and returning None means “don’t cache this call.”
def by_day(context, parameters) -> str:
return f"bookshop-transform-{parameters['day']}"
@task(cache_key_fn=by_day)
def transform(day: str, rows: list[dict]) -> list[dict]:
...Now the key is the day, not the payload. I ran this with two genuinely different rows lists for the same day, and the second call was a cache hit that returned the first call’s result — which is either exactly what you wanted or a catastrophe, depending on whether “the same day” really does mean “the same work” in your domain. cache_key_fn is the sharpest tool in the chapter and it has no guard.
Note the prefix in that key: bookshop-transform-, not just the day. When you write your own key, you own the namespace. A cache key is a key in a shared store; two different tasks that compute the same key against the same storage will collide, and one will cheerfully serve the other’s result. The built-in policies mostly protect you here because TASK_SOURCE makes two different functions produce two different hashes — which is the real reason to keep it in the policy rather than caching on INPUTS alone. Roll your own and that protection is gone. Namespace deliberately.
How INPUTS actually hashes, and where it bites
INPUTS builds the key by hashing the resolved arguments, and the mechanism has three sharp edges. I went looking for them.
It hashes via JSON first, then cloudpickle. hash_objects tries a JSON dump with sort_keys=True; if that fails it falls back to cloudpickle; if that fails too it raises. The sort_keys matters more than it sounds: {"a": 1, "b": 2} and {"b": 2, "a": 1} produce the same hash (I checked — 850dc833d975... both times), so dict ordering won’t spuriously bust your cache. Good.
A task with no arguments never caches under INPUTS. This one is a genuine trap and it’s four lines into the source: Inputs.compute_key returns None if the inputs dict is empty, and a None key means no caching. So:
@task(cache_policy=INPUTS) # zero-argument task
def fetch_config() -> dict:
return load_from_vault()…will run its body every single time, forever, and nothing will tell you. I ran it four times across two flow runs and got four executions. If you want a zero-arg task cached, key it on something else — TASK_SOURCE, FLOW_PARAMETERS, or a cache_key_fn returning a constant.
And when the hash fails, Prefect swallows it. This is the behaviour I most want you to know, because it is so quiet. Pass an argument that can’t be serialized — a threading.Lock, a live database connection, an open socket — and the key computation raises internally… and the engine catches it, logs
ERROR | Task run 'transform-b0d' - Error encountered when computing cache key -
result will not be persisted.…sets the key to None, and runs your task normally. Nothing fails. The flow is green. Your task is simply, permanently uncached, and the only evidence is one ERROR line in a log nobody reads at INFO-and-above volume. I confirmed this: the task returned its value happily while caching was silently switched off underneath it.
So if a task you cached is mysteriously always recomputing, grep the logs for Error encountered when computing cache key before you do anything else.
Now — the folklore fix for this is to wrap the offending argument in quote(), and the folklore is wrong, so let’s kill it here. from prefect.utilities.annotations import quote (that’s the import path; there is no from prefect import quote) does one documented thing: it tells Prefect not to introspect the wrapped object when it’s passed as a parameter. That’s a real optimisation — introspection walks large collections looking for futures to resolve, and on a big nested dict or a DataFrame that walk costs real time — and it disables dependency tracking for that argument as a side effect.
What it does not do is remove the argument from the cache key. I tested it directly: same task, cache_policy=INPUTS + TASK_SOURCE, one quoted argument, called twice with two different quoted values. If quote() excluded it from the key, the second call would be a hit. It wasn’t. The body ran both times. The quoted value is still in the hash.
The tool for excluding an argument from the key is the one designed for it:
@task(cache_policy=(INPUTS - "config") + TASK_SOURCE)
def transform(rows: list[dict], config: dict) -> list[dict]:
...Or a cache_key_fn that only looks at the parameters you care about. Use quote() for what it’s for — performance on large inputs — and - for the key. They are not the same knob, and I have seen quote() deployed as a caching fix in production code where it did precisely nothing.
Isolation and locking: the race nobody configures
The subtle bug in any cache is two runs computing the same key at the same time. Both check the store. Both see a miss. Both do the expensive work. One clobbers the other, and if the work had side effects, you did them twice. The scheduled 03:00 run and the engineer who clicked “Run” at 03:00:02 have just double-loaded your warehouse, and the cache you added to prevent duplicate work did nothing — because a cache without a lock is a suggestion.
Prefect 3 lets you make the cache transactional. A policy carries an isolation level, and SERIALIZABLE demands a lock manager:
from pathlib import Path
from prefect import task
from prefect.cache_policies import INPUTS, TASK_SOURCE
from prefect.transactions import IsolationLevel
from prefect.locking.filesystem import FileSystemLockManager
policy = (INPUTS + TASK_SOURCE).configure(
isolation_level=IsolationLevel.SERIALIZABLE,
lock_manager=FileSystemLockManager(lock_files_directory=Path("/srv/prefect/locks")),
)
@task(cache_policy=policy)
def transform(rows: list[dict]) -> list[dict]:
return [clean(r) for r in rows]The two levels, and there are only two:
READ_COMMITTED(the default) does no locking. It guarantees you won’t read a half-written result, and that is all it guarantees. Two concurrent misses on the same key will both proceed and both compute. Fine when your runs are naturally serialized — one schedule, one deployment, no overlap.SERIALIZABLEtakes a lock on the cache key for the life of the computation. Of two runs racing on the same key, one computes while the other blocks, and then reads the committed result. This is the guarantee you’d otherwise implement with an advisory lock in Postgres and a lot of care.
.configure() takes exactly three things — key_storage, lock_manager, isolation_level — and Prefect will not let you fake the last one. Ask for SERIALIZABLE without a lock manager and you get a hard failure at task start, not a warning:
ConfigurationError: Isolation level SERIALIZABLE is not supported by provided
configuration. Please ensure you've provided a lock file directory or lock
manager when using the SERIALIZABLE isolation level.Good. That’s a framework refusing to let you believe something false.
Three lock managers, and choosing wrong is the whole game:
MemoryLockManager()(prefect.locking.memory) coordinates threads within one process. Useful in tests and for concurrent tasks inside a single flow run. Useless across processes.FileSystemLockManager(lock_files_directory=Path(...))(prefect.locking.filesystem) coordinates every process that can see that directory. Note the kwarg — it’slock_files_directory, and it wants aPath. (It is notroot_dir; I have writtenroot_dirbefore and it does not exist, and I’d rather say so than let you find out.) On one box this coordinates everything. Across a fleet of Kubernetes pods that share no filesystem, it coordinates nothing — each pod locks a directory only it can see. “We run on K8s and use the filesystem lock manager” is a sentence that means “we have no lock.”RedisLockManagerfromprefect-redisputs the lock somewhere every worker can genuinely see. If your workers span machines and you needSERIALIZABLE, this is the answer and the others are theatre.
Match the lock’s blast radius to the race you’re actually trying to prevent. The race is almost always “two workers, two machines,” and exactly one of these three options addresses it.
A cache hit skips retries — and everything else
Tie the two chapters together with the order of operations, because it explains several behaviours at once.
On a cache hit, the task body never runs. Not “runs and returns early” — never runs. Which means:
- Retries never engage. There is nothing to retry, because nothing executed. A task with
retries=5and a cache hit has zero attempts, not one. timeout_secondsnever engages. You can’t time out work you didn’t do.retry_condition_fnis never called. No failure, no predicate.- The task’s side effects don’t happen. This is the important one, and it is the seam where caching becomes dangerous instead of merely useless: if your task writes to a table and returns a value, a cache hit gives you the value and skips the write. Which is either the idempotency you wanted or a data loss you didn’t, and the framework cannot tell the difference.
I ran this to be certain of the ordering. A task with cache_policy=INPUTS + TASK_SOURCE and retries=5, whose body raises on any call after the first: the first flow run computes and stores; the second returns the stored value with the body untouched — the RuntimeError sitting in there was never reached. Prefect checks the cache first, and only a miss proceeds to execute, at which point the whole failure-handling apparatus from the last chapter applies to that execution and only that one.
It also explains the flow-retry experiment. On the second flow attempt, every completed task is a cache hit and every incomplete one is a miss, so the retry re-runs exactly the unfinished work — provided the results were persisted. Flow retries don’t skip tasks. Caching skips tasks. The flow retry just re-executes the function and lets the cache do the rest.
In the UI, all of this shows up as a state named Cached — which is a successful terminal state (is_completed() returns True for it, and code that checks for success shouldn’t have to special-case it). Hold onto that word. It’s about to turn out to be a transaction.
Three idempotency mechanisms, kept straight
Prefect gives you more than one way to say “don’t do this twice,” and using the wrong one is the most expensive mistake in this part of the book. Disentangle them now:
Caching dedupes the execution of one task, by cache key. A hit returns the persisted result and skips the body. Its question is “did we already compute this?” It is the right tool when the work is pure enough that the same inputs genuinely mean the same output. It has no notion of undoing anything — it can skip a write, but it cannot un-write one.
Transaction keys dedupe a group of tasks that must land together. You wrap them in with transaction(key=...), and the block gets commit-or-roll-back semantics: if the middle blows up, the compensating hooks on everything that already succeeded fire, and the world goes back to how it was. Its question is “did we already commit this — and if half of it lands, how do I undo the half?” That’s the next chapter, and it’s the tool for side effects that must be all-or-nothing.
Manual idempotency keys dedupe the creation of a run, at the orchestration boundary — an idempotency_key passed when you trigger a deployment, so a retried webhook or a nervous cron produces one run and not two. It has nothing to do with task output. It’s about not launching the pipeline twice in the first place.
The trap is using caching where you meant a transaction. A cached task that has already written to a table is fine on re-run — the cache skips the write, the table is correct. A task that failed halfway through writing is exactly what caching cannot help you with: there’s no cache record (it failed), so a re-run redoes the write on top of a table that already has half of it. Caching answers “did we compute this.” Transactions answer “did we commit this.” When your task touches the outside world, you almost always meant the second one.
And now for the part that reframes everything you just read. Those are not two systems that happen to look alike. A task’s cache key is its transaction key — the same string, computed by the same compute_key call, written to the same store. Every task run in Prefect 3 opens a transaction whose key comes from its cache policy; a “cache hit” is that transaction discovering its record already exists and being born committed; refresh_cache is passed straight through as the transaction’s overwrite. Even the persistence rule falls out of it: the task’s transaction is created with write_on_commit=should_persist_result(), which is precisely why a task that doesn’t persist its result never writes a record, and therefore never hits a cache, and therefore re-runs on every flow retry — the mystery we opened this chapter with, solved by one keyword argument deep in the engine.
The Cached state you see in the UI is a transaction that found its record already written. That’s not an analogy for how caching works. That is how caching works.
Final thoughts
Results are the least glamorous chapter in this book and the one everything else is standing on. Get them wrong — leave persistence off, or point every worker at its own local disk — and Prefect’s three best features quietly degrade into nothing: retries restart instead of resuming, caches never hit, transaction records evaporate with the pod. Every one of those failures is silent. There is no error. Your pipeline just does more work than it should, forever, while the config file says otherwise.
Get them right and the payoff is immediate. INPUTS + TASK_SOURCE does, in one argument, what a sensor and a branch and a hand-written idempotency check used to do in a subgraph — and unlike that subgraph, it invalidates itself when you edit the code. That’s the trade this framework is offering: the thing you used to assemble becomes the thing you declare.
But declare it with your eyes open. Know where the results land. Know that an unhashable argument turns your cache off without telling you, and that SERIALIZABLE without a shared lock manager is a lock protecting one container from itself. Caching is not a performance tweak you sprinkle on. It’s a correctness claim about your task — “the same inputs mean the same output, and skipping this is safe” — and Prefect will take you completely at your word.
Which leaves the one thing caching can’t do: undo. A cache can skip work that already happened. It cannot reach into your warehouse and remove the half of a load that landed before everything went wrong. For that you need the same store, the same key, and one more guarantee.
Comments