Misk Configuration & Environments: YAML, Secrets, and Wisp

Series: Building Production Services with Misk — Part 21 of 24

You build one JAR. You ship it to a laptop, to CI, to staging, to production — and somehow the same bytes have to know they should talk to a local MySQL in one place and a real cluster in another, with different ports, different secrets, and different blast radius if you get it wrong. That’s the whole job of misk config: keep the artifact identical and let where it runs select what it reads. Part 2 introduced the Config interface, the $service-common.yaml / $service-$env.yaml pair, Secret<T>, and @Redact at a walking pace. This post goes underneath them — how misk environments are modeled, the exact order YAML files merge, how secrets actually resolve at load time, and the Wisp layer that does all of this without Guice.

Environments and Deployment

Misk does not have an Environment enum with five hard-coded values. It has Deployment, a Wisp data class describing where the process runs:

data class Deployment(
  val name: String,
  val isProduction: Boolean = false,
  val isStaging: Boolean = false,
  val isTest: Boolean = false,
  val isLocalDevelopment: Boolean = false,
)

The four booleans are mutually exclusive — the init block enforces it, so you can’t construct a deployment that’s both production and test. On top of them sit two derived flags worth knowing: isReal (a managed cluster — staging or production) and its inverse isFake (CI or a laptop). Reach for those in code that branches on “am I in a real cluster?” rather than re-deriving it from the raw booleans.

There are four presets — PRODUCTION, STAGING, TESTING, DEVELOPMENT — and a getDeploymentFromEnvironmentVariable() helper that reads the ENVIRONMENT variable, lowercases it, and looks it up in a small map. That map has aliases: test maps to TESTING, dev to DEVELOPMENT. Unset? You get DEVELOPMENT. This is the safe default — an unconfigured box behaves like a laptop, not like production.

The crucial bridge is Deployment.mapToEnvironmentName(), because this is what selects your YAML:

fun mapToEnvironmentName() =
  when {
    isProduction -> "production"
    isStaging    -> "staging"
    isTest       -> "testing"
    else         -> "development"
  }

So the Deployment decides the environment name, and the name decides the filename. A staging deployment looks for myservice-staging.yaml; a test deployment looks for myservice-testing.yaml. Note the asymmetry: there’s no myservice-test.yaml even though test is a valid ENVIRONMENT value — the alias collapses to testing before it ever touches a filename. Name your override files after the environment name, not the env var you set.

Config resolution & overrides

Part 2 told you “common loads first, then the environment file overrides it.” Here’s the mechanism, straight from MiskConfig.load. It builds an ordered list of file names:

private fun embeddedConfigFileNames(appName: String, deployment: Deployment) =
  listOf("common", deployment.mapToEnvironmentName().lowercase(Locale.US))
    .map { "$appName-$it.yaml" }

For an exemplar service in staging that’s exactly ["exemplar-common.yaml", "exemplar-staging.yaml"], each resolved as classpath:/<name>. Then any overrideResources you passed to load are appended after those. The merge itself is the part to internalize:

for ((key, value) in configYamls) {
  if (value == null) continue
  result = mapper.readerForUpdating(result).readValue(value)
}

It’s a Jackson readerForUpdating applied in sequence, so the rule is last writer wins, per leaf node. This is a deep merge, not a file-level replace: if common defines web.port and web.idle_timeout, and staging only sets web.port, the timeout from common survives. You override the leaves you care about and inherit everything else. After the embedded files come the override resources, and after those, an optional overrideValues: JsonNode gets merged dead last — handy for tests and for programmatic top-of-stack overrides.

Two things bite people here. First, a missing environment file is not an error — if exemplar-staging.yaml doesn’t exist, that map entry is just null and skipped. The only hard failure is when every file is absent (could not find configuration files). So a typo’d environment name fails silently by simply applying no overrides. Second, by default MiskConfig.load runs with failOnUnknownProperties = false: an unrecognized YAML key logs a warning (with a Levenshtein-based “did you mean?” suggestion) and is ignored rather than throwing. Convenient, occasionally maddening — a renamed-but-not-removed key sits there doing nothing while you wonder why your change isn’t taking.

Secrets, in depth

A Secret<T> is a one-method interface — val value: T — and in YAML the field doesn’t hold the secret, it holds a reference:

apiKey: "classpath:/secrets/api_key.txt"
api_secret: "filesystem:/etc/secrets/service/api_secret.txt"

At load time SecretDeserializer reads that string and resolves it through the ResourceLoader, wrapping the result in a RealSecret(value, reference). The scheme prefix selects the backend, and Misk ships several: classpath:, filesystem:, memory: (in-memory map, for tests), environment: (host env vars), and 1password:. So the same Secret<String> field reads from a baked-in classpath file on a laptop and a mounted filesystem:/etc/secrets/... path in a Kubernetes pod — you swap the reference per environment YAML, never the config class.

The file extension on the reference is load-bearing. A .txt reference must map to Secret<String> and is taken verbatim. A .yaml reference is parsed by Jackson into the secret’s type parameter, which is how you get a structured Secret<DatabaseCredentials> — the YAML file at the other end of the reference deserializes into that class. Other extensions fall back to primitive coercion, and a missing extension on a non-string type throws (needs a file extension for parsing). Access is always config.apiKey.value — the indirection is the point: the secret only materializes when you dereference it.

Two redaction layers sit on top, and they’re distinct:

  • @Redact marks a non-Secret field (or a whole class) so it serializes as ████████. But — and Part 2 glossed this — @Redact only affects serialized YAML. The plain value still shows in toString(). Use it for sensitive-but-not-secret fields that live inline in YAML.
  • Secret<T> is redacted in both serialized output and toString(). RealSecret.toString() is hard-coded to RealSecret(value=████████, reference=$reference). If you want a value invisible everywhere, that’s the type to reach for, not the annotation.

This matters for the admin dashboard’s config tab. ConfigMetadataProvider renders the effective config via MiskConfig.toRedactedYaml(config, ...), and the tab has three modes (ConfigMetadataAction.ConfigTabMode):

  • SAFE — shows nothing that could leak a Misk secret.
  • SHOW_REDACTED_EFFECTIVE_CONFIG — the merged, effective config with secrets and @Redact fields masked.
  • UNSAFE_LEAK_MISK_SECRETS — dumps the raw YAML files, unredacted. The name is the warning.

The redacted view is only as good as your annotations: a sensitive field that’s neither a Secret<T> nor @Redact-tagged will render in cleartext in SHOW_REDACTED_EFFECTIVE_CONFIG. The dashboard is showing you exactly what your types declared — no more, no less.

For non-secret values you can also pull from resources using YAML variable syntax — ${environment:PORT} or with a default ${environment:PORT:-9000} — handled by ResourceAwareDeserializer. That’s for non-sensitive data and framework config classes you don’t own; don’t smuggle secrets through it, because nothing redacts it.

Wisp: config & deployment without Guice

Deployment lives in wisp-deployment, and that’s not an accident. Wisp is Cash App’s set of Guice-free libraries — the substrate Misk is built on but that you can use standalone in a CLI, a Gradle plugin, or anything that has no DI container. wisp-deployment gives you Deployment, the presets, and getDeploymentFromEnvironmentVariable() with nothing Misk-specific attached. (For tests, wisp-deployment-testing ships a FakeEnvironmentVariableLoader so you can pin ENVIRONMENT without touching the real environment.)

wisp-config is the parallel story for config loading. It’s a thin wrapper over the Hoplite library, and its model is config sources, not Misk’s appName-plus-environment convention:

val configSources = listOf(
  ConfigSource("classpath:/myapp-config.yaml"),
  ConfigSource("classpath:/myapp-defaults.yml"),
)
val myConfig: MyConfig = WispConfig.builder()
  .addWispConfigSources(configSources)
  .build()
  .loadConfigOrThrow<MyConfig>()

The precedence here is the opposite gotcha from MiskConfig, so read carefully: the first source added wins. If you want an environment file to override defaults, add the environment file first. (Compare that to MiskConfig, where the file listed last wins.) Two libraries, two directions — knowing which one you’re holding is the difference between an override that works and one that silently doesn’t. Environment variables, system properties, and user settings are always consulted first, on top of everything.

When do you reach for wisp-config over MiskConfig? When there’s no Guice. Inside a Misk service, MiskConfig.load plus ConfigModule.create is the path — it binds your config into the injector. In a standalone tool that just needs to read a YAML file into a data class, wisp-config is lighter and doesn’t drag in the framework.

Production notes & gotchas

  • Override files are named after the environment, not the env var. ENVIRONMENT=test resolves to the testing environment, which loads service-testing.yaml. There is no service-test.yaml. Get the name wrong and you get silent no-op overrides, not an error.
  • @Redact is not Secret<T>. @Redact hides a field in serialized YAML only; its value still leaks through toString() and logs. If a value must never appear anywhere, make it a Secret<T>. Don’t assume the annotation is enough for a credential.
  • The redacted dashboard reflects your annotations, not your intentions. A sensitive inline field that’s neither Secret<T> nor @Redact shows up in cleartext in the config tab. Audit your config class against the SHOW_REDACTED_EFFECTIVE_CONFIG view before trusting it, and keep the tab out of UNSAFE_LEAK_MISK_SECRETS in any shared environment.
  • Unknown keys are ignored by default. failOnUnknownProperties is off, so a misspelled or renamed key logs a warning and does nothing. Grep your build logs for the “not found in” warnings; they’re the difference between “my override isn’t working” and “my override has a typo.”
  • Mind the merge direction. MiskConfig merges last-wins; wisp-config is first-wins. If you ever move config between the two, the precedence flips and your overrides invert.
  • Secrets resolve at load, not lazily. MiskConfig.load dereferences every Secret<T> reference up front through the ResourceLoader. A missing filesystem:/etc/secrets/... mount fails the service at startup — which is what you want, but it means a misconfigured secrets volume is a boot failure, not a runtime surprise.

What’s next

Config decides how your service behaves; the next question is whether you can see what it’s doing once it’s running. In Part 22: Misk Observability we’ll wire up metrics, tracing, and logging — the Prometheus config you’ve been quietly carrying in your YAML this whole time finally earns its keep.

Target keywords: misk config, misk environments.

Comments