Release the Bytes You Tested

The orders release passed its tests in staging. Production was then built from the same Git tag, with a different Maven profile that filtered a properties file into the archive. When the warehouse host turned out to be wrong, the team could name the commit but could not say which bytes had passed the tests.

Two archives built from the same commit can contain different bytes — archive timestamps, resource filtering and resolved dependencies all matter. A release needs the identity of the tested file, not just a source branch name.

The companion now includes a validation workflow. It checks the example catalogue, runs the actual lesson MUnit suites, requires their reports, packages the recovery candidate and records a digest. It has no deployment credentials and performs no platform publication.

Review a release from the editor

Open the companion root in ACB Explorer and inspect the pipeline file referenced in Example 070. Use the integrated terminal for its local verification commands. The ACB Testing view is useful while changing a flow; the pipeline repeats those tests without an interactive editor and retains reports with the candidate artifact.

Before packaging, save all modified canvas components and inspect the source-control diff. ACB serializes the visual changes into the application source. Include those changes, resources, dependencies and the version manifest in the release input; a screenshot cannot rebuild an application. When comparing a rollback candidate, use Explorer’s Select for Compare / Compare with Selected on configuration files and the recorded artifact digests, then follow the controlled promotion procedure below.

Inspect the executable validation job

Every stage of a release should be able to say what it proved — and keep the report that supports the claim. This one is deliberately modest, and saying so plainly is part of what makes it useful.

Example 070 — Validate the companion in CI

Source: ../.github/workflows/book-validation.yml.

name: Book companion validation
on:
  push:
    paths: ['book/**', '.github/workflows/book-validation.yml', 'settings.xml']
  pull_request:
    paths: ['book/**', '.github/workflows/book-validation.yml', 'settings.xml']
  workflow_dispatch:
permissions:
  contents: read
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: '17'
          cache: maven
      - uses: stCarolas/setup-maven@v5
        with:
          maven-version: 3.9.4
      - name: Check catalog and complete sources
        run: python3 book/check_catalog.py
      - name: Run the actual lesson MUnit suites
        run: |
          python3 book/run.py test 6
          python3 book/run.py test 10
          python3 book/check_reports.py
      - name: Package the recovery candidate
        run: |
          python3 book/run.py package 30
          python3 book/release_digest.py book/checkpoints/30-recovery-capstone/target/orders-learning-2.0.0-mule-application.jar
      - name: Retain evidence
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: book-validation-evidence
          if-no-files-found: error
          path: |
            book/checkpoints/06-first-tests/target/surefire-reports/
            book/checkpoints/10-enrichment-tests/target/surefire-reports/
            book/checkpoints/30-recovery-capstone/target/*.jar
            book/actual/release.json

The workflow uses Java 17 and Maven 3.9.4, matching the local build. It calls the same helper commands a reader can run. The chapter 6 suite checks greeting and arithmetic; the chapter 10 suite tests targeted enrichment. Those are deliberately small lesson tests. The independent HTTP checks exercise the complete local application and its real H2, file, queue and cache boundaries. A mocked dependency verifies the flow’s reaction to a convenient object — it cannot verify connector authentication, TLS trust, database driver behavior or broker acknowledgement.

The workflow does not start a licensed standalone runtime or claim that packaging the capstone reruns its integration matrix. A production release gate must execute its required integration and deployed acceptance stages under the appropriate runtime/account entitlement. Keeping that distinction visible prevents a green unit-test job from inheriting unrelated local evidence.

Example 071 — Reject missing or empty test evidence

Source: check_reports.py.

#!/usr/bin/env python3
"""Fail if required lesson MUnit reports are absent, empty or failing."""
from pathlib import Path
import xml.etree.ElementTree as ET
B = Path(__file__).resolve().parent
for folder, minimum in [('06-first-tests', 2), ('10-enrichment-tests', 1)]:
    reports = list((B / 'checkpoints' / folder / 'target/surefire-reports').glob('TEST-*.xml'))
    assert reports, f'No MUnit reports for {folder}'
    counts = dict(tests=0, failures=0, errors=0, skipped=0)
    for report in reports:
        root = ET.parse(report).getroot()
        for key in counts:
            counts[key] += int(root.get(key, '0'))
    assert counts['tests'] - counts['skipped'] >= minimum, (folder, counts)
    assert counts['failures'] == counts['errors'] == counts['skipped'] == 0, (folder, counts)
    print(folder, counts)

A successful process exit can occur with tests skipped, or with no matching suites at all — which is where an exit code stops being evidence. This check requires report files, the expected minimum executed counts, and zero failures, errors or skips. It preserves the reason to fail even if a JAR was created.

The report gate itself should fail when a report is absent, empty, failing or entirely skipped. The local verification record includes those negative controls. GitHub-hosted execution is a separate result: checking in a workflow and running its commands locally does not prove that a remote runner has completed it.

Give the archive an identity

Example 072 — Record a candidate digest

Source: release_digest.py.

#!/usr/bin/env python3
"""Record the exact candidate bytes without rebuilding them."""
import hashlib, json, subprocess, sys
from pathlib import Path
B = Path(__file__).resolve().parent
artifact = Path(sys.argv[1]).resolve()
assert artifact.is_file(), artifact
record = {'artifact': artifact.name, 'sha256': hashlib.sha256(artifact.read_bytes()).hexdigest(),
          'commit': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=B, text=True).strip(),
          'runtime': '4.12.3', 'javaMajor': 17,
          'sourceWorktreeDirty': bool(subprocess.check_output(['git', 'status', '--porcelain'], cwd=B, text=True).strip()),
          'boundary': 'local candidate; not Exchange publication or cloud acceptance'}
(B / 'actual').mkdir(exist_ok=True)
(B / 'actual/release.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps(record, indent=2))

The script hashes the actual archive and records the source commit and runtime boundary. It does not rebuild. A friendly release name proves nothing about identity — the digest is what connects a deployed file to the tests it passed. Retain that record with the test reports and the archive. Before a deployed candidate is accepted, add its environment configuration version, secret-version identifiers, policy versions and smoke-test evidence.

The historical companion also recorded two offline rebuilds under Mule Maven Plugin 4.10.0 with different SHA-256 digests. That observation belongs to those old builds; it is not a reproducibility measurement of this new checkpoint. It is one concrete reason the release procedure consumes a fixed file.

Publication and promotion are different jobs

A promotion job can accidentally rebuild the artifact it was meant to deploy, which is the whole reason the difference between a lifecycle phase and a plugin goal is worth knowing. An authoritative release build can run tests and publish fixed coordinates to Exchange. Fetch those coordinates back and compare the digest before staging. Production then uses the same published release with its own complete configuration, through the direct deployment goal from chapter 25.

mvn clean deploy invokes lifecycle phases, including the build and bound tests. mvn mule:deploy invokes a plugin goal. The similar word does not make them interchangeable: a promotion job must not accidentally rebuild the candidate whose evidence it was approved to consume. Maven lifecycle.

Revoking a build runner should not require rotating every business client, so publication and deployment get a controlled machine identity of their own, with the required environment permissions and nothing beyond them. Pull-request validation has no need for production credentials. Masking in logs is a backstop — not disclosing the value at all is the control that works — so keep private Maven settings outside retained artifacts and avoid printing effective configuration with secrets expanded.

Rehearse restoration before calling it rollback

Rebuilding an old tag during an incident reintroduces dependency resolution and packaging uncertainty exactly when there is no time for either. “We can rebuild the old branch” is a weaker position than holding the previous archive, because it leaves rollback dependent on repositories, credentials and build tools being available during the outage. So retain that previous compatible artifact and its complete configuration, and confirm that both releases can read the schema and consume the event versions present during rollout. Database migrations and already committed effects are not reversed by copying an older JAR.

In a sandbox, introduce a dependency endpoint failure after deploying a candidate. Stop promotion when the declared failure rate or oldest-pending-age condition is reached. Restore the previous artifact/configuration pair, then process the pending synthetic event through its idempotency boundary. Record the receiver effect count as well as restored process health.

A revoked credential can make the old configuration unusable. A new event schema can make an old consumer unable to drain the queue. Plan compatibility and secret rotation with the rollback window, rather than discovering these dependencies during an incident. And when a release does fail, preserve the evidence before retrying: deployed artifact identity, effective configuration version, platform events, sanitized logs and the first failing correlation ID. A second deployment should test a specific explanation rather than erase the first attempt.

Try it

1. Hide a report. Why must the release gate fail when Maven succeeds but no required test report exists?

Show answer

The required evidence is missing. Packaging success cannot establish that the intended suites ran. Restore the report path or test selection, then rerun the gate.

2. Rebuild for production. What breaks the argument that staging tested the production artifact?

Show answer

A rebuild creates a new archive. Even equal source does not establish equal bytes or configuration. Promote the retained published candidate and compare its digest instead.

3. Restore a working release. Which state remains changed after artifact rollback?

Show answer

Committed database/receiver effects, message delivery history, dead-letter state and external actions remain. Compatible configuration and live credentials must be restored separately, and unresolved work still needs reconciliation or safe replay.

Not run. Exchange publication, GitHub-hosted workflow execution, CloudHub promotion and production rollback have not been performed as part of the local verification. The checked-in validation commands, report gate and candidate digest are independently executable.

Next: Decide Where an API Boundary Belongs