Appendix A: Version Four Is a Number, Not a Format

Two tables created by Spark 4.1.3 with Iceberg 1.11.0, one at format version 3 and one at format version 4, with the same two rows and the same merge-on-read delete. Then their metadata files, read off the store.

  v3 metadata.json: format-version 3, keys [current-schema-id, current-snapshot-id, default-sort-order-id, default-spec-id,
                    format-version, last-column-id, last-partition-id, last-sequence-number, last-updated-ms, location, next-row-id, table-uuid]
  v4 metadata.json: format-version 4, keys [current-schema-id, current-snapshot-id, default-sort-order-id, default-spec-id,
                    format-version, last-column-id, last-partition-id, last-sequence-number, last-updated-ms, location, next-row-id, table-uuid]

The key lists match — the version numbers differ. That alone was enough to change which engines could open the second table.

  Trino 483:         count(*) on v4 -> 1        (reads it)
  DuckDB 1.5.5:      count(*) on v4 -> 1        (reads it)
  PyIceberg 0.11.1:  load_table v4 -> FAILED: Input tag '4' found using 'format_version' | 'format-version'
                     does not match any of the expected tags: 1, 2, 3

This appendix is the state of format version 4 as of this edition, September 2026: what the jar says, what the spec says, and the distance between them. It is verified two ways and labelled accordingly. The behaviour of the pinned release was run on the lab. The state of the specification is taken from the project’s own record, the Iceberg Summit 2026 recap and the July 2026 status of the developer list, and is docs-verified. It will be updated per edition, and it asserts nothing about unreleased behaviour as behaviour.

What the jar says

  org.apache.iceberg.TableMetadata (iceberg-spark-runtime-4.1_2.13-1.11.0.jar):
    DEFAULT_TABLE_FORMAT_VERSION = 2
    SUPPORTED_TABLE_FORMAT_VERSION = 4
    MIN_FORMAT_VERSION_ROW_LINEAGE = 3
  CREATE TABLE … TBLPROPERTIES ('format-version'='5'):   Unsupported format version: v5 (supported: v4)
  ALTER TABLE v3 SET TBLPROPERTIES ('format-version'='4'):   OK, table reports 4

Book 1’s record noted that a format-version of 4 was accepted, and this is why. The library’s supported maximum is 4, one ahead of the released specification, because the version constant is raised when work on a version begins and not when it ships. A request for 5 is refused by name. A request for 4 is granted, and the table that results is a v3 table with a 4 written on it. The same metadata fields, the same row-lineage columns, the same delete files, nothing that v4 proposes.

The number is accepted — the format is not there. That is not a defect in the library, whose developers need to build v4 tables to build v4. It is a trap for anyone who reads “supported: v4” as an invitation.

What the specification says

The developer list’s status as of July 2026, which the Summit in the spring set out.

Voted into the specification. Relative paths, passed in May 2026, which allow every metadata location field to be relative to the table’s base location instead of an absolute URI. Chapter 16 spent a section on why that matters. A replica under a different bucket name registered and then failed on its first manifest list, and chapter 2’s repair for that was a full metadata rewrite. Relative paths make a relocated table readable without one. Also voted in May: a typed representation for content statistics, replacing the generic column-statistics maps with structures that preserve logical and physical type information. And in early June, a compact bitmap format, intended to make the bitmap structures behind deletion vectors leaner and more broadly useful.

In active design, with public syncs. The adaptive metadata tree and single-file commits, which are the proposals that would change chapter 9 most. A commit that writes one file instead of a manifest, a manifest list and a metadata file. A metadata structure that adapts its depth to the table’s size. The open question inside them — whether the partition tuple stays in the metadata or is reconstructed from column bounds — decides what chapter 7’s partition summaries become. Offloading snapshot history from the metadata file, which is chapter 9’s 558 KB problem addressed at the format rather than by retention. The open question there is how a reader discovers offloaded history for time travel.

Contested. Efficient column-level updates, and whether they belong in Iceberg, in Parquet, or both. That one decides whether chapter 12’s copy-on-write merge of a whole row to change one column has a cheaper future.

Release. No v4 specification is released, and no v4 table can be set on the released specification. The 1.11 line from May 2026 is a v3 release, with v3’s deletion vectors, row lineage and variant type as its production features.

What this means for a platform

Do not set format version 4 on any table that another engine will read. The lab’s PyIceberg refused the table by schema validation before reading a byte. PyIceberg is chapter 12’s consumer, chapter 14’s recovery tool and chapter 16’s replica catalog. Trino and DuckDB read the table because they check the fields rather than the number. That leniency is the reason the number is dangerous — a table can be v4 for months on two engines before the third one is pointed at it.

Chapter 5’s certification matrix includes format version, with v4 tested per engine. The mixed results above show why a successful Spark write cannot establish compatibility for the platform. When v4 is ready, the same tests will identify which readers can use it.

The proposals that are voted are the ones to plan for. Relative paths change chapter 16’s replication design. A replica under a different bucket becomes readable without a rewrite, and the rewrite step in that chapter’s game day becomes conditional on the table’s version. Content statistics change nothing operational. The compact bitmap changes chapter 8’s delete-file arithmetic at the margin. None of the three is available in the pinned release, and none of them is claimed by this book as behaviour.

The proposals in design are the ones to watch, not plan for. Single-file commits would rewrite chapter 2’s request table, chapter 9’s manifest section and chapter 11’s commit-rate arithmetic. Snapshot offloading would rewrite chapter 9’s metadata-file section and chapter 10’s retention-as-payload argument. If they land, the next edition of this book re-measures them. Until then the numbers in those chapters are the numbers.

What was run, and what was read

Run on the lab, against the pinned release: the jar’s constants by javap; the creation, write and merge-on-read delete of v3 and v4 tables through Spark; the metadata files read from the store. Also the reads by Trino, DuckDB and PyIceberg, and the upgrade of a v3 table to v4 by ALTER TABLE. Read from the project’s record: the votes of May and June 2026, the design status of July 2026, and the release status of the 1.11 line. Not run, because it does not exist to run: any v4 behaviour.

Final thoughts

The pinned release will write the number four on a table today — and one of the platform’s four engines will refuse to open it. The specification that gives the number a meaning has three pieces voted in, three in design and one in dispute, and no release. The right response is the one this book has taken to every version claim: the jar is the fact, the specification is the intent, and a platform runs on facts. When the two agree, this appendix gets rewritten.