Qiming Bao · Blog · 19 November 2026 · English / 中文

Do layout models need the table header? A negative result

In Long documents with LayoutLMv3 on a 16 GB GPU, one of the baselines splits each page into 512-token windows. A window that starts in the middle of a line-item table cannot see the table's header. Neither can a continuation page where the header is not printed again. It seems obvious that this should hurt: how does the model know a column is the rate and not the number of spots, if it never sees the word "Rate"?

I tested it on RealKIE FCC invoices (TV ad invoices with long line-item tables) with LayoutLMv3 and ERNIE-Layout. The short answer: the models learned to find the header reliably, and giving them the header did not help. Without any header, they already get about 86 line-item F1 on headerless chunks.

Finding the header

The first question is how the model knows which rows are the header. I tried three things on 30 hand-checked test pages:

The labels (202 training and 67 test documents) were made from page images with an AI model and checked against the 30 hand-checked pages, where all 36 header rows matched.

Feeding it to every window

The prefix method prepends the most recent header words to every window. They keep their x coordinates, so they still line up with their columns, and are moved to the top of the page. That header comes from earlier in the same page or from an earlier page. At test time it comes from the model's own predictions, with pages processed in order. Prefix words are not scored. An oracle variant uses the gold header instead.

Measuring it properly

The test set has only 23 real continuation pages without a header. On them, the first results looked good: the prefix gave LayoutLMv3 +2.1 points, winning on all 5 seeds. With ERNIE-Layout the picture was different: the tag alone did nothing and the prefix gave +2.0. But 23 pages are too few. The same ERNIE-Layout setting scored 82.7 in one run and 80.3 in a rerun.

So I built a bigger test: 436 synthetic headerless chunks. Each test page with a header is cut just below the header, or 4 or 8 rows further down. The rest of the page is moved to the top of a blank page, image and boxes alike, so it looks like a continuation page. Line-item F1 on these chunks, 5 seeds:

modelnonetagprefixoracle
LayoutLMv386.3685.5986.0486.03
ERNIE-Layout86.7286.5587.0087.02

Prefix against none is -0.3 for LayoutLMv3 (paired t-test p = 0.66) and +0.3 for ERNIE-Layout (p = 0.25). The tag alone is slightly worse for LayoutLMv3 (-0.8, all 5 seeds, p = 0.02). Overall F1 on the full test set does not move either (66 to 67.6 for every setting). The 2-point gains on the 23 pages were noise.

Why the header does not matter here

Two observations explain it:

The prefix with predicted headers also scored the same as the oracle, so header detection was not the limit.

Takeaways

Code, labels and results are in layoutlmv3-pretraining: scripts/table_header_prefix.py, scripts/build_headerless_cuts.py and data/table_header/. W&B runs for the 436-chunk evaluation: LayoutLMv3 none, tag, prefix; ERNIE-Layout none, tag, prefix.

I used an AI coding assistant (Claude) to help write the code, label the header rows and run the experiments, and I checked the results myself.