A model can score well on a spatial task for the wrong reason. The narrow hypothesis in this article is that raw pixel coordinates may let a small network fit the training canvas while quietly tying its decision boundary to the coordinate range. That dependence can remain invisible until the same geometry appears on a larger page or inside a shifted crop.

I tested that hypothesis with a deliberately small system. Two points define one of five relations: left, right, above, below, or near. The relation depends only on their displacement after dividing by canvas size. I trained the same MLP with four ways of writing the coordinates, then changed the canvas without changing the rule. I then repeated the intervention with a tiny transformer classifier built from PyTorch’s TransformerEncoderLayer.

The result was not that one encoding won everywhere. Raw pixels actually produced the best in-domain score. The useful signal appeared only after the translation intervention: its mean accuracy fell from 0.908 to 0.535, while normalized coordinates retained 0.817. That gap is the reason to treat coordinate tests as interventions, not ordinary held-out evaluation.

The probe: keep the geometry, change its address

The synthetic sample is four numbers,

Let the canvas width and height both be $C$. The normalized displacement is

Pairs closer than $0.25$ in normalized Euclidean distance receive the class near. Otherwise, the larger absolute component decides whether the second point is left, right, above, or below the first. Uniformly scaling every coordinate and the canvas leaves the label unchanged.

That last property gives us a clean intervention. We can train on a $256\times256$ canvas, then evaluate on $512\times512$ and $1024\times1024$ canvases. A second intervention moves all points into the upper-right half of a $1024\times1024$ canvas. The local relation is still generated by the same rule, but the absolute coordinate range is unfamiliar.

The four inputs are:

• Raw pixels: $(x_1,y_1,x_2,y_2)$.
• Normalized: each coordinate divided by $C$.
• 32-bin quantized: normalized coordinates rounded down to one of 32 bins, then rescaled to $[0,1]$.
• Fourier features: normalized coordinates plus sine and cosine features at four frequencies.

Discrete coordinate tokens are a real design choice in multimodal systems. Pix2Seq, for example, serializes bounding boxes as discrete tokens, while document models such as LayoutLMv3 combine text, image, and layout information. Fourier features offer another route: they map low-dimensional coordinates into periodic features that can help MLPs fit high-frequency functions [1–3]. None of those papers implies that one choice is universally best here; they establish that coordinate representation is part of the model, not harmless formatting.

The complete command trains both the MLP and the tiny transformer:

1
2

OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 \
python code/coordinate_probe.py --output-dir data

The retained summary from five seeds is:

1
2
3
4
5
6
7
8

mlp/fourier: in_domain=0.799±0.012, scale_2x=0.789±0.014, scale_4x=0.800±0.010, translated_crop=0.596±0.017
mlp/normalized: in_domain=0.883±0.025, scale_2x=0.882±0.027, scale_4x=0.887±0.016, translated_crop=0.817±0.073
mlp/quantized_32: in_domain=0.887±0.023, scale_2x=0.887±0.028, scale_4x=0.892±0.015, translated_crop=0.798±0.071
mlp/raw_pixels: in_domain=0.908±0.009, scale_2x=0.902±0.014, scale_4x=0.908±0.012, translated_crop=0.535±0.071
transformer/fourier: in_domain=0.923±0.007, scale_2x=0.918±0.009, scale_4x=0.919±0.011, translated_crop=0.650±0.043
transformer/normalized: in_domain=0.956±0.011, scale_2x=0.955±0.007, scale_4x=0.951±0.012, translated_crop=0.674±0.037
transformer/quantized_32: in_domain=0.952±0.006, scale_2x=0.950±0.011, scale_4x=0.950±0.007, translated_crop=0.679±0.024
transformer/raw_pixels: in_domain=0.348±0.018, scale_2x=0.345±0.020, scale_4x=0.342±0.012, translated_crop=0.246±0.015

The full transcript is in data/raw-output.txt, and every run is retained in data/results.csv.

The baseline gives the comforting answer

If we stopped at the ordinary held-out split, raw pixels would look best. Its mean in-domain accuracy was 0.908, compared with 0.887 for 32-bin coordinates and 0.883 for normalized coordinates. The difference is not large enough to support a universal ranking, but it creates a familiar temptation: keep the simplest input and move on.

The scale tests seem to confirm that choice. Raw pixels remained near 0.90 at both two and four times the training canvas size. This is the first point where intuition needs refinement. Scale alone did not expose the shortcut because the classifier could still use signs and relative magnitudes of coordinate differences. A network trained on raw values can discover a useful relation without learning the normalized rule we wrote down.

That is why a single out-of-distribution test is not enough. A good probe must alter the suspected nuisance factor while preserving the target property.

Moving the relation reveals the hidden dependence

The translated-crop test changes the absolute range without changing how labels are computed. All four coordinates now lie in the upper-right half of a larger canvas. Raw pixels fall to 0.535 mean accuracy. Normalized and quantized inputs retain 0.817 and 0.798 respectively.

Observation: the raw-pixel model is much more sensitive to the shifted range than it was to uniform scaling.

Interpretation: at least part of its decision surface depends on absolute coordinate location, not only on relative geometry. The intervention does not tell us exactly which hidden units carry that dependence, and it does not prove that every raw-coordinate model will fail this way. It does show that baseline and scale-only accuracy were insufficient evidence for the desired behavior in this fixture.

The normalized model also drops by roughly seven points. That matters. Normalization removes one obvious nuisance variable, but it does not force translation independence. The MLP still receives four absolute normalized positions. To build that property into the input, we could encode displacement and distance directly, augment with translated examples, or use an architecture whose pairwise relation is explicit.

Fourier features did not help here. Their translated score was 0.596, and their in-domain score was lower than the simpler normalized input. This is not a contradiction of the Fourier-feature literature. The target function in this experiment is mostly made of broad directional regions plus a circular near boundary; the chosen frequencies, optimizer, model size, and training budget may be poorly matched. More features also make optimization harder in a small fixed-budget probe.

A tiny transformer changes the model, not the contract

The transformer variant turns the two points into two sequence tokens. Each token carries its point coordinates under the selected encoding. A learned CLS token passes through one PyTorch TransformerEncoderLayer with four attention heads, then a linear head predicts the relation. This is still intentionally small, but it tests whether the conclusion was merely an MLP artifact.

It was not. With normalized and 32-bin coordinates, the transformer reached roughly 0.95 in-domain accuracy, higher than the MLP. Under the translated crop, those same encodings fell to 0.674 and 0.679. The model got stronger, but absolute normalized point positions were still visible to the classifier, and the origin shift still changed behavior.

Raw pixels were worse: the transformer reached only 0.348 in-domain accuracy and 0.246 on the translated crop. I do not read that as a clean generalization failure. It is an interface failure. Feeding unscaled pixel magnitudes directly into a tiny attention block made optimization poor before the OOD test even mattered.

What each encoding preserves—and what it throws away

The experiment becomes more useful when read as a set of trade-offs rather than a leaderboard.

Quantization deserves a separate caution. Thirty-two bins were enough for this coarse five-way label, but that says little about insertion points, small clearances, wall thickness, or nearly touching boxes. Two distinct coordinates in the same bin are indistinguishable to the downstream model. A serious evaluation should sweep vocabulary size and concentrate examples near decision boundaries.

Raw source units are not inherently wrong either. They may be required when physical dimensions matter. The mistake is to let the model infer which changes are meaningful. A floor-plan system might need both normalized page position and real-world distance, kept as separate fields with separate tests.

A better spatial evaluation contract

For document and floor-plan models, random train/test splits mostly answer whether the model interpolates within familiar layouts. They do not establish that the model uses geometry in the intended way. A stronger evaluation suite should preserve the semantic relation while changing one geometric nuisance at a time:

1. Canvas scale: resize coordinates and canvas together.
2. Translation: move the same local configuration across the page.
3. Crop and padding: alter the page frame while preserving object relations.
4. Rotation or reflection: apply only when the task definition says the label should transform or remain fixed.
5. Quantization boundary: place examples just above and below token-bin edges.
6. Mixed units: separate page-relative position from physical distance rather than silently combining them.

The test oracle must be explicit. For this article, the label function is public and exact, and the tests verify that uniform scaling leaves labels unchanged. The bundle includes six direct tests, including a counterexample that confirms raw pixels do not preserve the encoded values under scale even though the trained classifier happened to preserve accuracy.

Run them with:

1

PYTHONPATH=code pytest -q code/test_coordinate_probe.py

Expected result:

1

6 passed

In a production research platform, the same pattern can become a pre-merge evaluation: every new spatial encoding or tokenizer must pass relation-preserving transformations, report uncertainty across seeds, and retain failure slices. That is more informative than asking whether the aggregate validation number moved by half a point.

What the probe changed—and the next test

The baseline suggested that raw pixels were not merely adequate but slightly better for the MLP. The translated crop changed that conclusion, and the tiny transformer made the broader point sharper: a modern attention block does not remove the need to define the coordinate contract. I would no longer approve a coordinate representation for a spatial model based on in-domain accuracy plus a resize test. I would require controlled shifts of origin, crop, scale, and bin boundaries, with the target relation held fixed.

The tested boundary is narrow: a two-point synthetic classification task, a small MLP, a one-layer transformer, five seeds, four encodings, and CPU PyTorch. It does not settle how a large vision-language model should encode floor plans. Its value is diagnostic. It gives us a way to catch one class of shortcut before model size, image features, and language context make the failure harder to see.

The next discriminating experiment is to train on mixed canvas sizes and translated crops, then compare three inputs: absolute normalized coordinates, relative displacement features, and both together. Add rotation only after defining which labels should rotate. That experiment would tell us whether data augmentation repairs the observed dependence or whether the input should expose the desired geometry directly.

References

1. Chen et al., “Pix2Seq: A Language Modeling Framework for Object Detection,” 2021.
2. Huang et al., “LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking,” 2022.
3. Tancik et al., “Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains,” 2020.