Glance, then Select:Evidence-Tempered Visual Token Selectionfor Extremely Compressed Multimodal LLMs

Anonymous CVPR submission

Anonymous review manuscript

submitted to CVPR 2027

Glance-ETD training and inference. Training distills a glance-encoder student with DivPrune toward a frozen full-token teacher. Inference re-encodes a diverse quarter of the image at native resolution, then Evidence-Tempered Diversity keeps a strict five percent of tokens. — Glance-ETD separates two decisions: where to spend native-resolution vision compute, and which visual tokens should reach the language model.

Abstract

At a 5% visual-token budget a multimodal LLM sees about one token in twenty, and two costs remain: the vision encoder still processes every patch of a large image, and the few surviving tokens must still contain the answer. Glance-ETD decides where to look and which tokens to keep. A glance encoder encodes the whole image at a quarter of the patches, picks 25% of the coarse cells by max–min diversity, and re-encodes only those at native resolution, producing a native-size token grid at a sixth of the encoder FLOPs. Evidence-Tempered Diversity (ETD) then selects the final tokens by multiplying each token’s max–min diversity gain by its relative evidence (Np_i)^βt, where p and the text confidence t come from a 1.9M-parameter scorer trained on box annotations with the MLLM frozen. With the MLLM frozen, ETD raises retention at strict 5% from 75.8% (DivPrune) to 84.2%. On Qwen3.5-4B trained with on-policy self-distillation, the same recipe with DivPrune retains 88.8%; ETD raises it to 91.7%, and the glance encoder then trades 1.4 points for a 2.9× shorter time-to-first-token on 4K images, giving 90.3±0.8%. The released LT-OPD checkpoint retains 85.4% at the same budget. On held-out document benchmarks ETD raises DocVQA from 52.5 to 74.5.

Method

The student is trained with on-policy self-distillation and DivPrune. At inference the glance encoder chooses where to look, and ETD chooses which tokens to keep. Every selector on this page uses the same strict budget, K = ⌈0.05N⌉ tokens per image.

Glance encoder

Encode the image at a quarter of the patches, keep 25% of the coarse cells by max–min diversity, and re-encode only those cells at native resolution.

Evidence scorer

A 1.9M-parameter scorer reads the image tokens and the question. It predicts a relative evidence map p and a text confidence t from box supervision, with the MLLM frozen.

Evidence-Tempered Diversity

ETD keeps the DivPrune loop and tilts it: i^⋆ = arg max d_i · (Np_i)^βt. Flat evidence returns DivPrune; a confident reading question moves the selection toward the answer.

Motivation at a strict five percent budget: time to first token on HR-Bench 4K, one TextVQA example, and the share of answer tokens kept by each selector. — Pruning after the vision encoder still spends 2.59 s in the encoder on HR-Bench 4K. The glance encoder cuts that stage to 0.65 s. On TextVQA, ETD keeps answer tokens and surrounding context.

Results

Retention is the mean, over nine benchmarks, of each score divided by the non-thinking full-token score. Qwen3.5-4B, strict 5% tokens per image. Glance-ETD at 5% is the mean of three training seeds; the other budgets use one seed.

90.3±0.8%Glance-ETD retentionQwen3.5-4B · three training seeds

91.7%Dense encoder + ETDSame recipe, without the glance encoder

84.2%Frozen MLLM + ETD8.4 points above frozen DivPrune

Retention across visual-token budgets on Qwen3.5-4B. Solid lines are trained models and dashed lines are training-free. — Qwen3.5-4B. At 5% of the tokens, Glance-ETD keeps 90.3% of the full-token score, 14.5 points above DivPrune.

Retention of Gemma 4 E4B-it across visual-token budgets for ETD and DivPrune. — Gemma 4 E4B-it, same token count per image. Filled markers average nine benchmarks. At 5%, ETD is 10.4 points above DivPrune. The open marker at 25% omits MMBench.

Efficiency of Qwen3.5-4B at a strict five percent budget: retention against speed-up, peak memory, prefill speed-up, and end-to-end speed-up. — On HR-Bench 4K the glance encoder cuts prefill from 373 to 65 TFLOPs and time to first token from 3305 to 1376 ms, at a cost of 1.4 retention points against the dense encoder with the same recipe.

On held-out document benchmarks, ETD raises DocVQA from 52.5 to 74.5 over DivPrune. On 4,123 TextVQA validation questions, ETD keeps 62.9% of answer-box tokens and at least one such token for 99.0% of questions. The same pipeline retains 89.5±1.1% on Gemma 4 E4B-it.

Design choices

Look cheaply first

A coarse pass chooses a quarter of the image before native-resolution vision work begins, and the output keeps the native token grid.

Keep diversity in the loop

Evidence moves the selector toward text and answer regions. Max–min distance still covers the scene when the question is not about reading.

Use one budget everywhere

Compared selectors keep the same K per image. The difference is which tokens survive, not how many.