← Blog
Week of

Retrospective reconstruction · Published · Weekly archive

Keep the old tests in the new evaluation

A generation change should not erase the tasks the application already depends on.

Field NotesRetrospective

Retrospective field note for 2026-10-19, reconstructed on 16 November 2026. The archive date is not a claim of publication at the time.

It is easy to build an evaluation around the exciting new capability and quietly leave the old workload out.

Keep the existing tests. Add cases for the new behaviour alongside them. The application still needs valid outputs, intact permissions and the same respect for user data after the model name changes.

Watch for shifts that are harder to score. A more proactive model may choose a tool where the old one answered directly. A more elaborate response may be worse for a short interaction. Those changes can be useful, but they should be intentional in the product.

Roll out in a way that can be inspected and reversed. Collect the actual failures rather than relying only on aggregate usage. Know what restoring the previous route involves.

Gemini 4 Argon provides the occasion for another evaluation. Carry the earlier acceptance criteria forward. If the new route improves the difficult work while preserving the ordinary obligations, there is a concrete reason to switch. The new generation label does not supply that evidence by itself.

Google released Gemini 4 Argon on 13 October. Source.

All posts Work with me