HighWalk Benchmark

Goal: Assess LLMs' ability to update a technical specification document from code changes.
Required capabilities: Strong code analysis, technical abstraction and writing skills, all within an existing document.
Scope: 46 commits, 95 modified files (+4,587 / -945 lines), Laravel project.

Why HighWalk

Over the past few months, I've worked on automating technical specification updates for projects in their run phase. The challenge is to produce documents for human teams (such as IT departments) with the right level of abstraction: covering architecture and business rules without getting lost in implementation details.

To achieve this, I designed and refined a dedicated Agent Skill. The main challenge lies in assessing the real impact of code changes, then writing and integrating them smoothly into an existing document.

I quickly found that models performing very well in coding benchmarks could struggle with this exercise. HighWalk was created to measure this specific ability to step back: a benchmark that evaluates broad understanding and documentation, beyond writing code or specifications for other agents.

Protocol, criteria, and ranking

Task assessed: Update an existing technical specification document from 46 commits in a Laravel project (95 modified files, +4,587 / -945 lines).

Environment: Runs performed with OpenCode and OpenRouter, with locked providers (the model publisher's provider, and NovitaAI for DeepSeek).

Raw quality: Weighted functional evaluation across 11 criteria. It measures the ability to identify relevant changes, abstract them at the right level, and integrate them consistently into the document. Omissions and unjustified additions are penalized.

Operational efficiency: A score weighted equally between run duration and cost. The 10-minute and $1.50 caps reflect practical usage constraints, exceeding both results in a zero score.

Overall score: 80/20 combination of raw quality and operational efficiency. Prioritizes reliable documentation while valuing configurations that are usable day to day.

Repetition: Each configuration (model, reasoning effort, and provider) was run twice. The best run is retained in order to compare the operational potential of the models.

Limitations: This benchmark covers one use case, one project, and one reference document. It is neither a comprehensive statistical measure nor a performance guarantee in other contexts.

Exclusion: Qwen 3.7 Max excluded from the ranking because it critically altered the document.

Ranking 1: Best overall

Overall ranking. Weighting: 80% raw quality / 20% operational efficiency.

Grok 4.5 (high) ranks first overall: it combines one of the best quality scores with unmatched time, cost, and token-consumption efficiency.
Two mid-range models follow: GPT 5.6 Terra (medium) and Claude Sonnet 5 (high).
In fourth place, we have an open-weight model: GLM 5.2 (xhigh).

Ranking 2: Raw quality

This ranking measures an agent's ability to rigorously meet the brief, regardless of time or cost.

GPT 5.6 Sol (xhigh) leads on raw quality, followed by Claude Sonnet 5 (high) and Grok 4.5 (high).
Claude Sonnet 5's high variant performs about 10% better than its xhigh variant. This is also seen across several models: the highest reasoning effort does not necessarily produce the best result.
Gemini 3.1 Pro (high) and the Mistral models are closing this ranking with very low quality scores.

Ranking 3: Operational efficiency

A normalized ranking based on run time and dollar cost (subject to the $1.50 and 10-minute caps). Considered without quality, it is not meaningful on its own but reveals two observations.

Kimi K3 has a relatively low operational-efficiency score, well below Kimi K2.6 and GLM 5.2 (which also outperforms it on raw quality).
GPT 5.6 Sol (xhigh), first on raw quality, is also last on operational efficiency: it is by far the most expensive and slowest. Its efficiency score is zero because it exceeds both the cost and duration caps.

Ranking 4: Completed tasks

This isolates models that did not completely fail any criteria. They are the strongest allies for human code review: the agent has at least identified and outlined 100% of the required elements.

GPT 5.6 Terra (xhigh) and Claude Fable 5 (high) stand out with 100% criteria coverage. Their complete coverage does not put them in first place: their lower raw-quality and overall scores suggest that exhaustiveness can sometimes come at the expense of relevance.

Observations and takeaways

Grok 4.5 (high) emerges as the best overall thanks to an unmatched combination of raw quality and operational efficiency. GPT 5.6 Sol (xhigh) achieves the highest raw quality, at the cost of operational efficiency dropping to 0 after exceeding the budget and time caps.
Mid-range models - Claude Sonnet 5 and GPT 5.6 Terra - perform particularly well.
These measurements are based on API cost. With a subscription, some models may become economically more attractive. The final choice also depends on how the service is accessed.
More reasoning effort does not guarantee a better result: Claude Sonnet 5 performs better in high than in xhigh, a trend also observed with DeepSeek V4 Flash/Pro and GPT 5.6 Luna. Claude Opus 4.8 shows the opposite. Reasoning effort is therefore not an option to maximize by default.
Open-weight models should not be dismissed. GLM 5.2 (xhigh) reaches fourth place overall, with a strong raw-quality/efficiency. Other models in this category, despite their reputation, struggle with this task.
At the other end of the scale, Gemini 3.1 Pro (high), DeepSeek V4 Flash (xhigh), and the Mistral models obtain very low raw-quality scores.
Finally, two configurations have no hard failures: GPT 5.6 Terra (xhigh) and Claude Fable 5 (high). They are the most reassuring starting points for human review.

Updated on July 26, 2026