Grounder › Benchmark

Does grounding actually make a local model more correct?

Published July 15, 2026 · A directional first run - method and raw arms below

The whole thesis: a small model on your own hardware, with a tiny context window, answers current questions like a cloud model if it's handed dense, trustworthy, token-capped evidence - and fails if it's handed raw pages. If a local model plus grounder doesn't clearly beat the alternatives, there's no product. Here is the first run - including where it lost.

The result - Qwen2.5-7B, 8,192-token window

Non-reasoning 7B instruct, 8k context, on an M5/32GB laptop. 8 current-fact questions, three arms.

B and C read the same pages - so B-vs-C isolates packaging (the product) and A-vs-rest isolates retrieval. Grounder is ahead on every measure here: more correct (+2 over the naive dump, +4 over the bare model), ~4× fewer tokens, 0 overflows, and 2.6× faster than the naive dump (fewer tokens = faster prefill).

It holds across model sizes - and grows as the model shrinks

The same 8 questions on a bigger local model (Qwen3.6-27B, a reasoning model, same 8k window):

Grounder is 7/8 on both. The bare model gets worse as it shrinks (4/8 → 3/8) - a smaller model knows less and hallucinates more - so grounder's margin over "model alone" widens on exactly the hardware the customer runs. That's the opposite of a result that only works on a big model.

And it's not just local - a frontier cloud model too

The overflow story is local-specific (it needs a small window). The correctness story is universal: every model is frozen at its training cutoff. Same current-fact questions, DeepSeek-chat (a large cloud model, big context) alone vs. with a grounder pack:

Unaided it got every current fact wrong, confidently: "Tavily was acquired by Snowflake in 2024" (Nebius), "Python 3.13.1" (stale), "cheapest plan is Starter, $49/mo" ($30 Project), "pay-as-you-go is $0.10/credit" ($0.008). Grounding corrected the acquirer, the rate limits, and the price. A huge context window doesn't fix a frozen one - so grounder's value (current + correct + cited) is universal, and the token-cap is an additional win for the small-window local case.

The three findings

1. Raw pages don't fit an 8k window - and the model doesn't degrade, it returns nothing. The naive dump's fetched payload averaged ~6,252 tokens and peaked at 22,759 (2.8× the window), against grounder's flat ~1,000. That's the content it tries to send; the table's 3,794 is lower because it's the average of what actually reached the model across the seven questions that fit - the eighth overflowed and sent nothing. When a payload exceeds the window the model hard-fails - e.g. "CEO of Anthropic": the raw pages came to ~22,759 tokens and returned HTTP 400: n_keep 29992 >= n_ctx 8192; grounder answered "Dario Amodei" from 3,454 chars. (How often it overflows is run-dependent - the 7B run overflowed 1/8, the 27B run 3/8. The stable claim isn't a fixed count; it's that the naive payload routinely approaches or blows the window while grounder's pack is a small constant that fits.)

2. The smaller the model, the more it invents - and grounding stops it cold. Left alone, the 7B produced confident, specific fabrications on 5/8: "Tavily acquired by ServiceNow in 2021" (Nebius), "free tier includes a $50 monthly allowance" (1,000 credits), "cheapest paid plan $4.99/mo, 100 credits" ($30, 4,000), "pay-as-you-go 10 cents/credit" ($0.008). Grounding replaced every one with the sourced answer - the sharpest version of "we installed search to reduce hallucination," and it's stronger on the small model than the big one.

3. max_tokens makes the input a known quantity - and that makes it fast. Grounder's prompt stayed in a 756-1,254 token band across all 8 questions; the naive dump swung from 59 to 22,759. Sending ~4× fewer tokens is why grounder answered in 5.0 s vs the naive dump's 13.1 s (prefill scales with tokens). Predictable-and-fast vs occasionally-catastrophic.

Where grounder lost or was lucky - reported, not buried

• The SUPPORTED / NOT_FOUND verdict this run stress-tested is gone. deep_search had reported NOT_FOUND on two questions while the returned passages contained the answer. Chasing a reliable verdict turned out to be the wrong fix: whether evidence "answers" a question depends on how the caller phrased it (one identical Tavily pack judged no for "cheapest plan" but yes for "how much does it cost"), which only the caller's model knows. So grounder no longer returns a verdict - it hands back the ranked, cited passages and the model decides. In these two cases the answer was in the pack all along, which is exactly what the model now reads.
• Grounder's remaining miss (7/8) is a retrieval gap: the Python-version question. The returned pack didn't cleanly carry the current version, so the model had nothing solid to answer from. The fix is retrieval (surface python.org's version better) - the one honest hole left, on both models.
• The naive arm's one "win" we discount: a question where its page fetches returned 0 chars and it answered from the model's own memory. Scored a hit; wasn't one for retrieval.
• Substring scoring is lenient - and it favored the naive arm, not grounder. One naive answer called the free plan "the cheapest" but matched on a later "$30" - a generous hit. If anything the naive arm's 5/8 is flattering.

Caveats - a directional proof, not yet a publishable benchmark

1. n = 8, single run, one grader. Ground truth was pinned from primary sources (Tavily's own docs, python.org) before results were seen. Directional, not statistical - re-run with more questions and seeds before trusting a headline.
2. The naive arm is the "SearXNG + scraper pattern," not literally SearXNG. Same shape - search then dump full pages - but no "beats SearXNG" claim until SearXNG is in the harness.
3. One machine, one grounder key, live web. Retrieval quality reflects grounder on this day; the overflow counts move with whatever pages the live SERP returns.

Reproduce it

The harness is bench/bench_local.py. It needs a local OpenAI-compatible server (LM Studio / llama.cpp) at 127.0.0.1:1234 with an 8k-context model loaded, and a grounder key in the env. It runs all three arms over the question set and prints the table; raw per-question results go to bench_results.json.

LM_MODEL=qwen2.5-7b-instruct GROUNDER_API_KEY=gnd_live_... python bench/bench_local.py

Transparency note: Grounder is a commercial product we build, so we have an interest here. That's exactly why the losses, the lenient scoring, the fixed judge bug, and the small sample are on this page rather than left off it. It's evidence for the mechanism, not a leaderboard.