Benchmarks
Reproduce with pnpm bench, which needs an API key. The raw output lives in
bench/results/bench.json.
| arm | off | native | precog |
|---|---|---|---|
| navigation median | 147 ms | 145 ms | 68 ms |
| navigation p95 | 404 ms | 397 ms | 395 ms |
| hit rate | n/a | n/a | 29% |
| top guess correct | n/a | n/a | 17% |
| speculative loads / session | 18.3 | 24.8 | 33.4 |
| wasted loads / session | 2.0 | 2.0 | 2.6 |
| wasted kB / session | 6.9 | 6.9 | 10.3 |
| jev calls / session | 0 | 0 | 14.9 |
| jev answered from cache | n/a | n/a | 17% |
| jev latency p50 / p95 | n/a | n/a | 372 / 622 ms |
| tokens / session | 0 | 0 | 10398 |
Every figure is measured. Timings are wall-clock milliseconds from the click to the new page's heading, the predictions come from Jev, and the token counts are what the service reported.
What the numbers say
The median navigation is about twice as fast. The p95 does not move, because the tail is the navigations the model got wrong, and prediction does not fix those.
off is not "nothing". The playground sets takeOverNuxtLinkPrefetch, so NuxtLink still
prefetches a link the cursor rests on in every arm. off means "Chrome and Nuxt on their own",
which is the baseline worth beating.
native is the free competitor. Document speculation rules at eagerness: moderate need no
model and no key. Here they tie off, because a visitor who clicks soon after the cursor lands
does not give the browser enough hover to work with. That gap is what precog is for; if your
visitors hover for a second before clicking, native already covers you.
It costs about 15 calls and 10,000 tokens per session, with a sixth of the calls answered from the server cache. Jev itself answers in 372 ms at the median and 622 ms at the p95, which is why the client deadline defaults to 1500 ms.
The visitor model
Each session is driven by a seeded generator, so all three arms face the same visitor:
- read the page in two to four scroll steps with pauses,
- pick a link with weight
1 / (rank + 1)over the links in the list, - move the cursor to it over a couple of waypoints,
- dwell 90 to 210 ms, then click.
The dwell is short on purpose. A long hover lets the browser's own prefetch-on-interaction finish every time, and then no arm is measuring anything but the hover.
Hit rate and top-guess accuracy are measured against this generator, not against a real audience. The timings do not depend on it: a warmed navigation is fast whoever asked for it.