AnyJev releases tools to calibrate language-model decisions

The Apache-licensed Python package, from researchers affiliated with Nokia and Tencent, addresses a modest but common task: choosing among defined options, such as which team should receive a support ticket.

Instead of asking for a paragraph and extracting an answer, the software reads a model's preference among supplied choices. Its simplest correction rotates the options to reduce position bias. A second level uses labelled examples to adjust confidence. A third fits a small decision head to the model's internal representation while leaving the underlying model weights untouched.

The distinction between getting an answer right and knowing when to trust it is central. In the authors' Qwen3-8B test on 300 banking-query examples, accuracy rose from 74.7% for the raw readout to 80.7% after bias correction and calibration. The reported calibration error fell from 0.240 to 0.095. These are developer measurements on a particular test, not an estimate of performance on a reader's workload.

The package is marked pre-alpha. Its documentation reports failures as well as gains, and the decisions were evaluated individually rather than inside complete agent workflows. Some other benchmark labels came from a teacher model. The release record and pinned documentation are available for inspection; we did not run the software.

For a first trial, choose one reversible classification task with examples a human can check. The useful question is how often the system should abstain, as well as how often it chooses correctly.