UCLA and Apple show a visual agent harness with multimodal memory cuts token cost by 63%
An October 19 UCLA and Apple preprint finds that pairing nine atomic visual and search tools with persistent multimodal memory lifts scores across 22 benchmarks at ≤37% of heavy code-agent token cost.
Multimodal agent frameworks often grant vision-language models unconstrained code interpreters while accumulating redundant full-resolution image crops in context.
In an October 19 arXiv preprint (arXiv:2610.26860), researchers from UCLA and Apple systematically ablate agent harness design across 22 visual perception, OCR, counting, and document-reasoning benchmarks, introducing the open-source OpenVLHarness framework. [1][2]
OpenVLHarness pairs nine core capabilities—five perception tools (Zoom-in, Visual Grounding, OCR, Depth Estimation, Camera Trajectory), three web search tools (Text Search, Image Search, Webpage Visit), and a Python coding agent—with a capability layer and persistent multimodal memory that reuses image handles and structured bindings instead of raw token dumps. Across 22 benchmarks and eight backbones, OpenVLHarness improved average scores by 9.7 to 12.6 percentage points over no-tool baselines (e.g. Qwen3-VL-32B rose from 38.30 to 50.92; Kimi K3 rose from 56.4 to 66.1). [1]
At medium reasoning effort ($0.92 per 1,000 questions), GPT-6 Luna with OpenVLHarness scored 56.5, outperforming max-effort Base (48.0), Codex (48.6) and Visual Sketchpad (48.8) by 7.7 to 8.5 points while consuming at most 37% of their model-token expenditure. The preprint is evaluated on 22 static 2D benchmarks rather than streaming video; engineers can inspect its tool interfaces and memory pruning via GitHub. [1][2][3]
Take it further
Explore the OpenVLHarness repository and tool definitions For engineers building vision or document-extraction agents. Inspect the 9-tool capability schema and persistent multimodal memory bindings.
Sources & details 3 sources
Discuss in your voice app
Take the reporting, visual explanations and original sources into a conversation. Copy the brief, paste it into your preferred assistant, then start its voice mode.
Paste the brief into your voice app to discuss this reporting.Reporting sources
- arXiv · OpenVLHarness: Real World Visual Agentic Tool-Use (2610.26860)
Original October 19 preprint by UCLA and Apple researchers evaluating 9 tools, persistent multimodal memory, and token cost across 22 benchmarks.
- GitHub · uclanlp/OpenVLHarness code and evaluation suite
Public repository providing the OpenVLHarness capability layer, multimodal memory manager, and benchmark evaluation suite.
- OpenVLHarness · Project documentation and tables
Companion project site with per-benchmark accuracy, tool ablation matrices, and token-efficiency breakdowns.
Images & figures
OpenVLHarness beat heavy code-interpreter agents across 22 visual benchmarks at ≤37% of the token cost.
Euangelion · original data figure · Source · CC BY 4.0 · Download figure