News
DeepMind and Artificial Analysis launch the MKA visual knowledge-work benchmark
A 200-task visual knowledge-work benchmark puts Fable 5.2 first and Gemini 4 Carbon close behind at a much lower published cost.
Google DeepMind and Artificial Analysis launched the Multimodal Knowledge Agent (MKA) benchmark on November 10, 2026. It tests whether agents can finish visually complex professional work, not whether they can answer from text alone.
A visual job, not a text quiz
Google’s launch post says most knowledge-work benchmarks can be solved without looking at an image or a page layout. Everyday work often cannot: people cross-reference architectural blueprints, trace engineering schematics, and turn findings into decks, reports, and spreadsheets. Artificial Analysis adds clinical scans to that list of inputs and says the files are required to complete the task, not extra context. Expected deliverables include slide decks, spreadsheets, PDFs, Word documents, and marked-up images.
Domain experts across 10 verticals and 83 occupations built the set. Google names architects, biologists, underwriters, and data scientists among them, and lists Business, Construction, Education, Engineering, Finance, Healthcare, Law & government, Logistics & mapping, Science, and Software & data. There are 200 tasks. Each is a multi-step project a seasoned professional would spend three to eight hours on: a realistic prompt, a bundle of multimodal files, and an expert rubric of roughly 70 yes/no checks, weighted 75% factual accuracy and 25% visual design.
Artificial Analysis splits those checks into Regression criteria, baseline items models should pass, and Climb criteria, harder items meant to separate strong models. The headline score is Climb weighted satisfaction rate (Climb WSR): the weighted share of Climb criteria completed.
Same harness, private tasks
Every model runs in Stirrup, the open-source agent harness, with the same tools, sandbox, and turn budget. Artificial Analysis specifies code execution, web search, web fetch, and image viewing, up to 250 turns, and says it grades deliverables with GPT-6 Luna (medium) as an agentic judge. Google describes an autorater built on a non-Google model that opens every file, renders pages, inspects spreadsheet formulas, and scores the work against the expert rubric. When domain experts graded outputs themselves, they agreed with that autorater 96% of the time.
Inside that shared setup, behavior still diverges. Google says the models it tested used between 30 and 130 turns on average on the same tasks, that strong perception is necessary, and that models try and fail to compensate when vision is removed.
Artificial Analysis runs the 200-task set and keeps it private so the tasks cannot leak into training data or be found by search. It says the evaluation is exclusive to Artificial Analysis. Google has also released 10 public sample tasks on Hugging Face so anyone can inspect the format and test an agent on the public examples.
Fable 5.2 first, Carbon close, costs far apart
Artificial Analysis ranks Claude Fable 5.2 (max) first at 57% Climb WSR, Gemini 4 Carbon (high) second at 54%, and Claude Opus 5.5 (max) third at 53%. Google also places Carbon second in quality among the models it tested.
The cost lines are separate statements, and they are not reconciled here. Google says Carbon completes tasks at 6x lower average cost than the top-scoring model, and that Carbon sits on the quality-price Pareto frontier. Artificial Analysis publishes average cost per task of $31.97 for Fable 5.2, $6.29 for Carbon, and $1.11 for GPT-6.1 Sol. It calls the Fable figure more than 5x Carbon’s for about two extra points, and about 29x Sol’s, and says cost per task spans nearly 30x within 10 points of the top score. Below Carbon, it places GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6 Luna on the cost-performance Pareto frontier, along with Claude Haiku 5.5 and MiMo-V2.6-Pro.
No model leads all 10 verticals. Fable 5.2 leads Software & Data, Engineering, Logistics & Mapping, and Business, and leads Finance alongside Claude Sonnet 5.5. Claude Opus 5.5 (max) leads Education and Construction. Gemini 4 Argon (high) leads Science. Carbon (high) leads Healthcare. Claude Sonnet 5.5 (max) narrowly leads Law & Government.
Output type moves the score less than industry does. Of six file types, Fable 5.2 leads PDF, Word document, spreadsheet (narrowly), and presentation alongside Opus 5.5 (max). Gemini 4 Argon (high) and Carbon (high) lead image output. DeepSeek V4.1 Flash 1017 leads text files. Word documents and text files are the hardest outputs: no model passes 52% Climb WSR on either.
Read Google’s post for task design, the private set, and the 6x cost claim, then Artificial Analysis for the Climb WSR order and the published cost per task. Inspect the 10 public Hugging Face samples before treating the private leaderboard as a full picture of the work.