Anthropic's Fable 5.2 completes 41 percent of real freelance jobs

Anthropic's Fable 5.2 completed 41.3 percent of real freelance jobs at professional quality on the Remote Labor Index, the top score ever recorded on the benchmark and more than 16 times what the best system managed when the test launched roughly a year ago. The result puts Fable 5.2 far ahead of every competitor, including Anthropic's own Opus 5.5 at 21.25 percent and OpenAI's GPT 6 Astra at 20.83 percent.

The Remote Labor Index (RLI), run by Scale Labs together with the Center for AI Safety (CAIS), measures something most AI benchmarks do not. It tests how often AI agents complete actual freelance projects well enough that a paying client would accept the work. That framing matters. It moves the evaluation away from isolated puzzle questions and toward the messy, multi-step reality of contracted labor — the market where AI's economic impact on white-collar and creative work will be decided.

How big is the jump?

The year-over-year numbers describe a steep curve. When the benchmark launched about a year ago, the leading system at the time, Manus, completed just 2.5 percent of tasks. Fable 5.2 now completes more than 16 times as many professional jobs.

The improvement is also dramatic against more recent measurements. In July, Fable 5 — an older model from the same family — led the leaderboard at about 16 percent. That July score was itself a milestone: within eight months, the best agents had more than quadrupled their share of jobs completed at professional quality.

Four months later, Fable 5.2 hit 41.3 percent. That is roughly two and a half times Fable 5's July score. The direct predecessor, Fable 5.1, trails far behind at just under 18 percent, meaning Fable 5.2 completes more than twice as many jobs as the model released immediately before it.

Where does the competition stand?

The gap between Fable 5.2 and the rest of the field is wide. The current leaderboard, as reported:

• Fable 5.2 (Anthropic): 41.3 percent
• Opus 5.5 (Anthropic): 21.25 percent
• GPT 6 Astra (OpenAI): 20.83 percent
• Gemini Argon 4 (Google): 17.08 percent, in fifth place
• Older models such as GPT-5.2 and Gemini 3 Pro: below three percent

Fable 5.2 works almost twice as successfully as the next-best systems on the market. The placement also shows how quickly previous flagships fall behind: models that led the field months ago now sit in the same scoring band as systems released years apart from them.

What can Fable 5.2 actually do?

Scale Labs said on X that Fable 5.2 makes its biggest gains in three categories: editing audio and music, designing architecture and interior spaces, and creating 3D models. The agent also handles long, complex projects that previously top-ranked models regularly failed to finish.

Nearly 50 of Fable 5.2's successful completions were jobs that no AI agent in the test had ever solved before.

The model is not universally capable, though. Scale Labs says the AI can still barely handle certain task types, especially industrial product design and animations. When agents fail in the test, they typically deliver amateur-quality work, incomplete results, broken files, or contradictory designs, according to the researchers. In other words, the failure mode is not refusal — it is output that looks plausible until someone opens the file.

What jobs does the benchmark use?

Unlike conventional AI tests that often check only isolated skills, the RLI research paper covers complete work assignments. The dataset is built from real transactions:

• 240 completed projects drawn from the Upwork platform
• 358 verified professionals who originally delivered the work
• About $144,000 in total contract value

A single project costs an average of $632.60 and takes a human freelancer an average of just under 29 hours to complete. Tasks range from video editing to CAD design, game development, and data analysis.

To work on these jobs, AI agents get access to a virtual workspace stocked with common professional software — Blender, GIMP, Audacity — along with specialized AI tools for image, audio, and video generation. The setup is designed to give the agent roughly the tools a human contractor would use, rather than testing raw question-answering ability.

Who grades the work?

Trained human experts manually check whether an AI agent completed each job successfully. They open the submitted files in the relevant professional software and compare the AI's output directly against the work the human freelancer delivered for the same assignment.

That verification process is labor-intensive, and attempts to automate it have failed so far. CAIS tried using another AI as the reviewer and ran into a telling problem: AI reviewers can barely operate the professional software themselves. As a result, they are fooled by submissions that look polished on the surface but are technically flawed, and they rate the models far too generously.

That finding carries weight beyond benchmark logistics. If automated reviewers cannot reliably distinguish professional-grade work from superficially attractive junk, then quality control for AI-generated deliverables remains a human job — at least for now.

Why the numbers matter

A 41.3 percent completion rate on real, paid freelance work changes the terms of the AI-and-jobs debate. A year ago, a 2.5 percent success rate supported the argument that agents were nowhere near replacing contractors. At 41.3 percent — on jobs averaging $632.60 and 29 human hours — the economics of outsourcing work to agents starts to look concrete rather than theoretical.

The benchmark's structure also sharpens what comes next. The remaining 58.7 percent of jobs still end in amateur output, broken files, or contradictions, and whole categories like industrial product design and animation remain essentially unsolved. The open question for the next release cycle is whether the frontier models close those specific gaps, or whether the failure modes prove harder than the gains achieved so far.