Google now runs six automated gates, including an AI reviewer, on every agent skill

Google's Agent Skills repository has crossed 21,000 GitHub stars, and every change to it now passes through six automated gates — including an AI reviewer — before a human expert signs off. The company detailed the new review and validation system this week. The repository now contains more than 170 skills, up from the 15,000-star milestone Google cited in the earlier phase of the project, and it hosts contributions from teams across the company distributed to multiple AI platforms.

The stakes are straightforward. Google no longer treats a skill as static documentation. It treats every update as a product release, because even small wording changes can alter how an AI agent reasons, selects tools or executes tasks. That shift is what the whole pipeline is built around.

What does the new review process actually check?

Each pull request in Google's internal repository now runs through six automated gates before human approval. They cover:

• structural conformance
• skill linters
• semantic versioning
• evaluation suite validation
• link checks
• an AI review agent

Only after all six pass does a human reviewer step in. That final approval typically falls to the skill's subject matter expert, or for brand-new skills, a member of Google's Central Skills team.

A central piece of the process is strict semantic versioning. Every skill carries a version number in its metadata, and any behavioural change to instructions, scripts or references requires a version bump before the pull request can proceed. New skills must start at 1.0.0. Standard semantic versioning reset rules apply to major and minor changes.

The version number serves developers and package tooling rather than the model at runtime. It lets teams compare their local installations against the upstream catalogue and decide when to update.

Google also validates the evaluation suites that measure agent performance with and without a given skill. The checks confirm that referenced prompt files exist and are not empty, that each expectation tests a single criterion, and that duplicate prompts or conflicting expectations get flagged. Every pull request must link to a fresh evaluation report based on the latest commit.

What does the AI reviewer do?

Alongside linters and schema checks, Google has introduced an AI review agent to assess instruction quality and design standards. It uses a dedicated review skill to inspect proposed changes and catch problems that syntax checks miss.

One of its main tests is whether a task belongs in natural-language instructions at all. Google says authors often fall into what it calls the "deterministic trap" — rigid command-line sequences written directly into Markdown instructions when they belong in executable scripts. The reviewer's rule is simple: deterministic operations such as command-line sequences, parsers and deployment steps sit in scripts. Markdown instructions focus on work that requires judgement, such as diagnosis, architecture choices or synthesis.

The AI reviewer also enforces structure. Skills must follow a three-level layout: concise metadata, a compact main instruction file, and separate supporting reference documents. The main SKILL.md body has a hard 500-line limit. The reviewer blocks chained references that force agents to follow multiple document hops. And it flags interactive helper scripts, requiring that scripts accept command-line flags or environment variables, provide help documentation, and separate structured output from error messages.

Why did Google decide manual review wasn't enough?

Scale. The repository grew from 15,000 to 21,000 GitHub stars, according to Google, and it now aggregates skills from teams across the company. Google argues that at this scale manual review has become less effective. Unvetted edits can introduce subtle bugs, and broken links and weak evaluation design can erode trust in the repository over time.

Link decay gets its own treatment. Google validates relative links before merge and runs scheduled external link checks across the public repository. Broken links are reported to skill owners for remediation.

What did Google learn from six months of running it?

Google outlined several lessons from roughly six months of repository history, and they read like a field guide for anyone building similar pipelines.

First, CI checks need careful scoping. Early rules blocked minor typo fixes while failing to inspect some newly added skills buried in deeper subdirectories. Both failure modes are exactly backwards for a review system, and Google says it had to recalibrate.

Second, evaluation expectations must be atomic. Compound expectations — a single check testing multiple things at once — produced inconsistent ratings from LLM-based judging systems. Breaking them into single checks made results clearer and easier to debug.

Third, documentation drift sets in fast in a multi-contributor repository. Google now relies on generated files and CI checks to keep core indexes current, rather than asking contributors to maintain them by hand.

How does Google keep the pipeline's cost down?

Cost control shaped the design directly. Fast deterministic checks run first and stop the process if they fail, so heavier LLM-based reviews and evaluations never run unnecessarily on broken changes. The ordering reduces token use and CI runtime as contributors make frequent small edits during review, while preserving the stricter checks for changes that survive the initial gates.

That's a notable data point for the broader industry. Agent skill marketplaces and instruction repositories are proliferating across AI platforms, and Google is effectively publishing a reference architecture for keeping them trustworthy: cheap checks first, expensive model-based review second, humans last.

The signal to watchers is that release engineering discipline — versioning, evaluation gates, linting — is migrating from software packages to natural-language agent instructions. As skills become the unit of deployment for AI agents, expect more repositories to adopt Google's pattern of treating every wording change like a shipped product.