CodeRabbit says Codex helped it ship 20x more software

By Sophie Lindqvist

CodeRabbit says OpenAI's Codex helped it ship roughly 20 times more software over six months without expanding its engineering team. Co-founder and CEO Harjot Gill also estimates that he personally pushed 2,500 to 3,000 pull requests in the 30 days before the interview.

Those figures quantify the review problem CodeRabbit wants to solve. The company says over 20,000 organizations and more than 150,000 open-source projects use its AI-powered governance and control platform, which uses OpenAI models to review, explain and secure code produced by AI agents.

Why has code review become the bottleneck?

Gill described the constraint plainly: “The first bottleneck that every company hits as soon as you adopt coding agents is code reviews,” he said. As agents increase code output, teams still have to decide which changes are correct, secure and ready for production.

CodeRabbit built its platform around that gap. According to the company, it helps teams validate and trust AI-generated code, understand proposed changes and decide what to ship. That need grows because speed from coding agents creates a second workload: human judgment must keep pace.

CodeRabbit says better price-performance from OpenAI models lets it add more agents to reviews and pass those benefits to customers. Its own use of Codex provides the clearest productivity claim: roughly 20 times more software with the same engineering team.

In plain terms, CodeRabbit is selling the review capacity needed to make agent-written code usable.

How did OpenAI models reshape the product?

CodeRabbit's first product used GPT‑3.5 Turbo. Gill said the arrival of GPT‑4 made the product effective at reviewing code, and the company adopted OpenAI o1 and OpenAI o3 early.

Because CodeRabbit runs reviews asynchronously in the cloud, it can use models that spend longer reasoning about a problem in exchange for stronger analysis. That approach lets the company favor deeper issue-finding over immediate response time.

The company evaluates new models against its own benchmark of known issues. At the time of the interview, Gill estimated that GPT‑6 Astra caught 50% of the issues in that dataset, ranking it above other models in the benchmark.

Gill tied those gains to OpenAI's release cadence: “With each OpenAI model release, especially with the reasoning models, we have seen massive improvements in the capabilities of our product to find issues at scale.”

Gill also said: “The nature of bugs keeps evolving, so does our benchmark as the models get better.” The benchmark changes because agents introduce new kinds of mistakes, while fixes can expose problems that earlier reviews missed.

CodeRabbit also attributes product expansion to lower model prices and caching. The company says those improvements let it run more AI, add more subagents and pass the gains to customers. When CodeRabbit started, it used only one or two subagents for code navigation.

Now, lower-cost GPT‑6 Luna supports more subagents throughout the platform. Models such as GPT‑6 Astra handle planning and orchestration, which lets the system divide a larger review across specialized parts.

Gill said: “Now we are using agents everywhere on the use cases we couldn't even afford in the past.” According to him, that reach expands both the validation CodeRabbit performs and the explanations it gives developers about changes they did not write themselves.

The practical shift is from a single reviewer to a coordinated group of cheaper and more specialized agents.

How did Codex change CodeRabbit's delivery process?

Using Codex exposed limits beyond coding. Gill said running multiple agents forced the team to handle laptop memory and compute constraints, testing demands, software tools that compile and package builds, and dependency management.

CodeRabbit streamlined its release process and reconsidered where human approvals belonged. It uses CodeRabbit to automatically ship low-risk changes that do not require human intervention.

Gill said: “We had to actually completely rethink the entire software delivery.” He also sees cloud-based development as a way to reduce local-machine constraints and give teams stronger controls over access and auditing.

Under this model, agents prepare and ship routine work, while people retain approval over the changes that carry greater risk. CodeRabbit's revised process treats a pull request as something an agent can often assemble, check and submit on its own.

How is Codex running the company as code?

CodeRabbit built a Codex-powered Slack bot that triages production incidents, Slack messages and support tickets. The bot can return a pull request for the team to review and merge.

The company also uses Codex for work beyond engineering, including marketing and operational tasks. Gill said: “We are running the company as code as much as possible.”

In marketing, CodeRabbit manages Google Ads through pull requests. The team opens and merges a request to deploy ads, turning an advertising change into the same reviewable workflow used for software.

A separate content studio repository lets Codex create slide decks and customer-facing documents. Gill said: “If you can translate your marketing problems to code problems, you will run 10x faster.”

The claimed gain comes from turning business tasks into pull requests that agents can prepare and people can review. CodeRabbit applies one underlying mechanism across software delivery, advertising and content production.

How did internal security testing become a product?

CodeRabbit has run agents in sandbox environments since 2023. The agents clone repositories and write code to help perform reviews, giving the company a way to test automated analysis against real codebases.

The company uses Daybreak to secure both its own stack and customers' code. More recently, CodeRabbit used OpenAI's Cyber model through Daybreak to test and secure its own platform. The team used the model for internal penetration testing, probing its systems for paths an attacker might exploit.

Gill said: “We found multiple vulnerabilities, advanced hacks that we could not even imagine could happen.” He said early access to the model helped CodeRabbit secure its own stack before exposing the capabilities to customers.

CodeRabbit then turned the work into CodeRabbit Secure, its latest offering. The product uses Daybreak with Codex models to find and fix vulnerabilities through security-focused agents.

A two-week promotional offer covered $1M in free deep scans for users. CodeRabbit reports that the scans found more than 43,000 security vulnerabilities across 13,000 deep scans, all powered by Codex models. Customers have already fixed thousands of those vulnerabilities, improving their code security.

In plain terms, CodeRabbit turned internal penetration testing into a commercial security workflow.

What changes as background agents take over?

CodeRabbit says background agents now begin responding to alerts and incidents without a person initiating every task. Gill expects teams to need more help deciding which changes to take through to completion.

The company is expanding toward what it calls agentic software development. Its stated role is to validate, prioritize, explain, secure and verify agent-generated changes at scale.

Gill frames the opportunity around control. His pitch is that developers can remain in charge while agents do more building, provided validation, explanations and maintenance advance at the same pace as the code itself.

On CodeRabbit's account, coding agents increase the need for a control layer around generated code. The next test is whether those controls can preserve accuracy and trust as background agents make more changes without waiting for a person to start every task.