Models

Runway open-sources Praxis-1, a video-trained model that controls robots

Runway open-sources Praxis-1 under Apache 2.0. The video-trained robot controller scores 46.2% on RoboLab-120, beating FLUX 3 Action, but still needs robot data and has limits.

Runway's open-source Praxis-1 turns video AI into a robot controller
Runway's open-source Praxis-1 turns video AI into a robot controllergeralt / pixabay
By Rebecca Stone5 min read

Why it matters

  • Runway released Praxis-1 under the Apache 2.0 license, allowing commercial use, following a partner-only debut in late September.
  • Praxis-1 scores 46.2 percent on Nvidia's RoboLab-120 benchmark, ahead of FLUX 3 Action at 42.9 percent and fifth overall.
  • On a real Franka arm, Praxis-1 completed 88 of 100 task attempts versus 80 for FLUX 3 Action.
  • Adapting the model used about 277 hours of DROID data for a Franka arm and up to 2,600 hours for a dual-arm robot.
  • After roughly 95 hours of fine-tuning on an unseen robot platform, Praxis-1 hit 80 percent versus Google RT-1's 52.4 percent.

Runway has released its robot control model Praxis-1 with open weights and source code under the Apache 2.0 license, which permits commercial use. The release marks the first public availability of a model the company first showed to select partners in late September, and it arrives with a detailed research report containing new performance tests.

Praxis-1 turns Runway's video research into robot control. The company, best known for generative video models such as Gen-4.5, calls Praxis-1 a World Action Model: it predicts how a scene will change and derives matching robot movements from those predictions. Runway sees this as a way to reduce dependence on expensive robot-specific datasets that have traditionally been collected solely to train such systems.

How does the model work?

Praxis-1 takes in camera images, text instructions, and information about a robot's position. From those inputs, it predicts how the environment will change and calculates the right movements. As new camera frames arrive, the model continuously adjusts its control output.

The model builds on a video model that learned how objects move and interact from ordinary footage. Runway then fine-tuned that foundation with videos of human actions and robot data. According to Runway, Praxis-1 can be adapted to new robot systems through additional training.

Since September, three robotics companies — Standard Bots, Noble Machines, and Ultra — have been testing Praxis-1 on different platforms, including a humanoid robot, an industrial robot arm, and a dual-arm warehouse robot.

Where does Praxis-1 rank on benchmarks?

On Nvidia's RoboLab-120 leaderboard, Praxis-1 scores a 46.2 percent success rate. The benchmark tests how reliably robot models can sort, grasp, or stack objects across 120 tasks in simulation. FLUX 3 Action, also built as a World Action Model, scores 42.9 percent under the same conditions. That result places Praxis-1 at the top of the World Action Model category. On the overall leaderboard, it ranks fifth, behind two AI agents and two robot models with different architectures.

Runway ran all of these tests itself. The results have not been independently verified.

The company also compared the two World Action Models on a real Franka robot arm after fine-tuning both on the DROID dataset, a public research collection of robot recordings. The test consisted of ten tasks, including placing a banana on a plate and closing a drawer, with ten attempts per task. Praxis-1 succeeded 88 times. FLUX 3 Action managed 80.

How much robot data does it still need?

Praxis-1 has not eliminated the need for specialized robot data. To adapt the model to a Franka robot arm, Runway used roughly 277 hours of recordings from the public DROID research dataset, according to its research report. Those recordings show robots grasping and moving objects. For a dual-arm robot, the company used between 1,400 and 2,600 hours of training material.

Runway had already incorporated DROID data and data from other robot systems during earlier training stages. Whether the approach actually requires less robot data overall than comparable models without general video pretraining remains an open question, one the research report does not settle.

Can it transfer to robots it has never seen?

Runway tested how well Praxis-1 transfers to a robot type absent from its training data: a mobile robot platform with a gripper arm that Google had previously used to develop RT-1. Recordings from this robot were not part of Praxis-1's earlier training.

After fine-tuning on about 95 hours of robot data, the model reached an average success rate of 80 percent in simulated tests, according to Runway. The tests covered four types of tasks, including grabbing soda cans, opening and closing drawers, and placing an apple in a drawer. Google's original RT-1 model scored 52.4 percent on the same benchmark.

What are the model's limits?

Runway's research report is candid about weaknesses. The model sometimes confuses similar-looking objects and struggles with vague instructions. For longer tasks, individual steps need to be spelled out, because Praxis-1 cannot yet reliably plan complex action sequences on its own.

Hardware demands set another constraint. The standard model version requires a GPU with 80 GB of memory. A compressed version should fit on an RTX 5090 with 32 GB of VRAM based on Runway's memory estimates, though the GitHub documentation states that actual peak memory usage has not been measured yet.

How can developers get it?

Developers can download Praxis-1 from Hugging Face after signing up and agreeing to the access terms. The source code is available on GitHub. Pre-adapted versions for different robots are already available alongside the base model, which itself needs additional training before it can work with new robot systems.

Why the release matters

The release puts a full robot control stack, licensed for commercial reuse, into the hands of any team with suitable hardware. The open-weights approach contrasts with the partner-only access Runway used at the September debut, and it gives academic and industrial labs a directly testable competitor to models like FLUX 3 Action and Google's RT-1 lineage.

Runway frames Praxis-1 primarily as a system for fast execution of robot movements. In the future, the company says, it could work alongside general-purpose AI models that break complex tasks down into individual steps — a division of labor that positions video-world models as the motor layer of future robotics stacks rather than the planner.

Original: the-decoder.de

Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Correspondent covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.

612 articles

Related articles

  1. Nvidia’s GR00T 2 lifts selected robot tasks from 40% to 95%
  2. Black Forest Labs enters robotics with open model FLUX 3 Action
  3. NVIDIA's GR00T 2 Learns Robot Skills From 400,000 Hours of Human Video
  4. Google DeepMind's Gemini Robotics 2 Gives Humanoids Whole-Body Control
  5. Odyssey-3 world model opens as free public research preview

« Previous articleNext article »