Research3 min read

AI Models Are Learning to Run Robot Arms, but Slowly

By , Senior AI ConsultantPublished

General-purpose AI models are getting good enough to steer real robot arms from camera images, and what holds them back is speed and cost, not understanding the job.

Nine of the leading AI models, including versions of GPT, Claude and Gemini, were given control of two real robot arms with three cameras and asked to do tasks like putting objects in a safe, capping a pen and sorting items into baskets. Each model picked where the grippers should go and how far to open them, looked at the result, and chose the next move, up to 40 times. From July to October 2026, the best score on those tasks doubled about every 54 days.

Language models have been able to plan robot jobs for a while: Google's robotics models, for example, already point at objects and write code that the robot runs. What is different here is that a general model does the whole loop itself, from seeing to deciding where each gripper goes, with no robot-specific training, and gets better with every release.

The judgment is there, the speed is not

The best models still only get about 45% to 48% of the way through a typical task. But they rarely fail because they are confused. They fail because they run out of moves, or because they make a mistake that cannot be undone: a safe door knocked shut with objects still outside, or a pen cap squeezed so hard that it shoots out of reach. Careful checking, such as lifting a bowl to see whether it is really held, is what separates the models that finish from the ones that do not.

Speed and price are the gaps, and they trade against each other. The fastest setup takes 1.1 minutes per step and costs $4.51 a step, which is about $245 an hour in fees. The cheapest costs $0.38 a step but takes 4.18 minutes, about $5.50 an hour. A remote human operator steering the same arms needs 25 to 40 seconds for a whole task, so today a person beats both. But the best speed rose 4.4 times and the lowest cost fell 6.2 times in under two months, and no one model has yet combined the two. If those lines keep going, models reach a remote operator's speed around the middle of 2027. Trends this young rarely run that long, so read that as the earliest date, not the expected one.

Specialists lead first, then the general model takes over

The best robot-specific model is still ahead, about 1.3 times better and 1.7 times faster on the same tests. That looks like the situation in language in 2020. Back then, a company that wanted a computer to sort support emails or translate a manual had to train a separate model for that job on thousands of labelled examples. Then GPT-3 showed that one general model could do many such jobs from an instruction and a few examples. It was often worse than the specialists at first, but it needed no training project, and a new job cost one sentence instead of a new model.

Robots carry the same cost today. Every new task on a factory or warehouse robot means an integrator, a programming quote and weeks of testing, and the result only works for that task. A general model that can be told "put the candle and the card in the box, then close it" removes that quote. A likely setup is a general model that plans the job while a specialist handles the fine hand movements, because the specialist is faster at small motions.

Slow work goes first

A model that takes four minutes a step loses to a person on a packing line. It does not lose on the night shift. A small lab could leave a cheap model running a bench overnight, preparing sample plates: ten hours at about $5.50 an hour is $55, with nobody waiting on it, and a changed protocol is a changed paragraph of instructions. A returns warehouse could do the same with a bin of mixed items to sort before morning. My estimate is that unattended work like this is where general models first earn their cost, well before they compete on speed.

There is one limit to put in place first. Another team stopped its trials after a model repeatedly commanded unsafe moves and damaged the hardware. Force limits and no-go zones belong in the robot's own controller, where the model cannot override them. When a supplier offers a general-model robot, ask what happens when the task changes and where the safety limits live. If the first answer is a reprogramming quote, the pricing is about to look old.

Share this

STAY INFORMED

Get AI intelligence like this delivered to your inbox.

Free forever · Unsubscribe anytime


You May Also Find Valuable