Silas Beck
Co-founder, Robotensor
← Writing
· Note 02

Why robot scores should be public

The work, the results, and the state of the art should be out in the open, where anyone can check them.

Most claims about robot models are made in a lab and reported on a slide. You cannot rerun them. You cannot see what was left out. Mostly you have to trust the person presenting.

That is not good enough for a field that wants machines working next to people.

Scores belong in public, where anyone can check them: the task, the model, and the result. If a new model beats the old one, everyone should be able to see it. If it does not, everyone should see that too.

That is why our training runs on Subnet 58, an open network where people compete to teach the best model. There is always a current best. When a challenger takes a run at it, I report whether the king fell.

Public scores do two things. They tell everyone the real state of the industry, not the marketing version. And they make progress compound, because the next person starts from what actually works.

Silas Beck
Silas BeckCo-founder, Robotensor · silas@robotensor.ai