The Sink or Float Problem

We built a robot that guesses whether an object sinks or floats. It looks like a toy. It took seven falsified measurement channels and a calibration plate to get halfway.

1.0 g/cm³

Hand a seven-year-old an object and ask whether it will float. They turn it over, feel its weight, squeeze it, and answer in about a second. They are usually right, and they cannot tell you how they did it.

The physics is trivial. An object floats below 1 g/cm³, and density is mass over volume. Two numbers. We spent weeks assembling them from a camera and a tactile gripper, and nearly everything we learned was about the integration rather than any single measurement.

A photograph has no scale

Ask a vision model to estimate volume from an image and it fails for an obvious reason: there is nothing in a photograph to measure against. A bottle could be a miniature or a five-litre jug. Volume errors also cube, so a 20% scale error becomes a 1.7× density error, enough to cross the waterline.

The fix was physical. We put a 12-inch white square in frame and perspective-warped its corners onto a true square, so every frame the model sees spans exactly 304.8 mm. The question changes from "how big is this?" to "what fraction of this square does it cover?", which vision models answer well.

None of that is vision getting better. It is scaffolding so vision can answer a question it could not answer unaided.

Seven ways to not weigh something

If vision gives size, touch should give weight. The gripper has two tactile pads, nine sensing pillars each, reporting 3-axis force at 1 kHz. We tried seven times.

Channels one through six share one physical cause. The object was never hanging from the pads. It rested on a stand or in someone's hand, so its weight went into the support, and every signal we found was contact geometry that happened to correlate with the two objects in front of us.

So we inverted the gripper, jaws at the floor, object dangling free. Channel seven then looked promising: monotonic with mass, R² of 0.94, slope within 8% of gravity. Then we re-placed the same object three times.

Re-gripping the same object moves the reading further than the quantity being measured. Within a single grip the sensor is steady to 0.06 N. The error is all in the bedding-in: every grip finds a slightly different contact, then holds it perfectly.

Five of the six earlier failures produced a number that separated the first two objects tested. What killed them was always a third data point.

That is from our own notes, written after failure four. The rule it encodes, never believe a channel until you have re-placed the object, is what stopped a fake scale reaching a public demo.

So we measure size to a couple of percent and cannot measure mass at all. The honest fix is a different sensor: a load cell on the mount would read weight directly. Until then the demo claims no weight and nothing on screen shows grams.

What two instruments can do that one cannot

The camera, rectified against the plate, measures the outline but is blind to height. The gripper measures one dimension, the one it closes across, but sees no outline. Neither produces a volume. Together they do:

On a block of known dimensions this lands within 0.7%. On an irregular object it degrades to ±40%, because we assume one measured thickness holds across the whole outline. That band is displayed rather than hidden: it is decisive for foam at 0.1 g/cm³ and useless near 1.0.

The most useful bug we found

A rigid ceramic ring was called a float. It had not compressed at all. Being round in cross-section, it met the jaws along a line, and as force built the contact patch spread and the jaws closed several millimetres. From width alone that is indistinguishable from the object giving.

The fix came from data we had been discarding. The pads report each of their eighteen pillars individually and we were reading only the summed total. A flat face loads most pillars; a line contact loads three or four. When the machine cannot get a valid reading it now says so:

That abstention is more useful than a confident wrong answer, and it came free from data the sensor was already sending.

Integration is the whole problem

None of these measurements are research problems. Finding a square in an image, applying a perspective transform, reading a force sensor, asking a language model what an object is. All solved, all off the shelf.

What is not solved is making them agree about a physical object well enough to act on. The camera needed a physical target before its numbers meant anything. The gripper needed seven falsified hypotheses before we knew which channel to trust. Together they produce a volume neither could produce alone, and the thing they still cannot produce, mass, is the one a child gets for free by lifting.

That is the gap: not perception or reasoning, but the calibration work required to get several imperfect instruments to describe the same object consistently enough to do arithmetic on. On this evidence it is most of what physical AI is.

Built on a Contactile GAL2 tactile gripper with a PapillArray sensor pair, a fixed overhead camera, and Claude for vision. All measurements are from the instrumented rig, not simulation. The weight question remains open.