moWerk

asteroid-docking-bay, or what happens when you let a machine write all of it

i have a drawer of smartwatches. thirty two of them work today, the rig is built for forty plus, and most of them run AsteroidOS. they sit in 3d printed cradles on a stack of usb hubs that can switch power per port. they need charging, draining, flashing, measuring, and occasionally rescuing. doing that by hand stopped being possible somewhere around watch number eight.

so there is a tool now. it is called asteroid-docking-bay, it is about twenty thousand lines of python, and i have not written a single one of them.

The asteroid-docking-bay web interface showing rows of watches grouped by usb hub, each with power state, connection, battery and a menu
the fleet view. one row per port, grouped by hub. shelved means the port is powered off and the watch is asleep in its cradle.

what it actually does

it keeps watches alive. the main job is unglamorous: hold every battery in a sensible band instead of letting it sit at 100 percent for months, which is how you kill lithium cells. it knows which port each watch is on, it can cut and restore power per port, and it can tell the difference between a watch that is off and a watch that is drawing nothing while pretending to charge.

beyond that it flashes nightlies to the whole drawer at once, runs drain tests to work out which watch is actually wearable for a day, dumps and restores stock roms, and talks to watches over usb, ssh, wifi and fastboot without me having to care which one is in use today.

Close up of six watch rows showing port number, power toggle, ppps badge, connection state, watch name, status icons and battery percentage
a closer look at the rows. almost everything lives in the row itself, because i got tired of hunting through panels.

and then it stopped being unglamorous

here is the part i did not plan. every time i found myself typing some long diagnostic command chain into a watch over adb, we added it. not as a script. as a panel, in the interface, that knows which watch you are looking at and what state it is in.

that went on for months. it has ended up somewhere i did not expect: a-d-b now reads kernel diagnostics that the watch images ship no tools for at all. there is no systemd-analyze on these builds, no powertop, no proper journal viewer. so we rebuilt the readings ourselves, gathered in one round trip over whatever link is up, and parsed on the host.

the control center has seven tabs now. identity, vitals, connect, power, diag, analysis and settings. the diag tab alone shows:

and the analysis tab draws the boot. systemd's own accounting, per service, as a waterfall, next to our host side measurement of the same boot: vbus on, kernel start, adb up, interface up. so you can see the bootloader phase and the userspace phase as separate things on one chart. i did not set out to build a boot analyser for watches. it turned up because i kept asking the same question manually.

underneath that there are eighty one named operations. every single thing the interface can do is one entry in a table, which is also the security boundary when the backend runs split off in a container. adding a capability means adding a line to that table in a diff somebody can read. a test fails if the list changes without anyone noticing.

The control center panel showing a live screenshot of the skipjack watch composited into a photo of the physical watch
the live screen, composited into a photo of the actual watch. this is skipjack. it is the feature with the least practical value and i use it constantly.

the argument i kept having with the machine

this is the part i most want to write down, because it took me a long time to articulate and it is the thing i think is actually interesting here.

over and over, i would ask for a feature and get told it was dangerous. flashing. dumping. restoring a stock rom. writing settings to a watch. the objection was always the same shape: this is an irreversible alteration to someone's device, so we should be careful about offering it.

and over and over, it had the baseline wrong. the alternative to a-d-b doing it is not that nobody does it. the alternative is somebody pasting a half remembered fastboot sequence out of a forum thread at one in the morning, getting a step wrong, and turning up in chat with a watch stuck on a spinner. that is the real world comparison. withholding the automation does not prevent the risky act, it just makes sure it happens worse.

so these are not dangerous features. they are the safety feature. taking a chain of manual commands that a newcomer will get wrong, doing it the same correct way every time, and putting a calm panel in front of it so the person is not guessing.

our flash procedure runs checks that experienced people routinely skip. not because they are careless, but because doing them by hand every time is tedious:

what happens before a single byte is written
check why it is there
sha512 against the release's own checksum file a truncated download looks completely fine in a directory listing
image filenames read from that same checksum file the names changed between releases. hardcoding them means flashing a 404
partition table compared to the live watch names and offsets must match exactly before anything is written
the image must name your device grepped out of the system partition, because two different watches can share a byte identical partition table and the layout check alone will pass both
per device partitions are backed up first, and never written sensor calibration and radio identity are the only data no other dump can replace. writing someone else's reports no error and you find out months later
the plan builder cannot express a bad restore it is structurally incapable of writing a per device partition or a populated userdata, so the dangerous restore is not a thing you can select by accident
it refuses a watch another operation is using and it pins the exact watch by its socket, never by codename, because two watches in the drawer can share a codename

i was right about this and the machine was wrong, repeatedly, and it took me a while to stop being talked out of it. that is worth saying out loud given everything else on this page is about the machine doing the work well.

the record is public, including the embarrassing parts

every investigation ships inside the repository as a dated document in docs/audits, with the methodology and the dead ends in it, not just the conclusion. there are seventeen of them so far. they are not press releases. they contain the things that went wrong, because a record that only lists wins is not a record.

some of what those audits turned up
what the finding
flashing and dumping both menu items were silently dead in the interface for over two weeks. the code was fine. nothing opened it.
usb brittleness my laptop's usb controller has 32 device slots. two hub cascades eat about 25 of them before a single watch is plugged in. months of "flaky hardware" was arithmetic.
power versus data the physical buttons on one hub cut power but not the data lines, so a watch can be fully reachable over adb at zero power. the led tells you about power, never presence.
mass power on a hub re-enumeration quietly restores power to every port, waking every shelved watch at once. the tool logged nothing, because it had never written down what the shelf was supposed to look like.
a benchmark that lied our own fps benchmark reported 67 frames per second on a panel capped at 60. it was dividing by the second it asked for instead of the second it got.

that last one is my favourite, because a maintainer spotted it in public and asked the obvious question, and the answer was that our instrument was wrong rather than our watches being fast. the fix and the numbers before and after are in benchymark.

the part that is unusual

i am a sushi chef. i am not a programmer and i am not going to become one. what i am reasonably good at is thoroughly observing and testing, feeling when i cope around something and articulating precisely what is wrong, which turns out to be most of the job.

every line of this project was written by claude, under my direction, since the end of june. i do not mean autocomplete and i do not mean a function here and there. i mean the architecture, the tests, the commit messages, the audits, all of it. my contribution is the hardware, the problem statements, and saying "no, that is wrong" a great many times.

i am aware how that sounds. i was sceptical too. so the whole thing is deliberately set up so you do not have to take my word for any of it.

the rules it follows came from ed beroset

ed beroset reviewed this a couple of times, somewhere around 0.5 to 0.7, and found four real bugs in a single day. that was not a lot of reviewing and it changed almost everything about how the project is built: the commit rules it follows are his, the code style rules are his, and the habit of validating a test by planting the bug it is supposed to catch is his. that last one keeps catching tests that passed against the exact defect they existed to find.

he is the single biggest influence on how this thing reviews, tests and commits, which is why it is not simply a machine talking to itself. llms are, in his words, generally terrible at coherent commits, and he was right, and it is fixable once somebody tells you.

where it is now

version 1.0 is close but not out. there are 883 tests, 37 modules and 660 commits. 0.1 to 0.8 are on the releases page with notes for each, and 0.9 is tagged.

Terminal output of the asteroid-docking-bay map command, discovering two Lenovo dock hubs and identifying watches on each port by codename and serial
the mapping step, which works out which watch is on which port by switching ports off and on and watching what disappears.

two smaller things fell out of it and got their own homes. benchymark is a rendering benchmark for watches, twelve phases, each isolating one cost. a-d-b-wanze is a probe that records battery and sleep behaviour on a watch you are wearing, built on one rule: never wake the watch, only record when something else already did. the gaps in its data are the measurement.

what i am not claiming

this is one person, one drawer of watches, one reviewer. the tests pass on my hardware. the audits are written by the same agent that wrote the code, which is a conflict of interest i can describe but not remove, and that is exactly why the method and the dead ends are published alongside the result. somebody re-running it is the only thing that fixes that.

i also do not think this generalises to everything. it works here because the problem is narrow, the feedback is brutal and immediate (the watch either enumerates or it does not), and because there is an author on the review side who is not impressed by volume.

i have been keeping a separate thing about how to disclose llm involvement honestly, graded rather than binary, at llmgd-specs. this project and its spin-offs carry a grade. it is self assessed, which is stated on the badge.

if you want to poke at it

the repository is here. it needs linux, a usb hub that supports per port power switching, and at least one watch you do not mind experimenting on. the readme has the setup. the audits are more interesting than the code.

if you only read one thing, read the flash validation record. it documents the first time the flashing feature was ever run against real hardware, which was long after it shipped, and it includes a correction of a claim the agent got wrong on the way. that is the shape of the whole project.