Calibrate

PhysiClaw watches the screen with a camera and taps it with a stylus. To do that, it needs to learn two mappings:

One is from the phone screen to the camera image: figuring out where a given button on the screen appears in the camera’s view.

The other is from the phone screen to the arm’s plane of motion: working out how far the stylus must move to reach that button.

With both mappings, PhysiClaw’s eye and hand agree. It sees the screen, then taps exactly the right spot.

Camera

Before calibrating, open a camera-preview app on your computer, look at what the camera captures, and adjust the camera angle so the phone screen:

• fills as much of the frame as possible
• stays flat
• isn’t tilted

Apps that show the live camera feed:

• macOS: Photo Booth
• Windows: Camera
• Linux: Snapshot (or Cheese)

Level the phone

The phone’s rear camera bump keeps it from lying flat. Level it with something shaped to fit underneath, like a piece of thin cardboard cut to shape or a soft pad, so the screen stays horizontal and doesn’t wobble.

Setup

Start the server:

macOS

bash

physiclaw

The first time it reads the camera, macOS asks for permission. Click Allow. You can also change it later under System Settings → Privacy & Security → Camera.

Windows

powershell

physiclaw

Linux

Reading the camera and accessing the serial port require the video and dialout groups. Add yourself to both, then log out and back in:

bash

sudo usermod -aG dialout,video $USER

(On Arch-based distros the serial group is uucp instead of dialout.)

Then start the server:

bash

physiclaw

It automatically opens a setup wizard in your browser that walks you through calibration step by step.

Auto mode

The wizard can run on its own. In auto mode it completes every calibration step for you.

If you’ve calibrated before and haven’t moved the hardware, you can turn on auto mode.

First run

On first run, once calibration is complete, PhysiClaw spends a few minutes learning the phone’s screen layout, working out where the messaging app’s input field, send button, and on-screen keyboard sit. This step is fully automatic, nothing for you to do.

Just make sure WhatsApp (or your messaging app of choice) is pinned to the Dock (see Prepare the phone) so PhysiClaw can find and open it. Once learning is done, it’s ready to take tasks.

Switched phones?

Run physiclaw reset to clear the calibration and the learned layout, then restart PhysiClaw. It will re-calibrate and relearn the new phone’s interface.