Enter the demo access code for AI, live captions, and cloud speech.
Captions off.
For a person who cannot speak, talking with someone who can hear. Show a hand sign, draw in the air with your fist, or shake the band: the page turns it into a sentence and says it out loud. What the other person answers is written on the screen, so the conversation keeps going both ways. How it works
Connect does nothing? Open this page through the web server:
http://localhost:3000/gesture-speak/ in Chrome or Edge.
Double-tap your band and press Connect.
Conversation
Nothing yet.
Draw in the air Camera off.
Hold up a hand sign and it is said. Make a fist to draw, move it, and open your hand when the drawing is done: it becomes a sentence that answers what the other person just said, and is spoken at once. A shake on the band works at any time. No camera? Draw on the board with the mouse or a finger. The board shows only your lines and a dot for your hand: the camera picture is never shown, and it stays on this computer.
Hand signs Hold one up to the camera for a moment.
Shapes Draw one with your fist. Anything else you draw becomes a sentence too.
How fast you are shaking not shaking
Band not connected.
3.5 shakes a second
Slow shakeTurn the wrist left and right slowly, about 2 a second, then stop.
Fast shakeShake as fast as you can for a moment, then stop.
Slow shake ×2A slow shake, stop for half a second, a slow shake again.
Fast shake ×2A fast shake, stop for half a second, a fast shake again.
| Phrase to say | Gesture | Colour | Examples | ||
|---|---|---|---|---|---|
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
 |
0 of 3 | ||||
|
0 of 3 | ||||
 |
0 of 3 | ||||
|
0 of 3 |
0.80
Demo script The other person says the line; you answer with the gesture.
Not connected
The last gesture the recogniser cut out of the motion is drawn here.
Red, green, blue: acceleration x, y, z. Orange, purple, cyan: turning speed x, y, z.