Skip to main content

Manual Gesture Control

Endpoints for streaming manual gestures from Android devices.

Gesture Streaming​

POST /api/manual/gesture-stream/start​

Starts a gesture streaming session.

Request Body:

{
  "deviceId": "device-123",
  "streamType": "realtime",
  "includeCoordinates": true
}

Parameters:

  • deviceId: Device identifier
  • streamType: Stream type - "realtime" or "buffered"
  • includeCoordinates: Include touch coordinates in stream

Response:

{
  "success": true,
  "sessionId": "stream-session-789",
  "streamUrl": "/api/manual/gesture-stream/sse/stream-session-789",
  "message": "Gesture streaming started"
}

POST /api/manual/gesture-stream/stop​

Stops a gesture streaming session.

Request Body:

{
  "sessionId": "stream-session-789"
}

Response:

{
  "success": true,
  "sessionId": "stream-session-789",
  "message": "Gesture streaming stopped"
}

GET /api/manual/gesture-stream/sse/:sessionId​

Server-sent events endpoint for real-time gesture streaming.

Parameters:

  • sessionId: Gesture streaming session ID

Response Format (SSE):

data: {"type": "gesture", "action": "touch", "x": 540, "y": 1200, "timestamp": 1640995200000}

data: {"type": "gesture", "action": "swipe", "startX": 100, "startY": 500, "endX": 100, "endY": 1000, "duration": 300}

data: {"type": "gesture", "action": "pinch", "centerX": 540, "centerY": 1200, "scale": 1.5}

Event Types:

  • touch: Single touch event
  • swipe: Swipe gesture
  • pinch: Pinch/zoom gesture
  • rotate: Rotation gesture
  • longpress: Long press event

Gesture Data Structure​

Touch Event​

{
  "type": "gesture",
  "action": "touch",
  "x": 540,
  "y": 1200,
  "pressure": 1.0,
  "timestamp": 1640995200000,
  "sessionId": "stream-session-789"
}

Swipe Event​

{
  "type": "gesture",
  "action": "swipe",
  "startX": 100,
  "startY": 500,
  "endX": 100,
  "endY": 1000,
  "duration": 300,
  "velocity": 500,
  "timestamp": 1640995200000,
  "sessionId": "stream-session-789"
}

Multi-Touch Event​

{
  "type": "gesture",
  "action": "multitouch",
  "touches": [
    {"x": 400, "y": 600, "pressure": 1.0},
    {"x": 680, "y": 600, "pressure": 1.0}
  ],
  "timestamp": 1640995200000,
  "sessionId": "stream-session-789"
}

Usage Examples​

Real-time Gesture Streaming​

const eventSource = new EventSource('/api/manual/gesture-stream/sse/stream-session-789');

eventSource.onmessage = function(event) {
  const gesture = JSON.parse(event.data);
  console.log('Gesture received:', gesture);
};

Start and Stop Streaming​

# Start streaming
curl -X POST http://localhost:3001/api/manual/gesture-stream/start \
  -H "Content-Type: application/json" \
  -d '{"deviceId": "emulator-5554"}'

# Stop streaming
curl -X POST http://localhost:3001/api/manual/gesture-stream/stop \
  -H "Content-Type: application/json" \
  -d '{"sessionId": "stream-session-789"}'