Skip to main content

MoonDream API Reference

Complete API reference for MoonDream Vision API endpoints, including request/response formats and usage examples.

Base URL​

http://localhost:20200

Authentication​

Currently, MoonDream API does not require authentication for local installations. For production deployments, consider adding API key authentication.

Endpoints​

MoonDream provides four main API endpoints for different computer vision tasks.

Health Check​

GET /​

Returns basic service status with load balancer statistics.

Response:

{
  "message": "Moondream Object Detection API is running!",
  "load_balancer_stats": {
    "total_models": 1,
    "current_index": 0,
    "device": "mps",
    "model_stats": {
      "0": {
        "requests": 150,
        "errors": 2,
        "last_used": 1704120600.5
      }
    }
  }
}

GET /health​

Returns health status with load balancer statistics.

Response:

{
  "status": "healthy",
  "load_balancer_stats": {
    "total_models": 1,
    "current_index": 0,
    "device": "mps",
    "model_stats": {
      "0": {
        "requests": 150,
        "errors": 2,
        "last_used": 1704120600.5
      }
    }
  }
}

Status Codes:

  • 200: Service is healthy
  • 503: Service is starting up or has issues

Visual Question Answering​

POST /v1/query​

Answers natural language questions about images.

Request:

  • Content-Type: multipart/form-data

Parameters:

  • question (string, required): Question about the image
  • init_image (file, required): Image file (PNG, JPEG, WebP)

Example:

curl -X POST "http://localhost:20200/v1/query" \
  -F "question=What do you see in this screenshot?" \
  -F "init_image=@screenshot.png"

Response:

{
  "answer": "I see a mobile app screenshot showing a login screen with username and password fields, and a blue login button.",
}

Error Responses:

  • 422: Validation Error (missing required parameters)

Image Captioning​

POST /v1/caption​

Generates descriptive captions for images.

Request:

  • Content-Type: multipart/form-data

Parameters:

  • init_image (file, required): Input image file

Example:

curl -X POST "http://localhost:20200/v1/caption" \
  -F "init_image=@screenshot.png"

Response:

{
  "answer": "..."
}

Error Responses:

  • 422: Validation Error (missing required parameters)

Object Detection​

POST /v1/detect​

Detects specific objects in images.

Request:

  • Content-Type: multipart/form-data

Parameters:

  • obj (string, required): The object to detect (e.g., 'face')
  • init_image (file, required): Input image file

Example:

curl -X POST "http://localhost:20200/v1/detect" \
  -F "obj=login button" \
  -F "init_image=@app_screenshot.png"

Response:

{
  "objects": [...],
  "request_id": 0
}

Error Responses:

  • 422: Validation Error (missing required parameters)

Point Detection​

POST /v1/point​

Finds precise coordinates for UI elements and objects.

Request:

  • Content-Type: multipart/form-data

Parameters:

  • obj (string, required): The element to point at (e.g., 'login button')
  • init_image (file, required): Input image file

Example:

curl -X POST "http://localhost:20200/v1/point" \
  -F "obj=search button" \
  -F "init_image=@mobile_app.png"

Response:

{
  "points": [...],
  "count": 0
}

Error Responses:

  • 422: Validation Error (missing required parameters)

Request/Response Formats​

Image Requirements​

Supported Formats​

  • PNG
  • JPEG/JPG
  • WebP
  • BMP
  • TIFF

Image Constraints​

  • Maximum Size: 10MB
  • Recommended Resolution: 720p - 1440p (1280x720 - 2560x1440)
  • Color Space: RGB
  • Aspect Ratio: Any (automatically handled)

Image Preprocessing​

MoonDream automatically:

  • Resizes large images for processing
  • Converts to RGB color space
  • Normalizes image data
  • Optimizes for model input requirements

Error Handling​

Common Error Responses​

Validation Error (422)​

{
  "detail": [
    {
      "loc": ["body", "question"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

Internal Server Error (500)​

{
  "detail": "Model inference error: ..."
}

HTTP Status Codes​

  • 200: Successful Response
  • 422: Validation Error (missing or invalid parameters)
  • 500: Internal Server Error (model/hardware issues)

Performance Characteristics​

Hardware Acceleration​

  • Apple Silicon (MPS): Automatic detection and optimization for macOS devices
  • NVIDIA GPUs (CUDA): GPU acceleration for compatible hardware
  • CPU Fallback: Works on any system without GPU acceleration

Load Balancing​

Requests are automatically distributed across available model instances using a round-robin algorithm. The number of model instances is controlled by the MOONDREAM_WORKERS environment variable.

Client Libraries​

JavaScript/Node.js​

const FormData = require('form-data');
const fetch = require('node-fetch');
const fs = require('fs');

async function detectObject(imagePath, object) {
  const form = new FormData();
  form.append('object', object);
  form.append('init_image', fs.createReadStream(imagePath));

  const response = await fetch('http://localhost:20200/v1/point', {
    method: 'POST',
    body: form
  });

  return await response.json();
}

Python​

import requests

def detect_object(image_path, object_name):
    with open(image_path, 'rb') as f:
        files = {'init_image': f}
        data = {'object': object_name}

        response = requests.post(
            'http://localhost:20200/v1/point',
            files=files,
            data=data
        )

    return response.json()

cURL Examples​

# Health check
curl http://localhost:20200/

# Image captioning
curl -X POST "http://localhost:20200/v1/caption" \
  -F "init_image=@screenshot.png"

# Question answering
curl -X POST "http://localhost:20200/v1/query" \
  -F "question=What UI elements do you see?" \
  -F "init_image=@app.png"

# Object detection
curl -X POST "http://localhost:20200/v1/detect" \
  -F "object=button" \
  -F "init_image=@interface.png"

# Point detection
curl -X POST "http://localhost:20200/v1/point" \
  -F "object=login button" \
  -F "init_image=@screenshot.png"

Monitoring and Debugging​

Request Logging​

All requests are logged with basic information including:

  • Model worker used for processing
  • Success/failure status
  • Error details (if any)

Load Balancer Statistics​

Access load balancer statistics via the health endpoints:

  • GET / - Basic stats with load balancer information
  • GET /health - Health status with load balancer stats

Statistics include:

  • Total number of loaded models
  • Current model selection index
  • Hardware device being used
  • Per-model request counts and error counts