Skip to main content

Sync Management

Endpoints for managing synchronization jobs and integration sync processes.

Sync Jobs​

GET /api/sync/jobs​

Gets all sync jobs.

Query Parameters:

  • status: Filter by status (pending, running, completed, failed)
  • integrationId: Filter by integration
  • limit: Maximum jobs to return (default: 20)

Response:

{
  "jobs": [
    {
      "id": "sync-job-123",
      "integrationId": "plane-integration-456",
      "type": "full_sync",
      "status": "completed",
      "progress": 100,
      "createdAt": "2024-01-01T12:00:00.000Z",
      "startedAt": "2024-01-01T12:00:05.000Z",
      "completedAt": "2024-01-01T12:02:30.000Z",
      "duration": 145000,
      "result": {
        "processed": 150,
        "created": 25,
        "updated": 100,
        "deleted": 5,
        "errors": 0
      }
    }
  ],
  "total": 1
}

GET /api/sync/jobs/:jobId​

Gets a specific sync job.

Parameters:

  • jobId: Sync job identifier

Response:

{
  "id": "sync-job-123",
  "integrationId": "plane-integration-456",
  "type": "incremental_sync",
  "status": "running",
  "progress": 65,
  "stage": "syncing_work_items",
  "createdAt": "2024-01-01T12:00:00.000Z",
  "startedAt": "2024-01-01T12:00:05.000Z",
  "estimatedCompletion": "2024-01-01T12:03:00.000Z",
  "stats": {
    "processed": 98,
    "created": 15,
    "updated": 75,
    "deleted": 2,
    "errors": 1
  },
  "logs": [
    {
      "timestamp": "2024-01-01T12:00:05.000Z",
      "level": "info",
      "message": "Starting incremental sync"
    },
    {
      "timestamp": "2024-01-01T12:01:00.000Z",
      "level": "info",
      "message": "Processed 50 work items"
    }
  ]
}

GET /api/sync/integrations/status​

Gets sync status for all integrations.

Response:

{
  "integrations": [
    {
      "integrationId": "plane-integration-456",
      "name": "Plane.so Integration",
      "status": "active",
      "lastSync": "2024-01-01T12:00:00.000Z",
      "nextSync": "2024-01-01T12:05:00.000Z",
      "syncInterval": 300,
      "currentJob": {
        "id": "sync-job-123",
        "status": "running",
        "progress": 65
      },
      "stats": {
        "totalJobs": 24,
        "successRate": 0.92,
        "averageDuration": 120000
      }
    }
  ]
}

Job Control​

POST /api/sync/initialize​

Initializes the sync system.

Response:

{
  "success": true,
  "message": "Sync system initialized",
  "schedulerStatus": "running",
  "integrationsFound": 2
}

POST /api/sync/jobs​

Creates a new sync job.

Request Body:

{
  "integrationId": "plane-integration-456",
  "type": "full_sync",
  "options": {
    "force": false,
    "includeArchived": false,
    "batchSize": 50
  }
}

Response:

{
  "success": true,
  "jobId": "sync-job-789",
  "status": "queued",
  "estimatedDuration": 180000
}

POST /api/sync/jobs/:jobId/resume​

Resumes a paused or failed sync job.

Parameters:

  • jobId: Sync job identifier

Response:

{
  "success": true,
  "jobId": "sync-job-123",
  "status": "running",
  "message": "Job resumed"
}

POST /api/sync/jobs/:jobId/stop​

Stops a running sync job.

Parameters:

  • jobId: Sync job identifier

Request Body:

{
  "graceful": true  // Allow current operation to complete
}

Response:

{
  "success": true,
  "jobId": "sync-job-123",
  "status": "stopping",
  "message": "Job stop requested"
}

POST /api/sync/jobs/stop-all​

Stops all running sync jobs.

Request Body:

{
  "integrationId": "plane-integration-456"  // Optional: stop jobs for specific integration
}

Response:

{
  "success": true,
  "stoppedJobs": 3,
  "jobs": ["sync-job-123", "sync-job-456", "sync-job-789"]
}

Integration Sync Control​

POST /api/sync/integrations/:integrationId/start​

Starts sync for a specific integration.

Parameters:

  • integrationId: Integration identifier

Request Body:

{
  "syncType": "incremental",
  "scheduleImmediate": true
}

Response:

{
  "success": true,
  "integrationId": "plane-integration-456",
  "jobId": "sync-job-789",
  "message": "Sync started"
}

POST /api/sync/integrations/:integrationId/stop​

Stops sync for a specific integration.

Parameters:

  • integrationId: Integration identifier

Response:

{
  "success": true,
  "integrationId": "plane-integration-456",
  "activeJobsStopped": 2,
  "message": "Integration sync stopped"
}

PUT /api/sync/jobs/:jobId/interval​

Updates sync interval for a job.

Parameters:

  • jobId: Sync job identifier

Request Body:

{
  "interval": 600  // New interval in seconds
}

Response:

{
  "success": true,
  "jobId": "sync-job-123",
  "oldInterval": 300,
  "newInterval": 600,
  "nextRun": "2024-01-01T12:10:00.000Z"
}

DELETE /api/sync/jobs/:jobId​

Deletes a sync job.

Parameters:

  • jobId: Sync job identifier

Request Body:

{
  "deleteLogs": false  // Keep job logs for auditing
}

Response:

{
  "success": true,
  "deleted": "sync-job-123"
}

Sync Types​

Full Sync​

  • Synchronizes all data from scratch
  • Used for initial setup or data repair
  • More resource intensive but ensures complete sync
  • Can be forced even if recent sync exists

Incremental Sync​

  • Synchronizes only changed data since last sync
  • More efficient for regular updates
  • Requires successful baseline sync
  • Default sync type for scheduled jobs

Selective Sync​

  • Syncs specific data types or projects
  • Useful for targeted updates
  • Can include/exclude specific entities
  • Good for troubleshooting sync issues

Usage Examples​

Start Manual Sync​

curl -X POST http://localhost:3001/api/sync/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "integrationId": "plane-integration-456",
    "type": "incremental_sync"
  }'

Check Sync Status​

curl http://localhost:3001/api/sync/integrations/status

Stop All Syncs​

curl -X POST http://localhost:3001/api/sync/jobs/stop-all

Change Sync Interval​

curl -X PUT http://localhost:3001/api/sync/jobs/sync-job-123/interval \
  -H "Content-Type: application/json" \
  -d '{"interval": 600}'

Resume Failed Job​

curl -X POST http://localhost:3001/api/sync/jobs/sync-job-123/resume

Monitoring and Troubleshooting​

Job Statuses​

  • queued: Job is waiting to start
  • running: Job is currently executing
  • completed: Job finished successfully
  • failed: Job encountered an error
  • paused: Job was paused (can be resumed)
  • stopping: Job is in the process of stopping
  • stopped: Job was manually stopped

Common Issues​

Sync Not Starting​

  • Check integration configuration
  • Verify API credentials and permissions
  • Check system resources (CPU, memory, network)

Sync Failing​

  • Review job logs for error details
  • Check external service status
  • Verify network connectivity
  • Try full sync to reset state

Performance Issues​

  • Reduce sync frequency
  • Use selective sync for large datasets
  • Schedule syncs during off-peak hours
  • Monitor resource usage

Best Practices​

Scheduling​

  • Use incremental syncs for frequent updates
  • Schedule full syncs weekly or monthly
  • Avoid overlapping sync jobs
  • Monitor sync performance and adjust intervals

Error Handling​

  • Enable automatic retries for transient errors
  • Set up alerts for failed syncs
  • Keep detailed logs for troubleshooting
  • Implement circuit breakers for unstable integrations

Resource Management​

  • Limit concurrent sync jobs
  • Configure appropriate timeouts
  • Monitor API rate limits
  • Use batching for large datasets