Manage Batches

Submit asynchronous bulk requests and inspect batch execution status.

Submit asynchronous bulk requests across supported model endpoints using the OpenAI-compatible Batch API.

POST/openai/v1/batches

Request Body

FieldTypeRequiredDescription
input_file_idstringYesUploaded file identifier with purpose set to batch.
endpointstringYesTarget endpoint (/v1/chat/completions, /v1/completions, /v1/responses, or /v1/embeddings).
completion_windowstringYesCompletion window, currently fixed at 24h.
metadataobjectNoOptional key-value pairs for tracking.

Input files must be UTF-8 JSONL with at most 50,000 items and a maximum file size of 200 MiB. Each JSONL line must be a valid JSON object containing a unique non-empty custom_id, method: "POST", url matching the batch endpoint, and an object body (with a required model field for strict native-prepaid batches). For native prepaid batches, the gateway validates stored file checksums and item execution policies before funding; all item reservations commit atomically upon admission before provider submission. Insufficient aggregate funds return HTTP 402 without partial submission. Incompatible provider configurations cannot be split within a single batch, and item-level cross-provider failover is not performed.

Batch Lifecycle

GET/openai/v1/batches/{batch_id}
POST/openai/v1/batches/{batch_id}/cancel
  • validating: Verifies input file formatting, JSONL syntax, and item schema compliance.
  • in_progress: Dispatches items to pinned provider credentials without cross-provider failover.
  • finalizing: Aggregates completion responses and records usage settlement.
  • completed: Successfully finishes processing, generating output_file_id and optional error_file_id.
  • failed: Terminal failure if input file validation or batch orchestration fails completely.
  • cancelling: Halts processing on request and transitions to cancelled state.
  • cancelled: Marks batch execution stopped before final completion. Failed, cancelled, or missing item results do not prove zero provider cost; settlement retains funding for reconciliation against provider records. Canonical status in AnyInt is cancelled.
  • expired: Triggers when processing exceeds the 24-hour completion window; unconfirmed or in-flight executions require reconciliation against provider records and do not establish zero provider cost.

Batch Operations and Output Downloads

GET/openai/v1/batches
  • List batches: GET /openai/v1/batches lists active and historical batch jobs for the authorized organization or project.
  • Object storage & downloads: Admitting files with purpose batch and running batch operations requires shared object storage (OBJECT_STORE_MODE). Once completed, output and error JSONL files can be retrieved via the files API at GET /openai/v1/files/{file_id}/content.

Compatibility Aliases

  • The gateway accepts /v1/batches for compatibility. Use /openai/v1/batches for new integrations.