> For the complete documentation index, see [llms.txt](https://docs.gradientj.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.gradientj.com/deployment/batch-api.md).

# Batch API

The Batch API is composed of two endpoints:\
1\. `/api/v0/deploy/create-batch-inference`\
2\. `/api/v0/deploy/check-batch-inference`

The first one mimics the API of the regular [inference endpoint](/deployment/inference-api.md) `/api/v0/deploy/inference` in terms of specifying the `model_id` and `deployment_version` except instead of sending a `params` dict, it accepts a `batches` argument that consists of an array of `params` for each of the batch requests.

The response to `/api/v0/deploy/create-batch-inference` will include a `batch_id` that can be used to poll the status with the second endpoint. Requests can take multiple hours to complete depending on OpenAI's load. 

It can be as quick as a few minutes, but in some cases can take hours.

Note that batch inference isn't supported for any providers except OpenAI and only chat models.

{% openapi src="/files/juHKwwoqJMXjksY7EJRW" path="/api/v0/deploy/create-batch-inference" method="post" %}
[batch\_api.yaml](https://3805607136-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfqfxUHftUkncczjFSv2d%2Fuploads%2FULnKeuzdED0EEgY6odMj%2Fbatch_api.yaml?alt=media\&token=2b21fe94-f458-4434-af7c-a1bcb0dcab07)
{% endopenapi %}

{% openapi src="/files/juHKwwoqJMXjksY7EJRW" path="/api/v0/deploy/check-batch-inference" method="post" %}
[batch\_api.yaml](https://3805607136-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FfqfxUHftUkncczjFSv2d%2Fuploads%2FULnKeuzdED0EEgY6odMj%2Fbatch_api.yaml?alt=media\&token=2b21fe94-f458-4434-af7c-a1bcb0dcab07)
{% endopenapi %}