Tasks and asynchronous operations

Many operations in Meilisearch are processed asynchronously. These API requests are not handled immediately—instead, Meilisearch places them in a queue and processes them in the order they were received.

Which operations are asynchronous?

Every operation that might take a long time to be processed is handled asynchronously. Processing operations asynchronously allows Meilisearch to handle resource-intensive tasks without impacting search performance.

Currently, these are Meilisearch's asynchronous operations:

• Creating an index
• Updating an index
• Swapping indexes
• Deleting an index
• Updating index settings
• Adding documents to an index
• Updating documents in an index
• Deleting documents from an index
• Canceling a task
• Deleting a task
• Creating a dump
• Creating snapshots

Understanding tasks

When an API request triggers an asynchronous process, Meilisearch creates a task and places it in a task queue.

Task objects

Tasks are objects containing information that allow you to track their progress and troubleshoot problems when things go wrong.

A task object includes data not present in the original request, such as when the request was enqueued, the type of request, and an error code when the task fails:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

For a comprehensive description of each task object field, consult the task API reference.

Summarized task objects

When you make an API request for an asynchronous operation, Meilisearch returns a summarized version of the full task object.

{
  "taskUid": 0,
  "indexUid": "movies",
  "status": "enqueued",
  "type": "indexCreation",
  "enqueuedAt": "2021-08-11T09:25:53.000000Z"
}

Use the summarized task's taskUid to track the progress of a task.

Task status

Tasks always contain a field indicating the task's current status. This field has one of the following possible values:

• enqueued: the task has been received and will be processed soon
• processing: the task is being processed
• succeeded: the task has been successfully processed
• failed: a failure occurred when processing the task. No changes were made to the database
• canceled: the task was canceled

Global tasks

Some task types are not associated with a particular index but apply to the entire instance. These tasks are called global tasks. Global tasks always display null for the indexUid field.

Meilisearch considers the following task types as global:

• dumpCreation
• taskCancelation
• taskDeletion
• indexSwap
• snapshotCreation

NOTE

In a protected instance, your API key must have access to all indexes ("indexes": [*]) to view global tasks.

Task queue

After creating a task, Meilisearch places it in a queue. Enqueued tasks are processed one at a time, following the order in which they were requested.

Task queue priority

Meilisearch considers certain tasks high-priority and always places them at the front of the queue.

The following types of tasks are always processed as soon as possible:

1. taskCancelation
2. taskDeletion
3. snapshotCreation
4. dumpCreation

All other tasks are processed in the order they were enqueued.

Task workflow

When you make a request for an asynchronous operation, Meilisearch processes all tasks following the same steps:

1. Meilisearch creates a task, puts it in the task queue, and returns a summarized task object. Task status set to enqueued
2. When your task reaches the front of the queue, Meilisearch begins working on it. Task status set to processing
3. Meilisearch finishes the task. Status set to succeeded if task was successfully processed, or failed if there was an error

Canceling and deleting tasks

You can cancel a task while it is enqueued or processing by using the cancel tasks endpoint. Doing so changes a task's status to canceled.

Meilisearch does not automatically delete tasks once their status is succeeded, failed, or canceled. These tasks remain visible in the task list. To delete them, use the delete tasks route.

NOTE

Terminating a Meilisearch instance in the middle of an asynchronous operation is completely safe and will never adversely affect the database.

Tasks are not canceled when you terminate a Meilisearch instance. Meilisearch discards all progress made on processing tasks and resets them to enqueued. Task handling proceeds as normal once the instance is relaunched.

Examples

Suppose you add a new document to your instance using the add documents endpoint and receive a taskUid in response.

When you query the get task endpoint using this value, you see that it has been enqueued:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

Later, you check the task's progress one more time. It was successfully processed and its status changed to succeeded:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "succeeded",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
            "receivedDocuments": 67493,
            "indexedDocuments": 67493
    },
    "error": null,
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

Had the task failed, the response would have included a detailed error object:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "failed",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
            "receivedDocuments": 67493,
            "indexedDocuments": 0
    },
    "error": {
        "message": "Document does not have a `:primaryKey` attribute: `:documentRepresentation`.",
        "code": "internal",
        "type": "missing_document_id",
        "link": "https://docs.meilisearch.com/errors#missing-document-id"
    },
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

If the task had been canceled while it was enqueued or processing, it would have the canceled status and a non-null value for the canceledBy field.

After a task has been deleted, trying to access it returns a task_not_found error.

Filtering tasks

Querying the get tasks endpoint returns all tasks that have not been deleted. Use query parameters to filter tasks based on uid, status, type, indexUid, canceledBy, or date. Separate multiple values with a comma (,).

Filter by uid

The following code sample returns tasks with uids 5, 10, and 13:

curl \
  -X GET 'http://localhost:7700/tasks?uids=5,10,13'

client.getTasks({ uids: [5, 10, 13] })

client.get_tasks({'uids': ['5', '10', '13']})

$client->getTasks((new TasksQuery())->setUids([5, 10, 13]));

TasksQuery query = new TasksQuery().setUids(new int[] {5, 10, 13});
client.getTasks(query);

client.get_tasks(uids: [5, 10, 13])

client.GetTasks(&meilisearch.TasksQuery{
  UIDS: []int64{5, 10, 13},
})

await client.GetTasksAsync(new TasksQuery { Uids = new List<int> { 5, 10, 13 } });

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_uids([5, 10, 13])
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(uids: [5, 10, 13])) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(uids: [5, 10, 13]));

Filter by status

The following code sample returns tasks with the failed and canceled statuses:

curl \
  -X GET 'http://localhost:7700/tasks?statuses=failed,canceled'

client.getTasks({ statuses: ["failed", "canceled"] })

client.get_tasks({'statuses': ['failed', 'canceled']})

$client->getTasks((new TasksQuery())->setStatuses(['failed', 'canceled']));

TasksQuery query = new TasksQuery().setStatuses(new String[] {"failed", "canceled"});
client.getTasks(query);

client.get_tasks(statuses: ['failed', 'canceled'])

client.GetTasks(&meilisearch.TasksQuery{
  Statuses: []string{"failed", "canceled"},
})

await client.GetTasksAsync(new TasksQuery { Statuses = new List<TaskInfoStatus> { TaskInfoStatus.Failed, TaskInfoStatus.Canceled } });

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_statuses(["failed", "canceled"])
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(statuses: ["failed", "canceled"])) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(statuses: ['failed', 'canceled']));

Filter by type

The following code sample returns dumpCreation and indexSwap tasks:

curl \
  -X GET 'http://localhost:7700/tasks?types=dumpCreation,indexSwap'

client.getTasks({ types: ["dumpCreation", "indexSwap"] })

client.get_tasks({'types': ['dumpCreation', 'indexSwap']})

$client->getTasks((new TasksQuery())->setTypes(['dumpCreation', 'indexSwap']));

TasksQuery query = new TasksQuery().setTypes(new String[] {"dumpCreation", "indexSwap"});
client.getTasks(query);

client.get_tasks(types: ['dumpCreation', 'indexSwap'])

client.GetTasks(&meilisearch.TasksQuery{
  Types: []string{"dumpCreation", "indexSwap"},
})

await client.GetTasksAsync(new TasksQuery { Types = new List<TaskInfoType> { TaskInfoType.DumpCreation, TaskInfoType.IndexSwap } });

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_types(["dumpCreation", "indexSwap"])
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(types: ["dumpCreation", "indexSwap"])) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(types: ['dumpCreation', 'indexSwap']));

Filter by indexUid

The following command returns all tasks belonging to the index movies. Note that the indexUid is case-sensitive:

curl \
  -X GET 'http://localhost:7700/tasks?indexUids=movies'

client.getTasks({ indexUids: ["movies"] })

client.get_tasks({'indexUids': ['movies']})

$client->getTasks((new TasksQuery())->setIndexUids(['movies']));

TasksQuery query = new TasksQuery().setIndexUids(new String[] {"movies"});
client.getTasks(query);

client.get_tasks(index_uids: ['movies'])

client.GetTasks(&meilisearch.TasksQuery{
  IndexUIDS: []string{"movie"},
})

await client.GetTasksAsync(new TasksQuery { IndexUids = new List<string> { "movies" } });

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_index_uids(["movies"])
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(indexUids: ["movies"])) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(indexUids: ['movies']));

Filter by canceledBy

Use the canceledBy filter to view all tasks canceled by one or more taskCancelation tasks:

curl \
  -X GET 'http://localhost:7700/tasks?canceledBy=9,15'      

client.getTasks({ canceledBy: [9, 15] })

client.get_tasks({'canceledBy': ['9', '15']})

$client->getTasks((new TasksQuery())->setCanceledBy([9, 15]));

TasksQuery query = new TasksQuery().setCanceledBy(new int[] {9, 15});
client.getTasks(query);

client.get_tasks(canceled_by: [9, 15])

await client.GetTasksAsync(new TasksQuery { CanceledBy = new List<int> { 9, 15 } });

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_canceled_by([9, 15])
  .execute()
  .await
  .unwrap();

Filter by date

You can filter tasks by their enqueuedAt, startedAt, and finishedAt fields. To do so, prepend either before or after to each field name:

• enqueuedAt → beforeEnqueuedAt or afterEnqueuedAt
• startedAt → beforeStartedAt or afterStartedAt
• finishedAt →beforeFinishedAt or afterFinishedAt

This filter accepts dates formatted according to RFC 3339 (opens new window):

• YYYY-MM-DD
• YYYY-MM-DDTHH:MM:SSZ
• YYYY-MM-DDTHH:MM:SS+01:00

curl \
  -X GET 'http://localhost:7700/tasks?afterEnqueuedAt=2020-10-11T11:49:53.000Z'

client.getTasks({ afterEnqueuedAt: '2020-10-11T11:49:53.000Z' })

client.get_tasks({'afterEnqueuedAt': '2020-10-11T11:49:53.000Z'})

$client->getTasks((new TasksQuery())->setAfterEnqueuedAt(new \DateTime('2020-10-11T11:49:53.000Z'));

SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
Date date = format.parse("2020-10-11T11:49:53.000Z");
TasksQuery query = new TasksQuery().setBeforeEnqueuedAt(date);

client.getTasks(query);

client.get_tasks(after_enqueued_at: DateTime.new(2020, 10, 11, 11, 49, 53))

client.GetTasks(&meilisearch.TasksQuery{
 AfterEnqueuedAt: time.Date(2020, time.October, 11, 11, 49, 53, 0, time.UTC),
})

await client.GetTasksAsync(new TasksQuery { AfterEnqueuedAt = DateTime.Parse("2020-10-11T11:49:53.000Z") });

let date = OffsetDateTime::parse(
    "2020-10-11T11:49:53.000Z",
    &::time::format_description::well_known::Rfc3339,
)

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_after_enqueued_at(&date)
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(afterEnqueuedAt: dateFormatter.date(from: "2020-10-11T11:49:53.000Z"))) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(afterEnqueuedAt: DateTime(2020, 10, 11, 11, 49, 53)));

The above code sample will return all tasks enqueued after 11:49:53:00 on 11 Oct 2020.

NOTE

Date filters are equivalent to < or > operations and do not include the specified value. It is not possible to perform ≤ or ≥ operations with a task date filter.

Combine filters

You can combine task filters. Use the ampersand character (&) to combine filters, equivalent to a logical AND.

The following code sample returns all tasks in the movies index that have the type documentAdditionOrUpdate or documentDeletion and have the status of processing.

curl \
  -X GET 'http://localhost:7700/tasks?indexUids=movies&types=documentAdditionOrUpdate,documentDeletion&statuses=processing'

client.getTasks({
  indexUids: ["movies"],
  types: ["documentAdditionOrUpdate","documentDeletion"],
  statuses: ["processing"]
})

client.get_tasks(
  {
      'indexUids': 'movies',
      'types': ['documentAdditionOrUpdate', 'documentDeletion'],
      'statuses': ['processing'],
  }
)

$client->getTasks(
  (new TasksQuery())
    ->setStatuses(['processing'])
    ->setUids(['movies'])
    ->setTypes(['documentAdditionOrUpdate', 'documentDeletion'])
);

TasksQuery query =
        new TasksQuery()
                .setStatuses(new String[] {"processing"})
                .setTypes(new String[] {"documentAdditionOrUpdate", "documentDeletion"})
                .setIndexUids(new String[] {"movies"});

client.getTasks(query);

client.get_tasks(index_uids: ['movies'], types: ['documentAdditionOrUpdate', 'documentDeletion'], statuses: ['processing'])

client.GetTasks(&meilisearch.TasksQuery{
  IndexUIDS: []string{"movie"},
  Types:     []string{"documentAdditionOrUpdate", "documentDeletion"},
  Statuses:  []string{"processing"},
})

var query = new TasksQuery { IndexUids = new List<string> { "movies" }, Types = new List<TaskInfo> { TaskInfo.DocumentAdditionOrUpdate, TaskInfo.DocumentDeletion }, Statuses = new List<TaskInfoStatus> { TaskInfoStatus.Processing } };

await client.GetTasksAsync(query);

let mut query = TasksQuery::new(&client);
let tasks = query
  .with_index_uids(["movies"])
  .with_types(["documentAdditionOrUpdate","documentDeletion"])
  .with_statuses(["processing"])
  .execute()
  .await
  .unwrap();

client.getTasks(params: TasksQuery(indexUids: "movies", types: ["documentAdditionOrUpdate", "documentDeletion"], statuses: ["processing"])) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(indexUids: ['movies'], types: ['documentAdditionOrUpdate','documentDeletion'], statuses: ['processing']));

OR operations between different filters are not supported. For example, you cannot view only tasks which have a type of documentAddition or a status of failed.

Paginating tasks

By default, Meilisearch returns a list of 20 tasks for each request. You can adjust the number of tasks returned using the limit parameter, and control where the list begins using the from parameter.

For each call to this endpoint, the response will include the next field. When you call the endpoint again, pass this value as the from parameter to view the next set of results.

The following command returns two tasks at a time, starting from task uid 10:

curl \
  -X GET 'http://localhost:7700/tasks?limit=2&from=10

client.getTasks({ limit: 2, from: 10 })

client.get_tasks({
  'limit': 2,
  'from': 10
})

$taskQuery = (new TasksQuery())->setLimit(2)->setFrom(10));
$client->getTasks($taskQuery);

TasksQuery query = new TasksQuery()
      .setLimit(2)
      .setFrom(10);

client.index("movies").getTasks(query);

client.tasks(limit: 2, from: 10)

client.GetTasks(&meilisearch.TasksQuery{
  Limit: 2,
  From: 10,
});

ResourceResults<TaskInfo> taskResult = await client.GetTasksAsync(new TasksQuery { Limit = 2, From = 10 });

let mut query = TasksSearchQuery::new(&client)
    .with_limit(2)
    .with_from(10)
    .execute()
    .await
    .unwrap();

client.getTasks(params: TasksQuery(limit: 2, from: 10)) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(limit: 2, from: 10);

Response:

{
  "results": [
    …
  ],
  "limit": 2,
  "from": 10,
  "next": 8
}

To view the next set of results, you would repeat the same query, replacing the value of from with the value of next:

curl \
  -X GET 'http://localhost:7700/tasks?limit=2&from=8

client.getTasks({ limit: 2, from: 8 })

client.get_tasks({
  'limit': 2,
  'from': 8
})

$taskQuery = (new TasksQuery())->setLimit(2)->setFrom(8));
$client->getTasks($taskQuery);

TasksQuery query = new TasksQuery()
      .setLimit(2)
      .setFrom(8);

client.index("movies").getTasks(query);

client.tasks(limit: 2, from: 8)

client.GetTasks(&meilisearch.TasksQuery{
  Limit: 2,
  From: 8,
});

ResourceResults<TaskInfo> taskResult = await client.GetTasksAsync(new TasksQuery { Limit = 2, From = 8 });

let mut query = TasksSearchQuery::new(&client)
    .with_limit(2)
    .from(8)
    .execute()
    .await
    .unwrap();

client.getTasks(params: TasksQuery(limit: 2, from: 8)) { result in
  switch result {
  case .success(let taskResult):
    print(taskResult)
  case .failure(let error):
    print(error)
  }
}

await client.getTasks(params: TasksQuery(limit: 2, from: 8);

When the returned value of next is null, you have reached the final set of results.