Queue

Demo: queue

Source: demo/queue/app.py in the repository.

Background work: one task per item, in its own container, with retries. Use it for anything you do not want a caller to wait for: file processing, transcription, batch scoring.

cd demo/queue

# from Python: enqueue five tasks and wait until each is COMPLETE
python app.py
# 703e7c4e-… COMPLETE   (what a task prints is in its logs: levers9 task list, then the dashboard's Apps page)

# deploy it so other systems can enqueue over HTTP
levers9 deploy app.py:resize --name resize

TOKEN=$(awk -F' = ' '/^\[default\]/{f=1} f&&/^token/{print $2; exit}' ~/.levers9/config)
curl -s -X POST localhost:1994/queue/resize/v1 \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"image_id": "img-from-curl", "width": 1024}'
# {"task_id": "..."}

# watch the work
levers9 task list
levers9 task list --status COMPLETE
levers9 task stop `task id`

Notes

• retries=2 re-runs a task if the container crashes; timeout caps how long one task may take.
• keep_warm_seconds keeps a worker container alive between tasks so a steady trickle does not pay a cold start each time; autoscaler=QueueDepthAutoscaler(...) adds containers when the queue grows.
• on_start= runs once per container before the first task, for loading a model or warming a cache.
• The dashboard’s Apps page shows the queue with its tasks, and Containers shows the workers.