Parameters

The knobs every decorator takes, and what each one changes

Every abstraction (job, endpoint, queue, cron, Box, Sandbox, VLLM) accepts the same resource, blueprint and storage parameters. The ones that differ are listed at the end.

Resources

parameter	default	what it does
cpu	1.0 (jobs 0.125)	cores per container; fractions allowed, or a string like "500m"
memory	128 MiB	per container; "8Gi" style strings work
gpu	none	a type ("T4", "L4", "A10G", "L40S", "A100-40", "A100-80", "H100", "H200", "RTXPro6000") or a preference list; the scheduler takes the first with capacity
gpu_count	0	GPUs per container; more than one for multi-GPU training
pool	scheduler’s choice	pin to a worker pool by name
timeout	3600 s (endpoints 180)	after which a task is killed and marked timed out
retries	3	retries on failure for jobs, crons and queues; retry_for=[SomeError] limits which exceptions retry
task_policy		TaskPolicy(max_retries=, timeout=, ttl=) as one object

Blueprint and files

parameter	what it does
image	a Blueprint(python_version=, python_packages=[], commands=[]); see Blueprint. Blueprint.from_dockerfile(path) and Blueprint.from_registry("...") also work, and .with_docker() enables Docker-in-Docker for sandboxes
volumes	a list of Volume(name=, mount_path=) or CloudBucket(name=, mount_path=, config=CloudBucketConfig(...)); see Volumes
disks	DurableDisk block storage on local machines
secrets	names of workspace secrets to inject as environment variables; see Secrets
env	extra environment variables, in plain text
name / app	the app the function belongs to in the dashboard; defaults to the script name
description	what the function is for, shown under its name in the dashboard; defaults to the first paragraph of the handler’s docstring (280 characters), "" shows nothing
show_args	jobs show each call with the arguments it was called with, whole, in the calls table, on the call’s own panel and in levers9 task list --format json. A config dict shows every key. A keyword argument or a dict key named like a credential (token, api_key, password, …) is redacted at any depth, and so is a string that looks like one (hf_..., sk-..., a JWT). Only a payload is shortened: a string past 2,000 characters, a container past 200 items, bytes past their first twelve, and the whole line past 8,000 characters, where it says how much is missing. False sends nothing. Default True

The current directory is synced into every container. Put a .levers9ignore file next to your script to leave things out (same syntax as .gitignore).

Anyone in the workspace can read the synced folder in the dashboard, so keep credentials out of it. Files that look like secrets (.env, .env.*, *.pem, *.key, id_rsa*, id_dsa*, id_ecdsa*, id_ed25519*, .aws/, .netrc, .npmrc, .pypirc) are still uploaded today, and every run, serve and deploy names them in a warning. A later release leaves them out. Put the values in a secret instead, or, if a file really has to ship, keep it with a line such as !.env in .levers9ignore. LEVERS9_SYNC_SECRET_FILES=skip leaves them out now. The dashboard’s code view lists such files but doesn’t show what’s in them.

Serving

Endpoints, queues, boxes and VLLM keep containers alive between calls.

parameter	default	what it does
keep_warm_seconds	endpoints 180, queues 10, boxes -1 never	how long an idle container stays up; 0 stops it as soon as it is idle, -1 never stops it. Whatever is left pays a cold start on the next call
workers	1	worker processes per container
max_pending_tasks	100	how many calls may be waiting or running before a new one is answered 429 (exceeded max pending tasks). An endpoint call stops counting as soon as it ends, when its caller hangs up while it waits, and when its container turns it away without starting it
autoscaler	QueueDepthAutoscaler()	QueueDepthAutoscaler(max_containers=, tasks_per_container=); LLMTokenPressureAutoscaler for VLLM
authorized	True	require the workspace token on the URL; False for public endpoints
on_start		a function that runs once per container before the first request (load a model, open a connection); its return value is passed to the handler as context.on_start_value
on_deploy		a function that runs at deploy time (warm a cache, download weights)
checkpoint_enabled	False	snapshot a warm container so later cold starts restore instead of booting; on a GPU the container must be sized to hold a copy of the card, see Cold starts
callback_url		a URL the controlplane posts each task’s result to. A caller can also pass its own callback_url with a call; that one is checked (see callback_hosts)
callback_hosts		hosts a caller may name in a per-call callback_url, besides the host of callback_url; "*.example.com" covers subdomains. Any other per-call URL has to be https to a public address. This release only logs, in the container’s output, a per-call URL that would be refused; env={"LEVERS9_CALLBACK_ENFORCE": "true"} refuses it now, and the result then goes to callback_url alone
inputs / outputs		a Schema for validated, typed endpoints

What a request brings with it

An endpoint, box or sandbox port gets the caller’s headers and cookies as they were sent, with one exception: the dashboard’s own cookies (levers9_session, levers9_oauth and levers9_cli) are taken out on the way in, so your code never sees someone’s dashboard sign-in. Your app’s own cookies get through untouched.

On a box reached through an authorized URL (/box/<name>/... or /box/id/...), the Authorization header is the caller’s levers9 token that let the request in. Don’t build on reading it: a controlplane can be set to drop it before it reaches the box (LEVERS9_BOX_STRIP_AUTHORIZATION), and that’s where it’s heading. On a public box the header is your app’s own and always gets through.

Outputs

Output saves a file produced inside a container and gives you a URL for it:

from levers9 import Output, job

@job()
def render():
    out = Output(path="chart.png")      # or Output.from_pil_image(img), Output.from_file(fh)
    out.save()
    return out.public_url(expires=3600)

Where each one is described

Jobs

.remote(), .map(), deploy as HTTP

Endpoints

serve with hot reload, keep-warm, autoscaling

Queues

.put(), retries, workers

Crons

when="0 2 * * *"

Boxes

ports, keep_warm_seconds, box.deploy()

Sandboxes

processes, files, network rules, snapshots