Decision API

Typed questions about a live call, answered in tens of milliseconds by a decision model on a GPU, and the Decide page that shows it

The Decision API answers typed questions about a call while the call is still going. A decision model reads the transcript and a set of questions and answers all of them in one forward pass, each with probabilities. It does not write text; it scores. That is what makes it fast enough for a live call.

type	answer	example
noul	P(true) of a yes/no question	Has the customer asked to be called back?
choice	one of a few labels, with a probability each	How will this call be labelled?
score	an ordinal level	How urgent is this, 0 to 2?

Lev is Levers’ own fine-tuned decision model: a checkpoint on the laya-lab volume, registered in your workspace’s Models registry as lev-<tenant>-<version>. The endpoint also serves the public bases, so you can compare.

model	backend	what it is
laya-lab/models/<name>, served as lev-<name>	laya	a Lev registered in this workspace; the page lists every one the registry holds
convaiinnovations/laya-multilingual	laya	the base encoder, 100+ languages; not trained on your calls
jaredpalmer/kev-0.8b, kev-4b, kev-9b	kev	Qwen-based decision models (Apache-2.0); not trained on your calls

A base model is not trained on your calls, and on your own outcome labels it is usually not far from a random guess. It is fast and its probabilities look confident; that is not the same as being right. The Decide page shows a banner for as long as a base model is serving. Measure any model on your workspace’s held-out calls (Decision models) before you trust a number it gives.

What a Lev is trained on

A Lev is trained on the question its registry entry records per tenant (read from the checkpoint’s rl_agent_config.json, or pinned when it was registered), in exact words and option keys. The endpoint reports the tenants it was trained for (trained_questions on /health), and the dashboard tells it which ones the registry names when it deploys. A checkpoint trained on the customer’s sentiment as well records that question beside the tenants’ ones, under sentiment (the same for every tenant). When a Lev serves, the Decide page asks the Outcome forecast card, and the sentiment card when there is a sentiment question, in exactly those words and option keys, so it gets the trained behaviour, and marks every other card not trained yet: those are asked of the same model with no training behind them. The note on the trained card is your own latest run of that model on that tenant, never a number shipped with the product. The page puts the forecasts first and keeps the untrained moves and signals in a closed section below them. Through the API and the CLI, moves that are not trained stay out of the plan field unless asked for ("include_untrained": true on /api/rl/decide, --include-untrained on the CLI).

The Decide page

Lab → Decide is the Decision API made visible. Pick a world in the page header (the worlds of your workspace that hold real calls; the replay list, the outcome labels and the trained question are all per world). The model is picked on the bar beside it: every model in the registry is listed, your Levs first, and Add a model goes to Decision Engine → Models. Picking one while the API runs opens the deploy dialog for it; nothing changes until you deploy.

The composer sits at the bottom of the screen, always in view: the header, the case and the forecast scroll in their own region above it, which ends where the composer begins, so nothing is ever behind it. In it:

• Type the call, one turn per line (agent: …, customer: …; a line with no speaker is the customer’s). Every keystroke re-scores, debounced by 250 ms. The empty composer shows an example call as its placeholder.
• The chips (Ready to go ahead, Price objection, Wrong person, Call me later) type a short scripted call out for you, in Saudi Arabic by default or in English (the عربي / English switch). A model fine-tuned on your calls answers best in the language it learned from.
• Replay a call… lists real calls of the world (“Call 2060 · 4:30 · first line · platform label”). Picking one types it into the composer a turn at a time, about a turn a second (all at once with reduced motion), and the case re-scores after each turn. The status line shows the platform’s own label for the call (“Platform label: Booked visit”), so the forecast can be held against what really happened. Typing or pressing a chip ends the replay. Phone numbers are masked to their last three digits and no customer name is shown.

On a phone the composer drops its title and hint, and the chips scroll sideways on their own row.

The page is built around the forecast: where the call is heading. Above the composer:

• The case (left, “Where this call is heading”): the most likely outcome and the most likely sentiment always go in, drawn large (“Most likely: Booked visit, 73% of this call”). Any other label at or above the threshold goes in too, smaller and most likely first; the threshold is 25% by default and set with the slider under the case. A bar per forecast shows how sure the model is (its top probability). Packing is motion you can follow: a label’s drawing is lifted off its tile, carried along an arc and dropped into the case, which settles under it (several one after another), and an item that drops out is carried back to its tile. With reduced motion turned on in the system, items appear and leave in place. You have the final say: tap an item to set it aside (it stays aside until the model lets go of it too, or until you put it back), and the plus on any tile puts that label in by hand. Before anything is typed the case is empty and says so.
• The forecast (right): one tile per label, for every outcome label of the world (read from its real calls) and for each sentiment (positive, neutral, negative). Each tile has a drawing chosen from words in the label, the label in words, how likely it is for this call (“73% of this call”), and a check when it is in the case or a plus when it is not. The info button shows the question and whether the serving model was trained on it; a forecast it was not trained on is marked not trained yet. Below them, Will they buy? is named but not forecast (marked needs CRM outcomes, no number, never in the case): whether a lead reserves or pays happens after the call, so it becomes a trained forecast once each call is joined to the client’s CRM outcomes.
• Not trained yet (closed at first): the move and signal cards, each one typed question with its probability, its threshold and a small line of that probability across the turns. They are asked of the same model with no training behind them, and nothing in this section ever enters the case. The info button on a card shows the question and lets you move its threshold.

Every debounced keystroke sends a request without waiting for the one before, and only the newest answer is shown, so the page keeps up even when the round trip is near a second. Numbers and bars ease from one answer to the next, and a dot pulses while a request is in flight. With reduced motion turned on in the system, nothing flies or eases: the numbers change in place.

Next step, at the top of the case, is a hand-written rule over the forecast, not a prediction, and the page says so. Negative sentiment asks for a human sales agent; otherwise the most likely outcome decides: a booking, a visit or a hot lead goes to a human sales agent, a request for information to a WhatsApp with the details, a call back later to an AI agent that calls back, a warm lead to a WhatsApp and then an AI follow-up call, a cold lead or a no to one soft WhatsApp and then nothing, and anything else to verifying the number. Outcome labels are read by their words, so the rule works for any world’s labels. Its info button lists the whole rule. It lives in one small module (components/rl/decide/next-step.ts) so a learned policy can replace it.

The footer says what answered: model, device and the model’s own latency, with the round trip beside it. Under the hood in the footer (or Raw in the composer’s status line) opens a drawer with the latest answer as the Decision API saw it: Request, the body the dashboard sent (its route, the call as a list of turns, and every question with its type, instructions and options, in the trained words where the model was trained), and Response, the endpoint’s own JSON (answers with probabilities and confidence, usage, latency, model). Each has a Copy button. The workspace token never reaches the browser, so it is never in either.

Deploy it

From the page: Deploy, pick a model and the number of cards. The deployment is named lev, so it answers at …/asgi/lev/vN. From a terminal:

levers9 decide serve                                   # the first Lev in the registry, else the base
levers9 decide serve --model jaredpalmer/kev-4b        # a Kev base, for comparison
levers9 decide serve --keep-warm -1                    # always on: no cold start, holds its cards
levers9 decide status                                  # where it is, which model, its System One URL
levers9 decide stop
levers9 decide start                                   # the same version back, at the same URL

Manage it from Endpoints

The Decision API sits under Endpoints → Your endpoints beside the vLLM endpoints, and is managed the same way:

• Logs opens its live container’s output; its name opens the app it was deployed into (containers, calls).
• Stop and Start. Start brings the same deployment back, so the URL and the internal key stay the same.
• Reconfigure changes the model, the number of cards or how long it stays warm (5 minutes to 2 hours, or always on). A deployment cannot change in place, so this deploys a new version: its URL ends in the next version number, and a caller that pins the version must be updated. Callers that use the lev-latest alias still have to follow the new base URL.
• Try in Decide opens Lab → Decide on it.

Or by hand, from lab/rl:

cd lab/rl
DECISION_MODEL=laya-lab/models/acme-ft DECISION_TRAINED_TENANTS=acme levers9 deploy decision_endpoint.py:decide --name lev

DECISION_MODEL picks the checkpoint, DECISION_TRAINED_TENANTS names the tenants it was trained for (the dashboard sets it from the registry), DECISION_BACKEND (laya or kev) is inferred from the model, DECISION_GPU_COUNT defaults to 2 (on the Dammam pool a one-card request is placed on a node that cannot start it; the model itself needs a fraction of one card), and DECISION_KEEP_WARM_SECONDS to an hour. The endpoint needs the HF_TOKEN secret and mounts the laya-lab volume. Every deploy is a new version (/v2, /v3); the dashboard follows the newest one it deployed.

The first start pulls the image and loads the weights, a few minutes. Warm, one call with a dozen questions is answered in tens of milliseconds on an RTX PRO 6000; the round trip through the gateway and the dashboard adds network time on top, which the footer shows separately.

Call it

Through the dashboard

The dashboard calls the endpoint for you, with your own token (the browser never sees it); name a tenant and its cards are the questions.

The Decision API belongs to the workspace that deployed it. Its members can call, stop, start, redeploy and repoint it; from any other workspace there’s no Decision API to see, and a change is refused with a sentence saying it belongs to another workspace. A record from before this was filed is filed once, under the workspace its stored token belongs to. Pointing it at an endpoint by hand (PUT /api/rl/decision-service) takes a deployment on the platform’s own gateway; moving it to another host or path leaves the stored token behind unless a new token of the same workspace comes with it.

curl -s -X POST $DASH/api/rl/decide -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json' -d '{
  "tenant": "acme",
  "state": ["agent: Good evening, this is Acme.", "customer: Can we meet on Thursday to go ahead?"]
}'

{
  "cards": [{"id": "next_step", "p": 0.81, "checked": true, "word": "next step"}, "..."],
  "plan": [],
  "model": "lev-acme-ft", "base": "convaiinnovations/laya-multilingual", "backend": "laya",
  "fine_tuned": true, "device": "NVIDIA RTX PRO 6000", "latency_ms": 31.4, "roundtrip_ms": 104.0,
  "request": {"route": "POST /decide", "state": ["..."], "questions": {"outcome": {"type": "choice", "instructions": "...", "criteria": {"...": "..."}}}},
  "raw": {"model": "lev-acme-ft", "answers": {"...": "..."}, "latency_ms": 31.4}
}

request is the body sent to the endpoint and raw its answer as it came; neither ever carries the token.

Pass questions instead of tenant to ask your own. POST /api/rl/decide/batch takes states (up to 200) and answers every one in one pass: every prefix of a call at once, for a line per card.

Always pass a call as a list of turns, never one joined string. The model truncates a list from the left, so a long call keeps its newest turns, which are the ones a decision about the next move is about.

For the model that also reads a lead’s history, the tutorial walks one lead through a call, its journey, the events behind each answer and a recommendation, with the real response to each request.

Explain: what the model read

Add "explain": true to POST /api/rl/decide, /api/rl/decide/batch or /api/rl/recommend and a Fin release that reads the journey as text returns an explain block beside the answers, exactly as the model built it:

"explain": {
  "read": [{"i": 0, "speaker": "event", "text": "#1 day -14 [CRM | stage] Inbound lead ... (confirmed, 14 days before)", "event_id": "ev-1"}, "..."],
  "windows": [{"index": 0, "turns": [0, 11]}],
  "evidence_all": {"next_step": [{"id": "ev-2", "weight": 0.4419}, {"id": "ev-1", "weight": 0.3093}, {"id": "ev-late", "weight": 0.0}]},
  "attention_on_history": {"next_step": 0.6294},
  "dropped": [{"id": "ev-late", "why": "at or after the decision time (...): not read"}]
}

• read: every line and turn in the order the model read it, with the event id for an event line.
• windows: the encoder’s windows, as inclusive ranges of read positions.
• evidence_all: for each answer, every event you sent with its share of the chosen option’s attention on the event lines (they sum to 1; an event not read gets 0). The first three are the answer’s own evidence.
• attention_on_history: the share of that attention on the journey rather than the call.
• dropped: events at or after decision_time, and events folded into the summary line past the line budget.

Attention shows what the model read most for an answer. It is a useful clue, not proof of cause; the twin test on the Decide page is the check for that. Without explain nothing changes. A release or backend that cannot explain (an older Fin, a Lev, Kev) answers as before with explain_unsupported: true and an explain_note; redeploy with levers9 decide serve to get it.

The endpoint directly: TypeSafe’s System One API

The endpoint speaks TypeSafe’s System One API, the one Jev and Kev serve, so a client written against it switches by changing its base URL:

curl -s -X POST https://levers9.uselevers.com/asgi/lev/v1/v1/systemone \
  -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json' -d '{
  "model": "jev-latest",
  "state": ["customer: I am driving, call me after the evening prayer"],
  "questions": {
    "callback": {"type": "noul", "instructions": "Has the customer asked to be called back?"},
    "outcome": {"type": "choice", "instructions": "How will this call be labelled?",
                "criteria": {"Call Back": "asked for a call later", "Booked": "a meeting was booked", "Other": "anything else"}}
  }
}'

{
  "model": "jev-latest",
  "answers": {
    "callback": {"type": "noul", "noul": 0.93, "confidence": 0.86, "probabilities": {"false": 0.07, "true": 0.93}},
    "outcome": {"type": "choice", "choice": "Call Back", "confidence": 0.8, "probabilities": {"Call Back": 0.87, "Booked": 0.05, "Other": 0.08}}
  },
  "usage": {"input_tokens": 61, "output_tokens": 0},
  "latency_ms": 18.2
}

jev-latest, kev-latest and lev-latest are aliases for whatever the deployment serves, as is its own name (lev-acme-ft); any other model name is a 404. GET /v1/models describes the model (its name, whether it is fine-tuned, the card), and every response carries an x-typesafe-request-id header.

A caller with a workspace token is told the served name, whether it is fine-tuned and the card, never what it was built from or trained on. The base, backend, checkpoint and trained questions go only to the dashboard, which generates a key at every deploy (DECISION_INTERNAL_KEY) and sends it as x-levers-internal. So you can hand the endpoint to someone outside for a test with a restricted token (levers9 token create --token-type workspace_restricted) without telling them which model it is built on.

import os
from typesafe_sdk import TypeSafeClient   # pip install typesafe-sdk

client = TypeSafeClient(base_url="https://levers9.uselevers.com/asgi/lev/v1",
                        api_key=os.environ["LEVERS9_TOKEN"], model="jev-latest")

Or with plain requests:

import os, requests

r = requests.post("https://levers9.uselevers.com/asgi/lev/v1/decide",
                  headers={"Authorization": f"Bearer {os.environ['LEVERS9_TOKEN']}"},
                  json={"state": ["customer: too expensive for us"],
                        "questions": {"price": {"type": "noul", "instructions": "Is the customer pushing back on price?"}}})
print(r.json()["answers"]["price"]["noul"], r.json()["latency_ms"])

POST /decide and POST /decide/batch are the same answers with the model, whether it is fine-tuned and the device beside them. GET /health answers once the model is loaded. For a Fin release it also says what the release knows, by name: trained_fields, the id of every field it was trained on, and actions, the moves it has a chance of its own for. levers9 decide status prints both. A question under one of those ids, asked with the release’s own option keys, is one it was trained to answer; a candidate under one of those moves gets a chance from the model itself (Where the chances come from).

The cards

A tenant’s cards live in the Arena’s registry and are editable for the workspace:

curl -s "$DASH/api/rl/decision-presets?tenant=acme"                      # cards, scenarios, outcome labels
curl -s -X PUT $DASH/api/rl/decision-presets -d '{"tenant": "acme", "cards": [...]}'
curl -s -X POST "$DASH/api/rl/decision-presets/reset?tenant=acme"       # the shipped ones back

A yes/no card is asked as a two-option choice with neutral keys (a for yes, b for no) by default: the Laya model card warns that noul can under-report true. Set "ask_as": "noul" on a card to ask it as a noul. A choice card takes 2 to 20 options. The outcome forecast’s options are the world’s own platform labels, read from its real calls when the cards are built; the shipped cards name no tenant.