Open-weight decision model

Skip the chatter.
Get the verdict.

Bernoulli takes unstructured state and typed questions and returns calibrated probabilities you can branch on. One forward pass per question. No generated tokens, no parsing, no JSON repair, no retries.

Open source. Self-host it in your VPC — your data stays on your infra.

Live demo runs the dev-tier Qwen2.5-VL-7B backbone on prepaid HF credits — same model as the benchmark numbers. Sleeps when idle, ~45 s to wake on first click. Fair-use session cap applies.

~145 msp50 per question
0.025ECE on SST-2
0outbound calls
bernoulli decide1201 ms · 3 questions
state.text
Where is my package? I ordered it two weeks ago and nothing has arrived. This is unacceptable service.
What does the customer want?choice
tracking0.942
other0.041
refund0.015
exchange0.001
Is this urgent?binary
truep = 0.947
How angry is the customer? (1–5)rating
1
2
3
4
5
expected = 4.30
Why

Your app needs a branch, not a paragraph.

Traditional LLM calls stuff instructions, context, and the question into one prompt, then parse the generated text. Bernoulli splits context (state) from queries (questions) and reads the probability directly. Same backbone — different reading mechanism.

Traditional LLM one call per question · parse what comes back
// one HTTP call per question
POST /chat/completions
{
  "messages": [
    {"role": "system",
     "content": "Reply with one word: refund, exchange, tracking, other."},
    {"role": "user",
     "content": "<the email>\nWhat does the customer want?"}
  ]
}

// response — now parse it
response.choices[0].message.content === "refund"
                                 // or "refund."
                                 // or "The customer wants refund."
                                 // or a refusal, or malformed JSON...
// confidence? ask the model to guess one, maybe parse that too
Bernoulli many questions per call · typed decision out
// one call, as many questions as you want
POST /v1/decide
{
  "state": {"text": "<the email>"},
  "questions": [
    {"id": "intent", "type": "choice",
     "prompt": "What does the customer want?",
     "options": ["refund", "exchange", "tracking", "other"]}
  ]
}

// response — already typed
response.decisions.intent.answer       === "refund"
response.decisions.intent.confidence   === 0.72
response.decisions.intent.distribution.tracking === 0.24
~70ms

Per added question

State tokens are processed once per call. Ask 20 things about one document — pay for the document once, each question adds ~70 ms on a 7B.

p=0.72

Typed, not text

Branch on confidence > 0.9 or expected ≥ 4.0. No regex, no "did it say Yes or yes.", no JSON repair, no retry on refusal.

1×N

Composable

Same state, different question configs. A/B test wording, swap in a calibrated temperature per type, add a question — no re-engineering.

1token

One output token

API pricing is weighted toward output: 3–5× input rates. Classification generates one token (max_tokens=1) — the output line item drops to the floor.

Bernoulli is for decisions, not content. If you need prose — summaries, explanations, rewrites — use a generative LLM.

What you build with it

Decisions worth making a thousand times a day.

Classification, triage, moderation, grading — anywhere your code would branch on a model's answer. State is the document; questions are the queries. Multiple questions per request share one forward pass over the state.

Support triage

Route tickets by intent, urgency, and anger.

Classify what the customer wants, how time-critical it is, how upset they sound, and whether it needs a senior agent — all in one call against the email thread.

choiceratingratingbinary
Backed by CLINC150-OOS — Bernoulli NLL 19× better than text-and-parse. "None of these" probability discriminates in-scope vs out-of-scope.
LLM guardrails

Judge your model's output before shipping it.

Ask whether a response is safe, on-topic, factually risky, or over-refusing. Threshold the probability to auto-ship, send to human review, or block.

binarybinaryratingchoice
Backed by XSTest + ToxicChat — Bernoulli <1% false-refusal rate on XSTest (4 of 500 prompts); NLL 10–15× better than text-and-parse across both.
Content moderation

Keep the review queue focused on the gray area.

Classify category and severity. Auto-ship clear-safe content, auto-reject clear violations, send only the uncertain cases to a human reviewer.

choiceratingbinarybinary
Backed by Yelp 1-5 stars — Bernoulli MAE 0.42, lowest of any baseline (Generative 0.46, BGE+LR 0.53). Uses the full predicted distribution, not just argmax.

Full write-ups — realistic state, request shapes, routing pseudocode — in the use cases guide, including code review severity.

See 25 real lenses across 5 industries → Salesforce, Terms of Service, Financials, Software Contracts, Shopify — each with five production-shaped question definitions.
Is this right for you?

A quick fit check — honest, benchmark-backed.

Four reasons to pick us, four reasons to pick something else. If any "pick something else" row matches your workload, use the named alternative — we'd rather you solve your problem than use our thing.

Pick Bernoulli when
  • You need calibrated probabilities for threshold-based auto-decisions
  • Your content is novel / recent / proprietary (not in public training sets)
  • You ask binary Yes/No questions over semantic content
  • You need multiple questions over the same input
Pick something else when
  • You have 2000+ labels & fixed label space → BGE-m3 + LR
  • Classical NLP (sentiment, topic) → DeBERTa-v3-zeroshot
  • Production safety priority → Llama Guard / ShieldGemma
  • You need generated text → chat LLM
See the full fit assessment → Full USECASES.md on GitHub
Features

Small surface. Strong guarantees.

A·B·C

Three typed questions

choice over 2–26 options, binary yes/no, and rating on an integer scale with full distribution and expected value.

p ≈ 0.9

Calibrated probabilities

Per-question-type temperature scaling fit on held-out data. When Bernoulli says 0.9, it should be right about 90% of the time.

⇄

Position debiasing

Re-scores with the option order reversed (or all cyclic shifts) and averages, cancelling the model's preference for "the first option".

1×

One forward pass

No decoding loop. State goes first and questions last, so many questions over one state share a cached prefix.

⌂

Self-host in your VPC

Open weights, your hardware, your network. Zero outbound calls at inference time. Offline mode is a first-class target.

◫

Multimodal state Soon

Text plus up to 8 images per state: screenshots, documents, photos. Long context up to the backbone's native window.

How it works

From state to decision in five steps.

The model never writes an answer. Its next-token distribution, restricted to a handful of single-token labels, is the answer.

01

Build the prompt

…state…
Q: What does the customer want?
A) refund  B) exchange
C) tracking D) other
Answer:

State first, question last, options labeled with letters. The assistant turn is pre-filled so the next token is the answer.

02

One forward pass

logits over ~152k tokens

No decoding loop. Read the scores for every possible next token at the last position.

03

Keep only labels

4 single-token labels

Throw away everything except A–D. Each label is verified at startup to be exactly one token.

04

Debias

orderABCD
reverseABCD
↳ map back · average

Score again with the options reversed, so "tracking" sits in a different slot, then average. Position bias cancels out.

05

Calibrate

÷ T = 1.35
tracking.94
other.04
refund.02
exchange.00

Softmax with a fitted temperature per question type. Out comes a typed decision with an honest probability.

Benchmarks

Same accuracy. Honest confidence.

Nine benchmarks — five academic, four use-case — against the same backbone prompted to answer in a word and parsed. Bernoulli stays within a point or two of accuracy and the NLL gap runs 7–18× in Bernoulli's favor.

BenchmarkBernoulli accGenerative accNLL ratio (B vs G)
SST-2 · binary sentiment0.910.928.8×
AG News · 4-way topic0.830.777.7×
PAWS · adversarial paraphrase0.850.8412.0×
CLINC150-OOS · in-scope0.450.2919×
Yelp · 1-5 star rating0.620.5813×
arXiv post-cutoff · papers the model can't have seen0.950.9210.7×

All rows at N=500 · Qwen2.5-VL-7B-Instruct · academic 2026-10-15, use-case 2026-10-16. arXiv row is the anti-contamination test: papers published Sept-Oct 2026, strictly after any plausible training cutoff — DeBERTa-v3-zeroshot drops to 0.53 on the same examples. See benchmarks.html for all 9 benchmarks across every metric, the DeBERTa + BGE-m3+LR columns, methodology, and known caveats. Raw per-benchmark result markdowns live at github.com/shyamsfo/bernoulli/benchmarks.

0.95 / 0.53

Bernoulli vs DeBERTa-v3-zeroshot on arXiv papers published strictly after both models' training cutoffs. The strongest "did the model memorize this?" test in the suite.

12×

better NLL than the same-model generative baseline on PAWS, at higher accuracy (0.85 vs 0.84). Adversarial paraphrase binary — the hardest calibration test in the suite.

0.42

Mean absolute error on Yelp 1-5 star rating — lowest of any baseline (Generative 0.46, BGE+LR 0.53). Uses the full distribution, not just argmax. Where "typed decisions, not single tokens" earns its keep.

See all 9 benchmarks → Try it on your own data → Raw results on GitHub
Know when it's wrong

Same 72 mistakes. Only one shows you where.

Each dot is one SST-2 review, 872 in all. Both methods get the same 800 right. The difference is what the confidence tells you about the other 72.

Prompted LLM generative

Every answer comes back at 1.00. The mistakes look exactly like the correct answers, so there's nothing to filter on.

Bernoulli calibrated

Sorted by confidence (darker = more sure). The mistakes cluster at the low-confidence end, exactly where you'd send them to a human.

correct (shade = confidence) wrong routed to review
20%60%100%
96.1%accuracy on auto-decided
0.83confidence threshold
–mistakes caught for review
–items sent to a human

Data: every prediction from the SST-2 validation run (Qwen2.5-VL-7B-Instruct, reverse debiasing, calibrated). The generative baseline has no usable confidence, so its accuracy stays at about 91.7% however much you hold back.

Interface

One request. Every question answered.

{
  "state": {"text": "Customer email: Where is my package?…"},
  "questions": [
    {"id": "intent", "type": "choice",
     "prompt": "What does the customer want?",
     "options": ["refund", "exchange", "tracking", "other"]},
    {"id": "urgent", "type": "binary",
     "prompt": "Is this urgent?"},
    {"id": "anger", "type": "rating",
     "prompt": "How angry?", "scale": [1, 5]}
  ],
  "options": {"debias": "reverse", "calibrated": true}
}

Typed in, typed out

Pydantic v2 discriminated unions with extra='forbid'. Invalid input is rejected with a 422, never silently guessed.

Your strings come back

Options are scored as letters internally; responses return your original option labels and integer rating keys.

Swappable backbone

A Scorer interface with HF and vLLM backends. Model id and revision are pinned in config, never hard-coded.

Calibrate on your data

Temperatures transfer imperfectly across domains, so domain re-calibration from your own labeled JSONL is planned.

Deployment

Three ways to run Bernoulli.

Bernoulli is open source, so the model and the code travel with you. Pick the mode that matches your data-residency, ops, and compliance posture.

Self-host

Run it yourself.

Clone the repo, pull the Docker image, point it at your GPU. Your infra, your network, your data — zero outbound calls at inference time.

Flagship. Apache 2.0 licensed.
Usage guide on GitHub →
Managed on your cloud

We operate it in your VPC.

You keep data residency; we handle deployment, GPU capacity, upgrades, monitoring, and incident response. Same sovereignty as self-host, without the ops.

For teams that want sovereignty without ops. Book a call →
Hosted by us

Call our endpoint.

No infra setup — hit our managed /v1/decide endpoint over HTTPS. Trades data residency for the fastest path to production.

For prototyping and low-sensitivity workloads. Book a call →

Stop parsing. Start deciding.

git clone it and run it yourself in minutes — or book a call if you'd rather we operate it for you.

git clone https://github.com/shyamsfo/bernoulli.git
Book a demo

See Bernoulli on your data.

Tell us a little about what you're deciding. We'll get back to you within a couple of business days.