Write the sentiment question you actually need
A noul question answers a yes/no probability over two criteria you write, such as the writer is reporting something wrong with the product against they are not. A score question places the message on 2–7 ordered levels, such as positive, neutral or mixed, negative, and blocking. Both answers are positions on your own definition, which is why the same message can honestly answer two different questions.
Keep the answer domain small. Four levels a reviewer can argue about beat ten overlapping emotion names, and an explicit neutral or mixed level stops the model from inventing certainty the message does not have.
{
"text": "The dashboard is fast now, but the CSV export still drops the last column.",
"questions": [
{"id": "negative", "type": "noul", "instructions": "Is the writer reporting something wrong with the product?", "criteria": {"true": "They describe a fault or a failure.", "false": "They only praise, ask or request."}},
{"id": "tone", "type": "score", "instructions": "How negative is the overall message?", "criteria": ["Positive", "Neutral or mixed", "Negative", "Blocking"]}
],
"threshold": 0.85
}Mixed, sarcastic and quoted text has no single answer
A message can praise one part of the product and report a defect in another. With no level for that, the model still has to choose one, and the answer looks more decided than the message is. Give mixed and unclear their own level, or ask a second noul question about the defect on its own.
Negation, sarcasm, a complaint quoted from someone else and text written in another language all move the probability without changing the words you keyed on. A threshold routes uncertain answers to a person: for a noul answer it applies to the probability itself, while a choice or a score is measured against confidence — and a confidence the model omits is null, which always needs review.
Choose confidence thresholds from evidence · Human-in-the-loop decisions
Cost, storage and honest limits
One successful request costs one credit whether it asks one question or four, and a failed account request is refunded. The decision history keeps the answer and its identifier, not the message you submitted, so hold the source text in your own system when a reviewer has to read it.
This is not a legal finding, a safety verdict or a clinical measure: it is one model answer under one rubric. A high probability is the model’s own number, not an accuracy measured on your traffic. Evaluate level by level on labeled examples from your own data, count errors in both directions, and keep a person responsible for whatever you act on.
Evaluate classification before automating work · Request limits and credit rules