CSV columns: message, intent, secondary, difficulty, route, urgency. Only message and intent are required.
Intent versus topic
An intent describes what a person wants to accomplish. A topic describes what the message discusses. “Where is my laptop order?” has a laptop topic but a delivery-tracking intent.
Single and multi-label classification
Single-label systems choose one primary intent. Multi-label systems can assign several valid intents. Routing usually depends on the primary requested action.
Confidence and thresholds
A confidence score estimates certainty, not truth. A threshold decides whether automation should act. Low-confidence cases should often be sent to a human.
False positives and false negatives
A false positive assigns an intent incorrectly. A false negative misses a real intent. Their business costs can differ significantly.
Class imbalance
Frequent intents can dominate training. Balanced examples, weighted metrics, and targeted data collection help protect smaller classes.
Ambiguity and unknown intents
Some messages contain several goals or fall outside known classes. An unknown label and human escalation reduce risky forced predictions.
Privacy and ethical use
Remove personal identifiers from learning data. Review labels for bias. Avoid automating high-impact decisions without suitable human oversight.
Scoring formula
Correct answers earn a base score. Confidence calibration, speed, streaks, secondary intent, and routing can add bonuses. Incorrect high-confidence answers receive larger penalties.