Training Data Builder Game

Mission Setup

Choose a task, difficulty, data source strategy, and operating constraints.

Dataset scenario

Difficulty

Game mode

Primary source

Collection budget

Mission time, seconds

Target dataset size

Random seed

Score

Budget

3,000

Time left

07:00

Collected

Label accuracy

—

Quality score

Mission progress0 / 60

Canvas Collection Pipeline

Select a card, inspect it, then collect, discard, review, or assign a label.

Shortcuts: C collect, D discard, R review

Selected Example

Select an incoming example on the canvas.

Assign label

Record label confidence

Confidence: 85%

Current Mission

Build a reliable starter dataset Collect 60 useful examples with balanced labels and fewer than three unresolved quality issues.

Diversity coverage0%

Privacy safety100%

Class balance0%

Annotation Guideline Builder

Label definitions

Boundary and ambiguity rules

Exclusion and escalation rules

Minimum confidence

Required reviewers

Dataset Records

Search, sort, relabel, flag, or remove collected examples.

	ID	Example	Label	Source	Confidence	Flags
No examples collected yet.

Quality Audit

Run an audit after collecting examples.

Privacy and Governance

Mask names and identifiers

Mask email addresses

Require consent or usage rights

Remove sensitive proxy fields

Retention policy

Feature and Target Review

Prediction target

Input features

Exclude record identifiers

Enable target leakage guard

Coverage and Fairness

Coverage dimension Minority coverage quota: 20%

Compare subgroup error rates

Request rare-case examples

Training, Validation, and Test Split

Configure a clean split while preventing duplicates, groups, and future information from leaking across sets.

Split method

Training: 70%

Validation: 15%

Keep test set locked

Prevent duplicate overlap

Training0Used to fit the model

Validation0Used for model selection

Test0Used once for final evaluation

Model Training Simulation

Train a simulated model and see how dataset quality affects generalisation.

Algorithm

Epochs

Dataset version

Version notes

Accuracy

—

Precision

—

Recall

—

F1 score

—

Generalisation gap

—

Subgroup reliability

—

Class Distribution

Dataset Growth and Quality

Label Confidence

Training Results

Game Event Log

Learning Summary

Representative data matters more than raw volume.
Ambiguous labels require written annotation rules.
Test data must remain independent and unseen.
Privacy, fairness, and lineage belong in dataset design.
Model errors reveal which examples to collect next.

Related Calculators