Analysis results
Your document-length report appears here.
Interactive document-length histogram
Select a bar to inspect documents inside that bin.
Document details
Search, filter, sort, and inspect each record.
| ID | Group | Text preview | Words | Characters | Sentences | Tokens | Unique | Selected length | Bin | Status |
|---|
Document data
Formula used
How to use
- Paste documents or upload a supported data file.
- Select the document-length measurement and tokenisation method.
- Choose preprocessing, binning, grouping, and outlier options.
- Set a percentile or fixed sequence-length recommendation.
- Calculate, inspect bins, review diagnostics, and export results.
Example data format
| id | group | text |
|---|---|---|
| DOC-001 | News | Machine learning systems improve with representative data. |
| DOC-002 | Reviews | The delivery was fast and the setup was simple. |
| DOC-003 | Support | Please reset my password and resend verification. |
Frequently asked questions
What is a document-length histogram?
It groups documents into numeric length intervals. Each bar shows how many records fall inside one interval.
Which length unit should I use?
Use tokens for model inputs. Use words or characters for general corpus exploration.
Why compare groups?
Group comparisons reveal class-specific length bias. They can expose sampling or collection differences.
What does a long right tail mean?
Most documents are short, but a smaller number are much longer. Truncation decisions require special care.
Which bin rule is best?
Freedman–Diaconis is robust for many datasets. Manual bins remain useful for operational thresholds.
How is a sequence length recommended?
The selected percentile covers a chosen share of documents. Longer records are marked for truncation.
What are empty documents?
They contain no measurable content after processing. Review them before model training.
Does preprocessing change results?
Yes. Removing punctuation, numbers, or stop words changes several length measurements.
Is the token estimate exact?
No. Exact counts depend on a model-specific tokenizer. This page provides configurable approximations.
How are duplicates detected?
Documents with identical normalized text are counted as duplicates. Their records remain available for inspection.
Is uploaded text private?
Processing occurs in your browser. The page does not transmit uploaded content to this server.