Skip to content

Remove duplicate lines

Clean up a list: remove repeats, sort it properly, strip blank lines. It tells you which lines were duplicated and how often, rather than silently shrinking your list.

  • Processed on your device
  • No upload, no waiting
  • No signup, no watermark
Sort
Tidy
Result

How it works

1

Paste your list

Emails, URLs, product codes, anything one per line. Or drop a text file in.

2

Choose what "the same" means

Ignore trailing spaces, ignore capitalisation, keep the first or the last copy — or remove every line that had a duplicate at all.

3

Sort and tidy

Sort naturally, reverse, shuffle, number the lines or strip blanks — then copy the result.

Frequently asked questions

Is my list uploaded?

No. Everything happens inside this page, on your own device — nothing is uploaded and no request is made. You can disconnect from the internet after the page loads and it keeps working.

Why are there still duplicates after removing them?

Almost always trailing whitespace. Two lines that end with a different number of spaces look identical on screen and are not, so an exact comparison keeps both — and you conclude the tool is broken. Trimming before comparing is switched ON here by default for that reason. Importantly it only affects the COMPARISON: the line that survives keeps its original text, because quietly editing your lines is not what "remove duplicates" means.

Which copy is kept?

The first, by default, and the order of the rest is untouched — that matters when a list is already in a meaningful order. You can keep the last instead, or remove every line that had a duplicate, leaving only the lines that appeared exactly once. That last one is how you find the entries unique to a list.

Why does sorting put item10 before item2?

That is ordinary alphabetical sorting, where "1" comes before "2" and the rest of the number is never reached. Switch on natural sorting and digit runs are compared as numbers, so item2 comes before item10 — the order a person expects. Accented text is handled properly too: a plain comparison sorts "Zürich" after "Zyzzyva", because it compares the underlying numbers rather than the letters.

Does it tell me what was removed?

Yes — the lines that appeared more than once are listed with their counts, commonest first. Knowing WHICH entry appeared four times is usually more useful than knowing that three lines went away.

Is the shuffle actually random?

Yes. It uses a Fisher-Yates shuffle with the browser's cryptographic random source. The common shortcut — sorting with a random comparison — is not a shuffle at all: sorting algorithms assume a consistent comparison, and with a random one the result is measurably biased towards the original order. If you are drawing a winner from a list, that difference matters.

Is there a size limit?

Your device's memory rather than an upload cap. Lists of hundreds of thousands of lines are fine.