uniq(1)
Filter or report on repeated lines in a file or input stream.
Synopsis
uniq [OPTION]... [INPUT [OUTPUT]]Description
Read lines from INPUT (default: stdin) and write to OUTPUT (default: stdout), either suppressing all but one copy of repeated consecutive lines, or counting and reporting occurrences. The input should typically be sorted first for meaningful results.
uniq compares adjacent lines only; non-consecutive duplicate lines are not treated as repeats. Use sort beforehand to group identical lines together.
Common options
| Flag | What it does |
|---|---|
-c | Prefix each line with the count of how many times it occurred |
-d | Print only duplicate lines (one line per group of duplicates) |
-u | Print only unique lines (lines that appear exactly once) |
-f N | Skip the first N fields when comparing lines |
-s N | Skip the first N characters when comparing lines |
-w N | Compare only the first N characters of each line |
-i | Ignore case differences when comparing lines |
-z | Use null character as line delimiter instead of newline |
Examples
Remove duplicate consecutive lines from a sorted file
sort data.txt | uniqCount and display each unique line with its occurrence count
sort data.txt | uniq -cShow only lines that appear more than once in the file
sort data.txt | uniq -dDisplay lines that appear exactly once (unique lines only)
sort data.txt | uniq -uCount occurrences and sort by frequency in descending order
sort data.txt | uniq -c | sort -rnSkip first 2 fields and compare remaining content for duplicates
uniq -f 2 data.txtRemove duplicates ignoring case differences
sort names.txt | uniq -i