uniq command is used to display or ignore duplicate lines in a file. It is a Linux command that is useful for filtering out duplicate lines from a file, especially when working with large datasets. The command is simple to use and can be used in combination with other commands to perform complex operations.
The basic syntax of the
uniq command is as follows:
uniq [option(s)] [input_file [output_file]]
uniq command reads input from a file or standard input, removes duplicate lines, and writes the output to standard output or a specified file.
uniq only removes adjacent duplicate lines. This means that if a file has duplicate lines that are not adjacent, they will not be removed. However, this behavior can be changed using some options.
- Display unique lines in a file:
- Ignore case while comparing lines:
uniq -i file.txt
- Display only duplicate lines:
uniq -d file.txt
- Count the number of occurrences of each unique line:
uniq -c file.txt
- Ignore the first ‘n’ characters while comparing lines:
uniq -s n file.txt
- Ignore the first ‘n’ fields while comparing lines:
uniq -f n file.txt
The following table lists the available options for the
||Prefix lines with the number of occurrences|
||Only print duplicate lines|
||Ignore the first ‘n’ fields while comparing lines|
||Ignore case while comparing lines|
||Ignore the first ‘n’ characters while comparing lines|
||Only print unique lines|
||Use null characters instead of newlines as separators|
- If you get an error message that says “No such file or directory”, make sure that the file you are trying to read exists in the specified location.
- If you are not getting the expected output, try using some of the available options to modify the behavior of the command.
uniqcommand only removes adjacent duplicate lines by default.
uniqcommand is case-sensitive by default.
uniqcommand can be used in combination with other commands to perform complex operations.