Bash sort command

With sort, you can order files based on the order in the dictionary or by numerical value, randomize file lines, remove duplicate lines, and check if a file is sorted.

You may be able to do other things with it but first, let’s worry about wrapping our heads around how to use sort in bash scripts.

What is sort?

Sort is an external command that concatenates files while sorting their contents according to a sort type and writes results of sort to standard output.

Sort command options for bash

The sort command comes with 31 options (13 main and 18 categorized as other). Most experienced bash programming (even experts) know only a few main sort options required to get by. Others are seldom touched. Lucky for you we have time to touch them all.

Main sort options

These are the options that help you get things done and sort (Sorting) in addition to manipulate sorted results (Post processing) and apply filters (Filters) prior to sorting.

Sorting

Sort comes with 5 different types of sorting. Here is A table showing each sort type with associated options.

Note that each type of sort has a long option ending with -sort. In addition to specific sort options, the –sort=WORD option may be used to sort by word. For example –sort=random may be used in place of –random-sort or -r.

Examples

Here are some sort command examples for each sorting method.

Example) Sorting names

Sort has no issues sorting lines alphabetically. Consider a list of famous people not sorted.

Function

Command line

Output

Example) General numeric sort

If we need to sort numeric values taking into fact scientific notation such as 99e2, we can use general numeric sort.

Function

Consider the sorted output using each method. Note that in addition to containing values 1 through 100, the list also includes ‘9e12′ (900) and ’99K’ (99000).

Command line

Output

What about 900 and 99000. That’s right it’s just numeric sort. Next.

Command line

Output

What about 900. That’s right it’s just human numeric sort. Next.

Command line

Output

What about 99000. That’s right it’s just general numeric sort. As you see no sorting method is compatible in this case; however, that doesn’t mean you can’t come up with a fix.

Command line

Output

Now that’s more like it.

Example) Human numeric sort

If we need to sort numerical values taking into fact the meaning of notations such as K, G, M, and E, we can use human numeric sort.

Command line

Output

Example) Numeric sort

If all we need is to sort integers numeric sort does the trick.

Command line

Output

Example) Month sort

Month sort allows you to order lines by month. It could prove useful for grouping lines together by month especially in the case that the option of sorting by time is not available.

Function

Suppose that are months are not sorted.

Command line

Output

We can always sort by month.

Command line

Output

Note that if we change Dec to any substring in November say ‘Novem’, it will appear after ‘Nov’ in the sorted output.

Example) Random sort – kill someone else’s terminal

As expected, random sort does the opposite of sorting, mixes up lines.

Suppose that for education purposes we want to kill another user. We would have to make sure it is not our pty and randomize the listings so that it is nicer and that we can say that ptys were selected at random.

Commands

Example) Version sort – sorting ips

As you know source files may be versioned using strings such as 1.0. Furthermore, versions may go deeper with version numbers like 1.0.0 such as seen in popular semantic version schemes.

Version sort allows you to sort version numbers. Great! Now what? Let’s test it out.

For this example, I’ve prepared a bash script to generate random ips so that we don’t have to go there. It’s in the repo. For those of us that don’t have the repo here’s a quick start.

Commands

Now that you are ready let’s get started.

Command line

Output

Okay, it works. Now let’s see what happens when we try and sort ips.

Command line

Output

At first glance, it appears to work but lines like 8.96.11.181 should appear elsewhere.

Commands

Output

As you see version sort allows you to sort version numbers when other sorting methods fail.

Example) Version sort – sorting file names with version numbers

Building on the last example, let’s use version sort a little closer to its intended use. As you know, version numbers commonly appear in filenames. See Details about version sort.

First, let’s transform ips into something else more project source file like.

Commands

Output

Exercise

Make the above commands run faster using xargs

See example in how to use xargs command in bash scripts.

This time, we won’t even bother using any of the other sorting methods.

Command line

Output

Now you see that version sort may be useful when sorting file names with version numbers.

Pre sort

Sort has four main options that affect actual sorting namely, –ignore-leading-blanks, –ignore-case, –ignore-nonprinting, and –dictionary-order, that may or may not overlap. Example using each option follow.

Sort ignoring leading blanks

Sort allows input leading blanks to be ignored as an option. Leading blanks are preserved in the sorted output.

Option

Usage

Commands

Output

Note that leading spaces in lines added to fp appear first in sort output.

To fix this we need to ignore leading blanks as follows.

Commands

Output

Alternatives

Note that the alternative does not preserve leading blanks in sort output.

Sort ignoring case

Sort allows input case to be ignored as an option. The case is preserved in the sorted output.

Option

Usage

Commands

Output

Note that leading spaces in lines added to fp appear first in sort output.

To fix this we need to ignore leading blanks as follows.

Commands

Output

Alternatives

Note that the alternative does not preserve case in sort output.

Sort ignoring nonprinting

Sort allows input nonprinting to be ignored as an option. Nonprinting is preserved in the sorted output.

Option

Usage

Commands

Output

Looks like we are missing an ‘Abe’ do to non-printing characters in sort input.

To fix this we need to ignore non-printing characters.

Commands

Sort dictionary order

Sort allows all input to be ignored except spaces and alphanumeric characters as an option. Input is preserved in the sorted output.

Post sort

Sort has one main option that doesn’t affect sorting namely, –reverse. However, it affects the output, allowing order to be toggled between ascending and descending. An example follows.

Sort reverse output

Sort allows output to be displayed in reverse order as an option.

Option

Usage

Command line

Output

Alternatives

Other options for sort

There are twenty-two other options for sort. Examples follow.

Sort check

Sort has an option that allows you to check if the input is sorted. It returns after the first instance of an unsorted line. It the case that input is required to be sorted but is likely already in order, using sort check is appropriate.

Option

Usage

Command line

Output

Command line

Output

Sort output

Sort has an option that allows you to specify a file to write to instead of using standard output or redirection. Its use may improve compatibility across scripting environments.

Option

Usage

Command line

Output

Sort null terminated

Sort has an option that allows you to set the line delimiter to null instead of a newline.

Option

Usage

Command line

Output

Sort stable

Sort has an option that allows you to disable last-resort comparison. As a result, more stable runtimes may be achieved in the case of large enough inputs that could cause sort to run unstable.

Option

Usage

Command line

Output

Sort buffer size

Sort has an option that allows you to set the amount of memory used as a buffer while sorting. It can be used to limit memory consumption sorting larger inputs. Performance may be affected.

Option

Usage

Command line

time seq 1000000 | sort –random-sort | sort –stable –buffer-size=64 >/dev/null

Output

Sort unique

Sort has an option that allows you to remove duplicate lines in sort output

Option

Usage

Command line

echo 1 2 2 4 5 | tr ’40’ ’00’ | sort –zero-terminated –unique

Output

1245

Alternatives

sort | uniq

Conclusion

Sort is an external command useful not only when used in combination with other external commands but also comes in handy when used with commands with no built-in ordering method such as a user-defined function or bash scripts in general.