This “pipe” command is readily available on UNIX/Linux platforms. This command pipes the output of the previous command to the next command. There are literally TONS of situations where this method offers serious value.Before jumping deeper, there’s something to know of. Every single program in the UNIX/Linux system has 3 built-in data streams.

STDIN (0) – Standard input
STDOUT (1) – Standard output
STDERR (2) – Standard error

When we’re going to work with “pipe” tricks, “pipe” will take the STDOUT of a command and pass it to the STDIN of the next command.

Let’s check out some of the most common ways you can incorporate the “pipe” command into your daily usage.

Pipe command

Basic usage

It’s better to elaborate on the working method of “pipe” with a live example, right? Let’s get started. The following command will tell “pacman”, the default package manager for Arch and all the Arch-based distros, to print out all the installed packages on the system.

pacman -Qqe

It’s a really LONG list of packages. How, about picking up only a few components? We could use “grep”. But how? One way would be dumping the output to a temporary file, “grep” the desired output and delete the file. This series of tasks, by itself, can be turned into a script. But we only script for very large things. For this task, let’s call upon the power of “pipe”!

pacman -Qqe | grep <target>

Awesome, isn’t it? The “|” sign is the call to the “pipe” command. It grabs the STDOUT from the left section and feeds it into the STDIN of the right section.

In the aforementioned example, the “pipe” command actually passed the output at the end of the “grep” part. Here’s how it plays out.

pacman -Qqe > ~/Desktop/pacman_package.txt
grep python ~/Desktop/pacman_package.txt

Multiple piping

Basically, there’s nothing special with the advanced usage of the “pipe” command. It’s completely up to you on how to use it.

For example, let’s start by stacking multiple piping.

pacman -Qqe | grep p | grep t | grep py

The pacman command output is filtered further and further by “grep” through a series of piping.

Sometimes, when we’re working with the content of a file, it can be really, really large. Finding out the right place of our desired entry can be difficult. Let’s search for all the entries that include digits 1 and 2.

cat demo.txt | grep -n 1 | grep -n 2

Manipulating list of files and directories

What to do when you’re dealing with a directory with TONS of files in it? It’s pretty annoying to scroll through the entire list. Sure, why not make it more bearable with pipe? In this example, let’s check out the list of all the files in the “/usr/bin” folder.

ls -l <target_dir> | more

Here, “ls” prints all the files and their info. Then, “pipe” passes it to “more” to work with that. If you didn’t know, “more” is a tool that turns texts into one screenful view at a time. However, it’s an old tool and according to the official documentation, “less” is more recommended.

ls -l /usr/bin | less

Sorting output

There’s a built-in tool “sort” that will take text input and sort them out. This tool is a real gem if you’re working with something really messy. For example, I got this file full of random strings.

cat demo.txt

Just pipe it to “sort”.

cat demo.txt | sort

That’s better!

Printing matches of a particular pattern

ls -l | find ./ -type f -name "*.txt" -exec grep 00110011 {} ;

This is a pretty twisted command, right? At first, “ls” outputs the list of all files in the directory. The “find” tool takes the output, searches for “.txt” files and summons “grep” to search for “00110011”. This command will check every single text file in the directory with the TXT extension and look for the matches.

Print file content of a particular range

When you’re working with a big file, it’s common to have the need of checking the content of a certain range. We can do just that with a clever combo of “cat”, “head”, “tail” and of course, “pipe”. The “head” tool outputs the first part of a content and “tail” outputs the last part.

cat <file> | head -6

cat <file> | tail -6

Unique values

When working with duplicate outputs, it can be pretty annoying. Sometimes, duplicate input can cause serious issues. In this example, let’s cast “uniq” on a stream of text and save it into a separate file.

For example, here’s a text file containing a big list of numbers that are 2 digits long. There are definitely duplicate contents here, right?

cat duplicate.txt | sort

Now, let’s perform the filtering process.

cat duplicate.txt | sort | uniq > unique.txt

Check out the output.

bat unique.txt

Looks better!

Error pipes

This is an interesting piping method. This method is used to redirect the STDERR to STDOUT and proceed with the piping. This is denoted by “|&” symbol (without the quotes). For example, let’s create an error and send the output to some other tool. In this example, I just typed some random command and passed the error to “grep”.

adsfds |& grep n

Final thoughts

While “pipe” itself is pretty simplistic in nature, the way it works offers very versatile way of utilizing the method in infinite ways. If you’re into Bash scripting, then it’s way more useful. Sometimes, you can just do crazy things outright! Learn more about Bash scripting.

Linux Commands