Shell command execution in UNIX-like operating systems

I have been using shell daily for almost two years now. I know some basic commands like cd, ls, and I can add a new PATH variable so that shell "understands" a new command...

But I have never truly understood shell's purpose and what happens when I add PATH=... into my ~/.zshrc file.

I spent some time researching and now I feel more confident when I type a command in a shell because I understand what happens in the background.

I decided to share my learnings to help other people demystify some of the concepts around shells.

Disclaimer

Most of the examples and references are about Z shell, but that's just because I use it as default :)

Prerequisite

Shell is an interface between a human and a computer. Back in the days when there was no GUI (graphical user interface) shell was a way for a user to tell the computer to do stuff, e.g. change a directory, create a file, etc.

It's important to understand that filesystems in UNIX-like operating systems such as macOS, Oracle and FreeBSD are different. For example, Windows OS uses the concept of internal and external drives, e.g. :C, :D , and we can choose where to store files or applications. In case we plug in an external device, e.g. a flash drive, we see a new drive appearing in the filesystem.

UNIX-like OS use different directories to store files and applications, but all of them are actually located under one root directory /, no matter internal or external. It can be visualized as a tree where each node is a directory and the root is the start of them all.

For example, if you open the Applications folder in shell and type pwd (print directory) you can see that its real location is /Users/yourusername/Applications

If you go outside the folders one by one you will end up in the / directory which is the root directory.

In the end, all your applications and binary files are actually stored next to each other, but with directories UNIX make it look as they were in separate locations.

Shell command execution

After writing a command in a shell and clicking the Enter key shell will parse the command to decide how to execute it. Shell has an operation sequence of what to do once you press the Enter key.

Commands like pwd, cd or which are not part of the operating system, they are native to shell. It means that each shell implements these commands and they can be found in that shell's source code. This also means that sometimes the same command for different shells, say zsh and bash can behave differently because of their unique implementation.

zsh: command not found:

Everybody has encountered command not found error at least once. This is because the shell doesn't know what to do with a command you typed. It will try to look up the command implementation and this is when it gets interesting.

To understand the logic behind the shells command lookup we need to learn about env command.

Shell envs

env command is a native shell command that shows the shared state for all processes that are started by this shell.

Open your shell and type env and you will see a list of environment variables. Something like this:

➜  zsh git:(master) env       
__CFBundleIdentifier=com.microsoft.VSCode
COMMAND_MODE=unix2003
PATH=/Users/msamadova/Desktop/work/scripts:/Users/msamadova/.nvm/versions/node/v12.16.1/bin:/Users/msamadova/.rbenv/shims
XPC_SERVICE_NAME=0
GIT_BINARY=/usr/bin/git
git_env_char=e
HOME=/Users/msamadova
SHELL=/bin/zsh
...

An environment variable is one of the ways to pass arguments to a process within a shell. If several shells are running on the computer, the scope of those environment variables is within the current shell. For example, if you change the env variable within a shell, the other opened shell won’t see the changed value, it will have the default one.

There are reserved env variables that have a special meaning. You can change it or add to it, but it should be done with caution!

One of them is PATH.

Shell uses PATH env variable to look up the path to external programs.

Type echo $PATH in your shell to see its current value. Most probably you will get a long string of directories with : sign between them. E.g.

➜  zsh git:(master) echo $PATH
/Users/msamadova/.nvm/versions/node/v12.16.1/bin:/Users/msamadova/.rbenv/shims:/usr/local/opt/mysql@5.7/bin:/Users/msamadova/bin:/usr/local/bin

These are locations where shell searches in when executing non-native to shell commands.

It splits this string by : , gets the array of strings, and iterates over all those locations till it finds the one that includes the implementation of your command.

If it doesn't find anything it will return the "beloved" command not found error. This is why you need to add newly installed command locations to PATH.

One of the ways to do so is to modify your shells rc file, which stands for (according to google :D) "run commands" because that's the main job of a shell.

When you open a new shell one of the things that it does before you will be able to type command is loading the rc file, in my case it's zshrc . It includes environment variables, aliases, plugins, etc. Depending on your setup, of course, it could be empty as well.

Be careful because

• export PATH= command will overwrite the path
• export PATH="/some_directory%PATH" will append the path

Use the second one if want your shell to have multiple locations to search for commands execution.

Conclusion

The purpose of this article was to give you a glimpse into the history of shell and to give more context about PATH variable - one of the most important in terms of shell.

There are so many things that you can learn about shells... ...and I am sure that you will have much more fun if you embark on that journey on your own.

I hope you learned something new today!

Have fun coding 😉