Cold Boot

Your First Shell Script

2025-01-26T00:00:00+00:00

You have been typing commands one at a time into the shell. That is fine for quick tasks, but when you need to run the same sequence of commands regularly -- or when the logic gets complex -- you write a shell script: a text file containing commands that the shell executes in order.</p>

A shell script is not a compiled program. It is not written in a special language. It is the same commands you have been typing interactively, saved in a file. The shell reads the file line by line and executes each command exactly as if you had typed it.</p>

The Shebang Line</h2>

Open your text editor and create a file called hello.sh</code>:</p>

#!/bin/bash
echo "Hello from a shell script"
</code></pre>
The first line -- #!/bin/bash</code> -- is called the shebang</strong> (or hashbang). It tells the operating system which program should interpret this file. When you execute the script, the kernel reads these two bytes (#!</code>) at the start, sees the path /bin/bash</code>, and launches bash with the script file as input.</p>

Key term: Shebang</strong>
The character sequence #! at the very beginning of a script file, followed by the path to the interpreter program. When the kernel sees #! at the start of an executable file, it runs the specified interpreter and passes the script as its argument. The name comes from "hash" (#) and "bang" (!).
</div>
The shebang must be the very first line. No blank lines before it. No spaces before the #</code>. If it is missing or wrong, the system may try to interpret the file with the wrong shell, or fail to run it at all.</p>
Common shebangs:</p>

#!/bin/bash</code> -- use bash specifically</li>
#!/bin/sh</code> -- use the system's POSIX shell (often dash)</li>
#!/usr/bin/env python3</code> -- use whatever python3 is in PATH</li>
</ul>
The #!/usr/bin/env</code> form is useful for portability because it searches PATH rather than hardcoding the interpreter's location.</p>


Fig. 22.0 -- How the kernel processes the shebang</div>
$ ./hello.sh</text></p>
  
  
  
  Kernel reads first 2 bytes</text>
  #!</text>
  
  
  
  Reads interpreter path: /bin/bash</text>
  from the rest of line 1</text>
  
  
  
  Executes: /bin/bash ./hello.sh</text>
  bash reads the file and runs each line</text>
  
bash ignores the #! line because # is a comment character in shell syntax.</text></p>
  
    
      
    </marker>
  </defs>
</svg>
When you run a script, the kernel inspects the first two bytes. If they are #!, it reads the rest of line 1 as the path to the interpreter. It then launches that interpreter with the script file as an argument. The interpreter reads the file from the beginning, but since # starts a comment in most shells, the shebang line is harmlessly skipped.</div>
</div>
</figure>
Making a Script Executable</h2>
Before you can run ./hello.sh</code>, you need to give the file execute permission:</p>
$ chmod +x hello.sh
$ ./hello.sh
Hello from a shell script
</code></pre>
The chmod +x</code> command sets the executable bit on the file. Without it, the kernel will refuse to run the file even though it contains valid commands. You can also run it explicitly through the interpreter:</p>
$ bash hello.sh
Hello from a shell script
</code></pre>
This does not require the execute bit because you are running bash</code> (which is already executable) and passing the script as an argument. But the ./</code> form is cleaner and more conventional for scripts you intend to reuse.</p>
Variables in Scripts</h2>
Shell variables work the same in scripts as they do interactively. Assign with =</code> (no spaces around the equals sign), and reference with $</code>:</p>
#!/bin/bash
name="Cold Boot"
count=22
echo "This is article $count of the $name series"
</code></pre>
You can capture the output of a command into a variable using command substitution:</p>
#!/bin/bash
today=$(date +%Y-%m-%d)
file_count=$(ls /usr/bin | wc -l)
echo "Today is $today"
echo "There are $file_count programs in /usr/bin"
</code></pre>
The $(...)</code> syntax runs the command inside and replaces itself with the command's output. The older backtick syntax `command`</code> does the same thing but is harder to read and cannot be nested.</p>

Key term: Command substitution</strong>
The syntax $(command) runs the enclosed command and replaces itself with the command's standard output. This lets you capture the result of a command into a variable or embed it in a string. It is one of the most commonly used shell features.
</div>
Quoting Rules</h3>
Quoting is a source of endless bugs in shell scripts. Here are the rules:</p>
Double quotes</strong> ("..."</code>) preserve the string as a single token but allow variable expansion:</p>
$ name="Cold Boot"
$ echo "Welcome to $name"
Welcome to Cold Boot
</code></pre>
Single quotes</strong> ('...'</code>) preserve the string literally. No variable expansion, no special characters:</p>
$ echo 'The price is $5.00'
The price is $5.00
</code></pre>
No quotes</strong> cause the shell to split the value on whitespace and expand wildcards. This is almost never what you want:</p>
$ files="one two three"
$ echo $files       # three separate arguments to echo
one two three
$ echo "$files"     # one argument containing spaces
one two three
</code></pre>
The practical rule: always double-quote your variables unless you have a specific reason not to. "$variable"</code> is safe. $variable</code> is a bug waiting to happen.</p>

Always double-quote variables in shell scripts: "$variable", not $variable. Unquoted variables are subject to word splitting and glob expansion, which causes subtle, hard-to-debug failures when values contain spaces or special characters.
</div>
Conditionals: if Statements</h2>
Shell scripts can make decisions. The if</code> statement tests a condition and runs different commands based on the result:</p>
#!/bin/bash
if [ -f /etc/hostname ]; then
    echo "Hostname file exists"
    cat /etc/hostname
else
    echo "No hostname file found"
fi
</code></pre>
The [ ... ]</code> is actually a command (it is an alias for the test</code> command). It evaluates the expression inside and exits with status 0 (true) or 1 (false). The if</code> statement checks the exit status.</p>
Common test expressions:</p>

[ -f file ]</code> -- true if file exists and is a regular file</li>
[ -d dir ]</code> -- true if directory exists</li>
[ -z "$var" ]</code> -- true if variable is empty or unset</li>
[ -n "$var" ]</code> -- true if variable is non-empty</li>
[ "$a" = "$b" ]</code> -- true if strings are equal</li>
[ "$a" != "$b" ]</code> -- true if strings differ</li>
[ "$x" -eq "$y" ]</code> -- true if integers are equal</li>
[ "$x" -gt "$y" ]</code> -- true if x is greater than y</li>
</ul>


Fig. 22.1 -- How if/then/else executes</div>
The if statement runs the test command (inside the brackets). If the test exits with status 0, the "then" block runs. If it exits with any other status, the "else" block runs. Both paths converge at "fi".</div>
</div>
</figure>
Exit Codes</h2>
Every command returns an exit code</strong> when it finishes -- a number between 0 and 255. By convention:</p>

0</strong> means success</li>
Anything else</strong> means failure</li>
</ul>
You can check the most recent exit code with $?</code>:</p>
$ ls /tmp
(file listing appears)
$ echo $?
0
$ ls /nonexistent
ls: cannot access '/nonexistent': No such file or directory
$ echo $?
2
</code></pre>
In your own scripts, you set the exit code with the exit</code> command:</p>
#!/bin/bash
if [ ! -f "$1" ]; then
    echo "Error: file $1 not found" >&2
    exit 1
fi
echo "Processing $1..."
exit 0
</code></pre>
Notice >&2</code> on the error message -- that sends the text to stderr, which is the correct channel for error output. The $1</code> is a special variable that holds the first command-line argument passed to the script.</p>

Key term: Exit code</strong>
A number from 0 to 255 returned by every process when it terminates. Zero means success; any non-zero value means failure. The shell stores the most recent exit code in the special variable $?. Exit codes are how if statements, && chains, and || chains make decisions.
</div>
Loops</h2>
The for Loop</h3>
The for</code> loop iterates over a list of items:</p>
#!/bin/bash
for fruit in apple banana cherry; do
    echo "I like $fruit"
done
</code></pre>
Output:</p>
I like apple
I like banana
I like cherry
</code></pre>
You can loop over files:</p>
#!/bin/bash
for file in /etc/*.conf; do
    echo "Config file: $file"
done
</code></pre>
Or over command output:</p>
#!/bin/bash
for user in $(cat /etc/passwd | cut -d: -f1); do
    echo "User: $user"
done
</code></pre>
The while Loop</h3>
The while</code> loop runs as long as a condition is true:</p>
#!/bin/bash
count=1
while [ "$count" -le 5 ]; do
    echo "Count: $count"
    count=$((count + 1))
done
</code></pre>
The $((...))</code> syntax performs arithmetic. Without it, the shell treats everything as strings.</p>
A common pattern is reading a file line by line:</p>
#!/bin/bash
while read -r line; do
    echo "Line: $line"
done < /etc/hostname
</code></pre>
The < /etc/hostname</code> at the end redirects the file into the while loop's stdin. The read -r</code> command reads one line at a time into the variable line</code>.</p>


Fig. 22.2 -- for loop vs. while loop execution flow</div>
for loop</text></p>
  
  list: [a, b, c]</text>
  
  
  next item?</text>
  
  
  yes</text>
  
  item=a</text>
  run body</text>
  
  
  
  
  
  
  
  no</text>
  
  done</text>
  
while loop</text></p>
  
  initial: count=1</text>
  
  
  count -le 5 ?</text>
  
  
  yes</text>
  
  echo, count++</text>
  run body</text>
  
  
  
  
  
  
  
  no</text>
  
  done</text>
  
Iterates over a fixed list</text>
Repeats while condition is true</text></p>
  
    
      
    </marker>
    
      
    </marker>
  </defs>
</svg>
A for loop steps through a fixed list of items, running the body once per item. A while loop re-tests its condition before each iteration and stops when the condition becomes false.</div>
</div>
</figure>
Putting It All Together</h2>
Here is a complete, practical shell script that combines everything from this article and the previous ones in the series. It checks disk usage on a set of directories and warns if any exceed a threshold:</p>
#!/bin/bash
# disk-check.sh -- warn about directories using too much space

THRESHOLD=80   # percent
DIRS="/home /var /tmp"

echo "Disk usage check -- $(date)"
echo "Threshold: ${THRESHOLD}%"
echo "---"

warnings=0

for dir in $DIRS; do
    if [ ! -d "$dir" ]; then
        echo "SKIP: $dir does not exist" >&2
        continue
    fi

    usage=$(df "$dir" | tail -1 | awk '{print $5}' | tr -d '%')

    if [ "$usage" -gt "$THRESHOLD" ]; then
        echo "WARNING: $dir is at ${usage}% (exceeds ${THRESHOLD}%)"
        warnings=$((warnings + 1))
    else
        echo "OK: $dir is at ${usage}%"
    fi
done

echo "---"
if [ "$warnings" -gt 0 ]; then
    echo "$warnings warning(s) found"
    exit 1
else
    echo "All directories within limits"
    exit 0
fi
</code></pre>
This script uses:</p>

A shebang (#!/bin/bash</code>)</li>
Variables (THRESHOLD</code>, DIRS</code>, warnings</code>, usage</code>)</li>
Command substitution ($(date)</code>, $(df ... | awk ...)</code>)</li>
A for loop</li>
Conditionals with if/else</code></li>
File tests ([ ! -d "$dir" ]</code>)</li>
Integer comparison ([ "$usage" -gt "$THRESHOLD" ]</code>)</li>
Stderr for errors (>&2</code>)</li>
Arithmetic ($((warnings + 1))</code>)</li>
Exit codes (0 for success, 1 for warnings)</li>
A pipeline (df | tail | awk | tr</code>)</li>
</ul>


Fig. 22.3 -- Data flow through the disk-check script</div>
The script loops over directories, runs a pipeline to extract usage percentages, compares each against a threshold, and exits with an appropriate status code. Every concept from the last five articles appears in this one script.</div>
</div>
</figure>
Script Arguments</h2>
Scripts can accept arguments from the command line. They are available as special variables:</p>

$0</code> -- the script's own name</li>
$1</code>, $2</code>, $3</code>, ... -- positional arguments</li>
$#</code> -- the number of arguments</li>
$@</code> -- all arguments as separate words</li>
</ul>
#!/bin/bash
echo "Script: $0"
echo "First argument: $1"
echo "Second argument: $2"
echo "Total arguments: $#"
</code></pre>
$ ./args.sh hello world
Script: ./args.sh
First argument: hello
Second argument: world
Total arguments: 2
</code></pre>
A well-behaved script checks that it received the right number of arguments:</p>
#!/bin/bash
if [ $# -lt 1 ]; then
    echo "Usage: $0 <filename>" >&2
    exit 1
fi
</code></pre>
Debugging Scripts</h2>
When a script does not work, add set -x</code> near the top. This makes bash print every command before it runs, showing you exactly what is happening:</p>
#!/bin/bash
set -x
name="world"
echo "Hello, $name"
</code></pre>
Output:</p>
+ name=world
+ echo 'Hello, world'
Hello, world
</code></pre>
Each line prefixed with +</code> is bash showing you the command after variable expansion but before execution. This is invaluable for finding quoting bugs and logic errors.</p>
Another useful setting is set -e</code>, which makes the script stop immediately if any command fails (returns a non-zero exit code). Combined with set -u</code> (treat unset variables as errors), these three settings catch most common scripting mistakes:</p>
#!/bin/bash
set -euo pipefail
</code></pre>
The pipefail</code> option makes a pipeline return the exit code of the last failing command, rather than the last command overall.</p>

Start your scripts with `set -euo pipefail` until you have a reason not to. These settings catch unset variables, failed commands, and broken pipes -- the three most common sources of silent script failures.
</div>
What You Have Learned</h2>
A shell script is a text file of commands with a shebang line that tells the kernel which interpreter to use. Variables hold values, and command substitution captures program output. Conditionals use [ ... ]</code> (the test command) and check exit codes. Loops iterate over lists (for) or repeat while a condition holds (while). Exit codes communicate success (0) or failure (non-zero) to the calling program. Script arguments arrive in $1</code>, $2</code>, and so on.</p>

This is the final article in the Cold Boot series. You have traveled from the first pulse of electricity through voltage rails and reset vectors, past the BIOS and bootloader, into the kernel and init system, through process management and permissions, and now into the shell where you write your own commands.</p>
The journey from power-on to a running shell script touches every layer of a computer. Electricity becomes bits. Bits become instructions. Instructions become firmware. Firmware finds a bootloader. The bootloader loads a kernel. The kernel starts init. Init launches services. A terminal opens. A shell starts its read-execute loop. And now you can write a script that orchestrates all of it.</p>
Every command you type from here forward sits on top of the entire stack you have just learned. You are not a passive user anymore. You understand what happens underneath.</p>
Back to the Cold Boot series index</a></p>

The Environment 2025-01-25T00:00:00+00:00 Every process on a Unix system carries a bag of named values. These values -- called environment variables -- tell programs where to find things, how to behave, and who is running them. They are the primary mechanism for passing configuration from a parent process to its children without modifying any files or passing command-line arguments.</p> You have already seen one: PATH, the list of directories the shell searches for programs. Now we will look at the full picture.</p> What an Environment Variable Is</h2> Think of an environment variable as a sticky note attached to a process. It has a name and a value, both plain text. The name is conventionally uppercase. The value can be anything: a directory path, a number, a comma-separated list, an empty string.</p> $ echo $HOME /home/user $ echo $USER user $ echo $SHELL /bin/bash </code></pre> The $</code> prefix tells the shell to substitute the variable's value in place of its name. Without the $</code>, the shell treats the text as a literal string:</p> $ echo HOME HOME $ echo $HOME /home/user </code></pre> Key term: Environment variable</strong> A named key-value pair carried by every process. Environment variables are inherited by child processes and are the standard Unix mechanism for passing configuration without using files or command-line flags. By convention, names are uppercase with underscores. </div> You can list every environment variable currently set in your shell:</p> $ env HOME=/home/user PATH=/usr/local/bin:/usr/bin:/bin SHELL=/bin/bash USER=user TERM=xterm-256color LANG=en_US.UTF-8 ... </code></pre> There may be dozens or even hundreds. Each one was set by something during your login process or shell startup.</p> Fig. 21.0 -- The environment as a key-value store</div> Process Environment Block</text></p> NAME</text> VALUE</text> HOME</text> /home/user</text> PATH</text> /usr/local/bin:/usr/bin:/bin</text> USER</text> user</text> SHELL</text> /bin/bash</text> LANG</text> en_US.UTF-8</text> TERM</text> xterm-256color</text> ... and dozens more</text></p> Every process gets its own copy of this table at birth.</text> </svg></p> The environment is a table of name-value pairs. Each process gets its own copy when it is created. Changes to one process's environment do not affect any other process.</div> </div> </figure> Shell Variables vs. Environment Variables</h2> The shell actually maintains two kinds of variables. Understanding the difference is important.</p> A shell variable</strong> exists only inside the shell process. It is not passed to child processes. You create one by simple assignment:</p> $ color=blue $ echo $color blue </code></pre> An environment variable</strong> is a shell variable that has been marked for export. It is copied into the environment of every child process the shell starts. You mark a variable for export with the export</code> command:</p> $ export color=blue </code></pre> Or export an existing variable:</p> $ color=blue $ export color </code></pre> The difference matters when you run programs. A program started by the shell only sees exported variables:</p> $ secret=hidden $ export visible=shown $ bash -c 'echo "secret=$secret visible=$visible"' secret= visible=shown </code></pre> The inner bash (a child process) can see visible</code> because it was exported. It cannot see secret</code> because it was a plain shell variable.</p> Key term: export</strong> A shell built-in command that marks a variable for inclusion in the environment of child processes. Without export, a variable exists only within the current shell. With export, it is inherited by every program the shell launches. </div> Fig. 21.1 -- Shell variables vs. environment variables</div> Shell variables:</text> secret=hidden</text> temp=working</text></p> Environment variables:</text> PATH=/usr/bin:...</text> visible=shown</text></p> fork+exec</text> Child process (ls)</text> Inherited environment:</text> PATH=/usr/bin:...</text> visible=shown</text></p> NOT inherited:</text> secret</text> (not exported)</text> temp</text> (not exported)</text></p> Exported (inherited by children)</text> Not exported (shell-only, invisible to children)</text> The child gets a COPY. Changes in the child do not affect the parent.</text></p> </marker> </defs> </svg> When the shell forks a child process, only exported variables are copied into the child's environment. Plain shell variables stay behind in the parent. The child gets a copy, not a reference -- changes in the child never propagate back to the parent.</div> </div> </figure> Inheritance flows one way: from parent to child. A child process can never modify its parent's environment. If you set a variable inside a script, the calling shell will not see the change after the script exits. This is a common source of confusion for beginners. </div> The Essential Variables</h2> Some environment variables are so common that nearly every program recognizes them. Here are the ones you need to know:</p> PATH</h3> The most important environment variable. It contains a colon-separated list of directories that the shell searches when you type a command name. We covered this in Article 18.</p> $ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin </code></pre> To add a directory, you append to PATH:</p> $ export PATH="$PATH:/home/user/bin" </code></pre> The $PATH</code> expands to the current value, and you concatenate your new directory after a colon. Put your addition at the end so that system programs are found first, or at the beginning if you want your version to take priority.</p> HOME</h3> The path to the current user's home directory. Programs use it to find configuration files. The ~</code> shortcut in the shell expands to this value.</p> $ echo $HOME /home/user $ echo ~ /home/user </code></pre> USER</h3> The name of the currently logged-in user. Programs use it for logging, file ownership, and access control decisions.</p> SHELL</h3> The path to the user's default login shell, as set in /etc/passwd</code>. Note that this is the default shell, not necessarily the shell currently running. If you start zsh from inside bash, $SHELL</code> still says /bin/bash</code>.</p> EDITOR and VISUAL</h3> The user's preferred text editor. Many programs (like git commit</code> with no -m</code> flag) consult these variables to decide which editor to open.</p> $ export EDITOR=vim </code></pre> LANG and LC_*</h3> Language and locale settings. They control how programs display dates, numbers, currency, and sort text. LANG=en_US.UTF-8</code> means English, United States, using the UTF-8 character encoding.</p> TERM</h3> The type of terminal emulator. Programs that draw to the screen (like vim</code> or top</code>) read this to know which escape codes to use for colors, cursor movement, and screen clearing.</p> Setting Variables for a Single Command</h2> You can set an environment variable for just one command without affecting your shell session:</p> $ LANG=C sort data.txt </code></pre> The LANG=C</code> assignment is placed before the command name. The variable is set only in the environment of sort</code> and reverts to its previous value (or unset state) when sort</code> exits.</p> This pattern is useful for overriding behavior temporarily:</p> $ TZ=UTC date $ LC_ALL=C grep "[A-Z]" file.txt </code></pre> How the Environment Is Built at Login</h2> When you log in, the environment does not appear from nowhere. It is constructed in layers, with each layer adding or overriding variables.</p> Layer 1: The kernel.</strong> The kernel sets a minimal environment when the login process starts. This includes the terminal type and some basic system information.</p> Layer 2: PAM and the login process.</strong> The login system reads /etc/environment</code> and sets variables found there. This is a system-wide configuration file.</p> Layer 3: The login shell.</strong> If you are using bash, the login shell reads /etc/profile</code> (system-wide defaults), then one of ~/.bash_profile</code>, ~/.bash_login</code>, or ~/.profile</code> (user-specific, in that priority order). These scripts typically set PATH, EDITOR, and other session-wide variables.</p> Layer 4: Interactive shell startup.</strong> Each new interactive bash shell (like opening a new terminal tab) reads ~/.bashrc</code>. This file typically contains aliases, shell functions, prompt customization, and other per-shell settings.</p> Fig. 21.2 -- Startup file loading order for bash</div> Login shell</text> (first shell after login)</text></p> /etc/environment</text> /etc/profile</text> ~/.bash_profile</text> (often sources ~/.bashrc)</text></p> Non-login interactive</text> (new terminal window)</text></p> ~/.bashrc</text> That's it.</text></p> # Typical ~/.bash_profile</text> export PATH="$HOME/bin:$PATH"</text> export EDITOR=vim</text> source ~/.bashrc</text> # load aliases and prompt</text> </marker> </defs> </svg> A login shell reads system-wide files first, then user-specific files. A non-login interactive shell reads only ~/.bashrc. A common pattern is for ~/.bash_profile to source ~/.bashrc so that all settings are available in both scenarios.</div> </div> </figure> The usual best practice is to put environment variable exports (PATH, EDITOR, etc.) in ~/.bash_profile</code> or ~/.profile</code>, and put aliases, functions, and prompt settings in ~/.bashrc</code>. Then have ~/.bash_profile</code> source ~/.bashrc</code>:</p> # In ~/.bash_profile: export PATH="$HOME/bin:$PATH" export EDITOR=vim # Load interactive settings too if [ -f ~/.bashrc ]; then source ~/.bashrc fi </code></pre> Key term: source</strong> A shell built-in that reads and executes a file in the current shell, rather than starting a new process. Unlike running a script normally, sourcing preserves variable changes and function definitions in the current shell. The dot command (.) is an alias for source. </div> Unsetting Variables</h2> To remove a variable from the environment, use unset</code>:</p> $ export DEBUG=1 $ echo $DEBUG 1 $ unset DEBUG $ echo $DEBUG </code></pre> After unset</code>, the variable no longer exists. This is different from setting it to an empty string:</p> $ export DEBUG="" </code></pre> An empty variable still exists in the environment. An unset variable is absent entirely. Some programs treat these differently.</p> A Process Cannot Modify Its Parent</h2> This point is so important it deserves its own section. When a process is created with fork()</code>, it gets a copy of the parent's environment. Changes to that copy are private. They do not propagate back to the parent.</p> This means if you run a script that sets a variable, the variable disappears when the script exits:</p> $ cat set_color.sh #!/bin/bash export COLOR=red echo "Inside script: COLOR=$COLOR" $ bash set_color.sh Inside script: COLOR=red $ echo $COLOR $ </code></pre> The variable COLOR</code> was set inside the child bash process that ran the script. When that process exited, its environment was destroyed. The parent shell never saw the change.</p> If you want a script to modify the current shell's environment, you must source</strong> it:</p> $ source set_color.sh Inside script: COLOR=red $ echo $COLOR red </code></pre> Sourcing reads the file and executes it in the current shell, not in a child. Now the export COLOR=red</code> command runs in your actual shell, so the variable persists.</p> Fig. 21.3 -- Running vs. sourcing a script</div> bash script.sh</text></p> Parent shell</text> COLOR=(unset)</text> unchanged</text> fork</text> Child bash</text> COLOR=red</text> dies on exit</text> X</text> no return</text></p> source script.sh</text></p> Same shell process</text> script runs HERE</text> COLOR=red persists</text> bash script.sh</text> New process. Variable changes</text> are lost when script exits.</text> source script.sh</text> Same process. Variable changes</text> persist in the current shell.</text> </marker> </defs> </svg> Running a script with "bash script.sh" creates a child process. Variables set inside it vanish when it exits. Sourcing with "source script.sh" (or ". script.sh") executes the commands in your current shell, so changes stick.</div> </div> </figure> If you want a script to set variables in your current shell, source it. If you want a script's variable changes to be isolated and temporary, run it normally. This is not a limitation -- it is a safety feature. Scripts cannot silently alter your working environment unless you explicitly allow it. </div> Practical Patterns</h2> Add a directory to PATH persistently.</strong> Edit ~/.bash_profile</code> (or ~/.profile</code>) and add:</p> export PATH="$HOME/.local/bin:$PATH" </code></pre> Then either log out and back in, or source the file: source ~/.bash_profile</code>.</p> Set a variable for development.</strong> Many programs read environment variables for configuration:</p> $ export DATABASE_URL="postgres://localhost/mydb" $ export DEBUG=1 $ ./my-app </code></pre> Check if a variable is set.</strong> Use the -z</code> test (we will use this more in Article 22):</p> $ if [ -z "$DATABASE_URL" ]; then echo "Not set"; fi </code></pre> Pass secrets without command-line arguments.</strong> Command-line arguments are visible in ps</code> output. Environment variables are not (by default). This is why database passwords and API keys are typically passed as environment variables rather than flags.</p> What You Have Learned</h2> Environment variables are key-value pairs carried by every process. The shell maintains both shell-only variables and exported environment variables. Only exported variables are inherited by child processes. Inheritance flows one way -- children cannot modify their parent's environment. The startup files ~/.bash_profile</code> and ~/.bashrc</code> build your environment at login and shell startup. The source</code> command executes a file in the current shell, which is the only way a script can modify the current shell's variables.</p> The environment is the invisible layer of configuration that sits between the operating system and every program you run. Understanding it demystifies why things "just work" -- and, more importantly, why they sometimes do not.</p> Next: Your First Shell Script</a></p> Pipes and Redirection 2025-01-24T00:00:00+00:00 In the previous article, you learned that every process has three standard channels and that programs do not know or care where those channels actually lead. Now we put that design to work. The shell gives you simple syntax for rewiring those channels: send output to a file, read input from a file, or connect the output of one program directly to the input of another.</p> This is the mechanism that turns a collection of small, single-purpose programs into a powerful system. Each program does one thing. You combine them.</p> Output Redirection: > and >></h2> The ></code> operator tells the shell to connect a program's stdout to a file instead of the screen. The shell opens (or creates) the file, then sets file descriptor 1 to point at it before starting the program.</p> $ echo "first line" > output.txt $ cat output.txt first line </code></pre> If the file already exists, ></code> destroys its contents and starts fresh. This is called truncation</strong>. The old data is gone the moment the shell processes the ></code>.</p> If you want to add to an existing file instead of replacing it, use >></code>:</p> $ echo "second line" >> output.txt $ cat output.txt first line second line </code></pre> The >></code> operator opens the file in append mode</strong>. New data is written after the existing contents. Nothing is deleted.</p> Key term: Redirection</strong> The shell's mechanism for changing where a process's standard channels (stdin, stdout, stderr) connect. The operators >, >>, <, and 2> rewire file descriptors before the program starts. The program itself is unaware that redirection has occurred. </div> Fig. 20.0 -- Output redirection with > and >></div> echo "hello" > file.txt</text></p> echo</text> stdout</text> old content</text> replaced with "hello"</text> TRUNCATED</text></p> echo "world" >> file.txt</text></p> echo</text> stdout</text> hello</text> world</text> APPENDED</text></p> > creates or overwrites</text> >> creates or appends</text> </marker> </defs> </svg> The > operator replaces the file's contents (truncation). The >> operator adds to the end (append). Both create the file if it does not exist.</div> </div> </figure> The `>` operator destroys existing file contents. This is the single most common way people accidentally delete data from the command line. If you are not sure whether you want to overwrite, use `>>` to append instead. </div> Redirecting Standard Error</h2> By default, ></code> redirects file descriptor 1 (stdout). To redirect file descriptor 2 (stderr), prefix the ></code> with the descriptor number:</p> $ ls /nonexistent 2> errors.txt $ cat errors.txt ls: cannot access '/nonexistent': No such file or directory </code></pre> The 2></code> syntax sends stderr to the file. Normal output still goes to the screen.</p> You can redirect both at the same time:</p> $ ls /home /nonexistent > listing.txt 2> errors.txt </code></pre> Now stdout goes to listing.txt</code> and stderr goes to errors.txt</code>. Clean separation.</p> If you want both stdout and stderr to go to the same file, the syntax is:</p> $ command > all_output.txt 2>&1 </code></pre> The 2>&1</code> means "redirect file descriptor 2 to wherever file descriptor 1 currently points." Since we already redirected fd 1 to all_output.txt</code>, fd 2 follows it there.</p> Order matters here. The shell processes redirections left to right. If you reversed it -- 2>&1 > all_output.txt</code> -- stderr would go to the screen (where fd 1 was pointing at the time) and only stdout would go to the file.</p> Key term: File descriptor duplication</strong> The syntax 2>&1 tells the shell to make file descriptor 2 point to the same place as file descriptor 1. This is called "duplicating" a file descriptor. It is how you merge stderr into stdout (or vice versa). </div> Input Redirection: <</h2> The <</code> operator connects a file to a program's stdin instead of the keyboard:</p> $ sort < names.txt Alice Bob Charlie </code></pre> The sort</code> program reads from file descriptor 0 as usual. It does not know that stdin has been rewired from the keyboard to a file. It just reads lines, sorts them, and writes the result to stdout.</p> Many programs accept a filename as an argument, so you can often achieve the same result with:</p> $ sort names.txt </code></pre> The difference is subtle but real. When you use <</code>, the shell opens the file and passes the file descriptor to the program. The program reads from stdin and has no idea a file is involved. When you pass the filename as an argument, the program opens the file itself.</p> In practice, the result is the same. But the <</code> form matters when you are building pipelines or working with programs that only read from stdin.</p> The Pipe: |</h2> The pipe operator is the most powerful tool in the Unix shell. It connects the stdout of one program directly to the stdin of the next. No temporary files. No manual copying. The data flows directly from one process to another through a kernel buffer.</p> $ ls /usr/bin | wc -l </code></pre> This command lists all files in /usr/bin</code> and sends that list (via a pipe) to wc -l</code>, which counts the number of lines. The result is the number of programs installed in that directory.</p> Here is what happens mechanically:</p> The shell creates a pipe -- a pair of connected file descriptors, one for reading and one for writing.</li> It forks two child processes.</li> In the first child, stdout (fd 1) is connected to the write end of the pipe, then ls</code> is executed.</li> In the second child, stdin (fd 0) is connected to the read end of the pipe, then wc</code> is executed.</li> Both programs run simultaneously. As ls</code> writes data, wc</code> reads it.</li> </ol> Fig. 20.1 -- Anatomy of a pipe</div> ls /usr/bin | wc -l</text></p> ls /usr/bin</text> writes to fd 1</text> (pipe write end)</text> kernel pipe</text> 4 KB buffer</text> wc -l</text> reads from fd 0</text> (pipe read end)</text> ls output:</text> bash</text> cat</text> chmod</text> ...</text></p> bytes flow through pipe</text> wc output:</text> 1847</text></p> </marker> </marker> </marker> </defs> </svg> The shell creates a kernel pipe with a small buffer (typically 4 KB on Linux, though it can grow to 64 KB). The first program writes to it; the second reads from it. Both run concurrently.</div> </div> </figure> The key insight is that both programs run at the same time. The pipe is not "run the first command, save its output, then feed it to the second command." It is a live connection. If the pipe buffer fills up because the writer is faster than the reader, the writer is paused until the reader catches up. This is called backpressure</strong>, and it means pipes work efficiently even with large amounts of data.</p> Building Pipelines</h2> You can chain as many pipes as you want:</p> $ cat /var/log/syslog | grep "error" | sort | uniq -c | sort -rn | head -10 </code></pre> This pipeline:</p> Reads the system log</li> Filters for lines containing "error"</li> Sorts those lines alphabetically (so duplicates are adjacent)</li> Counts consecutive duplicate lines</li> Sorts by count, highest first</li> Shows only the top 10</li> </ol> Six programs, each doing one small job, connected into a single data-processing chain. No temporary files. No programming language required.</p> Fig. 20.2 -- A six-stage pipeline</div> Data at each stage:</text></p> 1. All 50,000 log lines</text> </p> 2. 340 lines with "error"</text> </p> 3. 340 lines, sorted</text> </p> 4. 85 unique lines with counts</text> </p> 5. 85 lines, sorted by count</text> </p> 6. Top 10 lines</text> </p> Each stage reduces the data. The final output is tiny compared to the input.</text></p> </marker> </marker> </defs> </svg> A six-stage pipeline progressively filters and transforms data. Each program runs simultaneously, connected by kernel pipes. 50,000 lines go in; 10 come out.</div> </div> </figure> The Unix philosophy is "write programs that do one thing well, and connect them with pipes." This is not just a slogan. It is a practical engineering strategy that has worked for over fifty years. Small composable tools are easier to test, debug, and combine than large monolithic programs. </div> Combining Redirection and Pipes</h2> You can use redirection and pipes together. Redirection affects a single program's file descriptors; pipes connect programs to each other.</p> $ grep "error" < input.log | sort > sorted_errors.txt </code></pre> This reads input.log</code> into grep's stdin, pipes grep's stdout to sort, and redirects sort's stdout to a file. Each piece works independently.</p> You can also discard unwanted output in a pipeline:</p> $ find / -name "*.conf" 2>/dev/null | head -20 </code></pre> The find</code> command searches the entire filesystem, which generates many "Permission denied" errors on directories you cannot read. The 2>/dev/null</code> throws those errors away. Only the successful results (stdout) flow through the pipe to head</code>.</p> Here Documents</h2> Sometimes you want to feed multiple lines of text into a program's stdin without creating a file first. A here document</strong> (or heredoc) lets you embed the input directly in the command:</p> $ sort <<EOF cherry apple banana EOF apple banana cherry </code></pre> The <<EOF</code> tells the shell: "read everything from the next line until you see a line containing only EOF</code>, and feed it all into the program's stdin." You can use any word in place of EOF</code> -- it is just a delimiter.</p> Key term: Here document</strong> A form of input redirection that allows you to embed multi-line text directly in a shell command. The syntax is <<WORD, where WORD is any delimiter. The shell reads input lines until it encounters a line containing only that delimiter, then feeds all the collected text to the command's stdin. </div> The Tee Command: Splitting a Stream</h2> What if you want to save a pipeline's intermediate output to a file while also passing it along to the next stage? The tee</code> command reads from stdin, writes a copy to a file, and also writes to stdout:</p> $ ls /usr/bin | tee listing.txt | wc -l </code></pre> This counts the files in /usr/bin</code> (via wc -l</code>) and also saves the full listing to listing.txt</code>. The tee</code> command is named after a T-shaped pipe fitting that splits a water flow into two directions.</p> Fig. 20.3 -- The tee command splits the data stream</div> stdin</text> copy to file</text> stdout</text></p> 1847</text> </marker> </marker> </marker> </defs> </svg> The tee command acts like a T-junction in plumbing. Data flows in from the left. One copy goes down to a file. Another copy continues right to the next program in the pipeline.</div> </div> </figure> Common Patterns</h2> Here are redirection and pipe patterns you will use constantly:</p> Save command output to a file:</strong></p> $ dmesg > boot_messages.txt </code></pre> Append to a log:</strong></p> $ echo "$(date): backup complete" >> /var/log/backup.log </code></pre> Discard errors:</strong></p> $ find / -name "*.conf" 2>/dev/null </code></pre> Send both stdout and stderr to a file:</strong></p> $ make > build.log 2>&1 </code></pre> Count lines in a filtered result:</strong></p> $ grep -r "TODO" src/ | wc -l </code></pre> Find the ten largest files:</strong></p> $ du -sh /var/* 2>/dev/null | sort -rh | head -10 </code></pre> Each of these combines the same small set of operators -- ></code>, >></code>, <</code>, 2></code>, |</code> -- in different ways. The syntax is minimal. The power comes from composition.</p> What You Have Learned</h2> Output redirection with ></code> sends stdout to a file (truncating it). Append redirection with >></code> adds to the end. Input redirection with <</code> feeds a file into stdin. The 2></code> form redirects stderr specifically, and 2>&1</code> merges stderr into stdout. The pipe operator |</code> connects one program's stdout to the next program's stdin, allowing you to build multi-stage data processing pipelines. The tee</code> command lets you save a copy of data mid-pipeline.</p> These operators are the glue that holds the Unix tool ecosystem together. Small programs, each doing one thing, connected into arbitrarily complex workflows.</p> Next: The Environment</a></p> Standard I/O 2025-01-23T00:00:00+00:00 Every program needs to communicate. It needs to receive input, produce output, and report problems. Unix solved this decades ago with an elegant design: every process is born with three open communication channels. No setup required. No configuration files. They are just there.</p> These three channels are called standard input</strong>, standard output</strong>, and standard error</strong>. Together, they form the foundation of how every command-line program talks to the world.</p> Three Channels, Always</h2> Imagine a factory machine. It has a conveyor belt feeding raw materials in one side. It has a second conveyor belt carrying finished products out the other side. And it has an alarm buzzer that goes off when something goes wrong. The raw materials belt is standard input. The finished products belt is standard output. The alarm is standard error.</p> Every process on a Unix system starts with exactly these three channels open:</p> Standard input (stdin)</strong> -- where the program reads incoming data</li> Standard output (stdout)</strong> -- where the program writes its normal results</li> Standard error (stderr)</strong> -- where the program writes error messages and diagnostics</li> </ul> By default, stdin is connected to your keyboard (through the terminal), and both stdout and stderr are connected to your screen. But as we will see in the next article, these connections can be rewired.</p> Key term: Standard I/O</strong> The three default communication channels that every Unix process receives at birth: standard input (stdin) for reading data, standard output (stdout) for writing results, and standard error (stderr) for writing error messages. They are the universal interface between programs and the outside world. </div> Fig. 19.0 -- The three standard channels</div> fd 0</text> fd 1</text> fd 2</text></p> Default connections: keyboard feeds in, screen receives both outputs</text></p> </marker> </marker> </marker> </defs> </svg> Every process starts with three channels. By default, stdin reads from the keyboard and both stdout and stderr write to the screen. The numbers 0, 1, and 2 are file descriptors -- the kernel's internal names for these channels.</div> </div> </figure> File Descriptors: The Kernel's Numbering System</h2> The kernel does not think in terms of names like "stdin" and "stdout." It uses numbers. Each open channel in a process gets a sequential integer called a file descriptor</strong>.</p> The first three are always assigned the same way:</p> 0</strong> -- standard input</li> 1</strong> -- standard output</li> 2</strong> -- standard error</li> </ul> If a process opens additional files, they get the next available numbers: 3, 4, 5, and so on. But 0, 1, and 2 are reserved by convention, and every program assumes they are there.</p> Key term: File descriptor</strong> A small non-negative integer that the kernel assigns to each open file, socket, pipe, or I/O channel within a process. File descriptors 0, 1, and 2 are pre-assigned to stdin, stdout, and stderr. Every read or write operation in a Unix program ultimately references a file descriptor number. </div> This numbering system is important because it is how you target specific channels when redirecting. Want to redirect just error messages to a file while keeping normal output on screen? You need to reference file descriptor 2 specifically. We will do exactly that in the next article.</p> Standard Output: Where Results Go</h2> When a program wants to display results, it writes to stdout (file descriptor 1). The echo</code> command writes its arguments to stdout. The ls</code> command writes its file listing to stdout. The cat</code> command reads files and writes their contents to stdout.</p> $ echo "Hello, world" Hello, world </code></pre> The text "Hello, world" went to stdout, which was connected to your terminal, which displayed it on screen. Simple.</p> Here is the critical thing: the program does not know or care where stdout actually goes. It just writes bytes to file descriptor 1. Whether those bytes end up on a screen, in a file, or fed into another program is decided by whoever set up the file descriptors before the program started. Usually, that is the shell.</p> Fig. 19.1 -- stdout does not know its destination</div> The program writes to fd 1 the same way in all three cases.</text> Only the shell knows where the bytes actually go.</text></p> </marker> </defs> </svg> The echo command writes bytes to file descriptor 1 without knowing where they end up. The shell can connect that file descriptor to the screen (default), a file, or the input of another program.</div> </div> </figure> Programs do not write "to the screen" or "to a file." They write to a file descriptor. Where that file descriptor points is decided externally, usually by the shell. This separation is what makes pipes and redirection possible. </div> Standard Error: A Separate Channel for Problems</h2> Why do error messages need their own channel? Consider this situation: you run a program and send its output to a file. The program encounters an error halfway through. If the error message goes to stdout, it ends up mixed into the output file, corrupting the data. If the error message goes to stderr, it still appears on your screen even though stdout was redirected elsewhere.</p> $ ls /home /nonexistent /home: user1 user2 ls: cannot access '/nonexistent': No such file or directory </code></pre> In this example, the file listing goes to stdout and the error message goes to stderr. Both happen to be connected to your screen, so you see them interleaved. But if you redirect stdout to a file, the error still shows up on your terminal:</p> $ ls /home /nonexistent > listing.txt ls: cannot access '/nonexistent': No such file or directory </code></pre> The file listing.txt</code> contains the directory listing, clean and uncorrupted. The error message appeared on screen because stderr was not redirected -- only stdout was.</p> This is why stderr exists: to keep error messages out of the data stream.</p> Standard Input: How Programs Read</h2> Most interactive programs read from stdin (file descriptor 0). By default, this is connected to your keyboard through the terminal.</p> When you run cat</code> with no arguments, it reads from stdin:</p> $ cat Hello Hello World World </code></pre> Each line you type is echoed back because cat</code> reads stdin and writes to stdout. Press Ctrl+D to signal "end of input" -- this sends an EOF (end-of-file) marker that tells the program there is nothing more to read.</p> Programs that process data -- like sort</code>, grep</code>, wc</code> -- can all read from stdin when given no filename argument. This is what makes pipes work, but we are getting ahead of ourselves.</p> Fig. 19.2 -- stdin sources</div> Only one source is connected at a time.</text> The program reads from fd 0 the same way regardless of source.</text></p> </marker> </defs> </svg> File descriptor 0 (stdin) can be connected to the keyboard, a file, or the output of another program. The receiving program reads from fd 0 identically in all three cases.</div> </div> </figure> The /dev/null Black Hole</h2> Sometimes you want to throw output away entirely. Maybe a program prints progress messages you do not care about. Maybe you want to run a command silently.</p> Unix provides a special file for this: /dev/null</code>. It is a device file that accepts any data written to it and discards it immediately. Reading from it immediately returns end-of-file.</p> $ echo "This disappears" > /dev/null $ ls /nonexistent 2> /dev/null </code></pre> The first line sends stdout to /dev/null</code> -- the text vanishes. The second line sends stderr to /dev/null</code> -- the error message is suppressed. (The 2></code> syntax means "redirect file descriptor 2." We will cover this syntax fully in the next article.)</p> Key term: /dev/null</strong> A special device file that discards all data written to it and returns end-of-file on reads. It is the Unix "black hole" -- useful for silencing unwanted output or error messages. The name literally means the "null device." </div> Think of /dev/null</code> as a trash can with no bottom. You can throw anything in. Nothing ever comes back out. Nothing accumulates. It is bottomless and always empty.</p> Everything Is a File (Descriptor)</h2> One of the defining ideas in Unix is that I/O is uniform. Whether a program is reading from a keyboard, a file on disk, a network socket, or a pipe from another program, it uses the same system calls: read()</code> and write()</code>, operating on file descriptors.</p> This is why the standard channels are called "file" descriptors even though stdin often comes from a keyboard. The kernel presents all I/O sources through the same interface. A program that reads from file descriptor 0 does not know -- and does not need to know -- whether it is reading keystrokes, file contents, or output from another program.</p> This uniformity is not an accident. It is a deliberate design choice that makes it trivial to compose programs. If every program reads from stdin and writes to stdout using the same interface, then any program's output can be connected to any other program's input. No special adapters. No format conversion. Just bytes flowing through file descriptors.</p> The power of standard I/O comes from its uniformity. Every program reads and writes the same way, regardless of what is on the other end of the file descriptor. This uniformity is what makes the entire Unix pipeline model possible. </div> Fig. 19.3 -- Uniform I/O through file descriptors</div> Kernel System Call Interface</text></p> read(fd, buf, n)</text> write(fd, buf, n)</text> Program sees:</text> "just a number"</text></p> fd 0</text> Could be any of:</text></p> Terminal (keyboard)</text> Regular file on disk</text> Pipe from another program</text> Network socket</text> Same read()/write() calls work on any type of file descriptor.</text> </svg></p> A program calls read() and write() on file descriptor numbers without knowing what type of resource is on the other end. The kernel handles the translation between the uniform interface and the actual device or file.</div> </div> </figure> Seeing File Descriptors in Action</h2> You can inspect the file descriptors of a running process. Every process has a directory in /proc</code> that lists its open file descriptors:</p> $ ls -l /proc/self/fd lrwx------ 1 user user 64 Jan 23 10:00 0 -> /dev/pts/0 lrwx------ 1 user user 64 Jan 23 10:00 1 -> /dev/pts/0 lrwx------ 1 user user 64 Jan 23 10:00 2 -> /dev/pts/0 </code></pre> The /proc/self</code> directory always refers to the currently running process. Here, all three standard file descriptors point to /dev/pts/0</code>, which is a pseudo-terminal device. That is the terminal you are typing in.</p> If you redirect stdout to a file and check again, you will see file descriptor 1 pointing to that file instead of the terminal.</p> What You Have Learned</h2> Every Unix process starts with three open file descriptors: stdin (0) for reading input, stdout (1) for writing results, and stderr (2) for writing errors. Programs interact with these channels through uniform read and write system calls without knowing what is on the other end. Standard error exists as a separate channel so that error messages do not contaminate the data stream. The special file /dev/null</code> discards anything written to it.</p> This system of standard channels is not just a convenience. It is the architectural foundation that makes the next topic possible: connecting programs together with pipes and redirection.</p> Next: Pipes and Redirection</a></p> What a Shell Actually Is 2025-01-22T00:00:00+00:00 You have been typing commands into a terminal for the last few articles. You pressed keys, text appeared, programs ran. But what exactly was reading those commands? What was deciding which program to start, and how?</p> The answer is the shell</strong>. It is not the terminal. It is not the window on your screen. It is a separate, ordinary program that happens to be very good at one thing: taking text commands from a human and turning them into running processes.</p> The Terminal Is Not the Shell</h2> Think of a restaurant. The terminal is the dining room -- the table, the chair, the menu. The shell is the waiter. You tell the waiter what you want. The waiter carries your order to the kitchen. The kitchen does the actual cooking.</p> The terminal handles input and output. It draws characters on screen. It captures your keystrokes. But it does not know what ls</code> means. It does not know how to start a program. It is just a conduit.</p> The shell is the program running inside that terminal. It prints the prompt. It reads the line you type. It figures out what you meant. It asks the kernel to run the right program. When the program finishes, the shell prints the next prompt.</p> Key term: Shell</strong> A command-line interpreter -- a program that reads text commands, parses them, and executes them by asking the kernel to create new processes. It is called a "shell" because it wraps around the kernel, forming the outer layer that humans interact with. </div> Fig. 18.0 -- Terminal vs. shell vs. kernel</div> YOU TYPE HERE</text> </svg></p> The terminal displays text and captures keystrokes. The shell interprets commands. The kernel creates and manages the actual programs that do the work.</div> </div> </figure> You can prove this separation yourself. Open a terminal and type echo $SHELL</code>. It will print something like /bin/bash</code> or /usr/bin/zsh</code>. That is the path to a real file on disk -- the shell binary. You can even run a different shell inside your current one by typing its name, like sh</code> or zsh</code>.</p> What Happens When You Type a Command</h2> You type ls -l /tmp</code> and press Enter. Here is exactly what the shell does:</p> Step 1: Read.</strong> The shell reads the entire line of text from standard input.</p> Step 2: Parse.</strong> It splits the line into tokens. The first token is the command name: ls</code>. The remaining tokens are arguments: -l</code> and /tmp</code>.</p> Step 3: Find.</strong> The shell needs to locate the program called ls</code>. It searches a list of directories, one by one, looking for an executable file with that name. This list is the PATH variable, and we will cover it in detail in Article 21.</p> Step 4: Execute.</strong> The shell calls fork()</code> to create a copy of itself, then the copy calls exec()</code> to replace itself with the ls</code> program. The original shell waits for ls</code> to finish.</p> Step 5: Wait.</strong> When ls</code> exits, the shell collects its exit status -- a number that indicates success or failure. Then it prints the next prompt and goes back to Step 1.</p> Fig. 18.1 -- The shell's read-eval-execute loop</div> $ ls -l /tmp</text> ["ls","-l","/tmp"]</text> /usr/bin/ls</text> PID 4821 running</text> exit status: 0</text></p> </marker> </defs> </svg> The shell runs in an infinite loop: read a command, parse it, find the program, execute it, wait for it to finish, then repeat. This loop is sometimes called the REPL -- read, evaluate, print, loop.</div> </div> </figure> This loop is the core of every shell. Everything else -- variables, conditionals, scripting -- is built on top of this basic cycle.</p> The shell is a loop. It reads a line, figures out what program to run, asks the kernel to run it, waits for it to finish, and then reads the next line. That is the fundamental operation, and it never changes. </div> Built-in Commands vs. External Programs</h2> Not every command you type launches a separate program. Some commands are built into the shell itself.</p> Consider cd /tmp</code>. If the shell started a new process, changed that process's working directory, and then that process exited -- well, the shell's own working directory would be unchanged. The child process's directory change would die with it. So cd</code> has to be handled inside the shell itself, without forking a new process.</p> These are called built-in commands</strong> (or just builtins</strong>). Common builtins include:</p> cd</code> -- change the shell's working directory</li> echo</code> -- print text (also exists as an external program)</li> export</code> -- set environment variables</li> exit</code> -- terminate the shell</li> type</code> -- tell you whether a command is a builtin or an external program</li> </ul> You can check any command with type</code>:</p> $ type cd cd is a shell builtin $ type ls ls is /usr/bin/ls </code></pre> External programs are separate executable files stored somewhere on disk. When you run ls</code>, the shell finds /usr/bin/ls</code>, creates a new process, and runs that file. When you run cd</code>, the shell handles it internally and no new process is created.</p> Key term: Built-in command</strong> A command that the shell executes directly, without creating a new process. Built-ins exist because certain operations -- like changing the working directory or setting variables -- must modify the shell's own state, which a child process cannot do. </div> How the Shell Finds Programs: PATH</h2> When you type ls</code>, the shell does not magically know where /usr/bin/ls</code> lives. It searches through a list of directories in order. That list is stored in an environment variable called PATH</code>.</p> You can see yours:</p> $ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin </code></pre> The directories are separated by colons. The shell checks each one, left to right, for a file matching the command name. The first match wins.</p> If you type a command and get command not found</code>, it means the shell searched every directory in PATH and found nothing. The program either does not exist, is not installed, or is in a directory that is not listed in PATH.</p> You can bypass PATH entirely by specifying the full path to a program:</p> $ /usr/bin/ls </code></pre> Or a relative path:</p> $ ./my-program </code></pre> The ./</code> prefix tells the shell to look in the current directory specifically, not to search PATH.</p> Fig. 18.2 -- PATH search order</div> $ grep</text></p> /usr/local/bin/</text> not found</text> /usr/bin/</text> FOUND: /usr/bin/grep</text> Search stops</text> /bin/</text> not checked</text> /usr/sbin/</text> not checked</text> </marker> </marker> </defs> </svg> When you type "grep", the shell walks through PATH directories left to right. It finds /usr/bin/grep and stops searching. Later directories are never checked.</div> </div> </figure> The Major Shells</h2> The concept of a command-line shell is older than Linux. Here are the shells you are most likely to encounter:</p> sh -- The Bourne Shell</h3> Written by Stephen Bourne at Bell Labs in 1979. It established the basic syntax that all later shells build on: command arguments, pipes, redirection, variables, control structures. On modern Linux systems, /bin/sh</code> is usually a symlink to a lighter shell like dash</code>, not the original Bourne shell.</p> bash -- Bourne Again Shell</h3> The default shell on most Linux distributions. Written by Brian Fox for the GNU Project in 1989. It is backward-compatible with sh</code> and adds interactive features like command history, tab completion, and programmable completion. When people say "shell scripting" without qualification, they usually mean bash.</p> zsh -- Z Shell</h3> The default shell on macOS since 2019. It offers everything bash does plus more advanced features: better tab completion, spelling correction, extended globbing patterns, and a large plugin ecosystem. Its syntax is mostly compatible with bash.</p> dash -- Debian Almquist Shell</h3> A minimal, fast shell used by Debian and Ubuntu for system scripts. It implements only the POSIX sh</code> standard -- no bash extensions. When a system script says #!/bin/sh</code>, it often runs under dash because dash starts faster than bash.</p> All of these shells do the same fundamental job: read commands, find programs, run them. They differ in interactive features and scripting extensions, but the core loop is identical. If you learn bash, you can get by in any of them. </div> The Shell Is Just a Program</h2> This is the key insight that trips up many beginners: the shell is not special hardware. It is not an operating system component. It is an ordinary user-space program, no different in principle from ls</code> or firefox</code> or python3</code>.</p> The kernel does not know or care that bash is a "shell." From the kernel's perspective, bash is just another process that occasionally calls fork()</code>, exec()</code>, and wait()</code>. You could write your own shell in any language. Some people do -- it is a classic programming exercise.</p> What makes the shell feel special is that it is the first thing you interact with when you log in, and it is the thing that starts every other program you use from the command line. But that is a matter of convention, not privilege.</p> You can verify this by running a shell inside a shell:</p> $ bash $ echo "I am in a nested shell" I am in a nested shell $ exit $ echo "Back to the original" Back to the original </code></pre> The inner bash is a child process of the outer bash. When you type exit</code>, the inner shell terminates and you return to the outer one. There is no limit to how deep you can nest, though there is rarely a reason to go more than one level.</p> Fig. 18.3 -- Nested shells are just nested processes</div> $ exit kills PID 1001</text> returns to PID 1000</text> </svg></p> Running "bash" inside bash creates a child shell process. Programs you run in the inner shell are grandchildren of the outer shell. Exiting the inner shell returns you to the outer one.</div> </div> </figure> The Login Shell and Interactive Shells</h2> When you log in to a Linux system, the login process checks /etc/passwd</code> to find which shell to start for your user account. That shell is your login shell</strong>. It is the root of your session -- every other program you launch descends from it.</p> A login shell reads extra configuration files when it starts. For bash, that means ~/.bash_profile</code> or ~/.profile</code>. These files set up your environment: your PATH, your preferred editor, your prompt format.</p> When you open a new terminal window in a graphical desktop, you usually get a non-login interactive shell</strong>. It reads ~/.bashrc</code> instead. The distinction matters because some settings only need to happen once per login (like adding directories to PATH), while others should apply to every new shell (like aliases and prompt configuration).</p> We will explore these configuration files in detail in Article 21.</p> Key term: Login shell</strong> The shell started when you first log in to the system. It reads additional startup files (like ~/.profile) that set up your session environment. You can check if your current shell is a login shell by running "echo $0" -- a login shell's name starts with a hyphen, like "-bash". </div> What You Have Learned</h2> The shell is a program. It lives on disk at a path like /bin/bash</code>. It runs in a loop: read a command, parse it, find the program, fork a child process, execute the program in that child, wait for it to finish, repeat. Some commands are built into the shell itself because they need to modify the shell's own state. The shell finds external programs by searching directories listed in the PATH variable. There are several shells to choose from -- bash, zsh, dash, and others -- but they all implement the same fundamental loop.</p> The terminal is the window. The shell is the brain inside it.</p> Next: Standard I/O</a></p> Users, Groups, and Permissions 2025-01-21T00:00:00+00:00 A computer with one user and no network connection does not need access control. But the moment a second person can log in, or the machine connects to the internet, you need a way to separate what different people and programs are allowed to do. Who can read your files? Who can install software? Who can shut down the machine?</p> Linux answers these questions with a system that dates back to the original Unix: users, groups, and permission bits. Every file, every process, and every action on the system is governed by this model.</p> Users Are Numbers</h2> When you log into a Linux system, you type a username like "alice" or "admin." But internally, the kernel does not care about names. It tracks users by number. Every user has a UID</strong> -- a user identifier, which is an integer.</p> UID</th> Conventional Meaning</th></tr></thead> 0</td> root (superuser)</td></tr> 1-999</td> System accounts (services)</td></tr> 1000+</td> Regular human users</td></tr> </tbody></table> The mapping between names and numbers is stored in /etc/passwd</code>. Despite the name, this file does not contain passwords (not anymore -- it did in the 1970s). Each line describes one user:</p> alice:x:1000:1000:Alice Smith:/home/alice:/bin/bash </code></pre> The fields, separated by colons, are:</p> Username (alice</code>)</li> Password placeholder (x</code> -- actual password is in /etc/shadow</code>)</li> UID (1000</code>)</li> Primary GID (1000</code>)</li> Comment / full name (Alice Smith</code>)</li> Home directory (/home/alice</code>)</li> Login shell (/bin/bash</code>)</li> </ol> Key term: UID (User Identifier)</strong> A non-negative integer that uniquely identifies a user on the system. The kernel uses UIDs, not usernames, for all access control decisions. UID 0 is always the root superuser. </div> Fig. 17a -- The /etc/passwd file structure</div> /etc/passwd</text></p> alice:x:1000:1000:Alice Smith:/home/alice:/bin/bash</text> username</text> alice</text> pw</text> x</text> UID</text> 1000</text> GID</text> 1000</text> comment</text> Alice Smith</text> home dir</text> /home/alice</text> shell</text> /bin/bash</text> root:x:0:0:root:/root:/bin/bash</text> nobody:x:65534:65534:Nobody:/nonexistent:/usr/sbin/nologin</text> sshd:x:110:65534::/run/sshd:/usr/sbin/nologin</text> </svg> Each line in /etc/passwd maps a username to a UID, home directory, and login shell. System accounts use /usr/sbin/nologin to prevent interactive login.</div> </div> </figure> Passwords and /etc/shadow</h2> The actual password hashes live in /etc/shadow</code>, which is readable only by root. This separation exists because /etc/passwd</code> must be world-readable (programs need to look up usernames), but password hashes should not be.</p> A line in /etc/shadow</code> looks like:</p> alice:$6$rounds=5000$salt$hash...:19500:0:99999:7::: </code></pre> The $6$</code> prefix indicates SHA-512 hashing. The salt is a random string that ensures two users with the same password produce different hashes. The remaining fields control password aging -- when it was last changed, when it expires, and how much warning to give.</p> Key term: Password hash</strong> A one-way mathematical transformation of a password. The system stores the hash, not the password itself. When you log in, the system hashes what you typed and compares it to the stored hash. This means that even someone who reads /etc/shadow cannot directly recover the passwords. </div> Groups</h2> A group</strong> is a named collection of users, identified by a GID</strong> (group identifier). Groups exist so you can grant permissions to multiple users at once without listing them individually.</p> Every user has a primary group</strong> (set in /etc/passwd</code>) and can belong to additional supplementary groups</strong> (set in /etc/group</code>).</p> The /etc/group</code> file:</p> developers:x:1001:alice,bob,carol www-data:x:33:alice sudo:x:27:alice </code></pre> Each line has four fields: group name, password placeholder, GID, and a comma-separated list of members.</p> Check your own groups with:</p> id groups </code></pre> The id</code> command shows your UID, primary GID, and all supplementary groups. This is exactly what the kernel sees when it checks your permissions.</p> The Root User</h2> UID 0 is root</strong>, the superuser. Root bypasses almost all permission checks. Root can read any file, write any file, kill any process, mount filesystems, load kernel modules, and change the system clock. There is no higher authority on a Linux system.</p> This is both powerful and dangerous. A mistake as root can destroy the system. A compromised root account means an attacker controls everything. This is why modern practice avoids logging in as root directly. Instead, you log in as a normal user and use sudo</code> to run individual commands with root privileges.</p> sudo apt update # Run this one command as root sudo -i # Open a root shell (use with caution) </code></pre> sudo</code> checks /etc/sudoers</code> to determine whether the requesting user is allowed to elevate. On most desktop distributions, the first user created during installation is granted sudo access.</p> Root (UID 0) can do anything on the system. Normal users are limited by permission checks. Use sudo for temporary privilege escalation instead of logging in as root. This limits the damage from mistakes and compromises. </div> File Permissions</h2> Every file and directory on the system has three sets of permission bits: one for the owner</strong>, one for the group</strong>, and one for others</strong> (everyone else).</p> Each set has three bits:</p> Bit</th> Letter</th> On a file</th> On a directory</th></tr></thead> Read</td> r</td> Can read the file contents</td> Can list the directory contents</td></tr> Write</td> w</td> Can modify the file</td> Can create/delete files in the directory</td></tr> Execute</td> x</td> Can run the file as a program</td> Can enter (cd into) the directory</td></tr> </tbody></table> The ls -l</code> command shows permissions:</p> -rwxr-xr-- 1 alice developers 4096 Jan 18 10:30 script.sh </code></pre> Reading the permission string -rwxr-xr--</code>:</p> First character: file type (-</code> = regular file, d</code> = directory, l</code> = symlink)</li> Characters 2-4: owner permissions (rwx</code> = read, write, execute)</li> Characters 5-7: group permissions (r-x</code> = read, execute, no write)</li> Characters 8-10: others permissions (r--</code> = read only)</li> </ul> Fig. 17b -- File permission bits</div> -rwxr-xr-- 1 alice developers script.sh</text></p> type</text> -</text> owner (alice)</text> r</text> w</text> x</text> group (developers)</text> r</text> -</text> x</text> others</text> r</text> -</text> -</text> Octal Representation</text></p> r=4 w=2 x=1</text> 7</text> r=4 w=0 x=1</text> 5</text> r=4 w=0 x=0</text> 4</text> chmod 754 script.sh</text> sets -rwxr-xr--</text> Common: 644 (files), 755 (executables/dirs), 600 (private), 777 (everything -- avoid)</text> </svg></p> Permission bits are grouped into owner, group, and others. Each group has read, write, and execute. The octal number is the sum of the bit values (r=4, w=2, x=1).</div> </div> </figure> Octal Notation</h3> Each permission bit has a numeric value: read = 4, write = 2, execute = 1. You add them up for each group. The permissions rwxr-xr--</code> become 754</code>: owner gets 4+2+1=7, group gets 4+0+1=5, others get 4+0+0=4.</p> You can set permissions with chmod</code>:</p> chmod 755 script.sh # rwxr-xr-x chmod 644 document.txt # rw-r--r-- chmod 600 secret.key # rw------- </code></pre> You can also use symbolic notation:</p> chmod u+x script.sh # Add execute for owner chmod g-w file.txt # Remove write for group chmod o= file.txt # Remove all permissions for others </code></pre> Changing Ownership</h3> The chown</code> command changes the owner and group:</p> chown alice:developers file.txt # Set owner and group chown alice file.txt # Set owner only chown :developers file.txt # Set group only </code></pre> Only root can change the owner of a file. A regular user can only change the group to a group they belong to.</p> How the Kernel Checks Permissions</h2> When a process tries to access a file, the kernel follows a simple algorithm:</p> If the process runs as UID 0 (root), allow everything. Check done.</li> If the process's UID matches the file's owner UID, use the owner permission bits.</li> If the process's GID (or any supplementary GID) matches the file's group GID, use the group permission bits.</li> Otherwise, use the others permission bits.</li> </ol> This is a strictly ordered check. If you are the owner, the owner bits apply even if the group bits are more permissive. This occasionally surprises people: a file owned by alice with permissions ---rwxrwx</code> is unreadable by alice, even though both the group and others have full access.</p> Fig. 17c -- Permission check algorithm</div> The kernel checks UID 0 first (always allowed), then owner match, then group match, then falls through to others. Only one set of bits is ever applied.</div> </div> </figure> Special Permission Bits</h2> Beyond the basic rwx bits, there are three special bits that modify how files and directories behave.</p> setuid (Set User ID)</h3> When a program with the setuid bit set is executed, it runs with the UID of the file's owner rather than the UID of the user who launched it. This is how normal users can change their own passwords -- the passwd</code> command is owned by root with setuid set:</p> ls -l /usr/bin/passwd -rwsr-xr-x 1 root root 68208 Jan 18 10:30 /usr/bin/passwd </code></pre> Notice the s</code> where the owner's execute bit would be. When alice runs passwd</code>, the process runs with UID 0 (root), giving it permission to write to /etc/shadow</code>. When passwd</code> finishes, the elevated privileges disappear.</p> setuid is powerful and dangerous. A bug in a setuid-root program can give any user root access. This is why the set of setuid programs on a system is kept deliberately small, and security auditors pay special attention to them.</p> setgid (Set Group ID)</h3> Similar to setuid, but for the group. On a file, it makes the program run with the file's group. On a directory, it has a different effect: new files created inside the directory inherit the directory's group instead of the creating user's primary group. This is useful for shared project directories.</p> The Sticky Bit</h3> On a directory, the sticky bit prevents users from deleting files they do not own, even if they have write permission on the directory. The classic example is /tmp</code>:</p> ls -ld /tmp drwxrwxrwt 15 root root 4096 Jan 18 10:30 /tmp </code></pre> The t</code> at the end is the sticky bit. Everyone can create files in /tmp</code>, but you can only delete your own files. Without the sticky bit, anyone with write access to the directory could delete anyone else's files.</p> Key term: setuid</strong> A special permission bit on executable files. When set, the program runs with the privileges of the file's owner, not the user who executed it. This is how programs like passwd and sudo gain temporary root access. </div> Process Credentials</h2> Every process inherits the UID and GID of the user who started it. When Alice's shell (running as UID 1000) runs ls</code>, the ls</code> process also runs as UID 1000. When ls</code> tries to read a directory, the kernel checks UID 1000's permissions.</p> A process actually carries several IDs:</p> Real UID</strong> -- the user who started the process</li> Effective UID</strong> -- the UID used for permission checks (usually the same as real, unless setuid is involved)</li> Saved UID</strong> -- allows a process to temporarily drop and regain elevated privileges</li> </ul> When a setuid program runs, the effective UID changes to the file owner, but the real UID stays as the invoking user. This is how the program knows who actually ran it.</p> Permissions are checked against the process's effective UID and GID, not the username. Every file access, every signal delivery, every system call that requires authorization goes through this check. The entire security model rests on these numeric identifiers. </div> Why This Matters for Security</h2> The user/group/permission model is the first line of defense on a Linux system. It works because:</p> Isolation.</strong> Each user's files are protected from other users by default. A compromised web server running as www-data</code> (UID 33) cannot read files in /home/alice/</code> (owned by UID 1000, mode 700).</p> Least privilege.</strong> Services run as dedicated, unprivileged users. The SSH daemon runs as sshd</code>. The web server runs as www-data</code>. If one service is compromised, the attacker gets only that service's permissions, not root.</p> Audit trail.</strong> Every process carries a UID. Logs record which user performed each action. This makes it possible to trace what happened after a security incident.</p> The model is not perfect. It has no concept of fine-grained capabilities beyond the binary root/non-root distinction. (Linux capabilities and mandatory access control systems like SELinux and AppArmor add more granularity, but those build on top of the basic model.) It does not protect against a careless root user. And it requires discipline -- a single chmod 777</code> on a sensitive directory can undo all the protection.</p> But for a system designed in the 1970s, it has proven remarkably durable. Every modern Linux system, from phones to supercomputers, still uses these same UID and GID checks as the foundation of its security.</p> Essential Commands</h2> id # Show your UID, GID, and groups whoami # Show your username ls -la # Show file permissions chmod 640 file # Set permissions chown user:group file # Change ownership passwd # Change your password sudo command # Run a command as root su - username # Switch to another user getent passwd alice # Look up a user getent group developers # Look up a group </code></pre> These commands map directly to the concepts in this article. id</code> shows the kernel's view of who you are. ls -la</code> shows the kernel's view of who owns a file. chmod</code> and chown</code> change those values. Everything else in the system -- every permission error, every "access denied" message, every security policy -- traces back to these fundamentals.</p> Next: What a Shell Actually Is</a></p> The TTY and Terminal 2025-01-20T00:00:00+00:00 Open a terminal on your Linux system. You see a dark window with a blinking cursor, waiting for you to type. This seems simple -- you type, the computer responds. But between your keyboard and the program reading your input, there is a chain of abstraction layers with roots going back to the 1800s.</p> The word "terminal" and the abbreviation "TTY" both come from physical machines that no longer exist. Understanding where they came from explains why the system works the way it does today.</p> From Teletypes to Terminals</h2> In the 1960s and 1970s, computers did not have monitors. Users interacted with the machine through a teletype</strong> (also called a teleprinter or TTY) -- a mechanical device with a keyboard and a printer. You typed a command on the keyboard. The computer's response was printed on paper.</p> The teletype communicated with the computer over a serial line. Each keystroke was converted to an electrical signal (using a code called ASCII), sent down the wire, and received by the computer. The computer's response traveled back the same way, driving the print mechanism.</p> Key term: TTY</strong> Short for "teletype" or "teletypewriter." Originally a physical printing terminal connected to a computer via a serial line. In modern Linux, TTY refers to any terminal device -- physical or virtual -- that provides a text interface between a user and the system. </div> By the late 1970s, teletypes were replaced by video terminals</strong> -- screens with keyboards. The DEC VT100 (1978) became the standard. Instead of printing on paper, it displayed text on a CRT screen. But it still connected to the computer over a serial line, and the computer still treated it as a TTY.</p> The physical terminals eventually disappeared too, replaced by terminal emulators -- software programs like GNOME Terminal, Alacritty, or xterm that simulate a VT100 (or its successors) in a window on your screen. But the kernel still uses the same abstraction it used for physical teletypes. That abstraction is the TTY subsystem.</p> Fig. 16a -- Evolution of terminals</div> The hardware changed three times, but the kernel abstraction remained. Modern terminal emulators still speak the same protocol that physical teletypes used.</div> </div> </figure> Virtual Consoles</h2> Before looking at terminal emulators, there is a simpler kind of terminal built directly into the kernel: the virtual console</strong>. If you press Ctrl+Alt+F1 through Ctrl+Alt+F6 on a Linux system, you switch between virtual consoles. Each one is a full-screen text terminal, managed entirely by the kernel without any graphical environment.</p> These are the /dev/tty1</code> through /dev/tty6</code> devices. They are real TTY devices -- the kernel draws text directly to the video card's framebuffer. When your graphical desktop fails to start, these consoles are your fallback.</p> Your graphical desktop typically runs on one of the higher-numbered virtual consoles (often tty2 or tty7, depending on the distribution). The terminal emulator you open within your desktop is something different -- it uses a pseudoterminal</strong>.</p> The Pseudoterminal (PTY)</h2> A terminal emulator like GNOME Terminal or Alacritty is a regular graphical application. It draws a window, renders text using a font, and handles keyboard input. But the shell running inside it expects to talk to a TTY device. How does a regular program pretend to be a hardware terminal?</p> The answer is the pseudoterminal (PTY)</strong> -- a pair of virtual devices created by the kernel. One end is the master</strong> (also called the PTY master or ptmx). The other end is the slave</strong> (also called the PTY slave or pts).</p> Key term: Pseudoterminal (PTY)</strong> A pair of virtual character devices that simulate a serial terminal connection. The master side is held by the terminal emulator. The slave side is given to the shell. Data written to one end appears as input on the other, with the kernel's line discipline processing in between. </div> The terminal emulator opens the master side. The shell (and any programs the shell runs) open the slave side. When you type a key in the terminal emulator, the emulator writes the character to the master. The kernel's TTY layer processes it and makes it available for reading on the slave. When the shell prints output, it writes to the slave. The kernel passes it to the master, and the terminal emulator renders it on screen.</p> You can see your current PTY with:</p> tty </code></pre> This will print something like /dev/pts/0</code>. The number increases for each terminal you open. List all active pseudoterminals:</p> ls /dev/pts/ </code></pre> Fig. 16b -- The pseudoterminal pair</div> keyboard input</text> screen output</text></p> keystrokes</text> output</text> userspace</text> kernel</text> userspace</text> SSH uses the same PTY mechanism -- sshd holds the master, the remote shell gets the slave</text> </svg> The PTY pair connects a terminal emulator to a shell through the kernel. The line discipline sits in the middle, processing input and output.</div> </div> </figure> The Line Discipline</h2> Between the master and slave sides of the PTY sits the line discipline</strong> -- a kernel module that processes characters as they flow through the terminal. The line discipline is what makes a terminal feel like a terminal rather than a raw data pipe.</p> In its default mode (called canonical mode</strong> or cooked mode</strong>), the line discipline provides:</p> Line editing.</strong> When you type a character and press backspace, the line discipline removes the last character. You are editing a line buffer inside the kernel. The program on the other end does not see any of this -- it only receives a complete line when you press Enter.</p> Echo.</strong> When you type a character, the line discipline sends a copy back to the terminal so you can see what you typed. This is why you see your own keystrokes. Turn off echo (as passwd</code> does), and you type blind.</p> Signal generation.</strong> Certain key combinations are intercepted by the line discipline and converted to signals:</p> Ctrl+C generates SIGINT (interrupt the foreground process)</li> Ctrl+Z generates SIGTSTP (suspend the foreground process)</li> Ctrl+\ generates SIGQUIT (quit with core dump)</li> Ctrl+D on an empty line signals end-of-file</li> </ul> These key bindings are not hard-coded -- they are configurable with the stty</code> command. You can see the current settings:</p> stty -a </code></pre> The line discipline is the invisible layer that makes terminals usable. It handles line editing, character echo, and signal generation. Programs can disable it (raw mode) when they need to handle every keystroke themselves. </div> Raw Mode</h3> Some programs need to handle every keystroke directly -- text editors like vim, interactive programs like top, and anything that uses arrow keys for navigation. These programs switch the terminal to raw mode</strong> (technically, they disable canonical mode and echo). In raw mode, the line discipline does nothing: every keystroke is passed immediately to the program without buffering, editing, or signal translation.</p> This is why pressing Ctrl+C in vim does not kill vim. Vim has put the terminal in raw mode and handles Ctrl+C itself.</p> How Keystrokes Reach Your Shell</h2> Let us trace what happens when you press the letter "a" in a terminal emulator:</p> Your operating system's window manager detects the key press and sends a key event to the terminal emulator application.</li> The terminal emulator translates the key event to the byte 0x61</code> (ASCII for "a").</li> The terminal emulator writes 0x61</code> to the PTY master (/dev/ptmx</code>).</li> The kernel's line discipline receives the byte. In canonical mode, it adds it to the line buffer and echoes it back to the master (so the terminal emulator can display it).</li> When you press Enter (0x0A</code>), the line discipline marks the line as complete.</li> The shell, reading from the PTY slave (/dev/pts/0</code>), receives the complete line.</li> The shell processes the command and writes its output to the PTY slave.</li> The output passes through the line discipline (which does minimal processing on output) to the PTY master.</li> The terminal emulator reads the output from the master and renders it on screen.</li> </ol> Fig. 16c -- Journey of a keystroke</div> A keystroke makes a round trip: from the emulator to the kernel, through the line discipline, to the shell, and back. Echo happens before the shell ever sees the character.</div> </div> </figure> Terminal Escape Codes</h2> A terminal does more than display plain text. It can position the cursor, change text color, clear the screen, and scroll regions. It does all of this through escape sequences</strong> -- special byte sequences that the terminal interprets as commands rather than text.</p> An escape sequence begins with the ESC character (byte 0x1B</code>), followed by [</code>, then parameters and a command letter. These are called ANSI escape codes</strong> because they were standardized by ANSI (the American National Standards Institute) in the 1970s.</p> Some examples:</p> Sequence</th> Effect</th></tr></thead> ESC[2J</code></td> Clear the entire screen</td></tr> ESC[H</code></td> Move cursor to top-left</td></tr> ESC[31m</code></td> Set text color to red</td></tr> ESC[0m</code></td> Reset all formatting</td></tr> ESC[10;20H</code></td> Move cursor to row 10, column 20</td></tr> ESC[1A</code></td> Move cursor up one line</td></tr> </tbody></table> When a program like ls --color</code> outputs colored filenames, it is embedding these escape sequences in its output. The terminal emulator intercepts them and changes the rendering instead of displaying the raw bytes.</p> Key term: Escape sequence</strong> A series of bytes beginning with ESC (0x1B) that instructs a terminal to perform an action like moving the cursor, changing colors, or clearing the screen. Programs use these to create interactive text interfaces. </div> This is also why piping colored output to a file produces garbled-looking text -- the file contains the raw escape sequences, which only make sense to a terminal.</p> # This shows escape codes as visible text ls --color=always | cat -v </code></pre> The Terminal and Job Control</h2> The terminal layer is deeply connected to process management. Each terminal has a concept of a foreground process group</strong> -- the set of processes currently allowed to read from the terminal and receive keyboard signals.</p> When you press Ctrl+C, the kernel does not send SIGINT to every process. It sends it to the foreground process group of the controlling terminal. When you press Ctrl+Z, SIGTSTP goes to the foreground process group.</p> This is how job control works in the shell. When you run a command, the shell puts it in the foreground. When you press Ctrl+Z, it gets stopped. When you type bg</code>, the shell moves it to a background process group. Background processes that try to read from the terminal receive SIGTTIN and get stopped -- they are not allowed to compete with the foreground for terminal input.</p> Fig. 16d -- Terminal sessions and process groups</div> SIGINT</text> SIGTSTP</text> </svg></p> A terminal session contains one foreground and zero or more background process groups. Keyboard signals only reach the foreground group.</div> </div> </figure> The TTY subsystem is not just about displaying text. It manages which processes can read from the terminal, delivers keyboard-generated signals, and provides line editing. Understanding this layer explains why Ctrl+C, Ctrl+Z, and job control work the way they do. </div> SSH and Remote Terminals</h2> SSH uses the same PTY mechanism. When you SSH into a remote machine, the sshd daemon on the remote side allocates a PTY pair. sshd holds the master, and the remote shell gets the slave. Your keystrokes travel encrypted over the network to sshd, which writes them to the master. Output travels back the same way.</p> This is why interactive programs work over SSH -- the remote shell sees a real PTY slave and behaves exactly as if you were sitting at a local terminal. The TERM</code> environment variable (usually set to xterm-256color</code> or similar) tells programs which escape sequences the terminal understands.</p> Inspecting Your Terminal</h2> A few commands for examining the terminal system:</p> tty # Print your current terminal device stty -a # Show all terminal settings who # Show logged-in users and their terminals w # Show who is logged in and what they are doing ls -la /dev/pts/ # List active pseudoterminals echo $TERM # Show the terminal type </code></pre> The stty</code> command is particularly useful. It shows the current line discipline settings: which characters trigger signals, whether echo is on, the terminal dimensions (rows and columns), and the baud rate (a historical artifact that is always 38400 for PTYs).</p> Why This Still Matters</h2> Every time you open a terminal, type a command, press Ctrl+C to interrupt something, or SSH into a server, the TTY subsystem is doing the work. The abstraction is so good that you rarely think about it. But when something goes wrong -- when a program leaves your terminal in a broken state, when Ctrl+C does not work, when characters display as garbage -- understanding the PTY, the line discipline, and escape codes gives you the knowledge to fix it.</p> If your terminal ever gets into a bad state (no echo, garbled display), the fix is:</p> reset </code></pre> This sends the terminal reset escape sequence and restores the line discipline to sane defaults. It works because it operates at the same level as the problem -- the TTY layer.</p> Next: Users, Groups, and Permissions</a></p> The Process Model 2025-01-19T00:00:00+00:00 Every program running on your computer right now -- your browser, your text editor, the background service checking for software updates -- is a process. A process is the kernel's abstraction for "a running program." It is the unit of work that the operating system tracks, schedules, and controls.</p> The previous article described PID 1, the first process. But how does PID 1 start everything else? How does any process start another? The answer lies in three system calls that have defined Unix process management since the 1970s: fork()</code>, exec()</code>, and wait()</code>.</p> What a Process Is</h2> Think of a process as a sealed envelope. Inside the envelope is everything needed to run a program: the program's code, its current state, the data it is working on, and a record of which files it has open. The kernel keeps a table of all these envelopes and decides which one gets to use the CPU at any given moment.</p> Each process has:</p> A PID</strong> -- a unique integer identifier</li> A parent PID (PPID)</strong> -- the PID of the process that created it</li> A memory space</strong> -- its own private region of RAM</li> An execution state</strong> -- running, sleeping, stopped, or zombie</li> Open file descriptors</strong> -- references to files, sockets, and pipes</li> Credentials</strong> -- the user and group it runs as</li> </ul> Key term: Process</strong> An instance of a running program. The kernel assigns each process its own memory space, a unique PID, and a set of resources. Multiple processes can run the same program simultaneously -- each is an independent instance. </div> Fig. 15a -- Anatomy of a process</div> Each process is a self-contained unit with its own code, data, state, file descriptors, and saved CPU registers. The kernel manages all of these.</div> </div> </figure> The Process Table</h2> The kernel maintains a data structure called the process table</strong> (internally, a list of task_struct</code> structures in Linux). Every process on the system has an entry. You can see a snapshot of it with the ps</code> command:</p> ps aux </code></pre> This shows every process, its PID, its parent, how much CPU and memory it uses, and the command that started it. On a typical desktop system, you will see hundreds of entries. On a busy server, thousands.</p> Each entry costs memory. The kernel allocates a task_struct</code> (about 6 KB on a 64-bit Linux system) plus the process's page tables and kernel stack. This is why creating processes is not free -- though on modern systems, the overhead is small enough that you rarely worry about it.</p> fork(): Cloning a Process</h2> The fundamental operation in Unix process creation is fork()</code>. When a process calls fork()</code>, the kernel creates a near-exact copy of the calling process. The new process -- called the child</strong> -- gets a new PID, but everything else is a duplicate: the same code, the same data, the same open files, the same position in the program.</p> Think of it like photocopying a document. You now have two copies that are identical at the moment of copying, but changes to one do not affect the other.</p> After fork()</code> returns, two processes are running the same code at the same point. The only difference is the return value: the parent receives the child's PID, and the child receives zero. This is how each process knows which one it is.</p> pid_t pid = fork(); if (pid == 0) { // This is the child process printf("I am the child, PID %d\n", getpid()); } else { // This is the parent process printf("I am the parent, child PID is %d\n", pid); } </code></pre> Key term: fork()</strong> A system call that creates a new process by duplicating the calling process. The new child process has its own PID and memory space, but starts as an exact copy of its parent. Modern Linux uses copy-on-write to make this efficient. </div> In practice, the kernel does not actually copy all the memory immediately. It uses a technique called copy-on-write (COW)</strong>: both parent and child share the same physical memory pages, marked read-only. Only when one of them tries to write does the kernel copy just the affected page. This makes fork()</code> fast even for processes using gigabytes of memory.</p> Fig. 15b -- fork() creates a child process</div> Before fork()</text> After fork()</text></p> PID 500</text> code + data</text> memory: 40 MB</text> 3 open files</text> Parent PID 500</text> fork() returned 501</text> code + data (shared)</text> Child PID 501</text> fork() returned 0</text> code + data (shared)</text> Shared</text> pages</text> (COW)</text> fork()</text> </svg></p> fork() splits one process into two. The parent and child share physical memory through copy-on-write until one of them modifies a page.</div> </div> </figure> exec(): Replacing a Process</h2> fork()</code> alone just creates copies. To run a different program, a process calls one of the exec()</code> family of functions (execve</code>, execl</code>, execvp</code>, etc.). This replaces the process's code, data, and stack with a new program loaded from disk. The PID stays the same. The open file descriptors stay the same (unless marked close-on-exec). But the running program is entirely different.</p> This is the standard Unix pattern for starting a new program:</p> The parent calls fork()</code> to create a child.</li> The child calls exec()</code> to replace itself with the new program.</li> The parent continues running.</li> </ol> This two-step approach seems roundabout, but it is powerful. Between the fork()</code> and the exec()</code>, the child can rearrange its file descriptors, change its working directory, drop privileges, or set up any other environment the new program needs -- all without affecting the parent.</p> The fork-then-exec pattern is the foundation of process creation in Unix. fork() creates a copy, exec() replaces it with a new program. This two-step design lets the child set up its environment before the new program starts. </div> wait(): Collecting the Dead</h2> When a child process exits, it does not vanish immediately. The kernel retains its exit status -- a small integer indicating whether it succeeded (0) or failed (nonzero) -- in the process table. The parent must call wait()</code> or waitpid()</code> to read this status and release the entry.</p> This is the contract: the parent that created the child is responsible for collecting its exit status. Until it does, the dead child remains in the process table as a zombie</strong>.</p> pid_t pid = fork(); if (pid == 0) { // Child does its work, then exits exit(0); } else { // Parent waits for the child to finish int status; waitpid(pid, &status, 0); printf("Child exited with status %d\n", WEXITSTATUS(status)); } </code></pre> Process States</h2> A process is always in one of several states. The kernel tracks the current state and transitions the process between them:</p> Fig. 15c -- Process state transitions</div> A process moves through these states during its lifetime. The scheduler decides which ready process gets the CPU. A zombie exists only to hold an exit status until the parent reads it.</div> </div> </figure> Running</strong> -- currently executing on a CPU core. On a single-core system, only one process can be running at a time. On a multi-core system, one per core.</p> Ready (Runnable)</strong> -- able to run, waiting for the scheduler to give it a CPU time slice.</p> Sleeping</strong> -- waiting for something external. This is the most common state. A process reading from a network socket sleeps until data arrives. A process waiting for disk I/O sleeps until the read completes. There are two variants: interruptible sleep (can be woken by signals) and uninterruptible sleep (cannot).</p> Stopped</strong> -- paused by a signal, typically SIGSTOP or SIGTSTP (what happens when you press Ctrl+Z in a terminal). The process remains in memory but does not execute. SIGCONT resumes it.</p> Zombie</strong> -- the process has exited, but its parent has not yet called wait()</code>. The process table entry remains so the parent can read the exit status.</p> Zombies and Orphans</h2> These two situations arise from the parent-child relationship.</p> Zombie Processes</h3> A zombie is a process that has finished executing but still has an entry in the process table. The entry is tiny -- just enough to store the PID and exit status -- but it is still a slot in the kernel's table. A few zombies are harmless. Thousands of them can exhaust the PID space.</p> You can spot zombies in ps</code> output by the state letter Z</code>:</p> ps aux | grep Z </code></pre> The fix is always the same: the parent must call wait()</code>. If the parent is your code, fix it. If the parent is a third-party program, the only remedy is to kill the parent. When the parent dies, the zombies are adopted by PID 1, which reaps them immediately.</p> Orphan Processes</h3> An orphan is a process whose parent has exited. The kernel does not allow a process to have no parent, so it reassigns the orphan to PID 1 (the init process). PID 1 is required to periodically call wait()</code> for any adopted children, which prevents orphans from becoming permanent zombies.</p> This is one of PID 1's special responsibilities, as we discussed in the previous article. If PID 1 fails to reap orphaned zombies, the system slowly accumulates dead process entries that can never be cleaned up.</p> Key term: Zombie process</strong> A process that has exited but whose parent has not yet collected its exit status via wait(). It occupies a slot in the process table but consumes no CPU or memory beyond that small entry. The name comes from the fact that it is dead but still present. </div> The Process Tree</h2> Because every process (except PID 1) has a parent, the entire set of processes forms a tree. PID 1 is the root. You can see this tree with:</p> pstree -p </code></pre> On a typical system, the tree looks something like this: PID 1 (systemd) starts a login manager, which starts your desktop session, which starts a terminal emulator, which starts your shell, which starts the commands you type. Each layer is a parent-child relationship created by fork() and exec().</p> Fig. 15d -- The process tree</div> Every process has a parent, forming a tree rooted at PID 1. Your shell, your editor, and every command you run are branches on this tree.</div> </div> </figure> Signals</h2> Processes communicate with each other and with the kernel through signals</strong> -- small, numbered notifications. When a process receives a signal, it can handle it (run a custom function), ignore it, or accept the default behavior (which is usually to terminate).</p> The most commonly encountered signals:</p> Signal</th> Number</th> Default</th> Typical Source</th></tr></thead> SIGTERM</td> 15</td> Terminate</td> kill</code> command</td></tr> SIGKILL</td> 9</td> Terminate (cannot be caught)</td> kill -9</code></td></tr> SIGINT</td> 2</td> Terminate</td> Ctrl+C</td></tr> SIGTSTP</td> 20</td> Stop</td> Ctrl+Z</td></tr> SIGCONT</td> 18</td> Continue</td> fg</code> or bg</code></td></tr> SIGHUP</td> 1</td> Terminate</td> Terminal closed</td></tr> SIGCHLD</td> 17</td> Ignore</td> Child process exited</td></tr> SIGSEGV</td> 11</td> Terminate + core dump</td> Invalid memory access</td></tr> </tbody></table> Two signals cannot be caught or ignored: SIGKILL (9) and SIGSTOP (19). These are the kernel's guarantee that any process (except PID 1) can always be stopped or killed.</p> Signals are the Unix mechanism for process-to-process and kernel-to-process notification. SIGKILL and SIGSTOP cannot be caught or ignored. Every other signal can be handled by the receiving process. </div> Seeing It in Action</h2> You can observe the process model with a few commands:</p> # Show the process tree pstree -p # Show all processes with details ps aux # Watch processes in real time top # Trace the system calls a process makes (including fork and exec) strace -f -e trace=process bash -c "ls" </code></pre> The strace</code> example is particularly illuminating. You will see your shell call clone()</code> (Linux's version of fork()</code>), then the child call execve("/usr/bin/ls", ...)</code>. The parent calls wait4()</code> to collect the result. This is the fork-exec-wait pattern playing out in real time, exactly as described.</p> Why This Matters</h2> Every time you type a command at a shell prompt, the shell forks a child, the child execs the command, and the shell waits for it to finish. Every time a web server handles a request (in the traditional model), it forks a worker. Every time you start a program from a GUI menu, a process calls fork() and exec().</p> The process model is the foundation on which everything else in the operating system rests. Pipes, redirections, job control, services, containers -- they all build on these three system calls. Understanding fork(), exec(), and wait() is understanding how Unix works.</p> Next: The TTY and Terminal</a></p> PID 1: init and systemd 2025-01-18T00:00:00+00:00 In the previous articles, the kernel initialized hardware, loaded drivers, and mounted the root filesystem. The kernel itself is not a program you interact with. It is the platform on which programs run. Now it needs to hand control to a program that will build the rest of the system -- start services, mount additional filesystems, configure networking, and eventually present you with a login prompt.</p> That program is process number one. Every other process on the system descends from it.</p> The Handoff</h2> At the very end of kernel initialization, the kernel calls a function named kernel_init</code>. This function does one thing: it looks for an executable to run as the first userspace process. The kernel searches a short list of paths in order:</p> Whatever was passed as init=</code> on the kernel command line</li> /sbin/init</code></li> /etc/init</code></li> /bin/init</code></li> /bin/sh</code> (last resort)</li> </ol> When it finds one, the kernel creates a new process, assigns it PID 1</strong> (process identifier 1), and executes the program. From this moment forward, the kernel's job is to provide system calls and manage hardware. PID 1 is responsible for everything else.</p> Key term: PID</strong> A process identifier. The kernel assigns a unique integer to every running process. PID 1 is always the init process -- the first userspace program started by the kernel. </div> Fig. 14a -- The kernel to userspace handoff</div> exec()</text></p> kernel space | userspace</text> </svg></p> The kernel searches for an init binary, then executes it as PID 1. This is the boundary between kernel initialization and userspace.</div> </div> </figure> Why PID 1 Is Special</h2> PID 1 is not just the first process. The kernel treats it differently from every other process on the system.</p> It cannot be killed.</strong> The kernel will not deliver fatal signals to PID 1 unless PID 1 has explicitly registered a handler for that signal. If you run kill -9 1</code> as root, nothing happens. On any other process, signal 9 (SIGKILL) is instant, unavoidable death. PID 1 is immune. If PID 1 dies, the kernel panics and the system halts.</p> It adopts orphans.</strong> When a process dies, its children need a parent. The kernel reassigns orphaned children to PID 1. This prevents the process table from filling with unmanaged entries.</p> It reaps zombies.</strong> When a child process exits, it lingers as a zombie until its parent reads the exit status. PID 1 must periodically call wait()</code> to collect these dead children. If it does not, zombie processes accumulate.</p> These responsibilities make PID 1 the most critical userspace program on the system. Everything else can crash and be restarted. PID 1 must keep running from the moment it starts until the system shuts down.</p> PID 1 has three unique responsibilities: it bootstraps all other services, it adopts orphaned child processes, and it reaps zombie processes. If PID 1 dies, the kernel panics. </div> A Brief History of Init</h2> The concept of a first process dates back to the earliest Unix systems in the 1970s. Over the decades, several implementations have competed for the role.</p> SysV Init</h3> The original System V init, dating to the early 1980s, is the simplest design. It reads a file called /etc/inittab</code> that defines runlevels</strong> -- numbered operating modes for the system.</p> Runlevel</th> Purpose</th></tr></thead> 0</td> Halt (shut down)</td></tr> 1</td> Single-user mode (maintenance)</td></tr> 2</td> Multi-user, no networking</td></tr> 3</td> Multi-user with networking</td></tr> 5</td> Multi-user with networking and GUI</td></tr> 6</td> Reboot</td></tr> </tbody></table> SysV init starts services by running shell scripts sequentially. The scripts live in directories like /etc/rc3.d/</code> (for runlevel 3). Each script is named with a prefix like S01</code> or K99</code> -- the letter indicates start or kill, and the number sets the order.</p> This design is simple to understand but slow. Services start one at a time, in a fixed order, even when they have no dependency on each other. On modern hardware with dozens of services, booting with SysV init takes noticeably longer than it should.</p> Key term: Runlevel</strong> A numbered operating mode in SysV init. Each runlevel defines which services should be running. Switching runlevels starts and stops services to match the target configuration. </div> Upstart</h3> Ubuntu introduced Upstart in 2006 as an event-driven replacement for SysV init. Instead of fixed runlevels and sequential scripts, Upstart could react to events -- "when the network is up, start the web server." This was a significant improvement, but Upstart's event model was complex and its adoption remained limited to Ubuntu and a few other distributions.</p> systemd</h3> In 2010, Lennart Poettering and Kay Sievers created systemd. By 2015, virtually every major Linux distribution had adopted it. systemd is what you are almost certainly running today.</p> systemd replaced more than just the init process. It absorbed functionality that was previously scattered across dozens of separate tools: service management, logging, device management, login tracking, time synchronization, DNS resolution, and more. This consolidation is both its greatest strength and its most frequent criticism.</p> How systemd Works</h2> systemd organizes the system around units</strong>. A unit is a configuration file that describes a resource the system manages. There are several types:</p> Unit Type</th> File Extension</th> Purpose</th></tr></thead> Service</td> .service</code></td> A daemon or one-shot program</td></tr> Target</td> .target</code></td> A group of units (replaces runlevels)</td></tr> Mount</td> .mount</code></td> A filesystem mount point</td></tr> Socket</td> .socket</code></td> A network or IPC socket</td></tr> Timer</td> .timer</code></td> A scheduled task (replaces cron)</td></tr> Device</td> .device</code></td> A kernel device</td></tr> </tbody></table> Unit files live in /usr/lib/systemd/system/</code> (distribution defaults) and /etc/systemd/system/</code> (administrator overrides). When both exist for the same unit, the /etc</code> version wins.</p> Fig. 14b -- systemd unit dependency tree</div> --- requires/depends on</text> </svg></p> systemd resolves a tree of dependencies. Targets group related services. Services declare what they need, and systemd starts them in parallel where possible.</div> </div> </figure> Targets Replace Runlevels</h3> systemd does not use runlevels. Instead, it uses targets</strong> -- units that group other units together. The mapping is roughly:</p> SysV Runlevel</th> systemd Target</th></tr></thead> 0</td> poweroff.target</code></td></tr> 1</td> rescue.target</code></td></tr> 3</td> multi-user.target</code></td></tr> 5</td> graphical.target</code></td></tr> 6</td> reboot.target</code></td></tr> </tbody></table> The default target is usually graphical.target</code> on desktop systems and multi-user.target</code> on servers. You can check your system's default with:</p> systemctl get-default </code></pre> Parallel Startup</h3> The most visible improvement systemd brings is parallelism. SysV init starts services one at a time. systemd reads all the unit files, builds a dependency graph, and starts everything it can simultaneously.</p> If service A and service B have no dependency on each other, they start at the same time. If service C depends on A, it waits only for A and starts as soon as A reports ready. This can cut boot times in half or better.</p> A Service Unit File</h3> Here is a typical service unit file:</p> [Unit] Description=OpenSSH Daemon After=network.target [Service] Type=notify ExecStart=/usr/sbin/sshd -D ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure [Install] WantedBy=multi-user.target </code></pre> The [Unit]</code> section declares ordering -- this service should start after the network is available. The [Service]</code> section defines how to run the program. Type=notify</code> means the service will explicitly tell systemd when it is ready. Restart=on-failure</code> means systemd will restart it if it crashes. The [Install]</code> section says that when this service is enabled, it becomes part of multi-user.target</code>.</p> Managing Services with systemctl</h2> systemctl</code> is the command you use to interact with systemd. The essential operations:</p> systemctl start sshd # Start a service now systemctl stop sshd # Stop a service now systemctl restart sshd # Stop, then start systemctl status sshd # Show current state systemctl enable sshd # Start at boot systemctl disable sshd # Do not start at boot systemctl is-active sshd # Check if running </code></pre> enable</code> and disable</code> control what happens at boot. start</code> and stop</code> control what is happening right now. They are independent -- you can start a service without enabling it, or enable it without starting it immediately.</p> Fig. 14c -- Service lifecycle in systemd</div> systemd automatically restarts failed services when configured</text> </svg></p> A service moves through states. systemd can automatically restart failed services, creating a self-healing loop.</div> </div> </figure> The Boot Sequence Under systemd</h2> When the kernel starts systemd as PID 1, the boot proceeds through a predictable chain of targets:</p> sysinit.target</strong> -- mount essential filesystems, load kernel modules, activate swap, seed the random number generator.</li> basic.target</strong> -- start logging, timers, and socket activation.</li> multi-user.target</strong> -- start all non-graphical services: networking, SSH, databases, web servers.</li> graphical.target</strong> (if configured) -- start the display manager and desktop environment.</li> </ol> Each target pulls in the targets before it through dependency chains. systemd resolves the entire graph at once and starts everything it can in parallel, only serializing where an explicit ordering exists.</p> systemd replaced sequential shell scripts with a dependency graph. Services declare what they need, and systemd starts everything in parallel where possible. This is why modern Linux boots fast. </div> The Journal</h2> systemd includes its own logging system called the journal</strong>, managed by journald</code>. Instead of plain text files in /var/log/</code>, the journal stores structured, indexed, binary log entries.</p> journalctl -u sshd # Logs from sshd journalctl -b # Logs from current boot journalctl --since "1 hour ago" journalctl -f # Follow new entries (like tail -f) </code></pre> The journal captures both stdout and stderr from every service. This means you do not need to configure logging in each service individually -- systemd captures it all.</p> Controversy and Alternatives</h2> systemd's expansion beyond a simple init replacement has drawn criticism. Some argue it violates the Unix philosophy of small, focused tools. Others worry about the complexity of depending on a single project for so many system functions.</p> Alternatives exist. OpenRC</strong> is used by Gentoo and Alpine Linux. runit</strong> is used by Void Linux. s6</strong> is a supervision suite popular in container images. Each takes a more minimal approach, managing services without absorbing logging, networking, and device management.</p> For learning purposes, understanding systemd is essential -- it runs on the vast majority of Linux installations. But it is worth knowing that PID 1 does not have to be this complex. The kernel only requires that something runs as PID 1, reaps zombies, and adopts orphans. Everything else is a design choice.</p> What PID 1 Means for You</h2> When you sit down at a Linux system and see a login prompt, that prompt exists because PID 1 started a chain of services that made it possible. The network works because PID 1 started NetworkManager. Your SSH sessions work because PID 1 started sshd. Your disk is mounted because PID 1 triggered the mount units.</p> If a service is not running, systemctl status</code> will tell you why. If the system will not boot, understanding the target chain tells you where to look. PID 1 is the root of the entire userspace tree, and every other process on the system exists because, directly or indirectly, PID 1 started it.</p> Next: The Process Model</a></p> The Root Filesystem 2025-01-17T00:00:00+00:00 The kernel has initialized its subsystems, loaded its built-in drivers, and set up memory management and scheduling. But it still cannot do most of what an operating system needs to do. It cannot run programs from disk. It cannot read configuration files. It cannot load additional driver modules. All of these require a filesystem -- a structured way to organize, name, and access files stored on a device.</p> The first filesystem the kernel mounts is called the root filesystem. It is mounted at /</code> -- the single forward slash that is the top of the entire directory tree. Every file path on a Linux system starts from this point. Before the root filesystem exists, the system has no /bin</code>, no /etc</code>, no /home</code>. It has nothing.</p> This article explains how the kernel gets from "no filesystem at all" to a fully mounted root, and the intermediate step that makes it possible.</p> The Chicken-and-Egg Problem</h2> To mount the root filesystem, the kernel needs a storage driver -- the software that knows how to talk to the disk. But many storage drivers are compiled as loadable modules, stored as files on the root filesystem. You need the driver to read the disk. You need the disk to load the driver.</p> This circular dependency is one of the classic problems of operating system design. Linux solves it with a temporary filesystem that the bootloader loads into RAM alongside the kernel itself. This temporary filesystem is called the initramfs.</p> Key term: initramfs (initial RAM filesystem)</strong> A small, compressed filesystem archive that the bootloader loads into memory alongside the kernel. It contains the essential drivers, tools, and scripts needed to find and mount the real root filesystem. Once the real root is mounted, the initramfs is discarded. </div> Think of it like a toolbox you carry with you to a construction site. The site has all the heavy equipment, but the gate is locked. Your toolbox has the key. Once you open the gate and get the heavy equipment running, you do not need the toolbox anymore.</p> What is Inside the initramfs</h2> The initramfs is a compressed cpio archive -- a simple format that packs files and directories into a single stream. When the kernel starts, it unpacks this archive into a temporary in-memory filesystem (a tmpfs) and uses it as the initial root.</p> A typical initramfs contains:</p> Kernel modules</strong> for the storage controller (NVMe, AHCI, SCSI, USB storage)</li> Filesystem modules</strong> (ext4, XFS, Btrfs) so the kernel can understand the root partition's format</li> A small init script or binary</strong> that orchestrates the mounting process</li> Essential tools</strong> like busybox</code> (a single binary that provides dozens of standard Unix utilities)</li> Device manager</strong> components (a minimal udev or mdev) to create /dev entries for detected hardware</li> Cryptographic modules</strong> if the root partition is encrypted (LUKS/dm-crypt)</li> </ul> You can examine your system's initramfs:</p> $ lsinitramfs /boot/initrd.img-6.8.0-generic | head -10 . bin bin/busybox bin/cat bin/cp conf conf/initramfs.conf etc etc/modprobe.d lib </code></pre> The entire image is typically 20-60 MB compressed, expanding to perhaps 100-200 MB when unpacked. This is tiny compared to a full root filesystem, which may be tens of gigabytes. The initramfs carries only what is needed to get to the real root.</p> Fig. 13a -- initramfs boot sequence</div> The bootloader places both the kernel and the initramfs in RAM. The kernel unpacks the archive into a tmpfs, runs the /init script inside it, loads drivers, finds the real root device, mounts it, then switches root.</div> </div> </figure> The /init Script</h2> When the kernel finishes its own initialization, it looks for a program to execute as PID 1 -- the first userspace process. If an initramfs is present, the kernel runs /init</code> from within it. This is typically a shell script (on Debian/Ubuntu systems using initramfs-tools) or a compiled binary (on systems using dracut).</p> The init script performs a precise sequence:</p> Mount the /proc</code> and /sys</code> pseudo-filesystems so it can communicate with the kernel.</li> Start a minimal device manager (udev or mdev) to create /dev</code> entries for detected hardware.</li> Load the storage driver module for the root device.</li> Load the filesystem module for the root partition's format.</li> If the root is encrypted, prompt for the passphrase and set up the decryption layer.</li> If the root is on LVM or RAID, assemble the logical volume or array.</li> Mount the real root filesystem on a temporary mount point (like /root</code> or /mnt</code>).</li> Call switch_root</code> to replace the initramfs with the real root filesystem.</li> </ol> The initramfs /init script is a carefully ordered sequence. Each step enables the next. Missing a step -- failing to load the right storage driver, for example -- means the real root cannot be mounted, and the boot halts with the dreaded "unable to mount root fs" panic. </div> switch_root: The Handoff</h2> The switch_root</code> command (or pivot_root</code>, an older mechanism) performs the transition from the initramfs to the real root filesystem. It does three things:</p> Deletes everything in the initramfs to free the memory it was using.</li> Moves the mount of the real root filesystem from its temporary mount point to /</code>.</li> Executes the real init program (typically /sbin/init</code> or systemd</code>) from the new root.</li> </ol> After switch_root</code>, the initramfs is gone. The system's /</code> is now backed by the real disk partition. The real init program takes over as PID 1 and begins starting system services.</p> Key term: switch_root</strong> A command that transitions from the initramfs to the real root filesystem. It frees the initramfs memory, moves the real root mount to /, and executes the real init binary. This is a one-way operation -- there is no going back to the initramfs. </div> The Old Way: initrd</h2> Before initramfs, Linux used initrd (initial RAM disk). An initrd was a disk image -- a file containing a complete filesystem, formatted as ext2 or similar, loaded into a RAM-based block device. The kernel would mount this block device as the initial root.</p> initramfs replaced initrd because it is simpler and more flexible. An initramfs is just a cpio archive extracted into tmpfs. It does not need a block device driver, a filesystem driver, or a fixed size allocation. The kernel simply unpacks the archive into memory. Despite this, many systems still name the file initrd.img</code> for historical reasons, even though the contents are actually an initramfs cpio archive.</p> Filesystem Types</h2> The real root filesystem needs to be in a format the kernel understands. Linux supports dozens of filesystem types. The most common ones for root partitions:</p> ext4</strong> -- The fourth extended filesystem. The default for many distributions. It is mature, well-tested, and handles most workloads well. It uses a traditional block allocation scheme with inodes, block groups, and journals.</p> XFS</strong> -- Originally developed by SGI for large files and high throughput. Default on Red Hat Enterprise Linux and Fedora. Excels at parallel I/O on multi-core systems.</p> Btrfs</strong> -- A copy-on-write filesystem with built-in support for snapshots, checksums, compression, and multi-device spanning. Default on openSUSE and Fedora Silverblue. More features than ext4 or XFS, but also more complexity.</p> Each filesystem type is implemented as a kernel module (or built-in code). The initramfs must include the module for whatever filesystem type the real root partition uses. If you format your root as ext4, the initramfs needs ext4.ko</code>. Format it as Btrfs, and you need btrfs.ko</code>.</p> Fig. 13b -- The Virtual Filesystem Switch (VFS)</div> The VFS provides a single API for all filesystem types. Applications call open/read/write without knowing whether the file is on ext4, XFS, Btrfs, or an in-memory filesystem like tmpfs. The VFS dispatches each call to the correct filesystem implementation.</div> </div> </figure> The Virtual Filesystem Switch (VFS)</h2> Applications do not call ext4 or XFS code directly. They call generic kernel functions like open()</code>, read()</code>, write()</code>, and close()</code>. The layer that translates these generic calls into filesystem-specific operations is the Virtual Filesystem Switch, or VFS.</p> The VFS is an abstraction layer. It defines a set of standard operations that every filesystem must implement. When you call open("/etc/hostname")</code>, the VFS figures out which filesystem /etc/hostname</code> lives on, looks up that filesystem's implementation of the open operation, and calls it.</p> This is what makes it possible to mix filesystem types on a single system. Your root might be ext4 at /</code>, your home directory might be on a Btrfs partition mounted at /home</code>, and /tmp</code> might be a tmpfs in RAM. The VFS stitches them all into one seamless directory tree.</p> Key term: VFS (Virtual Filesystem Switch)</strong> The kernel layer that provides a uniform interface for all filesystem types. It defines standard operations (open, read, write, stat, etc.) and dispatches them to the appropriate filesystem implementation based on which mount point the file belongs to. Applications never interact with a specific filesystem directly. </div> The VFS maintains several key data structures:</p> Superblock</strong> -- represents a mounted filesystem. Contains metadata like block size, total size, and available space.</li> Inode</strong> -- represents a file or directory. Contains permissions, ownership, timestamps, and pointers to data blocks.</li> Dentry</strong> -- represents a directory entry, mapping a filename to an inode. The kernel caches dentries for fast path lookups.</li> File</strong> -- represents an open file, tied to a specific process. Contains the current read/write position and the operations table for that file's filesystem.</li> </ul> What "/" Actually Is</h2> The root filesystem is not just the first filesystem mounted. It is the anchor for everything else. Every other filesystem mount in the system attaches to a directory within the root filesystem's tree.</p> When you mount a partition at /home</code>, you are grafting that partition's directory tree onto the /home</code> directory of the root filesystem. The original contents of /home</code> (if any) on the root filesystem become hidden -- replaced by the contents of the mounted partition. Unmount the partition, and the original contents reappear.</p> Fig. 13c -- Mount points graft filesystems together</div> Three different filesystems -- ext4 on an NVMe partition, Btrfs on another partition, and tmpfs in RAM -- are grafted together into a single directory tree. Users and applications see one seamless hierarchy.</div> </div> </figure> This is the mount model. The root filesystem at /</code> is the trunk of the tree. Everything else is a branch grafted on. The kernel's mount table (visible in /proc/mounts</code>) records every active mount -- which device, which filesystem type, which mount point, and which options.</p> $ cat /proc/mounts | head -5 /dev/nvme0n1p2 / ext4 rw,relatime 0 0 /dev/nvme0n1p3 /home btrfs rw,relatime,compress=zstd 0 0 tmpfs /tmp tmpfs rw,nosuid,nodev 0 0 proc /proc proc rw,nosuid,nodev,noexec 0 0 sysfs /sys sysfs rw,nosuid,nodev,noexec 0 0 </code></pre> The root filesystem is not just one filesystem among many. It is the anchor point for the entire directory hierarchy. Every other mount attaches to a directory within it. Without a root filesystem, there is no place to attach anything, no way to find programs or configuration, and no operating system. </div> Pseudo-Filesystems</h2> Not everything mounted in the directory tree represents data on a disk. Linux uses pseudo-filesystems -- filesystems that exist only in memory and provide interfaces to kernel data structures.</p> proc</strong> (mounted at /proc</code>) -- Provides information about running processes and kernel state. /proc/cpuinfo</code> shows CPU details. /proc/meminfo</code> shows memory usage. Each running process has a directory under /proc/<pid>/</code>.</p> sysfs</strong> (mounted at /sys</code>) -- Exposes the kernel's device model as a directory hierarchy. Every device, driver, and bus has a directory with attribute files you can read or write.</p> tmpfs</strong> (often mounted at /tmp</code> and /run</code>) -- A filesystem that lives entirely in RAM. Files written to tmpfs are fast to access but disappear on reboot. The initramfs itself is a tmpfs.</p> devtmpfs</strong> (mounted at /dev</code>) -- A tmpfs where the kernel automatically creates device nodes for detected hardware. udev then applies rules to set permissions and create symlinks.</p> These pseudo-filesystems are just as "real" to programs as disk-backed filesystems. You open, read, and write them with the same system calls. The VFS makes them indistinguishable from files on disk.</p> Mounting the Root: Read-Only First</h2> The kernel typically mounts the root filesystem read-only during boot. This is a safety measure. If the system crashed previously, the filesystem might have inconsistencies -- writes that were in progress when power was lost. Mounting read-only prevents further damage while the system runs a consistency check.</p> The init system (systemd, OpenRC, or similar) later remounts the root filesystem read-write after verifying its integrity, either through a full filesystem check (fsck</code>) or by replaying the filesystem's journal. Journaling filesystems like ext4, XFS, and Btrfs maintain a log of in-progress operations. Replaying this journal recovers the filesystem to a consistent state without scanning every block.</p> [ 3.456789] EXT4-fs (nvme0n1p2): mounted filesystem with ordered data mode. Quota mode: none. [ 4.567890] EXT4-fs (nvme0n1p2): re-mounted. Quota mode: none. </code></pre> The first message shows the read-only mount. The second shows the remount as read-write. Between these two messages, the init system ran its checks and determined the filesystem was clean.</p> From Root to Running System</h2> With the root filesystem mounted read-write, the system finally has persistent storage. Programs can be loaded from /bin</code> and /usr/bin</code>. Configuration can be read from /etc</code>. Logs can be written to /var/log</code>. The operating system has a place to stand.</p> The next step is for PID 1 -- the init process that switch_root</code> launched -- to read its configuration and start bringing up system services: networking, logging, login prompts, and everything else that turns a booted kernel into a usable machine.</p> Next: PID 1: init and systemd</a></p> Device Drivers 2025-01-16T00:00:00+00:00 The kernel knows how to manage memory, schedule processes, and handle interrupts. But it does not inherently know how to talk to your specific network card, your particular SSD, or the exact GPU on your motherboard. Every piece of hardware has its own protocol -- its own set of registers, commands, and quirks. The software that translates between the kernel's generic interfaces and a specific device's protocol is called a device driver.</p> Without drivers, the kernel sees hardware as a collection of inscrutable chips on a circuit board. With drivers, it sees a network interface, a disk, a display. This article explains how drivers work, how they get loaded, and how the system creates the /dev</code> entries that let userspace programs interact with hardware.</p> What a Driver Actually Does</h2> Think of a driver as a translator at a diplomatic meeting. On one side is the kernel, which speaks a standard internal language: "read 4096 bytes from this block device" or "send this packet out this network interface." On the other side is the hardware, which speaks its own proprietary language: "write value 0x42 to register offset 0x18, then poll bit 3 of the status register until it goes high."</p> The driver translates between these two languages. It exposes a standard interface to the kernel -- a set of function pointers that implement operations like read, write, open, and close. Inside those functions, it talks to the hardware using whatever register-level protocol the manufacturer designed.</p> Key term: Device driver</strong> A piece of software that knows how to communicate with a specific piece of hardware. It implements the kernel's standard interface for that class of device (block device, network device, character device, etc.) and translates generic requests into hardware-specific register operations. </div> Fig. 12a -- The driver as translator</div> userspace</text> kernel</text> hardware</text></p> </marker> </defs> </svg> A read request flows from the application through the kernel's generic layer to the driver, which translates it into hardware-specific operations. The driver is the only component that knows the details of the NVMe protocol.</div> </div> </figure> This layered design means the kernel does not need to be rewritten every time a new piece of hardware appears. Someone writes a driver for the new device, the driver plugs into the kernel's standard interfaces, and everything above the driver works without modification.</p> Built-in vs. Loadable Modules</h2> A driver can be compiled directly into the kernel binary (built-in) or compiled as a separate file that the kernel loads at runtime (a loadable module). The choice has practical consequences.</p> Built-in drivers are available immediately at boot. The kernel needs certain drivers to get through early initialization -- the driver for the boot disk, for example, must be available before the kernel can mount the root filesystem. You cannot load a module from a disk you cannot yet read.</p> Loadable modules live in files with a .ko</code> extension (kernel object) on the root filesystem, typically under /lib/modules/<kernel-version>/</code>. The kernel loads them on demand -- when it detects hardware that needs a particular driver, or when an administrator explicitly requests it with modprobe</code>.</p> Key term: Loadable kernel module</strong> A compiled driver file (.ko) that can be loaded into or removed from a running kernel without rebooting. Modules let the kernel start small and add hardware support as needed, rather than carrying every possible driver in memory at all times. </div> Most Linux distributions compile a small set of essential drivers into the kernel and ship everything else as modules. A typical system has a few dozen built-in drivers and several thousand available as modules. Only the modules for hardware actually present in the system get loaded.</p> $ lsmod | head -5 Module Size Used by nvidia 2453504 42 snd_hda_intel 57344 4 iwlwifi 450560 1 nvme 49152 3 </code></pre> The lsmod</code> command shows currently loaded modules. Each module has a name, a size in memory, and a count of how many things are using it. A module with a use count of zero can be safely unloaded.</p> The Device Model</h2> The kernel maintains an internal model of every device in the system. This model organizes devices into a hierarchy that reflects how they are physically connected: the PCI bus hosts a network controller, which has a specific vendor and device ID, which is bound to a specific driver.</p> This hierarchy is exposed to userspace through the /sys</code> filesystem (sysfs). You can browse it to see every device the kernel knows about, which driver is bound to it, and what attributes it has.</p> Fig. 12b -- The device model hierarchy</div> Each device is bound to a driver. The kernel matches</text> devices to drivers using vendor/device ID tables.</text> </svg></p> The kernel's device model is a tree. Buses contain devices, and each device is bound to a driver. The binding is based on hardware identifiers -- the kernel looks up the device's vendor and product ID in each driver's supported-device table.</div> </div> </figure> When the kernel scans a bus (PCI, USB, platform) and discovers a device, it reads the device's identifier -- typically a vendor ID and device ID pair. It then searches all registered drivers for one that claims to support that identifier. If a match is found, the kernel calls the driver's probe function, which initializes the device and makes it available for use.</p> The kernel uses a match-and-bind model: devices advertise their identity, and drivers advertise which identities they support. The kernel's bus code matches them up automatically. This is why plugging in a USB device usually "just works" -- the kernel finds the matching driver and binds it without manual intervention. </div> /dev and Device Files</h2> Unix systems represent hardware devices as files in the /dev</code> directory. This is one of Unix's most powerful abstractions: programs that know how to read and write files can talk to hardware without knowing anything about how that hardware works internally.</p> There are two types of device files:</p> Block devices</strong> transfer data in fixed-size blocks. Disks and SSDs are block devices. Reading or writing any block is equally fast -- you can jump to any position without reading everything before it. Block devices are named things like /dev/sda</code> (first SCSI/SATA disk), /dev/nvme0n1</code> (first NVMe SSD), and /dev/sda1</code> (first partition on the first disk).</p> Character devices</strong> transfer data as a stream of bytes, one at a time. Serial ports, terminals, and random number generators are character devices. You read from them sequentially. /dev/tty</code> (the current terminal), /dev/null</code> (discards everything written to it), and /dev/urandom</code> (produces random bytes) are all character devices.</p> Key term: Device file (device node)</strong> A special file in /dev that represents a hardware device or virtual device. Programs interact with hardware by opening, reading, writing, and closing these files using standard file operations. The kernel maps these file operations to the appropriate driver functions. </div> Each device file has two numbers associated with it: a major number and a minor number. The major number identifies which driver handles the device. The minor number identifies which specific device that driver manages. You can see these numbers with ls -l</code>:</p> $ ls -l /dev/sda brw-rw---- 1 root disk 8, 0 Jan 14 10:00 /dev/sda </code></pre> The 8, 0</code> means major 8 (the sd driver), minor 0 (the first disk). /dev/sda1</code> would be 8, 1</code> -- same driver, different minor number for the first partition.</p> udev: Populating /dev Dynamically</h2> In early Linux systems, /dev</code> was a static directory containing thousands of device files for every conceivable piece of hardware, whether or not it was actually present. This was wasteful and confusing.</p> Modern Linux uses udev</strong>, a userspace daemon that creates and removes device files dynamically as hardware appears and disappears. When the kernel detects a new device -- at boot or when you plug something in -- it sends an event called a uevent. udev receives this event and creates the appropriate /dev</code> entry.</p> Fig. 12c -- udev device creation flow</div> When hardware appears, the kernel sends a uevent. udev receives it, applies rules to determine the device name and permissions, creates the /dev entry, and optionally creates symlinks for stable naming.</div> </div> </figure> udev also applies rules that control the device name, its permissions, and any symlinks. For example, udev rules create the stable symlinks under /dev/disk/by-uuid/</code> and /dev/disk/by-id/</code> that let you refer to a disk by its unique identifier rather than a name like sda</code> that might change between boots.</p> The rules files live in /etc/udev/rules.d/</code> and /lib/udev/rules.d/</code>. A rule looks like this:</p> SUBSYSTEM=="block", ATTR{size}!="0", ENV{ID_SERIAL}=="?*", \ SYMLINK+="disk/by-id/$env{ID_BUS}-$env{ID_SERIAL}" </code></pre> This says: for any block device with a non-zero size and a serial number, create a symlink using the bus type and serial number. This is how /dev/disk/by-id/ata-Samsung_SSD_860_S3Z9NB0K123456</code> gets created.</p> How Drivers Talk to Hardware</h2> At the lowest level, drivers communicate with hardware through two mechanisms: memory-mapped I/O and port I/O.</p> Memory-mapped I/O (MMIO)</strong> maps a device's registers into the system's address space. The driver reads and writes to specific memory addresses, but those addresses do not correspond to RAM -- they are wired to the device's control registers. Writing to address 0xFEA00004 might tell a network card to start transmitting. Reading from it might return the card's status.</p> Port I/O (PIO)</strong> uses special CPU instructions (in</code> and out</code> on x86) to communicate with devices through a separate I/O address space. This is an older mechanism, mostly used by legacy devices like PS/2 keyboards and serial ports.</p> Modern high-performance devices also use DMA -- Direct Memory Access. Instead of the CPU reading data from the device one word at a time, the device writes directly into RAM without CPU involvement. The CPU sets up a DMA transfer by telling the device where in memory to write, then does other work while the transfer happens. When the device finishes, it fires an interrupt to let the CPU know the data is ready.</p> Drivers bridge the gap between the kernel's abstract view of hardware (block devices, network interfaces, character devices) and the physical reality of register addresses, DMA buffers, and interrupt lines. The driver model lets the rest of the system treat all devices uniformly, regardless of how different they are at the hardware level. </div> The Class System</h2> Devices are also organized into classes based on their function. All network interfaces -- whether Ethernet, Wi-Fi, or cellular -- belong to the net</code> class. All block storage devices belong to the block</code> class. All input devices belong to the input</code> class.</p> Classes let userspace tools work with devices by function rather than by physical connection. The ip</code> command does not care whether your network interface is PCI, USB, or virtual. It uses the net</code> class interface. The mount</code> command does not care whether your disk is NVMe, SATA, or a network block device. It uses the block</code> class interface.</p> You can see device classes in /sys/class/</code>:</p> $ ls /sys/class/ block dmi hwmon misc net scsi_device tty dma gpio input mmc pci_bus scsi_host usb </code></pre> Each directory contains symlinks to every device of that class, regardless of how it is physically connected.</p> Drivers During Boot</h2> During the boot sequence, driver loading happens in two waves.</p> Wave 1: Built-in drivers.</strong> As the kernel initializes, it runs the init functions of all built-in drivers. These include drivers for the CPU itself, the interrupt controllers, the system timer, and the memory controller. Without these, nothing else can function.</p> Wave 2: Module loading.</strong> Once the kernel has mounted enough of a filesystem to read module files (typically from initramfs, which we cover in the next article), it can load additional drivers on demand. The kernel detects hardware on PCI, USB, and other buses, matches device IDs to module alias strings, and calls modprobe</code> to load the right module.</p> Fig. 12d -- Driver loading timeline during boot</div> CPU, IRQ, timer,</text> memory controller,</text> early console</text></p> initramfs</text> mounted</text> MODULE LOADING</text> Storage drivers,</text> filesystem modules,</text> network, GPU, USB</text></p> root</text> mounted</text> LATE MODULES</text> Bluetooth, audio,</text> webcam, sensors,</text> additional USB</text></p> udev processes events, creates /dev entries</text> </svg> Driver loading happens in waves. Built-in drivers run immediately. Modules load once the initramfs is available. Additional modules load after the real root filesystem is mounted.</div> </div> </figure> This staged loading is a chicken-and-egg solution. You need a storage driver to read module files from disk, but the storage driver itself might be a module. The answer is initramfs -- a temporary root filesystem loaded into RAM by the bootloader that contains the essential modules needed to access the real root. We cover initramfs in the next article.</p> Debugging Driver Problems</h2> When a device does not work, the first place to look is the kernel log:</p> $ dmesg | grep -i error [ 2.345] e1000e: probe of 0000:03:00.0 failed with error -5 </code></pre> The -5</code> is EIO</code> -- an I/O error during probe. The kernel also provides diagnostic information through /sys</code>. Every device directory in sysfs contains files like vendor</code>, device</code>, driver</code>, and uevent</code> that tell you exactly what the kernel sees and which driver (if any) is bound.</p> lspci -v</code> and lsusb -v</code> show detailed information about every PCI and USB device, including which driver is currently bound. If a device shows no driver, either the right module is not loaded or no driver exists for that hardware.</p> Key term: probe</strong> The function a driver calls to check whether it can handle a specific device and, if so, to initialize it. If probe succeeds, the driver is bound to the device. If it fails, the kernel tries other drivers or leaves the device unbound. </div> Drivers are the nervous system of the operating system. They bridge the abstract world of file operations and network sockets with the physical world of voltage levels, register writes, and DMA transfers. Every time you read a file, send a packet, or hear a sound, a driver is doing the translation.</p> Next: The Root Filesystem</a></p> The Clock Signal 2025-01-15T00:00:00+00:00 A CPU does not run continuously like water through a pipe. It advances in steps, like a marching band. Every step happens at the same instant, synchronized across billions of transistors. The thing that keeps them in lockstep is the clock signal -- a voltage that swings between high and low at a precise, unwavering frequency.</p> Without a clock, the CPU would not know when to read an instruction, when to add two numbers, or when to write a result to memory. The clock is the heartbeat of the machine. This article explains where it comes from, how it reaches every part of the processor, and what "clock speed" actually means.</p> The Crystal Oscillator</h2> The master clock starts with a small piece of quartz crystal, usually mounted in a metal can on the motherboard. Quartz has a useful physical property: when you apply voltage across it, it vibrates at a very precise frequency. This is the piezoelectric effect -- mechanical stress produces voltage, and voltage produces mechanical stress.</p> The crystal is cut to a specific shape and thickness that determines its natural resonant frequency. A typical motherboard crystal vibrates at around 25 MHz -- 25 million cycles per second. This is far slower than the CPU's final clock speed, but it is the starting point.</p> Key term: Crystal oscillator</strong> A circuit that uses the piezoelectric properties of a quartz crystal to generate an electrical signal at a very precise frequency. It provides the base timing reference for the entire computer. The frequency is stable across temperature changes and does not drift significantly over time. </div> Think of the crystal as a tuning fork. Strike a tuning fork and it vibrates at exactly 440 Hz every time, regardless of how hard you hit it. The quartz crystal does the same thing electrically. It produces a clean, stable wave that the rest of the system can rely on.</p> Fig. 11b-a -- From crystal to CPU clock</div> CPU cores 4.0 GHz</text> Memory 1.6 GHz</text> PCIe 100 MHz</text></p> 25 MHz base wave</text></p> </marker> </defs> </svg> The crystal provides a stable 25 MHz reference. A PLL multiplies this up to the CPU's operating frequency. A clock generator then divides this down to produce different frequencies for different parts of the system.</div> </div> </figure> The Phase-Locked Loop</h2> A 25 MHz crystal cannot directly clock a 4 GHz processor. You need a frequency multiplier. That is the job of the Phase-Locked Loop, or PLL.</p> A PLL is a feedback circuit that takes a low-frequency reference signal and generates a higher-frequency output that stays locked in phase with the input. "Locked in phase" means the output signal's edges line up precisely with the input signal's edges -- they do not drift apart over time.</p> Here is the basic idea. The PLL contains a voltage-controlled oscillator (VCO) that can run at high frequencies but is not very stable on its own. A feedback circuit divides the VCO's output frequency down and compares it with the crystal's reference signal. If the divided-down output runs too fast, the circuit slows the VCO. If it runs too slow, the circuit speeds it up. This feedback loop keeps the output frequency at an exact multiple of the reference.</p> Key term: Phase-Locked Loop (PLL)</strong> A feedback circuit that generates a high-frequency output signal locked to a lower-frequency reference. The output frequency is a precise integer multiple of the input. PLLs are used throughout a computer to create the various clock frequencies that different components need. </div> To get 4.0 GHz from a 25 MHz crystal, the PLL uses a multiplication factor of 160. The result is a high-frequency clock that is just as stable as the crystal, because any drift is immediately corrected by the feedback loop.</p> Modern CPUs contain multiple PLLs. One generates the core clock. Others generate clocks for the memory controller, the PCIe bus, and the integrated GPU. Each runs at a different frequency, but all are ultimately derived from the same crystal reference.</p> The Clock Tree</h2> The PLL produces a single clock signal at the target frequency. But a modern CPU die has billions of transistors spread across several square centimeters. The clock must reach every one of them at nearly the same instant. A delay of even a fraction of a nanosecond between two parts of the chip can cause incorrect computation.</p> The network of wires and buffers that distributes the clock signal across the chip is called the clock tree. It is one of the most carefully engineered structures in the processor.</p> Fig. 11b-b -- Clock tree distribution</div> The clock fans out in a balanced tree.</text> Each path from PLL to transistor has</text> the same length and same delay.</text> </svg></p> The clock tree fans out from the PLL through multiple levels of buffers. Engineers design each branch to have equal propagation delay, so the clock edge arrives everywhere on the chip at the same time.</div> </div> </figure> The tree is designed so that every path from the PLL to any transistor has the same electrical length. This property is called clock skew minimization. If one corner of the chip received the clock edge 0.1 nanoseconds before another corner, those two regions would briefly disagree about what "now" is, and logic that depends on both regions would produce wrong answers.</p> The clock tree is one of the most critical structures on a CPU die. It must deliver the clock signal to billions of transistors with nearly zero timing difference between any two points. Getting this wrong by even a fraction of a nanosecond causes computation errors. </div> What Happens on a Clock Edge</h2> The clock signal is a square wave -- it swings between low and high voltage at the clock frequency. Each transition from low to high is called the rising edge. Most digital logic is designed to do its work on the rising edge.</p> On each rising edge, a cascade of events happens simultaneously across the chip:</p> Flip-flops capture the values on their input wires and store them.</li> The stored values propagate through combinational logic (adders, multiplexers, comparators).</li> The results settle on the input wires of the next stage of flip-flops.</li> On the next rising edge, those results are captured, and the cycle repeats.</li> </ol> Key term: Flip-flop</strong> A circuit element that captures and stores one bit of data on the rising (or falling) edge of the clock signal. Between edges, it holds its value steady regardless of what happens on its input. Flip-flops are the basic storage elements inside a processor -- registers, pipeline stages, and caches are all built from them. </div> This is why clock speed matters. A 4 GHz clock produces 4 billion rising edges per second. Each edge advances the processor's state by one step. More edges per second means more steps per second means more work done.</p> But there is a limit. After each clock edge, the signals must propagate through logic gates and settle to stable values before the next edge arrives. If the clock runs too fast, the signals have not finished settling when the next edge captures them, and the processor computes garbage. This is why you cannot simply crank up the clock speed indefinitely -- the physics of signal propagation through transistors sets an upper bound.</p> Clock Speed vs. Instructions Per Cycle</h2> Clock speed alone does not determine how fast a processor runs. Two processors at the same clock speed can have very different performance. The other half of the equation is instructions per cycle (IPC) -- how much useful work the processor completes in each clock tick.</p> A simple processor might take five clock cycles to execute one instruction: one cycle to fetch it, one to decode it, one to read registers, one to execute, and one to write the result back. That processor has an IPC of 0.2.</p> A modern superscalar processor can execute multiple instructions simultaneously by using parallel execution units. It might complete four or more instructions every cycle, giving it an IPC of 4 or higher. This is why a modern 4 GHz chip vastly outperforms a 4 GHz chip from 2005 -- the modern chip does far more work per tick.</p> Fig. 11b-c -- Clock cycles and instruction execution</div> CLK</text> </p> C1</text> C2</text> C3</text> C4</text> C5</text> C6</text> C7</text> C8</text> C9</text></p> Simple</text> CPU:</text></p> FETCH</text> DEC</text> READ</text> EXEC</text> WRITE</text> instr #2 begins...</text> IPC: 0.2</text></p> Pipelined</text> CPU:</text></p> F</text> D</text> R</text> E</text> W</text> F</text> D</text> R</text> E</text> W</text> F</text> D</text> R</text> E</text> W</text> IPC: 1.0</text></p> Super-</text> scalar:</text></p> IPC: 3.0+</text> </svg></p> Three execution models at the same clock speed. A simple CPU completes one instruction every five cycles (IPC 0.2). A pipelined CPU overlaps stages to reach IPC 1.0. A superscalar CPU executes multiple instructions per cycle.</div> </div> </figure> The formula for raw throughput is simple:</p> Performance = Clock Speed x IPC</strong></p> A 4 GHz processor with IPC 4 completes 16 billion operations per second. A 5 GHz processor with IPC 2 completes only 10 billion. The slower clock wins because it does more per tick. This is why "GHz wars" ended around 2005 -- chip designers realized that increasing IPC was a more efficient path to performance than increasing clock speed.</p> Clock Domains and Crossing Boundaries</h2> Not everything in a computer runs at the same frequency. The CPU cores might run at 4 GHz, but the memory bus runs at 1.6 GHz, the PCIe links at 100 MHz (before serialization encoding), and USB at 480 MHz or 5 GHz depending on the version.</p> Each of these frequencies defines a clock domain. Within a domain, all logic runs synchronously -- every flip-flop sees the same clock edge at the same time. But when data crosses from one domain to another, there is a problem: the two clocks are not synchronized. A signal that is stable in one domain might be changing at the exact moment the other domain tries to read it.</p> Key term: Clock domain crossing</strong> The boundary where data passes between two parts of a system running at different clock frequencies. Special synchronization circuits are needed at these boundaries to prevent data corruption. Getting clock domain crossings wrong is one of the most common sources of hardware bugs. </div> Engineers solve this with synchronizer circuits -- typically a pair of flip-flops in the receiving domain that sample the incoming signal twice, giving it time to settle before the receiving logic uses it. This adds a small delay (usually two clock cycles of the receiving domain) but prevents data corruption.</p> Dynamic Frequency Scaling</h2> Modern processors do not run at a fixed clock speed. They adjust their frequency continuously based on workload and temperature. This is called dynamic frequency scaling, or DVFS (Dynamic Voltage and Frequency Scaling).</p> When the CPU is idle, it drops to a low frequency -- sometimes as low as 400 MHz -- to save power. When a demanding workload arrives, it ramps up to its maximum rated frequency. If it gets too hot, it throttles back down to prevent damage.</p> The operating system participates in this process. The kernel's cpufreq subsystem communicates with the CPU's power management hardware to set frequency targets. The "performance" governor locks the CPU at maximum speed. The "powersave" governor keeps it at minimum. The "schedutil" governor (default on modern Linux) adjusts frequency based on actual scheduler utilization data.</p> This is relevant to boot because during kernel initialization, the CPU typically runs at its base frequency. Turbo boost and dynamic scaling are configured later, once the kernel has set up the cpufreq subsystem and loaded the appropriate driver. The boot messages you see often include lines like:</p> [ 1.234567] cpufreq: Using governor schedutil </code></pre> That marks the moment the kernel takes control of the processor's clock speed.</p> A computer's timing system starts with a single quartz crystal and scales up through PLLs and clock trees to deliver precise timing to billions of transistors. Clock speed matters, but instructions per cycle matters just as much. Modern CPUs dynamically adjust their frequency thousands of times per second based on workload and thermal conditions. </div> Why This Matters for Boot</h2> Every step in the boot process we have covered so far -- fetching instructions from the BIOS, reading sectors from disk, decompressing the kernel, running start_kernel()</code> -- happens on the clock's rising edge. Every memory access, every comparison, every branch decision advances one tick at a time.</p> When you see a boot time of 3.5 seconds, that represents roughly 14 billion clock ticks at 4 GHz. Each tick is a single step. That the kernel can go from a blank slate to a running operating system in 14 billion steps -- setting up memory, interrupts, scheduling, drivers, filesystems, and launching userspace -- is a testament to both the speed of modern hardware and the efficiency of the kernel's initialization code.</p> The clock never stops. It never pauses. From the moment the crystal starts vibrating until the moment you pull the power cord, it ticks on, driving every computation the machine will ever perform.</p> Next: Device Drivers</a></p> Kernel Init 2025-01-14T00:00:00+00:00 The bootloader has done its job. It found the kernel image on disk, loaded it into memory, decompressed it, and jumped to its entry point. The processor is now executing kernel code. But the kernel is not yet an operating system. It is a large program that has just started running on bare hardware with almost nothing set up.</p> What happens next is a precisely ordered sequence of initialization steps. Each step builds on the one before it. Skip one, get the order wrong, and the machine hangs or panics. This article follows the kernel from its first instruction to the moment it launches the first userspace process.</p> The Entry Point</h2> On x86-64 Linux, the kernel's journey begins in architecture-specific assembly code. This code was written by hand, not generated by a compiler, because the machine is not yet in a state where compiled C code can run safely. There is no stack. The memory management unit may not be configured. Interrupts are disabled.</p> The assembly stub does the minimum work needed to get into C territory: it sets up a temporary stack, configures the page tables for the kernel's own memory, enables the memory management unit, and then calls the C function start_kernel()</code>.</p> Key term: start_kernel()</strong> The C function where Linux kernel initialization truly begins. Located in init/main.c in the kernel source tree, it is the single entry point for all of the kernel's high-level setup. Everything before it is architecture-specific assembly. Everything after it is portable C. </div> Think of it like building a house. The assembly stub pours the foundation and frames the first wall. start_kernel()</code> is the general contractor who arrives and begins coordinating every trade -- electricians, plumbers, roofers -- in the right order.</p> What start_kernel() Does</h2> The function is roughly 100 lines of sequential function calls. Each call initializes one kernel subsystem. The order matters because later subsystems depend on earlier ones. Here is a simplified view of the sequence:</p> Fig. 11a -- start_kernel() initialization sequence</div> A simplified view of the major calls inside start_kernel(). Each step depends on the ones above it. The kernel must complete them in order.</div> </div> </figure> Let us walk through the most important ones.</p> Setting Up the Architecture</h2> setup_arch()</code> is where the kernel learns what kind of machine it is running on. On x86, this function reads the memory map that the BIOS or UEFI firmware left behind, identifies the CPU model and its capabilities, and configures architecture-specific features.</p> The memory map is critical. It tells the kernel which regions of physical memory are usable RAM, which are reserved by firmware, and which are mapped to hardware devices. Without this map, the kernel cannot allocate memory safely -- it might overwrite firmware data or try to use an address that corresponds to a device register instead of RAM.</p> setup_arch()</code> also detects how many CPU cores are present and prepares data structures for each one. On a multi-core machine, only one core -- the bootstrap processor -- runs start_kernel()</code>. The other cores sit idle until the kernel explicitly wakes them later.</p> Key term: Bootstrap Processor (BSP)</strong> The single CPU core that the hardware selects to run the boot sequence. All other cores are Application Processors (APs). The BSP performs all kernel initialization and then wakes the APs one by one. </div> Memory Management</h2> mm_init()</code> builds the kernel's memory management system. Before this call, the kernel uses a simple boot-time allocator that hands out memory in chunks and never reclaims it. That works for early setup, but it is far too wasteful for a running system.</p> The full memory manager divides physical RAM into fixed-size pages, typically 4,096 bytes each. It tracks which pages are free and which are in use. It sets up the page tables that let the CPU translate virtual addresses -- the addresses that programs see -- into physical addresses where data actually lives in the RAM chips.</p> Fig. 11b -- Virtual to physical address translation</div> VIRTUAL</text> Process A: 0x4000</text></p> Process B: 0x4000</text> Kernel: 0xFFFF8000</text> PAGE</text> TABLES</text> PHYSICAL RAM</text> 0x00000 firmware</text></p> 0x12000 page A</text> 0x1F000 free</text> 0x34000 page B</text> 0x80000 kernel</text> </marker> </defs> Both processes use address 0x4000,</text> but page tables map them to different</text> physical locations. Each process sees</text> its own private address space.</text> </svg></p> Virtual memory gives each process the illusion of having its own private address space. The page tables translate virtual addresses to physical RAM locations. Two processes can use the same virtual address without conflicting.</div> </div> </figure> This translation mechanism is fundamental to everything the operating system does. It lets multiple programs run at the same time without stepping on each other's memory. It lets the kernel protect its own data from buggy or malicious programs. It enables features like swap, memory-mapped files, and copy-on-write.</p> The memory management subsystem is one of the first things the kernel sets up because almost everything else depends on it. Without the ability to allocate and track pages, the kernel cannot create processes, load drivers, or mount filesystems. </div> The Scheduler</h2> sched_init()</code> creates the process scheduler. A scheduler is the piece of the kernel that decides which program runs on which CPU core and for how long. At this point, no user programs exist, but the scheduler infrastructure must be in place before the kernel can create any processes -- including the very first one.</p> The scheduler maintains a run queue for each CPU core. A run queue is a list of processes that are ready to execute. When a core finishes its current time slice or a process blocks waiting for I/O, the scheduler picks the next process from the queue.</p> Linux uses the Completely Fair Scheduler (CFS) by default. The name reflects its design goal: give every process a fair share of CPU time proportional to its priority. CFS tracks how much CPU time each process has consumed using a virtual runtime counter. The process with the lowest virtual runtime runs next.</p> Key term: Scheduler</strong> The kernel subsystem that decides which process runs on each CPU core. It multiplexes many processes onto a limited number of cores, switching between them fast enough to create the illusion of simultaneous execution. </div> Interrupts</h2> init_IRQ()</code> sets up the interrupt system. An interrupt is a signal from hardware -- or from software -- that tells the CPU to stop what it is doing and handle an event. When you press a key, the keyboard controller sends an interrupt. When a network packet arrives, the network card sends one. When a timer fires, the timer hardware sends one.</p> Each interrupt has a number. The kernel installs a handler function for each number -- a piece of code that knows how to respond to that specific event. When interrupt number 42 fires, the CPU looks up handler 42 and runs it.</p> Before init_IRQ()</code>, the interrupt controllers are not configured and any hardware interrupt would either be ignored or crash the machine. After this call, the kernel can respond to hardware events.</p> The Console</h2> console_init()</code> sets up early text output. Every line of boot text you see scrolling by -- the [ 0.000000]</code> timestamped messages -- comes through this subsystem. The kernel needs a way to tell you what it is doing (and what went wrong) before the full display driver is loaded.</p> The early console typically talks directly to the VGA text buffer at a fixed memory address, or to a serial port. It is crude but reliable. Later in the boot process, a full framebuffer console or graphical display takes over.</p> rest_init() and the Birth of PID 1</h2> After all subsystems are initialized, start_kernel()</code> calls rest_init()</code>. This function does three critical things:</p> It creates a kernel thread called kthreadd</code> (PID 2), which will be the parent of all future kernel threads.</li> It creates the init</code> process (PID 1), which will become the first userspace process.</li> It turns the bootstrap code path into the idle thread (PID 0) -- the thread that runs when there is nothing else to do.</li> </ol> Fig. 11c -- The first three processes</div> PID 0 creates both PID 1 and PID 2. All userspace from 1, kernel threads from 2.</text> </svg></p> The kernel's first three processes form the root of the entire process tree. PID 1 (init) is the ancestor of every userspace program. PID 2 (kthreadd) parents all kernel worker threads.</div> </div> </figure> PID 1 is special. If it dies, the kernel panics. It is the root of the entire userspace process tree. Every daemon, every shell, every application is ultimately a descendant of PID 1. We will explore PID 1 in detail in a later article.</p> The Kernel Command Line</h2> Throughout initialization, the kernel reads parameters from its command line -- a string passed by the bootloader. You can see your system's kernel command line by reading /proc/cmdline</code>. Typical entries include:</p> root=/dev/sda2</code> -- which partition holds the root filesystem</li> ro</code> -- mount root as read-only initially</li> quiet</code> -- suppress most boot messages</li> init=/sbin/init</code> -- which program to run as PID 1</li> </ul> These parameters influence how nearly every subsystem initializes. The root=</code> parameter, for example, determines which block device the kernel will try to mount as the root filesystem. The init=</code> parameter lets you override the default PID 1 binary, which is useful for rescue situations.</p> The kernel command line is the bridge between the bootloader and the kernel. It carries configuration that the kernel cannot determine on its own, such as which disk partition holds the root filesystem or how verbose the boot messages should be. </div> The Kernel Log</h2> As each subsystem initializes, it writes messages to the kernel log buffer -- a ring buffer in memory. These are the dmesg</code> messages you can read after the system is running. Each message is timestamped relative to the start of boot.</p> The early messages tell you exactly what the kernel found and configured:</p> [ 0.000000] Linux version 6.8.0 ... [ 0.000000] Command line: root=/dev/sda2 ro quiet [ 0.004523] Memory: 16384MB available [ 0.012847] CPU: 8 cores detected [ 0.089234] Scheduler: CFS initialized [ 0.102456] Interrupt controller: APIC configured </code></pre> Reading dmesg</code> is one of the most useful debugging tools available. If a piece of hardware does not work, the answer is almost always in these messages.</p> From Kernel to Userspace</h2> Once rest_init()</code> spawns PID 1, the kernel's direct initialization work is largely done. PID 1 -- whether it is the traditional init</code>, systemd</code>, or another init system -- takes over the job of starting services, mounting filesystems, and bringing the system to a usable state.</p> But the kernel does not go away. It continues running underneath everything, handling interrupts, managing memory, scheduling processes, and mediating every interaction between software and hardware. The kernel is not a program that runs and exits. It is the permanent foundation on which everything else stands.</p> The transition from kernel initialization to userspace is the moment the computer stops being a machine running setup code and becomes an operating system. The next articles explore the subsystems that make this possible -- starting with the clock signal that times every operation.</p> Next: The Clock Signal</a></p> Decompressing and Handing Off 2025-01-13T00:00:00+00:00 GRUB has loaded the kernel image into memory and jumped to its entry point. But the code now running is not the Linux kernel. It is the decompression stub -- a small, self-contained program embedded at the front of the bzImage. Its job is to decompress the actual kernel, move it to its final address, and jump to it.</p> Between the bootloader's jump and the kernel's first real instruction, two major transitions happen. First, the compressed kernel must be extracted into a working executable. Second, the CPU must switch from the constrained 16-bit real mode it has been running in since power-on to the full 32-bit (or 64-bit) protected mode that the kernel requires. These two operations are deeply intertwined.</p> Why the Kernel Is Compressed</h2> A modern Linux kernel, compiled with a typical desktop or server configuration, produces an uncompressed vmlinux</code> binary of 30 to 80 megabytes. That is a lot of data to load from disk during boot. Disk reads are slow -- even on an NVMe drive, reading 60 MB takes noticeably longer than reading 12 MB.</p> Compression shrinks the kernel to roughly one-fifth its original size. The trade-off is CPU time: the decompressor must run after loading to recover the original data. But CPUs are fast and disk I/O is relatively slow, so decompressing in memory is almost always faster than loading the uncompressed image from disk. The net boot time is shorter with compression.</p> The Linux kernel build system supports several compression algorithms:</p> gzip</strong> -- the original default. Good compression ratio, moderate speed.</li> LZ4</strong> -- faster decompression, slightly larger output.</li> LZMA / XZ</strong> -- higher compression ratio, slower decompression.</li> Zstandard (zstd)</strong> -- good balance of ratio and speed. Increasingly the default on modern distributions.</li> </ul> The choice is made at kernel compile time (CONFIG_KERNEL_GZIP</code>, CONFIG_KERNEL_LZ4</code>, etc.). The decompression stub is built to match: it contains exactly one decompressor, for whichever algorithm was selected.</p> Key term: Decompression stub</strong> A small program concatenated to the front of the compressed kernel data inside the bzImage. It contains the decompression algorithm (gzip, LZ4, zstd, etc.), memory setup code, and the logic to place the decompressed kernel at its final address. It runs once and then jumps to the real kernel, never to be used again. </div> The First Instructions: Real-Mode Setup</h2> When GRUB jumps to the kernel's entry point, execution begins in the real-mode setup code -- the part of the bzImage that GRUB loaded to low memory (around 0x10000). This code runs with the CPU still in 16-bit real mode. It performs a series of preparedness checks and hardware queries before the transition to protected mode.</p> The real-mode setup code does the following:</p> Normalizes the CPU state.</strong> Sets segment registers to known values, clears the direction flag, and establishes a stack.</p> </li> Queries the BIOS for hardware information.</strong> This is the last chance to use BIOS services, because they only work in real mode. The code calls INT 0x15 to get the system memory map (the E820 map), INT 0x10 to query video modes, and other interrupts to detect hardware features. All results are stored in the boot parameters structure where the kernel will read them later.</p> </li> Enables the A20 line.</strong> On the original IBM PC, address line 20 was disabled to maintain compatibility with 8086 programs that relied on address wraparound. Enabling the A20 line allows the CPU to address memory above 1 MB. Without it, protected mode cannot access the full address space. The A20 enabling code tries several methods (keyboard controller, Fast A20 via port 0x92, BIOS INT 0x15) because different hardware requires different approaches.</p> </li> Prepares for the mode switch.</strong> The code loads a temporary Global Descriptor Table (GDT) and Interrupt Descriptor Table (IDT) -- the data structures that the CPU requires before it can enter protected mode.</p> </li> </ol> Key term: A20 line</strong> The 21st address line (bit 20) of the CPU's address bus. On the original 8086, only 20 address lines existed, and addresses above 1 MB wrapped around to zero. The IBM AT added the A20 line but kept it disabled by default for backward compatibility. Every boot sequence must explicitly enable it before the CPU can access memory above 1 MB. </div> Fig. 10b-1 -- Real-mode setup sequence</div> 16-bit Real Mode</text> </p> 32-bit</text> </p> Last chance for BIOS interrupts is step 2. After step 5, BIOS is gone.</text> </svg></p> The real-mode setup code runs a sequence of preparation steps, culminating in the switch to protected mode. BIOS services are only available during the first two steps.</div> </div> </figure> The Mode Switch</h2> The transition from real mode to protected mode is one of the most important moments in the boot process. In real mode, the CPU acts like a fast 1981 processor: 16-bit operations, 1 MB address space, no memory protection. In protected mode, the CPU gains 32-bit operations, a 4 GB address space, memory segmentation with privilege levels, and the ability to run the kind of code that a modern operating system requires.</p> The switch itself is a single instruction: setting bit 0 (the PE bit, for Protection Enable) in the CR0 control register. But that one instruction only works if the ground has been prepared. The GDT must be loaded. The A20 line must be enabled. Interrupts must be disabled (the real-mode interrupt handlers would crash if called in protected mode).</p> The sequence looks like this in pseudocode:</p> cli ; disable interrupts lgdt [gdt_pointer] ; load the Global Descriptor Table mov eax, cr0 ; read CR0 or eax, 1 ; set PE bit mov cr0, eax ; write CR0 -- NOW IN PROTECTED MODE jmp 0x10:protected_entry ; far jump to reload CS with a 32-bit selector </code></pre> The far jump immediately after setting CR0 is essential. It flushes the CPU's instruction pipeline and loads the code segment register (CS) with a selector that points to a 32-bit code segment in the GDT. Without this jump, the CPU would continue fetching instructions using the old 16-bit CS value, and the next instruction would be decoded incorrectly.</p> Key term: Global Descriptor Table (GDT)</strong> A table in memory that defines memory segments for the CPU in protected mode. Each entry describes a segment's base address, size, type (code or data), and privilege level. The CPU consults the GDT on every memory access to enforce segment boundaries and permissions. The kernel will replace the boot GDT with its own shortly after starting. </div> The switch from real mode to protected mode is a one-way door during boot. Once the PE bit is set in CR0, BIOS interrupts stop working, the address space expands from 1 MB to 4 GB, and the CPU enforces segment-based memory protection. The far jump immediately after the switch is mandatory to flush the instruction pipeline and load a valid 32-bit code segment. </div> On 64-Bit Systems: The Long Mode Jump</h2> Modern x86-64 systems take the mode switching further. The kernel needs 64-bit long mode to access the full address space and use 64-bit registers. On these systems, the decompression stub typically transitions through protected mode and then into long mode before decompressing.</p> The transition to long mode requires additional steps beyond protected mode:</p> Set up page tables.</strong> Long mode requires paging to be enabled. The stub creates a minimal identity-mapped page table -- a page table where virtual addresses equal physical addresses. This is temporary; the kernel will build its own page tables later.</p> </li> Enable PAE.</strong> Physical Address Extension (bit 5 of CR4) must be set. Long mode requires it.</p> </li> Set the LME bit.</strong> The Long Mode Enable bit in the EFER (Extended Feature Enable Register) MSR signals the CPU to prepare for 64-bit operation.</p> </li> Enable paging.</strong> Setting the PG bit (bit 31) in CR0, with LME already set, activates long mode.</p> </li> Far jump to 64-bit code.</strong> Like the real-to-protected switch, a far jump reloads CS with a 64-bit code segment selector.</p> </li> </ol> After this sequence, the CPU is in 64-bit mode with identity-mapped paging, and the decompressor can use the full address space.</p> Fig. 10b-2 -- CPU mode transitions during boot</div> BIOS / GRUB</text> transient</text> kernel runs here</text></p> Each transition requires a far jump to flush the instruction pipeline</text> </svg> The CPU transitions through three modes during boot. Real mode is used for BIOS interaction, protected mode is a stepping stone, and long mode is where the kernel runs on 64-bit systems.</div> </div> </figure> The Decompression</h2> With the CPU in the correct mode, the decompression stub can do its primary job: decompress the kernel.</p> The stub knows the exact location and size of the compressed data because these values were embedded in the image at build time. The compressed kernel payload sits immediately after the stub code in the bzImage. The stub also knows the expected size of the decompressed output.</p> The process is straightforward in concept:</p> Determine the output address. The stub decompresses the kernel to a location that does not overlap with the compressed data or the stub itself. The kernel build system calculates safe addresses at compile time.</p> </li> Call the decompressor. The stub invokes the appropriate decompression function (gzip's inflate</code>, LZ4's LZ4_decompress</code>, zstd's ZSTD_decompress</code>, etc.). This function reads the compressed data and writes the decompressed kernel to the output address.</p> </li> Verify the output. Some configurations check a CRC or other integrity value to confirm the decompression produced correct data.</p> </li> </ol> The decompressed output is vmlinux</code> -- the raw kernel ELF binary, stripped of debug symbols. This is the actual Linux kernel: the scheduler, the memory manager, the device driver framework, the system call interface, everything.</p> Fig. 10b-3 -- Decompression and relocation in memory</div> Physical Memory During Decompression</text></p> Real-mode setup code + boot params</text> ~0x10000</text> 1 MB</text> Decompression stub + compressed kernel</text> (loaded here by GRUB at 0x100000)</text> decompress</text> Decompressed kernel (vmlinux)</text> (placed at non-overlapping address, then relocated to final position)</text> initramfs (loaded by GRUB, untouched)</text> jump to</text> kernel</text> </svg> The decompression stub extracts the compressed kernel from within the bzImage and places the decompressed vmlinux at a safe address. After relocation to the final address, the stub jumps to the kernel entry point.</div> </div> </figure> Relocation</h2> The decompressed kernel needs to be at a specific physical address. On x86-64, this is typically 0x1000000 (16 MB) for a relocatable kernel, but the kernel's link address (compiled-in start address) determines the expectation.</p> If the decompressed data is not already at the correct address, the stub copies it there. Modern kernels are often built as position-independent executables (with CONFIG_RELOCATABLE=y</code>), which means they can run at addresses other than their link address. The kernel adjusts its internal references at startup. But the decompression stub still relocates the image to a clean region of memory before jumping.</p> The decompression stub is a single-use program. It transitions the CPU to the correct mode (protected or long mode), decompresses the kernel payload using the algorithm selected at compile time, places the result at the correct physical address, and jumps to the kernel entry point. After this jump, the decompression stub is never used again -- its memory will be reclaimed by the kernel. </div> The Handoff</h2> The final act of the decompression stub is a jump to the kernel's entry point -- the startup_64</code> function (on x86-64) or startup_32</code> (on 32-bit systems). At this instant, control passes from boot code to kernel code.</p> The kernel inherits a specific machine state:</p> CPU mode:</strong> 64-bit long mode (or 32-bit protected mode on older systems), with identity-mapped paging.</li> Interrupts:</strong> disabled. The kernel will set up its own interrupt handlers before re-enabling them.</li> Boot parameters:</strong> in a known memory location. These contain the memory map, command line, initrd location, video mode, and other data collected by the real-mode setup code and GRUB.</li> Memory contents:</strong> the decompressed kernel at its final address, the initramfs at its GRUB-assigned address, and the boot parameters structure in low memory.</li> </ul> The kernel does not depend on GRUB's code or data structures. It does not call back to the bootloader. From this point, GRUB might as well not exist. The kernel reads the boot parameters, initializes its own data structures, and begins the process of bringing up a full operating system.</p> The Boot Parameters (Zero Page)</h2> The boot parameters structure deserves a closer look, because it is the only communication channel between the bootloader world and the kernel world. It occupies the first page (4096 bytes) of the real-mode code segment -- historically called the "zero page" because it starts at offset zero of that segment.</p> The zero page contains:</p> The E820 memory map from BIOS (or EFI memory map on UEFI systems).</li> The kernel command-line pointer and length.</li> The initrd address and size.</li> The video mode and framebuffer information.</li> The number of setup sectors and the boot protocol version.</li> Hardware detection results from the real-mode setup code.</li> </ul> The kernel's first initialization code reads this structure to learn about the machine. Without it, the kernel would not know how much RAM exists, what video mode is active, where the initrd is, or what command-line parameters the user requested.</p> Key term: Zero page (boot parameters)</strong> A 4096-byte data structure that the bootloader and real-mode setup code fill in before jumping to the kernel. It contains the memory map, command line, initrd location, and hardware information. The kernel reads it during early initialization and never modifies or refers to it again after extracting the information it needs. </div> Summary of the Journey So Far</h2> From GRUB's jump to the kernel's first real instruction, a remarkable amount happens:</p> The real-mode setup code queries the BIOS for hardware information.</li> The A20 line is enabled.</li> A temporary GDT is loaded.</li> The CPU switches to protected mode, then to long mode.</li> The decompression stub extracts the compressed kernel.</li> The decompressed kernel is placed at its final physical address.</li> The stub jumps to the kernel entry point.</li> </ol> All of this happens in a fraction of a second. The user sees nothing -- perhaps a brief flash on screen if verbose boot is enabled. But in that fraction of a second, the machine transforms from a BIOS-era 16-bit environment into a 64-bit system ready to initialize a modern operating system.</p> The kernel is now running. It has the CPU. It has the memory map. It has the initramfs. The next phase -- kernel initialization -- is where Linux truly comes to life.</p> Next: Kernel Init</a></p> Finding the Kernel 2025-01-12T00:00:00+00:00 GRUB has read its configuration, shown a menu, and the user (or the timeout) has selected an entry. The linux</code> command in that entry specifies a path: something like /vmlinuz-6.2.0</code> or /boot/vmlinuz-6.2.0</code>. GRUB must now find that file on disk, figure out what kind of image it is, and load it into the right place in memory.</p> This sounds simple, and for GRUB it mostly is -- GRUB has filesystem drivers that can read ext4, XFS, and other formats. But the kernel image itself is not straightforward. It is a compressed, multi-part file with a specific internal structure that GRUB must understand. The name vmlinuz</code> hints at that structure, and the story behind it reaches back decades.</p> Where the Kernel Lives</h2> On most Linux systems, kernel images live in /boot</code>. This directory is sometimes on the root filesystem and sometimes on a separate small partition. A separate /boot</code> partition was historically necessary because older bootloaders could not read beyond certain disk boundaries, or could not understand the root filesystem's format. Today, GRUB can handle most filesystems and disk sizes, but the convention persists.</p> Inside /boot</code>, you will typically find several files for each installed kernel:</p> /boot/vmlinuz-6.2.0-36-generic /boot/initrd.img-6.2.0-36-generic /boot/config-6.2.0-36-generic /boot/System.map-6.2.0-36-generic </code></pre> The version number in each filename matches a specific kernel build. When you upgrade the kernel, your package manager installs new files with the new version number and runs grub-mkconfig</code> to regenerate the GRUB configuration with updated paths.</p> Key term: vmlinuz</strong> The compressed Linux kernel image. The name comes from "Virtual Memory LINUx" with a "z" appended to indicate compression (like gzip). Despite the name, vmlinuz is not a standard compressed file -- it is a specially structured image containing a decompression stub, a setup header, and the compressed kernel proper. </div> The symbolic link /vmlinuz</code> (in the root directory) often points to the latest installed kernel image in /boot</code>. GRUB configurations may reference either the full path or the symlink.</p> The Kernel Image Format</h2> The file vmlinuz</code> is not a simple executable. It is not an ELF binary that you could run with a normal program loader. It is a custom format specific to the Linux boot process, and it contains several distinct parts packed together.</p> Think of it like a shipping container. The outside of the container has labels and handling instructions (the setup header). Inside the container is a smaller box containing the actual cargo, wrapped in packing material (the compressed kernel, surrounded by the decompression stub). You cannot use the cargo until you unpack the box.</p> Fig. 10a-1 -- Structure of a bzImage file</div> GRUB reads this ---></text> to know how to load this ---></text> </svg></p> A bzImage contains three parts: the real-mode setup code with a header that describes the image, a small decompression stub, and the compressed kernel itself.</div> </div> </figure> The three parts serve different purposes:</p> The boot sector and setup header</strong> occupy the first portion of the file. The boot sector is a legacy artifact -- it allows the kernel to be booted directly from a floppy disk without a bootloader. Nobody does this anymore, but the structure remains. The setup header, however, is critical. It contains fields that GRUB reads to know the kernel's version, where to load it in memory, what boot protocol version it supports, and the size of the setup code.</p> The decompression stub</strong> is a small self-contained program. Its job is to decompress the next part. We will cover this in detail in the next article.</p> The compressed kernel</strong> is the actual Linux kernel -- the vmlinux</code> ELF binary, stripped and then compressed with gzip, LZ4, LZMA, XZ, or Zstandard (depending on the kernel's build configuration).</p> vmlinux vs. vmlinuz vs. bzImage</h2> These names are confusing because they evolved over decades and carry historical baggage. Here is the lineage:</p> vmlinux</strong> is the raw, uncompressed Linux kernel. It is an ELF binary -- the direct output of the kernel build's linking step. It contains debug symbols and section headers. On a modern kernel, it can be 30-80 MB or more. You will find it in the kernel build tree but almost never in /boot</code>.</p> zImage</strong> was the original compressed kernel format. The "z" stands for the compression. It was limited to 512 KB of compressed data because it loaded the kernel below the 1 MB mark in memory (the real-mode address space). This limit made it unusable for modern kernels.</p> bzImage</strong> stands for "big zImage." It lifts the size restriction by loading the compressed kernel above the 1 MB mark. Despite the name containing "bz," the compression is not bzip2 -- it was historically gzip, and modern kernels can use several algorithms. Nearly every Linux system today uses bzImage.</p> vmlinuz</strong> is the file you find in /boot</code>. It is a bzImage -- the same file, just installed with a different name.</p> Key term: bzImage</strong> The standard format for a bootable Linux kernel image. "Big zImage" loads the compressed kernel above the 1 MB boundary, allowing much larger kernels than the original zImage format. The file installed as /boot/vmlinuz is a bzImage. </div> Fig. 10a-2 -- Kernel image name lineage</div> The kernel build pipeline: vmlinux is compiled, stripped and compressed, then wrapped with a setup header and decompression stub to produce a bzImage. That bzImage is installed as /boot/vmlinuz.</div> </div> </figure> The Setup Header</h2> The setup header is the critical interface between the bootloader and the kernel. It is a binary structure starting at offset 0x01F1 in the kernel image, and it tells the bootloader everything it needs to know about how to load this particular kernel.</p> GRUB reads the following fields from the setup header:</p> setup_sects</strong>: How many 512-byte sectors the setup code occupies. This tells GRUB where the protected-mode kernel code begins in the file.</li> syssize</strong>: The size of the protected-mode code, in 16-byte paragraphs.</li> boot_flag</strong>: Must be 0xAA55 (same magic number as the MBR, for historical reasons).</li> header</strong>: Must be the magic string "HdrS" (0x53726448). This confirms the file is a valid Linux kernel image.</li> version</strong>: The boot protocol version. Different versions support different features. Modern kernels use protocol 2.15 or higher.</li> loadflags</strong>: Bit flags that control loading behavior. Bit 0 indicates whether the kernel was compiled as a bzImage (loaded high) or a zImage (loaded low).</li> cmd_line_ptr</strong>: The physical address where GRUB should place the kernel command-line string.</li> initrd_addr_max</strong>: The maximum physical address where the initrd can end. This prevents GRUB from loading the initrd above the kernel's addressable range.</li> ramdisk_image</strong> and ramdisk_size</strong>: Where GRUB loaded the initrd and how large it is. GRUB fills these in so the kernel can find the initrd later.</li> </ul> Key term: Boot protocol</strong> A versioned specification that defines the contract between the Linux kernel and the bootloader. The boot protocol defines which fields exist in the setup header, what values are valid, and what the bootloader must do before jumping to the kernel. The protocol version increments when new fields or capabilities are added. </div> Fig. 10a-3 -- Key setup header fields</div> Setup Header (starts at file offset 0x01F1)</text></p> setup_sects</text> Number of setup sectors (real-mode code size)</text> header</text> Magic "HdrS" = valid Linux kernel</text> version</text> Boot protocol version (e.g. 0x020F = 2.15)</text> loadflags</text> Bit 0: loaded_high (1 = bzImage)</text> cmd_line_ptr</text> Address for kernel command line string</text> ramdisk_image</text> Bootloader writes initrd load address here</text> ramdisk_size</text> Bootloader writes initrd size here</text> payload_offset</text> Offset to compressed kernel within the image</text> 0x01F1</text> 0x0202</text> 0x0206</text> 0x0211</text> 0x0228</text> 0x0218</text> 0x021C</text> 0x0248</text> </svg></p> The setup header contains fields that GRUB reads (yellow, blue) and fields that GRUB writes (orange) to communicate with the kernel. The protocol version determines which fields are present.</div> </div> </figure> How GRUB Loads the Image</h2> When GRUB processes the linux</code> command, it executes a precise loading sequence:</p> Open the file.</strong> GRUB uses its filesystem driver to open vmlinuz</code> from the specified partition and path.</p> </li> Read the setup header.</strong> GRUB reads the first few kilobytes of the file and parses the setup header starting at offset 0x01F1. It checks for the "HdrS" magic and the boot protocol version.</p> </li> Validate the image.</strong> GRUB checks that the loadflags</code> indicate a bzImage (it refuses to load a zImage on modern systems where the kernel would not fit below 1 MB). It verifies the protocol version is recent enough to support the features GRUB needs.</p> </li> Load the real-mode code.</strong> The first setup_sects + 1</code> sectors of the file (the boot sector plus the setup sectors) contain the real-mode kernel code. GRUB loads this to a low memory address, typically around 0x10000 (64 KB).</p> </li> Load the protected-mode code.</strong> Everything after the setup sectors is the protected-mode kernel -- the decompression stub followed by the compressed kernel. GRUB loads this at the 1 MB mark (address 0x100000) or higher if the setup header specifies a preferred address.</p> </li> Store boot parameters.</strong> GRUB writes the kernel command line to memory and records its address in cmd_line_ptr</code>. It fills in the memory map, video mode information, and other fields in the boot parameters structure (the "zero page" at the start of the real-mode code segment).</p> </li> </ol> The kernel image vmlinuz is a bzImage containing a setup header, a decompression stub, and the compressed kernel. GRUB reads the setup header to learn how to load the image, places the real-mode code in low memory and the protected-mode code above 1 MB, and fills in boot parameters so the kernel knows where everything is. The setup header is the contract between bootloader and kernel. </div> The initrd Companion</h2> When GRUB processes the initrd</code> command, it loads the initramfs image into memory above the kernel. The setup header field initrd_addr_max</code> tells GRUB the highest address the kernel can access for the initrd, and GRUB loads it as high as possible within that limit (to leave room for the kernel to decompress and expand).</p> GRUB then writes the initrd's physical address and size into the ramdisk_image</code> and ramdisk_size</code> fields of the boot parameters. The kernel will read these values during its initialization to find the initramfs.</p> The initrd is not optional on most systems. Without it, the kernel would need every driver compiled in -- SATA, NVMe, SCSI, USB, LVM, dm-crypt, every filesystem -- to mount the root partition. The initramfs provides exactly the drivers needed for this specific machine, keeping the kernel itself smaller and more generic.</p> Finding the Kernel on UEFI</h2> On UEFI systems, the process is slightly different but the kernel image format is the same. The UEFI version of GRUB reads files from the EFI System Partition (or from the /boot</code> partition, which GRUB can access via its filesystem modules). The kernel is still a bzImage with the same setup header.</p> One notable UEFI feature: modern Linux kernels (since about 2011, with the EFI boot stub) can be booted directly by the firmware without GRUB at all. The kernel's EFI stub makes the bzImage look like a valid EFI application. The firmware loads it, the EFI stub runs, sets up the boot parameters itself, and jumps to the kernel. This is the basis of systemd-boot and other simple UEFI boot managers.</p> But whether GRUB or the firmware loads the image, the kernel's internal structure is the same. The setup header, the decompression stub, and the compressed kernel are all present. The next step is always the same: decompress.</p> What Happens Next</h2> GRUB has found the kernel file, parsed its setup header, loaded its real-mode and protected-mode parts into the correct memory addresses, and filled in the boot parameters. The initramfs is in memory too. Everything is staged and ready.</p> Now the CPU must jump to the kernel's entry point. But the code it jumps to is not the kernel itself -- it is the decompression stub. The real kernel is still compressed, packed inside the bzImage. The stub must unpack it, and to do that, it first needs to leave real mode behind.</p> Next: Decompressing and Handing Off</a></p> Stage 2: GRUB 2025-01-11T00:00:00+00:00 Stage 1 loaded a small piece of code that can read a filesystem. That code loaded GRUB's main body into memory. Now, for the first time in the boot process, the machine is running a program that can do more than just load the next sector. GRUB can read configuration files, display a menu, accept keyboard input, and -- most importantly -- find and load a Linux kernel.</p> GRUB stands for GRand Unified Bootloader. The "unified" part matters: GRUB can boot Linux, Windows, BSD, and other operating systems from a single menu. But its core job on a Linux system is straightforward. Read a config file. Show a menu. Load the kernel and an initial RAM filesystem into memory. Hand control to the kernel.</p> GRUB's Architecture</h2> GRUB is not a single program. It is a modular system built from a small core and a collection of loadable modules. The core provides the command-line interpreter, the module loader, and basic device access. Modules add specific capabilities: filesystem drivers, compression support, graphical terminal rendering, network booting, and more.</p> Key term: GRUB module</strong> A loadable piece of code that adds a specific capability to GRUB. Modules are stored as files in /boot/grub/ (or /boot/grub2/ on some distributions). Common modules include ext2.mod (for reading ext2/ext3/ext4 filesystems), normal.mod (the menu system), and linux.mod (for loading Linux kernels). </div> When GRUB's core starts, it loads the normal</code> module, which provides the familiar menu interface. The normal</code> module then reads the configuration file and builds the menu entries. If the configuration file is missing or corrupt, GRUB drops to a rescue shell -- a minimal command line where you can manually type commands to boot the system.</p> Fig. 09a -- GRUB modular architecture</div> GRUB's core loads modules on demand. The normal module reads grub.cfg and builds the boot menu. Filesystem modules let GRUB read files from partitions.</div> </div> </figure> The Configuration File</h2> GRUB's behavior is controlled by a file called grub.cfg</code>, usually found at /boot/grub/grub.cfg</code> or /boot/grub2/grub.cfg</code>. You are not meant to edit this file by hand. It is generated automatically by the grub-mkconfig</code> command (or grub2-mkconfig</code> on Fedora/RHEL systems), which scans your system for installed kernels and writes the appropriate menu entries.</p> A typical grub.cfg</code> entry looks like this:</p> menuentry 'Ubuntu' --class ubuntu --class os { set root='hd0,gpt2' linux /vmlinuz-6.2.0 root=/dev/sda3 ro quiet splash initrd /initrd.img-6.2.0 } </code></pre> Each line does something specific:</p> set root='hd0,gpt2'</code> tells GRUB which partition to read files from. hd0</code> is the first disk, gpt2</code> is the second GPT partition.</li> linux /vmlinuz-6.2.0 ...</code> tells GRUB to load the kernel image from the specified path, and passes everything after the filename as kernel command-line parameters.</li> initrd /initrd.img-6.2.0</code> tells GRUB to load the initial RAM disk alongside the kernel.</li> </ul> Key term: initrd / initramfs</strong> An initial RAM filesystem loaded into memory alongside the kernel. It contains a minimal set of drivers and tools that the kernel needs to mount the real root filesystem. Without it, the kernel could not access a root partition on LVM, RAID, an encrypted volume, or any device requiring a driver not compiled into the kernel itself. </div> How GRUB Identifies Disks and Partitions</h3> GRUB uses its own naming convention for storage devices. The first hard disk is hd0</code>, the second is hd1</code>, and so on. Partitions are numbered starting from 1: hd0,msdos1</code> for the first MBR partition on the first disk, or hd0,gpt1</code> for the first GPT partition. This naming is independent of how Linux names the same devices (/dev/sda1</code>, /dev/nvme0n1p1</code>, etc.).</p> GRUB resolves these names using its disk and partition modules. It reads partition tables directly -- it does not rely on the operating system, because the operating system is not running yet. GRUB carries its own partition table parsers (part_msdos.mod</code> for MBR, part_gpt.mod</code> for GPT) and its own filesystem drivers.</p> How GRUB Reads Filesystems</h2> This is the capability that separates GRUB from the tiny Stage 1 code. Stage 1 can only read raw sectors by number. GRUB can read files by path.</p> GRUB includes filesystem driver modules for ext2/ext3/ext4, XFS, Btrfs, ZFS, FAT, NTFS, and others. When GRUB needs to read a file, it:</p> Identifies the disk and partition from the device name.</li> Reads the partition's superblock to determine the filesystem type.</li> Loads the appropriate filesystem module if it is not already loaded.</li> Navigates the filesystem's internal structures (inodes, directory entries, extent trees) to locate the file.</li> Reads the file's data blocks into memory.</li> </ol> This is the same work that the Linux kernel does when you run cat /boot/vmlinuz</code>, but GRUB does it with its own code, running before the kernel exists. GRUB's filesystem drivers are simpler than the kernel's -- they only need to read, never write -- but they must be correct. A bug in GRUB's ext4 driver means an unbootable system.</p> Fig. 09b -- GRUB reading a file from disk</div> GRUB uses its own filesystem driver to resolve file paths, reads the underlying sectors through BIOS services, and loads the kernel and initrd into specific memory addresses.</div> </div> </figure> The Menu and the Timeout</h2> When GRUB finishes loading its configuration, it displays a menu. If you have a single operating system, the menu might be hidden -- GRUB waits for a configurable timeout (often 5 or 10 seconds) and then boots the default entry automatically. Press Shift or Escape during this timeout to reveal the menu.</p> The menu lets you choose between different kernels (useful after a bad kernel upgrade), different operating systems (dual-boot setups), or recovery modes. Each menu entry is a menuentry</code> block in grub.cfg</code> containing the commands GRUB will execute when that entry is selected.</p> You can also press c</code> at the menu to drop into GRUB's command line, where you can type commands manually. This is invaluable for recovery. If your grub.cfg</code> is corrupt but you know where your kernel lives, you can type:</p> set root=(hd0,gpt2) linux /vmlinuz root=/dev/sda3 initrd /initrd.img boot </code></pre> Those four commands do exactly what a menuentry</code> block does, but typed by hand.</p> GRUB is a modular bootloader with its own filesystem drivers, command-line interpreter, and menu system. It reads a configuration file (grub.cfg) generated by your distribution's tools, loads the kernel and initramfs into memory, and then hands control to the kernel. If the configuration is broken, you can use GRUB's command line to boot manually. </div> Loading the Kernel and Initramfs</h2> When you select a menu entry (or the timeout expires), GRUB executes the commands in that entry. The linux</code> command is the critical one. It does several things:</p> Opens the kernel image file (vmlinuz</code>) from the specified partition and path.</li> Reads the kernel's setup header, which is embedded in the first few kilobytes of the image. The header tells GRUB the kernel's version, required load address, command-line format, and other parameters.</li> Copies the kernel image into memory at the address the header specifies.</li> Stores the kernel command-line string (everything after the filename on the linux</code> line) in a designated memory area.</li> </ol> The initrd</code> command then loads the initial RAM filesystem image into another region of memory and records its address and size so the kernel can find it later.</p> Fig. 09c -- Memory layout after GRUB loads kernel and initrd</div> After GRUB's linux and initrd commands execute, memory contains the kernel setup header at a low address, the compressed kernel at the 1 MB mark, and the initramfs image in high memory.</div> </div> </figure> The Boot Command</h2> After the linux</code> and initrd</code> commands have loaded their data, the boot</code> command (usually implicit at the end of a menuentry</code> block) triggers the actual handoff. GRUB fills in a data structure called the boot parameters (or "zero page") with everything the kernel needs to know: the address and size of the initrd, the command-line location, the video mode, the memory map from the BIOS, and more.</p> Then GRUB jumps to the kernel's entry point. At this moment, GRUB's job is done. It will never run again (until the next reboot). The kernel takes over, and the bootloader is gone.</p> GRUB on UEFI Systems</h2> Everything described so far assumes legacy BIOS booting. On UEFI systems, the picture is different. UEFI firmware can read FAT-formatted partitions natively, so GRUB does not need the Stage 1 / Stage 1.5 chain. Instead, GRUB is compiled as a UEFI application -- an EFI executable stored on the EFI System Partition (usually at /EFI/ubuntu/grubx64.efi</code> or similar).</p> The firmware loads this EFI application directly. There is no MBR boot code, no INT 0x13, no real mode. The UEFI version of GRUB uses UEFI boot services instead of BIOS interrupts for disk access and video output. But once GRUB is running, the user-facing experience is the same: same config file, same menu, same linux</code> and initrd</code> commands.</p> Key term: EFI System Partition (ESP)</strong> A FAT32-formatted partition that UEFI firmware can read directly. It contains bootloader executables (.efi files) and related configuration. On Linux systems, the ESP is usually mounted at /boot/efi or /efi. </div> When Things Go Wrong</h2> GRUB failures are common enough that every Linux administrator encounters them. The most frequent scenarios:</p> "error: unknown filesystem"</strong> -- GRUB cannot read the partition containing its modules or configuration. This happens when a filesystem is corrupted, when you resize a partition without updating GRUB, or when a GRUB module for the filesystem type was not installed.</p> "error: file not found"</strong> -- GRUB can read the filesystem but cannot find grub.cfg</code> or the kernel. This happens after a kernel upgrade that did not regenerate grub.cfg</code>, or after moving files around in /boot</code>.</p> Rescue shell</strong> -- GRUB loaded its core but could not load normal.mod</code>. You get a grub rescue></code> prompt with minimal commands. The fix is usually set prefix=(hd0,gpt2)/boot/grub</code> followed by insmod normal</code> and then normal</code>.</p> In all these cases, the fix involves booting from a live USB, mounting the installed system, and running grub-install</code> and grub-mkconfig</code> to rebuild GRUB from scratch.</p> GRUB's power comes from its modularity. It carries its own filesystem drivers, so it can read files by path -- not just by sector number. It reads grub.cfg to build a menu, loads the kernel and initramfs into specific memory addresses, fills in the boot parameter structure, and jumps to the kernel entry point. From that jump onward, the bootloader is finished and the kernel is in control. </div> What Happens Next</h2> GRUB has loaded the kernel image and the initramfs into memory. It has passed the kernel command line and a block of boot parameters. Now the kernel needs to actually start running. But the kernel image on disk is not a simple executable -- it is compressed, and it contains multiple parts. GRUB has found it; now the kernel must find itself.</p> Next: Finding the Kernel</a></p> Stage 1: The Bootloader 2025-01-10T00:00:00+00:00 In the previous article, the firmware found a boot device. It read the very first sector from that disk -- 512 bytes -- and loaded it into memory at address 0x7C00. Now the CPU's instruction pointer is aimed at that address, and execution begins.</p> Those 512 bytes are the Master Boot Record, and the tiny program inside it is the Stage 1 bootloader. It is the smallest useful program in the entire boot chain. It has one job: load something bigger.</p> The 512-Byte Constraint</h2> A single disk sector is 512 bytes. That is not a lot of space. To put it in perspective, this sentence alone is about 70 bytes of ASCII text. The entire Stage 1 bootloader has to fit in roughly seven sentences worth of raw data -- and that space is shared with other structures.</p> The 512-byte sector is divided into three regions. The first 440 bytes (sometimes 446, depending on the convention) hold executable machine code. The next 64 bytes hold the partition table -- four entries of 16 bytes each, describing how the disk is divided. The final 2 bytes hold the boot signature: 0x55 followed by 0xAA.</p> Key term: Master Boot Record (MBR)</strong> The first 512-byte sector of a disk formatted with the traditional partitioning scheme. It contains boot code, a partition table, and a magic signature. The firmware checks the last two bytes for 0x55AA before treating the sector as bootable. </div> Fig. 08a -- MBR layout (512 bytes)</div> Offset</text> Contents</text> Size</text></p> 0x000</text> Stage 1 Boot Code</text> 440 bytes</text> 0x1B8</text> Disk Signature + Padding</text> 6 bytes</text> 0x1BE</text> Partition Table (4 entries)</text> 64 bytes</text> 0x1FE</text> 0x55 0xAA (Boot Signature)</text> 2 bytes</text> </svg> The 512-byte MBR is divided into boot code, a disk signature, a four-entry partition table, and the 0x55AA magic number.</div> </div> </figure> The firmware does not care what the boot code does. It does not validate the instructions. It checks for 0x55AA, loads the sector, and jumps. From that point on, the boot code is in complete control.</p> Real Mode: The 1981 Prison</h2> When the Stage 1 code starts running, the CPU is in real mode. This is the processor's most primitive operating state -- the same mode the original IBM PC's 8086 processor used in 1981. Every x86 processor since then, no matter how advanced, starts in this mode for backward compatibility.</p> Key term: Real mode</strong> The CPU's initial operating mode, inherited from the 8086. It provides direct access to hardware, uses 16-bit registers, and can address only 1 MB of memory. There is no memory protection, no privilege separation, and no virtual memory. Any instruction can access any address or any hardware port. </div> Real mode imposes harsh constraints. The CPU uses 16-bit registers, which means a single register can hold a value between 0 and 65,535. Memory addresses use a segment:offset scheme where a 16-bit segment register is shifted left by 4 bits and added to a 16-bit offset, giving a 20-bit physical address. That is enough to address exactly 1,048,576 bytes -- one megabyte.</p> One megabyte sounds generous for 440 bytes of boot code, but the address space is not empty. The BIOS and hardware claim large chunks of it. The usable region for loading data is surprisingly small.</p> Fig. 08b -- Real mode memory map (first 1 MB)</div> 0xFFFFF</text> </svg></p> The first megabyte of address space. The MBR boot code runs from 0x7C00. Green regions are available for loading additional data. Red and blue regions are reserved by firmware and hardware.</div> </div> </figure> Notice where the MBR loads: address 0x7C00. This is not at the bottom of memory (reserved for the interrupt vector table) and not at the top (reserved for the BIOS ROM and video hardware). It sits in a gap of free memory, chosen decades ago for the original IBM PC and preserved ever since.</p> What the Boot Code Actually Does</h2> The Stage 1 boot code has 440 bytes to work with. That is too small for a filesystem driver, too small for a configuration parser, too small for anything complicated. The code does the absolute minimum needed to load the next piece of the chain.</p> Here is the typical sequence:</p> Set up segment registers.</strong> The code sets CS, DS, ES, and SS to known values. The firmware does not guarantee these are correct, so the boot code normalizes them immediately.</p> </li> Relocate itself.</strong> Many bootloaders copy themselves from 0x7C00 to a lower address (often 0x0600) to free up the area around 0x7C00 for loading the next stage. Then they jump to the relocated copy and continue from there.</p> </li> Scan the partition table.</strong> The code reads the four 16-byte partition table entries starting at offset 0x1BE. It looks for the entry marked as "active" -- the one with 0x80 in its first byte. This is the bootable partition.</p> </li> Load the first sector of the active partition.</strong> Using BIOS interrupt 0x13 (the disk read service), the code loads the first sector of the active partition into memory at 0x7C00 -- the same address the MBR was originally loaded to.</p> </li> Jump to the loaded code.</strong> The CPU begins executing the newly loaded sector. This is the Volume Boot Record (VBR) of the active partition, and it typically contains the start of a larger bootloader.</p> </li> </ol> Key term: BIOS Interrupt 0x13</strong> A firmware service for reading and writing disk sectors. The boot code places parameters in CPU registers (drive number, cylinder/head/sector address, buffer address) and triggers a software interrupt. The BIOS handles the actual hardware communication. This is the only way to read a disk in real mode without writing a hardware driver from scratch. </div> The BIOS Disk Interface</h2> Reading from disk in real mode means using BIOS services. The CPU cannot talk to a SATA controller or NVMe drive directly -- those protocols require setup that only the firmware has done. Instead, the boot code uses INT 0x13 with function 0x02 (read sectors).</p> The original interface uses Cylinder-Head-Sector (CHS) addressing. You specify which cylinder, which head, and which sector to read. CHS has a hard limit: it can address about 8 GB of disk space. For disks larger than that, an extended interface (INT 0x13 with function 0x42) uses Logical Block Addressing (LBA), where sectors are numbered sequentially from zero.</p> The Stage 1 code in a modern bootloader like GRUB uses LBA when available. It checks whether the BIOS supports the extensions (INT 0x13, function 0x41) and falls back to CHS if not.</p> The Stage 1 bootloader has 440 bytes of executable space. It runs in real mode with a 1 MB address limit. Its only job is to find the active partition, load its first sector using BIOS disk services, and jump to it. Everything else -- filesystem parsing, user menus, kernel loading -- must wait for a larger program. </div> The Chain Continues</h2> The code that Stage 1 loads from the active partition is sometimes called the Volume Boot Record. For simple bootloaders, this might be enough to start an operating system directly. But for modern systems with large kernels, compressed images, and configuration files, even the VBR is too small.</p> This is why GRUB and similar bootloaders use a multi-stage design. Stage 1 loads a small intermediate piece (often called Stage 1.5), which contains just enough code to read a filesystem. That intermediate piece then loads the full Stage 2 from a known location on the partition.</p> Fig. 08c -- Boot chain: Stage 1 to Stage 2</div> Kernel</text></p> BIOS loads</text> Sector 0 (LBA)</text> Sectors 1-62</text> /boot/grub/</text> </svg></p> The boot chain progresses from smallest to largest. Each stage loads the next, adding capabilities (filesystem access, configuration parsing, kernel loading) as it goes.</div> </div> </figure> Where Does Stage 1.5 Live?</h3> On an MBR-partitioned disk, the first partition typically does not start until sector 63 (or sector 2048 on modern tools). The sectors between the MBR (sector 0) and the first partition form a gap. GRUB installs its Stage 1.5 -- called core.img</code> -- in this gap. It contains enough code to understand a filesystem (ext4, XFS, Btrfs, or whatever /boot</code> uses) so it can find and load the full GRUB Stage 2.</p> On a GPT-partitioned disk, the partition entries occupy the sectors immediately after the protective MBR, so there is no convenient gap. Instead, GPT systems use a dedicated BIOS Boot Partition -- a small partition (typically 1 MB) with a specific type GUID, reserved for the bootloader's intermediate code.</p> The Fragility of Stage 1</h2> The MBR is a single point of failure. If those 512 bytes are corrupted -- by a bug, a careless dd</code> command, or malware -- the system will not boot. The firmware will either fail the 0x55AA check and move to the next device, or execute garbage instructions and crash.</p> This is why disk cloning tools and system administrators treat the MBR with care. Overwriting the first sector of a disk destroys the bootloader and the partition table in one stroke. Backup tools like dd if=/dev/sda of=mbr.bin bs=512 count=1</code> exist specifically to preserve it.</p> The MBR design dates to 1983. It was built for disks measured in tens of megabytes. The 4-entry partition table can describe at most four primary partitions, and CHS addressing limits disk access to about 8 GB. These limitations are why GPT and UEFI eventually replaced the MBR scheme -- but legacy BIOS booting with MBR is still common, and understanding it explains why the newer systems were designed the way they were.</p> Stage 1 is the smallest link in the boot chain. It occupies 440 bytes inside the 512-byte MBR, runs in 16-bit real mode, and uses BIOS interrupts to load the next stage from disk. Its only job is to hand off to something bigger. Every modern bootloader (GRUB, syslinux, Windows Boot Manager) starts with this same tiny seed. </div> What Happens Next</h2> Stage 1 has done its job. It found the active partition (or loaded GRUB's core.img from the post-MBR gap), read it into memory, and jumped to it. The CPU is still in real mode. It still has only 1 MB of address space. But now there is a much larger program running -- one with enough code to read filesystems, parse configuration files, and present a menu.</p> That program is GRUB Stage 2, and it is the subject of the next article.</p> Next: Stage 2: GRUB</a></p> Finding the Boot Device 2025-01-09T00:00:00+00:00 The firmware has tested the hardware, trained the memory, and enumerated every bus. It knows what devices are connected. Now it faces a practical question: which one has an operating system on it?</p> A typical system might have an NVMe SSD, a SATA hard drive, a USB flash drive, and a network adapter. Any of them could potentially contain bootable code. The firmware needs a reliable method for deciding which one to try first, and a way to recognize bootable media when it finds one.</p> The Boot Order</h2> Every firmware stores a list of boot devices, ordered by priority. This list lives in NVRAM -- a small region of non-volatile storage on the motherboard that persists across power cycles. When you enter the firmware setup screen and rearrange the boot order, you are editing this list.</p> The firmware works through the list from top to bottom. It tries the first entry. If that device is not present, or does not contain bootable data, the firmware moves to the next entry. If nothing on the list works, most systems display an error -- "No bootable device found" or similar -- and halt.</p> Key term: NVRAM (Non-Volatile Random-Access Memory)</strong> A small region of persistent storage on the motherboard, typically backed by flash memory or a battery-maintained CMOS chip. The firmware uses NVRAM to store configuration settings: boot order, date/time, hardware preferences, and (on UEFI systems) boot variables that point to specific files on specific partitions. </div> On legacy BIOS systems, the boot order is a simple list of device types: first hard disk, CD-ROM, USB, network. The firmware does not know about specific partitions or files. It just tries to read from each device in order.</p> On UEFI systems, the boot order is more specific. Each entry can point to a particular file on a particular partition of a particular disk. A UEFI boot entry might say: "load \EFI\ubuntu\shimx64.efi</code> from partition 1 of NVMe disk 0." This precision means UEFI systems can have multiple operating systems installed on the same disk, each with its own boot entry, without conflict.</p> Fig. 07a -- Boot order resolution</div> 1. NVMe SSD</text> 2. USB Drive</text> 3. SATA HDD</text> 4. Network (PXE)</text> 5. CD-ROM</text></p> Try NVMe SSD</text> Read sector 0</text> or locate ESP</text> Bootable?</text> YES</text> Load it!</text> NO</text> Try next entry</text> No bootable device found</text> </marker> </marker> </marker> </defs> </svg> The firmware walks the boot order list top to bottom, testing each device for bootable signatures or files. The first success wins. If nothing works, boot fails.</div> </div> </figure> MBR: The Legacy Partition Table</h2> The Master Boot Record is the original PC disk format, dating back to 1983. It occupies the very first sector of the disk -- 512 bytes at LBA 0 (Logical Block Address zero). Those 512 bytes are divided into three parts.</p> The first 446 bytes contain executable code -- the bootstrap program. The next 64 bytes contain the partition table: four entries of 16 bytes each, describing up to four primary partitions. The final 2 bytes are a magic number: 0x55AA</code>. This signature tells the firmware that the sector contains valid boot code.</p> Key term: MBR (Master Boot Record)</strong> The first 512 bytes of a legacy-partitioned disk. It contains a small bootstrap program, a table of up to four partition entries, and a two-byte signature (0x55AA). The firmware loads this sector into memory and jumps to the code at offset zero. </div> Each MBR partition entry contains a start address, a size (both in 32-bit LBA), a partition type byte, and a status byte. The status byte is either 0x80</code> (active, or "bootable") or 0x00</code> (inactive). Only one partition should be marked active at a time. The bootstrap code in the MBR typically scans the partition table for the active partition, loads the first sector of that partition (the Volume Boot Record), and jumps to it.</p> Fig. 07b -- MBR structure (512 bytes)</div> Offset:</text> 0x000</text> 0x1BE</text> 0x1FE</text></p> Bootstrap Code</text> 446 bytes of x86 machine code</text> Partition Table</text> 4 entries x 16 bytes = 64B</text> 0x55</text> 0xAA</text> Partition Entry Format (16 bytes each)</text></p> Status</text> 0x80</text> CHS Start</text> 3 bytes</text> Type</text> 0x83</text> CHS End</text> 3 bytes</text> LBA Start</text> 4 bytes</text> Size (LBA)</text> 4 bytes</text> 0x80 = active</text> 0x00 = inactive</text></p> 0x83 = Linux</text> 0x07 = NTFS</text> 0x82 = swap</text></p> 32-bit = max</text> 2 TiB disk</text> </svg></p> The MBR packs bootstrap code, four partition entries, and a boot signature into exactly 512 bytes. The status byte marks which partition is "active" -- the one the bootstrap code should chain-load.</div> </div> </figure> The Boot Flag</h3> The "active" status byte -- 0x80</code> -- is commonly called the boot flag. It is how the MBR bootstrap code decides which partition to load next. The firmware itself does not check the boot flag. The firmware only checks the 0x55AA</code> signature at the end of the sector. The boot flag is interpreted by the code within the MBR.</p> This is a subtle but important distinction. The firmware's job is to find a disk with a valid MBR signature and execute the code there. That code is then responsible for finding the active partition and loading its boot sector. The firmware does not know or care about partitions. It only knows about whole disks.</p> MBR Limitations</h3> MBR has two significant limitations. First, the four-partition limit. You can work around it using "extended partitions" -- one of the four primary entries can point to a chain of logical partitions -- but the scheme is fragile and complicates boot code. Second, the 32-bit LBA fields limit addressable disk space to 2 TiB with 512-byte sectors. There is no way to extend this without breaking the format.</p> GPT: The Modern Partition Table</h2> The GUID Partition Table, introduced alongside UEFI, solves both of MBR's problems. It uses 64-bit LBA fields, supporting disks up to 9.4 ZB (zettabytes) -- far beyond anything that exists today. It supports up to 128 partition entries by default, each identified by a 128-bit GUID.</p> A GPT disk starts with a protective MBR at LBA 0. This is a standard MBR with a single partition entry covering the entire disk, typed as 0xEE</code> (GPT protective). Its purpose is to prevent MBR-only tools from treating the disk as unpartitioned and overwriting the GPT data.</p> The actual GPT header lives at LBA 1. It contains a magic signature (EFI PART</code>), the disk GUID, the number of partition entries, and the LBA of the partition entry array. The partition entries follow the header, typically occupying LBA 2 through 33.</p> Key term: GPT (GUID Partition Table)</strong> The modern partition table format used on UEFI systems. Each partition has a 128-bit type GUID and a 128-bit unique GUID. GPT supports 64-bit addressing (no practical disk size limit) and up to 128 partitions. A backup copy of the header and entries is stored at the end of the disk. </div> GPT also stores a backup copy of the header and all partition entries at the end of the disk. If the primary header is corrupted, the firmware or OS can recover from the backup. MBR has no such redundancy.</p> The EFI System Partition and Boot Discovery</h3> On a UEFI system, the firmware does not execute code from the first sector. Instead, it looks for an EFI System Partition -- a partition with the specific type GUID C12A7328-F81F-11D2-BA4B-00A0C93EC93B</code>. This partition must be formatted as FAT32 (or FAT12/FAT16 for small media).</p> The firmware reads the UEFI boot variables from NVRAM. Each variable specifies a file path on a specific partition of a specific disk. For example: "disk 0, partition 1, \EFI\fedora\shimx64.efi</code>." The firmware mounts the FAT32 partition, navigates to that path, and loads the EFI executable.</p> If no boot variable matches, or the specified file is missing, the firmware falls back to a default path: \EFI\BOOT\BOOTX64.EFI</code> for 64-bit x86, \EFI\BOOT\BOOTAA64.EFI</code> for ARM64. This fallback is what makes USB installation media work without special NVRAM configuration -- the installer puts a bootloader at the default path, and any UEFI system will find it.</p> Fig. 07c -- UEFI boot device discovery</div> UEFI reads boot variables from NVRAM, locates the ESP on the specified disk, and loads the named EFI executable. If that fails, it falls back to the default boot path.</div> </div> </figure> Network Boot: PXE</h2> Not all boot devices are local disks. PXE -- Preboot Execution Environment -- allows a system to boot over the network. The firmware's network stack sends a DHCP request to get an IP address. The DHCP response includes the address of a TFTP server and the filename of a boot program. The firmware downloads that program over TFTP and executes it.</p> PXE is essential in data centers and labs where hundreds of machines need to be installed or re-imaged without anyone plugging in USB drives. It is also the fallback when local disks fail -- a system with a dead SSD can PXE boot into a recovery environment.</p> On UEFI systems, network boot loads an EFI executable over TFTP or HTTP. On legacy BIOS systems, it loads a 16-bit bootstrap program called a Network Bootstrap Program (NBP). In both cases, the network boot entry is just another item in the boot order list, tried in sequence like any disk.</p> The firmware discovers boot devices by walking the boot order stored in NVRAM. For legacy BIOS, it looks for the 0x55AA signature in the first sector of each disk. For UEFI, it reads structured boot variables that point to specific EFI executables on specific partitions. PXE allows booting over the network when no local disk is available or desired. </div> How the Firmware Talks to Disks</h2> The firmware needs storage drivers to read from disks. On a legacy BIOS system, these drivers are the INT 13h interrupt routines -- firmware-provided functions that can read and write disk sectors. INT 13h is slow (it reads one sector at a time through the CPU) and limited, but it works for the boot process.</p> On a UEFI system, the firmware loads Block I/O Protocol drivers during the DXE phase. These drivers can talk to NVMe controllers, SATA controllers, and USB mass storage devices. The firmware also loads a Simple File System Protocol driver for FAT32, which is how it reads files from the ESP. These drivers are full 64-bit code with access to all system memory -- a far cry from the 16-bit, 1 MB world of INT 13h.</p> The firmware does not need to understand every filesystem. It only needs FAT32 for reading the ESP. The operating system brings its own drivers for ext4, NTFS, XFS, Btrfs, and everything else. The firmware's job is strictly to get the bootloader loaded and running.</p> Removable Media and Hot-Plug</h2> USB drives and optical media add a complication: they can be inserted after power-on. The firmware must periodically re-scan USB buses to detect new devices. Most UEFI implementations do this automatically. Legacy BIOS systems vary -- some re-scan USB at boot menu time, others only detect devices that were present at POST.</p> This is why the timing of inserting a USB boot drive matters. On some systems, you need to plug in the USB drive before powering on. On others, you can insert it at the boot menu. UEFI systems are generally more flexible because their USB drivers run continuously during the firmware phase.</p> Fig. 07d -- Boot media detection methods compared</div> How Firmware Validates Boot Media</text></p> Legacy BIOS</text> 1. Read LBA 0 (512 bytes)</text> 2. Check bytes 510-511</text> Must be 0x55 0xAA</text> 3. Copy sector to 0x7C00</text> 4. Jump to 0x7C00</text> No signature check</text> No file path awareness</text></p> UEFI</text> 1. Read GPT, find ESP</text> 2. Mount FAT32 filesystem</text> 3. Locate .EFI file by path</text> 4. Verify Secure Boot sig</text> 5. Load into memory, execute</text> Cryptographic verification</text> Named file on known path</text> </svg></p> BIOS trusts a two-byte magic number. UEFI verifies a cryptographic signature on a named file. The difference in security posture is enormous.</div> </div> </figure> Putting It Together</h2> Here is the complete sequence, from the moment the firmware starts looking for a boot device to the moment it hands off to the bootloader:</p> The firmware reads the boot order from NVRAM.</li> For each entry, it checks whether the device is present.</li> If present, it reads the disk's partition table (MBR or GPT).</li> On a BIOS system: it checks the MBR for the 0x55AA</code> signature, loads the 512-byte sector to memory address 0x7C00</code>, and jumps to it.</li> On a UEFI system: it locates the ESP, mounts the FAT32 filesystem, finds the specified EFI file, optionally verifies its Secure Boot signature, loads it into memory, and calls its entry point.</li> If the boot attempt fails, it moves to the next entry in the boot order.</li> If all entries fail, it displays an error and halts (or offers a setup menu).</li> </ol> Finding the boot device is a search problem with a strict protocol. The firmware does not guess. It follows the boot order list, checks each device against known criteria (MBR signature or ESP with an EFI executable), and loads the first thing that passes the test. This is the last step the firmware performs on its own. From here, control passes to the bootloader -- the first piece of software that was chosen by the user or the OS installer, not by the hardware vendor. </div> The firmware's work is now done. The bootloader it loaded will take over, find the operating system kernel, and begin the process of bringing up a full OS. That is the next chapter in the boot sequence.</p> Next: Stage 1: The Bootloader</a></p> The Bus System 2025-01-08T00:00:00+00:00 A CPU by itself can compute, but it cannot do anything useful without talking to other hardware: disks to read from, displays to write to, network adapters to send packets through. The wires and protocols that connect the CPU to everything else are called buses. The bus system is how a computer becomes more than a processor in a void.</p> During boot, the firmware must discover every device attached to every bus, assign each one an address, and configure it so the operating system can use it later. This process is called bus enumeration, and it happens before the OS loads.</p> What a Bus Actually Is</h2> A bus is a shared communication channel. In the simplest case, it is a set of parallel wires connecting two or more components. One device puts data on the wires, another reads it off. A protocol defines who gets to talk when and how addresses are specified.</p> Think of a bus as a shared hallway in an apartment building. Anyone can walk down it, but there are rules: you do not block the hallway, you knock on the right door, and you wait your turn. The hallway itself is just space -- the rules are what make it work.</p> Key term: Bus</strong> A communication pathway shared by multiple devices. A bus includes the physical wires (or lanes), a protocol defining how data is transferred, and an addressing scheme so each device knows which messages are meant for it. </div> Early personal computers had a single bus for everything. The original IBM PC's ISA bus carried CPU instructions, memory access, and I/O device communication on the same set of wires. Modern systems split this into several specialized buses, each optimized for different types of traffic.</p> The Modern Bus Hierarchy</h2> A modern x86 system has a layered bus hierarchy. The CPU sits at the top. It connects to memory through dedicated high-speed channels (the memory bus, which we covered in the previous article). It connects to everything else through a hierarchy of buses that branch out like a tree.</p> Fig. 06a -- Modern bus hierarchy</div> Bandwidth decreases as you move away from the CPU</text> </svg></p> The CPU connects to memory directly and to everything else through PCIe. High-bandwidth devices (GPU, NVMe) connect to the CPU's PCIe root complex. Lower-speed devices connect through the chipset, which itself is a PCIe device.</div> </div> </figure> The key component between the CPU and most peripherals is the chipset, which Intel calls the Platform Controller Hub (PCH) and AMD calls the Fusion Controller Hub (FCH). The chipset connects to the CPU through a dedicated PCIe link and provides the buses for lower-speed devices: SATA ports, USB controllers, audio, Ethernet, and additional PCIe slots.</p> PCIe: The Backbone</h2> PCI Express (PCIe) is the dominant interconnect in modern computers. It replaced the older PCI and AGP buses. Unlike those parallel buses, PCIe uses serial point-to-point connections called lanes. Each lane is a pair of differential signal pairs -- one pair for sending, one for receiving -- so data flows in both directions simultaneously.</p> Key term: PCIe Lane</strong> A single bidirectional serial link made of two pairs of wires (four wires total). Each lane can transfer data in both directions at the same time. Devices are connected with x1, x4, x8, or x16 lanes depending on how much bandwidth they need. </div> A PCIe x1 link has one lane. A x16 slot has sixteen lanes. A single PCIe 4.0 lane delivers about 2 GB/s in each direction. A x16 GPU slot therefore provides roughly 32 GB/s of bandwidth to the graphics card. An NVMe SSD typically uses a x4 link, giving it about 8 GB/s.</p> PCIe is not a shared bus in the traditional sense. Each device has its own dedicated link to either the CPU's root complex or the chipset. There is no contention for wire time the way there was on the old PCI bus. If two PCIe devices want to send data at the same time, they can -- they are on separate wires.</p> PCIe Enumeration</h3> At boot, the firmware must discover every PCIe device in the system. It does this by walking the bus hierarchy.</p> PCIe uses a tree topology. The root complex is at the top. Below it are bridges (or switches) that fan out to more devices. Each device has a configuration space -- a block of registers at a known address that describes the device: its vendor ID, device ID, what class of device it is (storage, network, display, etc.), and what resources it needs (memory ranges, I/O ports, interrupt lines).</p> The firmware scans this tree by reading the configuration space at every possible bus/device/function address. If something responds, the firmware records it and assigns it resources. If nothing responds, the firmware moves on. This process is called enumeration.</p> Fig. 06b -- PCIe enumeration: scanning the tree</div> PCIe Configuration Space Scan</text></p> Root Complex (Bus 0)</text> Bus 0, Dev 0</text> GPU: 10de:2684</text> Bus 0, Dev 1</text> PCIe Bridge</text> Bus 0, Dev 2</text> NVMe: 144d:a80a</text> Bus 1</text> Bus 1, Dev 0</text> Ethernet: 8086:15b8</text> Bus 1, Dev 1</text> No response</text> Device found</text> No device (skip)</text> Bridge (scan sub-bus)</text> </svg> The firmware reads configuration registers at each possible address. Devices that respond get recorded and assigned resources. Bridges reveal additional buses to scan.</div> </div> </figure> Every PCIe device has a 256-byte (PCIe extends this to 4096 bytes) configuration space. The first 64 bytes are standardized. Byte 0 and 1 are the vendor ID. Bytes 2 and 3 are the device ID. The class code at offset 0x09-0x0B tells the firmware what kind of device it is -- mass storage, network controller, display adapter, and so on. The firmware does not need a device-specific driver to enumerate PCIe. It just reads these standard registers.</p> Address Spaces: Memory-Mapped I/O</h2> When the firmware assigns resources to a device, it gives that device a range of memory addresses. But these addresses do not point to RAM. They point to the device itself. When the CPU reads or writes to one of these addresses, the transaction travels over the PCIe bus to the device instead of to DRAM. This is called memory-mapped I/O, or MMIO.</p> Key term: Memory-Mapped I/O (MMIO)</strong> A scheme where hardware device registers are assigned addresses in the same address space as system memory. The CPU accesses device registers using the same load and store instructions it uses for RAM. The chipset routes the transaction to the correct bus based on the address. </div> From the CPU's perspective, the address space is one big range of numbers. Some ranges map to RAM. Other ranges map to PCIe devices. The chipset and root complex route each memory access to the right destination based on address ranges configured during enumeration. The firmware sets up these mappings; the OS can adjust them later.</p> This is why a 32-bit system with 4 GB of RAM might only report 3.2 GB available. The missing 800 MB of address space is reserved for MMIO -- GPU frame buffers, device registers, and the firmware itself. On 64-bit systems, MMIO regions are typically mapped above the 4 GB line, so all physical RAM is accessible.</p> The firmware enumerates every bus and assigns every device an address range before the operating system loads. The OS inherits this map and can modify it, but the initial discovery is the firmware's job. Without enumeration, the OS would have no idea what hardware is present. </div> USB: A Different Model</h2> USB (Universal Serial Bus) takes a different approach from PCIe. Where PCIe devices are discovered by scanning a static tree of addresses, USB devices announce themselves when they are plugged in. The USB host controller detects a voltage change on the port, then initiates a conversation with the new device to learn its type and capabilities.</p> USB is also hierarchical. A host controller connects to a root hub. The root hub has ports. External hubs can be connected to those ports, creating a tree up to seven levels deep with up to 127 devices. Each device gets a dynamic address assigned by the host controller during enumeration.</p> The firmware needs USB working early for a practical reason: keyboards and mice. If you want to enter the firmware setup screen or select a boot device, the firmware must have a USB driver running. UEFI firmware includes USB drivers as part of its DXE phase; BIOS implementations had to include USB support in their interrupt service routines.</p> SATA and Storage Buses</h2> SATA (Serial ATA) connects traditional hard drives and older SSDs. Like PCIe, it uses serial point-to-point links, but it has a simpler protocol optimized for block storage. Each SATA port connects to exactly one device. The SATA controller typically lives on the chipset and presents itself to the firmware as a PCIe device.</p> Modern NVMe SSDs bypass SATA entirely. They connect directly via PCIe, which gives them access to the full PCIe bandwidth and a much simpler command protocol. A SATA SSD maxes out at about 550 MB/s. An NVMe SSD on PCIe 4.0 x4 can reach 7,000 MB/s. The difference is not just speed -- NVMe supports 65,535 command queues compared to SATA's single queue, which matters enormously for parallel workloads.</p> Fig. 06c -- Storage bus bandwidth comparison</div> Storage Interface Bandwidth</text></p> SATA III</text> 0.6 GB/s</text></p> NVMe PCIe 3.0</text> 3.5 GB/s</text></p> NVMe PCIe 4.0</text> 7.0 GB/s</text></p> NVMe PCIe 5.0</text> 14.0 GB/s</text></p> 0</text> 7 GB/s</text> 14 GB/s</text> </svg> Each generation of PCIe roughly doubles the bandwidth available to storage. NVMe SSDs on PCIe 5.0 are more than 20 times faster than a SATA III drive.</div> </div> </figure> DMA: Letting Devices Access Memory Directly</h2> Normally, data moves through the CPU: the CPU reads from the device, then writes to memory, or vice versa. This works but wastes CPU cycles on what is essentially a copy operation.</p> DMA -- Direct Memory Access -- lets devices read from and write to system memory without involving the CPU at all. A network adapter receiving a packet can write the data directly into a buffer in RAM, then interrupt the CPU to say "your data is ready." The CPU never touches the data during the transfer.</p> Key term: DMA (Direct Memory Access)</strong> A capability that allows hardware devices to transfer data directly to or from system memory without CPU involvement. The device is given a memory address and a length, and it handles the transfer autonomously. The CPU is free to do other work during the transfer. </div> DMA is essential for high-throughput devices. A PCIe 4.0 NVMe drive can push 7 GB/s. If the CPU had to mediate every byte of that transfer, it would spend most of its time just copying data. DMA lets the drive write directly to RAM while the CPU runs application code.</p> The security implication is significant. A device with DMA access can read any physical memory address. A malicious PCIe device -- or a compromised firmware on a legitimate device -- could read encryption keys, passwords, or any other data in RAM. This is why modern systems include an IOMMU (Input/Output Memory Management Unit) that restricts which memory regions each device can access. The IOMMU is configured by the firmware and managed by the OS.</p> The bus system is the circulatory system of the computer. PCIe provides the high-bandwidth backbone, USB handles human-interface and peripheral devices, and SATA connects traditional storage. The firmware discovers all of these during enumeration, assigns addresses, and builds the device map that the operating system will inherit. DMA lets devices move data without burdening the CPU, but it requires careful access control. </div> With memory working and the bus system enumerated, the firmware knows what hardware is available. The next step is finding something to boot from.</p> Next: Finding the Boot Device</a></p> Memory Initialization 2025-01-07T00:00:00+00:00 When you press the power button, the processor wakes up and starts executing firmware from flash memory. But there is a problem: the system's main memory -- the DRAM installed in those slots on the motherboard -- does not work yet. It is not broken. It simply has not been configured.</p> This is one of the least understood parts of the boot process. The CPU needs memory to run software, but configuring memory requires running software. The firmware has to solve this chicken-and-egg problem before anything else can happen.</p> Why RAM Does Not Just Work</h2> A stick of RAM is not a simple storage device. It is a dense grid of capacitors and transistors organized into rows and columns across multiple internal banks. Each capacitor holds one bit of data as an electrical charge. Reading or writing any bit requires activating the correct row, waiting for the signals to stabilize, then selecting the correct column.</p> The timing of every one of these operations matters. How long the row signal must be held before the data is valid. How many clock cycles must elapse between activating one row and activating another. How frequently each row must be refreshed before the capacitors leak their charge. These timings are measured in clock cycles and nanoseconds, and they vary between manufacturers, speeds, and even production batches.</p> Key term: DRAM (Dynamic Random-Access Memory)</strong> The main system memory in virtually all computers. "Dynamic" means each bit is stored as a charge on a tiny capacitor that must be periodically refreshed -- re-read and re-written -- or the data disappears. This is in contrast to SRAM (Static RAM), which holds its state without refreshing but uses more transistors per bit. </div> If the memory controller sends commands with the wrong timing, the data that comes back is garbage. Worse, incorrect timings can cause electrical stress that damages the memory chips over time. The firmware must discover the correct timings for the specific memory modules installed, then program the memory controller to use them.</p> SPD: The Memory's Identity Card</h2> Every DDR memory module carries a small chip called an SPD EEPROM. SPD stands for Serial Presence Detect. This chip stores a data sheet about the module: its capacity, the number of ranks, the supported clock speeds, and a table of timing parameters specified by JEDEC -- the standards body that defines how DDR memory works.</p> Key term: SPD (Serial Presence Detect)</strong> A small read-only memory chip on each DRAM module that stores the module's specifications. The firmware reads SPD data over a simple two-wire bus (SMBus or I2C) to learn what memory is installed and what timings it supports. </div> The firmware reads SPD data at a very early stage, using a simple serial protocol called SMBus (System Management Bus). This bus works even before the main memory controller is configured because it runs on a separate, low-speed interface. Think of it as asking the memory stick to introduce itself before you try to have a conversation.</p> Fig. 05a -- SPD data read from a DDR module</div> The firmware reads identity and timing data from the SPD chip over a slow serial bus before the main memory interface is usable.</div> </div> </figure> The SPD data includes JEDEC-standard timing parameters. You may have seen these on memory packaging as numbers like "22-22-22-52." Those four numbers represent CAS latency (tCL), row-to-column delay (tRCD), row precharge time (tRP), and row active time (tRAS), all measured in clock cycles. They tell the memory controller the minimum number of cycles it must wait between each type of operation.</p> Memory Training: The Calibration Process</h2> Reading the SPD data tells the firmware what the memory modules are. But it does not tell the firmware how the signals actually behave on this specific motherboard, with this specific CPU, at the current temperature, with these specific trace lengths on the circuit board.</p> Electrical signals take time to travel along the copper traces between the CPU and the memory slots. The length of each trace is slightly different. The impedance varies with temperature. At DDR4 and DDR5 data rates -- billions of transfers per second -- even a fraction of a nanosecond of misalignment between the clock signal and the data signals will corrupt data.</p> This is why the firmware must "train" the memory. Training is a calibration process where the firmware writes known patterns to memory, reads them back, and adjusts the timing parameters until the reads return correct data consistently. It is like tuning a radio dial: you sweep through the range, find the sweet spot where the signal is clearest, then lock it in.</p> Fig. 05b -- Memory training: finding the timing window</div> DQS/DQ Eye Diagram (simplified)</text></p> TIMING OFFSET (ps)</text> VOLTAGE</text> FAIL</text> FAIL</text> PASS</text> (timing eye)</text> optimal sample point</text> firmware sweeps delay values</text> </marker> </marker> </defs> </svg> The firmware sweeps through possible timing offsets, looking for the window where data reads back correctly. It then sets the sample point in the center of that window for maximum reliability.</div> </div> </figure> Training involves multiple phases. Write leveling adjusts the timing of write commands to each DRAM chip individually. Read leveling does the same for reads. Gate training determines the exact moment to start capturing data from the bus. Each phase runs independently for each byte lane -- each group of eight data pins -- because the trace lengths differ.</p> The entire process can take hundreds of milliseconds. That does not sound like much, but remember: this happens before the system has usable memory. The firmware runs the training code from the CPU's internal cache, which is repurposed as temporary RAM. This technique is called Cache-as-RAM, or CAR.</p> Key term: Cache-as-RAM (CAR)</strong> A technique where the CPU's L1 or L2 cache is configured as a small block of read/write memory. This gives the firmware a few hundred kilobytes to work with -- enough to run the memory training algorithm -- before DRAM is available. Intel calls this mode "No-Eviction Mode." </div> The Memory Controller</h2> The component responsible for sending correctly-timed commands to the DRAM is the memory controller. In modern systems, the memory controller is built directly into the CPU die. Older systems had it in a separate chip called the northbridge.</p> The memory controller does several things. It translates memory addresses from the CPU into the bank, row, and column addresses that the DRAM chips understand. It schedules commands to maximize throughput -- reordering reads and writes to avoid unnecessary row activations. It manages the refresh cycle, periodically re-reading and re-writing every row so the capacitors do not lose their charge.</p> Fig. 05c -- Memory controller and DRAM organization</div> CMD/ADDR</text> DATA (DQ)</text> CLOCK</text></p> DDR channel</text></p> DRAM Module (1 DIMM)</text> Bank 0</text> rows x cols</text> Bank 1</text> rows x cols</text> Bank 2</text> rows x cols</text> ...</text> Access Sequence</text> 1. Activate row (tRCD wait)</text> 2. Read column (tCL wait)</text> 3. Precharge (tRP wait)</text> Refresh: every row re-read every 64 ms</text> Miss a refresh cycle and data silently corrupts</text> </svg> The memory controller on the CPU translates addresses into bank/row/column commands. Each access follows a strict sequence of timed operations. Refresh runs continuously in the background.</div> </div> </figure> After training completes, the firmware programs all the calibrated timing values into the memory controller's configuration registers. From this point forward, the CPU can read and write main memory normally. The training results are often saved to flash storage so that the next boot can reuse them, making subsequent boots faster. This is why changing or rearranging your RAM sticks sometimes triggers a longer boot -- the firmware detects the change and retrains from scratch.</p> DDR Generations</h2> The DDR standard has gone through several generations. Each generation roughly doubles the data transfer rate and reduces power consumption while increasing complexity.</p> DDR3 ran at 800 to 1066 MHz base clock with a peak transfer rate around 17 GB/s per channel. DDR4 pushed that to 1200-1600 MHz base, adding bank groups and finer timing granularity. DDR5, the current generation, doubled the channel width by splitting each module into two independent 32-bit channels, added on-die ECC (error correction within each chip), and moved the voltage regulator from the motherboard onto the module itself.</p> Each generation requires a different training algorithm. The firmware for a DDR5 system is substantially more complex than for DDR4 because DDR5 has more timing parameters, more per-bit adjustments, and the two-channel-per-DIMM architecture that requires independent calibration of each sub-channel.</p> RAM does not work at power-on. The firmware must read each module's SPD data, run a training algorithm that sweeps through timing parameters to find the reliable operating window, and program the memory controller with the results. Only then does the system have usable main memory. Everything before this point runs from flash or CPU cache. </div> Dual Channel and Interleaving</h2> Most consumer systems support two memory channels -- two independent paths between the CPU and the DRAM. When you install matching modules in both channels, the memory controller can interleave accesses: it sends even-addressed cache lines to one channel and odd-addressed cache lines to the other, effectively doubling the available bandwidth.</p> This is why memory kits are sold in pairs. Two 8 GB modules in dual channel will outperform a single 16 GB module in most workloads, even though the total capacity is the same. The firmware detects the channel configuration during training and enables interleaving automatically when the modules match.</p> Server systems take this further. A dual-socket server might have eight or twelve memory channels per processor, with multiple DIMMs per channel. The firmware must train every module on every channel, which is why enterprise servers often take noticeably longer to boot than desktops.</p> ECC: When Bits Flip</h2> Cosmic rays, alpha particles from chip packaging, and electrical noise can flip bits in DRAM. In a desktop, a flipped bit might crash an application or corrupt a file. In a server, it could corrupt a database or crash a hypervisor managing hundreds of virtual machines.</p> ECC memory -- Error-Correcting Code memory -- adds extra data bits to each word. For every 64 bits of data, ECC adds 8 bits of parity information. This allows the memory controller to detect and correct any single-bit error, and detect (but not correct) any two-bit error.</p> ECC requires support in the memory controller, the DRAM modules, and the firmware. The training process for ECC memory is slightly longer because the error-correction logic must also be calibrated. Consumer platforms often do not support ECC, while server and workstation platforms require it.</p> Memory initialization is one of the most time-consuming parts of the boot process and one of the most critical. Without trained, working DRAM, the system has nothing but a few hundred kilobytes of cache to work with. Everything that follows -- loading the bootloader, decompressing the kernel, building the page tables -- depends on the memory controller being correctly configured during this early phase. </div> What Happens Next</h2> Once memory training completes, the firmware copies itself from the cramped cache-as-RAM environment into proper DRAM and continues execution with megabytes of memory available. The next task is discovering all the other hardware in the system: storage controllers, network adapters, USB hubs, and graphics cards. That discovery happens through the bus system.</p> Next: The Bus System</a></p> UEFI vs Legacy BIOS 2025-01-06T00:00:00+00:00 The firmware is the first software that runs when you press the power button. For decades, that software was BIOS. Then a replacement arrived called UEFI. Most computers sold today use UEFI, but the older system is still everywhere -- in virtual machines, embedded devices, and the assumptions baked into millions of lines of existing code.</p> Understanding both systems is not optional. You will encounter both, and the differences between them shape everything from disk layout to operating system security.</p> The Original BIOS</h2> BIOS stands for Basic Input/Output System. IBM introduced it with the original PC in 1981. Its job was straightforward: test the hardware, find a disk with a boot program on it, load that program into memory, and hand over control.</p> The original BIOS ran in 16-bit real mode -- the same mode the Intel 8088 processor started in. It could address one megabyte of memory. It used interrupt-based service routines that an operating system could call to read a disk sector or write a character to the screen. The interface was simple, stable, and wildly successful.</p> Key term: Real mode</strong> The processor mode that all x86 CPUs start in. It uses 16-bit registers and can address only 1 MB of memory. Every x86 chip, even a modern 64-bit processor, wakes up in this mode for backward compatibility. </div> For twenty years, BIOS barely changed. Vendors added features -- boot menus, PCI scanning, ACPI tables -- but the core interface remained 16-bit, interrupt-driven, and limited to 1 MB of addressable memory. The disk layout it used, called MBR, capped out at 2 terabytes. By the early 2000s, these limits were becoming painful.</p> Why BIOS Had to Go</h2> Three problems forced the industry to act.</p> First, the 2 TB disk limit. MBR partition tables use 32-bit logical block addresses. With 512-byte sectors, that gives you 2^32 times 512 bytes -- exactly 2 TiB. Server vendors were already shipping disks larger than that.</p> Second, the 1 MB memory ceiling. BIOS code had to run in real mode or use awkward workarounds to access more memory. Writing firmware in 16-bit assembly was slow, error-prone, and hostile to features like network stacks or graphical setup screens.</p> Third, security. BIOS had no mechanism to verify that the code it loaded from disk was legitimate. Any program written to the first sector of a disk would be executed without question. Rootkits that replaced the boot sector were a real and growing threat.</p> Fig. 04a -- BIOS limitations at a glance</div> Legacy BIOS Constraints</text></p> Memory</text> 1 MB</text> 16-bit real mode</text> 20-bit address bus</text> No protected mode</text> firmware access</text> Disk</text> 2 TiB</text> 32-bit LBA in MBR</text> 512-byte sectors</text> 4 primary partitions</text> maximum</text> Security</text> None</text> No code signing</text> No verification</text> Any sector 0 code</text> runs unchecked</text> </svg> The three hard limits that made BIOS replacement inevitable: memory, disk size, and security.</div> </div> </figure> Enter UEFI</h2> UEFI stands for Unified Extensible Firmware Interface. Intel started the project in the mid-1990s under the name EFI for their Itanium processor, which had no legacy BIOS at all. The specification went through several revisions before being handed to an industry consortium called the UEFI Forum in 2005. Today, UEFI is the standard firmware interface for virtually all x86 and ARM platforms.</p> Key term: UEFI (Unified Extensible Firmware Interface)</strong> A specification that defines the interface between a computer's firmware and its operating system. Unlike BIOS, UEFI runs in 32-bit or 64-bit mode, can address all of system memory, and includes a driver model, a shell, and a boot manager. </div> UEFI is not a single piece of software. It is a specification -- a contract that describes how firmware should behave, what services it must provide, and how operating systems should interact with it. Different vendors write their own UEFI implementations. What they share is the interface.</p> The practical differences from BIOS are large. UEFI firmware runs in 32-bit or 64-bit protected mode from the start. It can address gigabytes of memory. It has a built-in boot manager that understands file systems, so it can load a boot program from a specific file path on a specific partition rather than blindly executing whatever sits in the first disk sector. It supports a driver model, a network stack, and a command-line shell -- all before the operating system loads.</p> GPT: The New Partition Table</h2> UEFI introduced a new disk layout called GPT -- the GUID Partition Table. Where MBR uses 32-bit addresses and supports four primary partitions, GPT uses 64-bit addresses and supports up to 128 partitions by default.</p> Fig. 04b -- MBR vs GPT disk layout</div> MBR Layout</text> Boot Code</text> 446 bytes</text></p> Partition Table</text> 4 entries x 16B</text> Sig</text> Partition 1</text> Partition 2</text> ...</text> Sector 0 (512 bytes) holds EVERYTHING: code + table + signature</text></p> GPT Layout</text></p> Protective</text> MBR</text> GPT Header</text> LBA 1</text> Partition Entries</text> 128 entries</text> Partitions ...</text> Backup GPT</text> Header+Entries</text> Backup copy at end of disk = redundancy</text> MBR: 4 partitions, 2 TiB max</text> GPT: 128 partitions, 9.4 ZB max</text> </svg></p> MBR crams everything into a single 512-byte sector. GPT spreads the metadata across multiple sectors and keeps a backup copy at the end of the disk.</div> </div> </figure> The first sector of a GPT disk contains a "protective MBR" -- a fake MBR entry that marks the entire disk as a single partition of an unknown type. This prevents old MBR-only tools from misinterpreting the disk as empty and overwriting the GPT data. It is a compatibility shim, nothing more.</p> Each GPT partition entry includes a 128-bit GUID (Globally Unique Identifier) as its type code and another GUID as its unique partition identifier. The operating system uses these GUIDs to identify partitions reliably, without depending on partition order or drive letters.</p> The EFI System Partition</h3> UEFI-booted systems have a small FAT32 partition called the EFI System Partition, or ESP. This is where the firmware looks for bootloader programs. The ESP typically sits at the beginning of the disk and is between 100 MB and 550 MB in size.</p> Key term: EFI System Partition (ESP)</strong> A FAT32-formatted partition with a specific GPT type GUID that UEFI firmware knows how to read. It contains bootloader executables, firmware updates, and other files the firmware needs before the operating system loads. </div> Inside the ESP, bootloaders live at well-known paths. For example, the default fallback path is \EFI\BOOT\BOOTX64.EFI</code> on a 64-bit x86 system. A Linux distribution might install its bootloader at \EFI\ubuntu\shimx64.efi</code>. The UEFI boot manager stores these paths in NVRAM variables and presents them as boot options.</p> The UEFI Boot Flow</h2> The boot process under UEFI looks different from BIOS. Instead of loading a 512-byte sector and jumping to it, UEFI firmware reads a full executable file from the ESP.</p> Fig. 04c -- UEFI boot flow vs BIOS boot flow</div> Legacy BIOS</text></p> POST (hardware test)</text> Load MBR sector 0</text> Execute 446-byte code</text> Chain to stage 2</text> Load OS kernel</text> UEFI</text></p> SEC + PEI (init)</text> DXE (load drivers)</text> BDS (find boot entry)</text> Load .EFI from ESP</text> Load OS kernel</text> Blind sector load</text> No filesystem awareness</text> File-based boot</text> Full driver model</text></p> </marker> </marker> </defs> </svg> BIOS loads raw bytes from a disk sector. UEFI loads a named file from a filesystem on a known partition. The UEFI path is longer but far more flexible.</div> </div> </figure> UEFI defines several boot phases internally. The Security (SEC) phase runs first, initializing the CPU cache as temporary RAM. The Pre-EFI Initialization (PEI) phase trains the memory and gets DRAM working. The Driver Execution Environment (DXE) phase loads firmware drivers for storage, network, and graphics. Finally, the Boot Device Selection (BDS) phase reads the boot variables from NVRAM and loads the selected bootloader from the ESP.</p> The operating system never sees most of this. By the time the bootloader runs, it has access to a rich set of UEFI services -- memory allocation, file I/O, network, console output -- that it can call through a well-defined function table. These services remain available until the OS calls ExitBootServices()</code>, at which point the firmware gets out of the way and the OS takes full control of the hardware.</p> Secure Boot</h2> Secure Boot is a UEFI feature that addresses the third problem: unsigned code execution. When Secure Boot is enabled, the firmware will only load and execute EFI binaries that are signed with a trusted cryptographic key.</p> The trust chain works like this. The firmware ships with a database of trusted keys, called the "db." It also has a database of revoked keys, called the "dbx." When the firmware loads an EFI binary from the ESP, it checks the binary's digital signature against these databases. If the signature matches a trusted key and has not been revoked, the binary runs. Otherwise, the firmware refuses to load it.</p> Secure Boot creates a chain of trust from firmware to bootloader to operating system. Each stage verifies the signature of the next before handing over control. This prevents bootkits -- malware that hides in the boot process below the operating system. </div> In practice, most systems ship with Microsoft's keys in the trust database because Microsoft signs the bootloaders for Windows and for the "shim" program that Linux distributions use. The shim is a small signed bootloader that contains the distribution's own key. It loads GRUB, checks GRUB's signature against that key, and GRUB in turn verifies the kernel. The chain extends from firmware to shim to GRUB to kernel.</p> You can add your own keys to the firmware database if you want to sign your own bootloaders. You can also disable Secure Boot entirely. The specification requires that users be able to turn it off on general-purpose PCs.</p> CSM: The Compatibility Bridge</h2> Many UEFI systems include a feature called the Compatibility Support Module, or CSM. The CSM is essentially a legacy BIOS emulator built into the UEFI firmware. When enabled, it lets the system boot from MBR disks using the old BIOS method.</p> The CSM exists because the transition from BIOS to UEFI took over a decade. Operating systems, bootloaders, and recovery tools all needed time to add UEFI support. The CSM gave users and vendors a way to run old software on new hardware.</p> Modern systems are dropping CSM support. Intel removed it from their reference firmware starting with 11th-generation processors. If you are installing a current operating system on current hardware, you should use native UEFI boot with GPT. The only reason to use CSM today is if you need to run old operating systems that lack UEFI bootloaders.</p> Why Both Still Exist</h2> BIOS is dead in new hardware. But it is alive and well in virtual machines, older servers, and embedded systems. QEMU and VirtualBox still default to BIOS-style boot in many configurations. Legacy operating systems -- anything before Windows Vista SP1 on the x86-64 side, or older Linux installers -- expect BIOS.</p> More importantly, the concepts from BIOS did not disappear. UEFI still performs POST. It still has to initialize memory, enumerate buses, and find a boot device. It does these things differently -- in 64-bit mode, with drivers and a filesystem -- but the sequence is recognizable. The BIOS boot process is the skeleton that UEFI built on.</p> UEFI replaced BIOS as the standard firmware interface, bringing 64-bit operation, GPT disk support, Secure Boot, and a proper driver model. But the fundamental job is the same: test the hardware, find something bootable, and load it. The next step in both paths is the same -- initializing system memory so the rest of the boot process has somewhere to work. </div> Understanding BIOS teaches you what firmware must accomplish. Understanding UEFI teaches you how modern systems accomplish it. You need both.</p> Next: Memory Initialization</a></p> Why Linux? 2025-01-05T00:00:00+00:00 This series is about what happens between the moment you press the power button and the moment a shell prompt appears. We have covered the hardware and firmware layers so far. From here on, we need to talk about an operating system -- the software that takes control after the firmware finishes its work.</p> We are going to use Linux. This article explains why.</p> This is not a value judgment. It is not an argument that Linux is "better" than Windows or macOS or FreeBSD. It is a practical decision based on one overriding requirement: you should be able to see, read, and understand every piece of the system we discuss.</p> The Requirement: Inspectability</h2> When this series describes a piece of the boot process, you should be able to open the source code and see exactly what happens. Not a summary. Not a vendor's documentation of what they say happens. The actual code that runs on the machine.</p> With proprietary operating systems, this is impossible. The Windows kernel source code is not available to the public. macOS is built on an open-source foundation (XNU/Darwin), but large portions of the system are closed. When we say "the kernel does X at this stage of boot," you would have to take our word for it.</p> Key term: Open source</strong> Software whose source code is publicly available for anyone to read, modify, and redistribute, subject to the terms of its license. Linux is released under the GNU General Public License version 2 (GPLv2), which requires that anyone distributing the software also make the source code available. </div> With Linux, every claim in this series is verifiable. When we say the kernel initializes the memory manager at a certain point during boot, you can open mm/init.c</code> and read the function. When we say the init system starts services in a particular order, you can read the systemd source code or the init scripts. Nothing is hidden.</p> This matters because understanding a computer is not about memorizing facts. It is about building a mental model that you can verify against reality. If the model and the reality disagree, you need access to the reality to figure out which one is wrong.</p> Where Linux Runs</h2> Linux is not a niche operating system. It is the dominant operating system in several of the largest computing domains.</p> Fig. 01 -- Linux deployment across computing domains</div> Understanding Linux boot means understanding how most</text> of the world's computing infrastructure starts up.</text> </svg></p> Linux dominates servers, cloud, supercomputing, containers, and embedded devices. Android phones run the Linux kernel. Learning the Linux boot process gives you knowledge that applies across the broadest range of real systems.</div> </div> </figure> Servers</strong>: Roughly 80% of public-facing web servers run Linux. If you visit a website, there is a strong chance the machine serving the page is running a Linux kernel.</p> Cloud computing</strong>: AWS, Google Cloud, and Azure all offer Linux instances, and the vast majority of cloud workloads run on Linux. AWS even built their own Linux distribution (Amazon Linux) for their infrastructure.</p> Supercomputers</strong>: Every single system on the TOP500 list of the world's fastest supercomputers runs Linux. One hundred percent. This has been true since 2017.</p> Embedded systems</strong>: Your home router almost certainly runs Linux. Many smart TVs, security cameras, automotive infotainment systems, and industrial controllers run Linux. The kernel's configurability makes it adaptable to hardware ranging from a microcontroller with 4 MB of RAM to a server with 4 TB.</p> Mobile</strong>: Android, which runs on roughly 72% of the world's smartphones, uses the Linux kernel. The userspace is different from what you would find on a server, but the kernel -- the part we will study in this series -- is the same codebase.</p> Containers</strong>: Docker and Kubernetes, the dominant container technologies, run on Linux. Container isolation uses Linux-specific kernel features (namespaces and cgroups) that do not exist in other kernels.</p> If you understand how Linux boots, you understand how most of the world's computing infrastructure starts up.</p> You Can Read the Source</h2> The Linux kernel source code is available at kernel.org</a>. As of early 2025, it contains roughly 36 million lines of code across approximately 80,000 files. That sounds overwhelming, but the boot path -- the code that executes between the kernel being loaded and the first user process starting -- is a small, well-defined subset.</p> Key term: Kernel</strong> The core of an operating system. The kernel manages hardware resources (CPU time, memory, devices), enforces security boundaries between processes, and provides the system call interface that all programs use to interact with hardware. In Linux, the kernel is a single binary (vmlinuz) loaded by the bootloader. </div> Here is a concrete example. During boot, the Linux kernel prints messages to the console. You have probably seen them -- lines scrolling by during startup, each prefixed with a timestamp. The function that handles this is printk()</code>, defined in kernel/printk/printk.c</code>. You can read it. You can see how the ring buffer works, how log levels are handled, how early boot messages are stored before the console is initialized.</p> This level of transparency is unique. No other widely-deployed operating system gives you complete access to its internals. FreeBSD and OpenBSD do, and they are excellent systems, but they do not have Linux's reach across servers, cloud, embedded, and mobile.</p> The Tooling Ecosystem</h2> Linux comes with tools that let you inspect the boot process while it is happening and after it has completed:</p> dmesg</strong> -- Prints the kernel's message buffer, showing every hardware detection, driver initialization, and boot event in chronological order.</li> journalctl</strong> -- On systemd-based systems, shows the complete log of the boot process, including both kernel and userspace messages.</li> /proc and /sys</strong> -- Virtual filesystems that expose kernel state at runtime. Want to know what the kernel detected about your CPU? Read /proc/cpuinfo</code>. Want to see the memory map? Read /proc/iomem</code>.</li> strace</strong> -- Traces system calls made by any process, letting you see exactly how programs interact with the kernel.</li> ftrace / perf</strong> -- Kernel tracing tools that can instrument the boot process itself, showing you the exact sequence of function calls and their timing.</li> </ul> Fig. 02 -- Inspecting the boot process with Linux tools</div> Linux provides tools to inspect every phase of the boot process, from kernel ring buffer messages to systemd timing analysis to runtime kernel tracing.</div> </div> </figure> These tools are not afterthoughts. They are built into the system specifically because Linux is designed to be understood. The kernel developers themselves use these tools to debug boot issues, and they are available to anyone running a Linux system.</p> Linux provides complete source code and built-in inspection tools (dmesg, journalctl, /proc, /sys, ftrace) that let you verify every claim about how the system works. No other widely-deployed OS offers this combination of transparency, tooling, and market reach. </div> What "Linux" Means in This Series</h2> When people say "Linux," they sometimes mean just the kernel, and sometimes mean an entire operating system including the kernel, system libraries, package manager, desktop environment, and applications. In this series, we will be precise about which we mean.</p> The Linux kernel</strong> is the software written by Linus Torvalds and thousands of contributors, maintained at kernel.org. It handles hardware management, process scheduling, memory management, filesystems, networking, and security. When we say "the kernel does X," we mean this specific piece of software.</p> A Linux distribution</strong> (distro) is a complete operating system built around the Linux kernel. It includes a bootloader (often GRUB), an init system (often systemd), system libraries (glibc or musl), a package manager (apt, dnf, pacman), and a collection of user applications. Examples include Debian, Fedora, Arch Linux, Ubuntu, and Alpine.</p> Fig. 03 -- Components of a Linux distribution</div> This series</text> </svg></p> A Linux distribution is a complete system built in layers. This series covers the full stack from bootloader through kernel initialization to the first shell prompt.</div> </div> </figure> This series covers the boot process from firmware handoff (where the bootloader takes over) through kernel initialization to the moment the first user process runs. That path touches the bootloader, the kernel, the init system, and the early userspace -- components found in every Linux distribution.</p> Where distribution-specific details matter (such as how GRUB is configured or how systemd orders its services), we will note which distribution we are using and point out where other distributions differ.</p> Why Not Windows? Why Not BSD?</h2> Windows is a fine operating system. It boots, it runs applications, it handles hardware. But its boot process is closed. You cannot read the Windows kernel source code. You cannot see what ntoskrnl.exe</code> does during initialization. You cannot trace the exact sequence of operations from bootloader to desktop. You can study Microsoft's documentation, which is extensive, but documentation is not source code.</p> FreeBSD and OpenBSD are open-source, well-documented, and technically excellent. Their boot processes are worth studying. But they serve a smaller user base. If you learn the FreeBSD boot process, you understand how a few percent of servers start up. If you learn the Linux boot process, you understand how the majority of the world's servers, all of its supercomputers, most of its cloud infrastructure, and billions of mobile devices start up.</p> Key term: Distribution (distro)</strong> A complete operating system package that includes the Linux kernel, bootloader, init system, system libraries, package manager, and applications. Different distributions make different choices about these components, but they all share the same kernel. Debian, Fedora, Arch, and Ubuntu are all Linux distributions. </div> This is not about allegiance. It is about reach. The knowledge you gain from studying the Linux boot process transfers to more real-world systems than any alternative.</p> What You Will Need</h2> To follow along with this series, you will want access to a Linux system. This could be:</p> A physical machine</strong> running any Linux distribution. Debian, Fedora, Ubuntu, Arch -- any will work.</li> A virtual machine</strong> using VirtualBox, QEMU/KVM, or VMware. This is the safest option for experimentation, since you can snapshot and restore.</li> A cloud instance</strong> on any provider. A small Debian or Ubuntu instance is inexpensive and gives you full root access.</li> Windows Subsystem for Linux (WSL2)</strong> -- usable for many exercises, though it does not boot a real Linux kernel in the traditional sense (the kernel is provided by Microsoft's lightweight VM).</li> </ul> The specific distribution does not matter much for this series. We will point out distribution-specific details when they arise. What matters is that you have root access and can inspect the system freely.</p> This series uses Linux because it is open-source (you can read every line of code), dominant (it runs most of the world's servers, cloud, and mobile devices), and well-tooled (dmesg, /proc, ftrace give you direct visibility into the boot process). It is a practical choice, not a tribal one. </div> What Comes Next</h2> We have covered the hardware layer (power delivery, the reset vector) and the firmware layer (POST and the BIOS). We briefly discussed why the legacy BIOS has limitations. The next article tackles the firmware that replaced it: UEFI.</p> UEFI changes the boot process significantly. It replaces the 512-byte MBR with a proper filesystem on a dedicated partition. It runs in 32-bit or 64-bit mode instead of real mode. It provides Secure Boot to verify the integrity of everything in the boot chain. And it standardizes the interface between firmware and operating system in ways the legacy BIOS never did.</p> Next: UEFI vs Legacy BIOS</a></p> POST and the BIOS 2025-01-04T00:00:00+00:00 The CPU has fetched its first instruction from the reset vector and jumped to the main body of the BIOS firmware. No operating system is running. No drivers are loaded. The firmware is alone with the bare hardware, and it has one job: get the system into a state where an operating system can take over.</p> The first thing the firmware does is test whether the hardware is working at all. This test has a name that has survived four decades of computing: the Power-On Self Test, or POST.</p> What Is the BIOS?</h2> BIOS stands for Basic Input/Output System. It is firmware -- software permanently stored on a chip on the motherboard. Unlike programs you install on a hard drive, the BIOS is present from the factory. It is the first code that runs on the machine.</p> The original IBM PC BIOS, released in 1981, fit in 8 KB of ROM. It provided basic routines for reading the keyboard, writing characters to the screen, and accessing floppy disks. Modern BIOS firmware (or its successor, UEFI) is measured in megabytes and handles tasks that the original designers never imagined: USB, PCIe, SATA, network boot, Secure Boot, and more.</p> Key term: BIOS (Basic Input/Output System)</strong> Firmware stored on a chip on the motherboard that initializes hardware and provides basic services during the boot process. The BIOS runs before the operating system and is responsible for bringing the hardware from a raw, uninitialized state to a state where an OS can be loaded. </div> Regardless of how complex modern firmware has become, the first task remains the same: test the hardware.</p> The Power-On Self Test</h2> POST is a series of diagnostic checks that the firmware performs immediately after gaining control of the CPU. These tests verify that the most fundamental hardware components are working before the firmware attempts anything more complex.</p> POST is not a single test. It is an ordered sequence of tests, and the order matters. The firmware tests the most basic components first, because later tests depend on earlier components working. You cannot test RAM if the memory controller is broken. You cannot display an error message if the video card is dead.</p> Fig. 01 -- POST test sequence</div> No RAM, no display</text> Verify BIOS not corrupt</text> System timer (PIT/HPET)</text> Direct Memory Access</text> Read SPD, train DIMMs</text> Write/read patterns</text> First visual output</text></p> Silent</text> Beep</text> codes</text> RAM</text> available</text> Display</text> </marker> </defs> </svg> POST tests are ordered by dependency. The earliest tests run with no RAM and no display. Only after memory is initialized and tested can the firmware use a stack or show anything on screen.</div> </div> </figure> CPU Register Test</h3> The very first test checks whether the CPU's own registers work correctly. The firmware writes known values to each register and reads them back. If a register cannot hold a value, the CPU is fundamentally broken and nothing else can proceed.</p> This test runs without RAM (which has not been initialized yet) and without any display output. If it fails, the system halts silently or the motherboard signals the failure through a POST code display or a pattern of beeps from the speaker.</p> ROM Checksum Verification</h3> Next, the firmware verifies its own integrity. It reads through the entire BIOS ROM and computes a checksum -- a mathematical summary of all the bytes. If the checksum does not match the expected value, the firmware itself is corrupt. This can happen due to a failed firmware update, a degraded flash chip, or cosmic ray bit-flips (rare but real).</p> Memory Controller Initialization</h3> This is one of the most complex and important tasks POST performs. The firmware must:</p> Detect which DIMM slots have memory installed.</li> Read the SPD data from each DIMM to learn its specifications.</li> Configure the memory controller with the correct speed, timings, and voltage.</li> "Train" the memory interface -- adjusting signal timing to account for the physical characteristics of the traces on the motherboard.</li> </ol> Key term: SPD (Serial Presence Detect)</strong> A small EEPROM chip on each memory DIMM that contains information about the module's capacity, speed, timings, and voltage requirements. The BIOS reads SPD data during POST to configure the memory controller correctly. </div> Memory training is a process where the firmware sends test patterns through the memory interface and adjusts timing parameters until data can be read back reliably. This process can take several hundred milliseconds -- it is one reason why turning on a computer is not instantaneous.</p> Once memory is initialized and tested, the firmware finally has RAM to work with. It can set up a stack, allocate buffers, and store variables. The execution environment goes from "CPU registers only" to something resembling a normal programming environment.</p> RAM Test</h3> After the memory controller is configured, the firmware tests the RAM itself. The classic RAM test writes a pattern to every address (often alternating 0x55 and 0xAA, which represent alternating bit patterns) and reads it back. If any location fails to hold its value, the firmware knows that RAM module is defective.</p> Modern systems often perform a quick RAM test rather than testing every byte, because testing 64 GB of memory byte-by-byte would take a very long time. The BIOS setup menu usually has an option to enable a thorough memory test if you suspect a problem.</p> Video Initialization</h3> Once RAM is working, the firmware initializes the video system. On a machine with a discrete graphics card, this involves loading and executing the video card's option ROM -- firmware stored on the graphics card itself. On systems with integrated graphics, the main BIOS handles video initialization directly.</p> This is the moment when you first see something on the screen. Before this point, the monitor has been receiving no signal. Now, the BIOS can display its manufacturer logo or a text-mode screen showing POST progress.</p> POST follows a strict order: CPU test, ROM checksum, timer and DMA setup, memory initialization, RAM test, then video. Each step depends on the previous one succeeding. The earliest tests run with no RAM and no display -- if they fail, the only feedback is through beep codes or diagnostic LEDs. </div> Beep Codes: When the Screen Is Not Available</h2> What happens when POST detects a failure before the video system is initialized? The firmware cannot display an error message because the display is not working yet. Instead, it uses the only output device available at that stage: the PC speaker.</p> The BIOS generates a pattern of beeps to indicate which test failed. These patterns are called beep codes, and they vary by BIOS manufacturer.</p> Fig. 02 -- Common beep codes (Award BIOS)</div> Beep codes encode error information in patterns of long and short tones. One short beep means success. Longer or repeated patterns indicate specific hardware failures detected during POST.</div> </div> </figure> A single short beep from an Award BIOS means POST completed successfully. One long beep followed by two short beeps means the video card failed. Continuous long beeps mean no memory was detected. Each BIOS vendor (Award, AMI, Phoenix) has its own set of codes.</p> Modern motherboards often replace or supplement beep codes with a two-digit hexadecimal POST code display -- a pair of seven-segment LEDs on the board that show the current POST stage. If the system hangs, the displayed code tells you exactly which test it got stuck on. These codes are listed in the motherboard manual.</p> Beyond POST: Hardware Enumeration</h2> Once POST confirms that the basic hardware is functional, the BIOS moves on to more complex initialization. This phase detects and configures all the devices attached to the system.</p> PCI/PCIe Bus Enumeration</h3> The BIOS scans the PCI and PCIe buses to discover what devices are installed -- graphics cards, network adapters, storage controllers, sound cards. Each device has a set of configuration registers that the BIOS reads to identify the device and assign it resources (memory-mapped I/O regions, interrupt lines, and so on).</p> Option ROM Execution</h3> Many expansion cards carry their own firmware, called an option ROM. The video card's option ROM initializes the GPU and provides basic display services. A network card's option ROM might provide PXE boot capability. A RAID controller's option ROM provides access to its disk arrays.</p> The BIOS scans for option ROMs in a specific memory range (0xC0000 to 0xDFFFF), validates their checksums, and executes them one by one. This is why you sometimes see a separate splash screen from your graphics card or RAID controller before the main BIOS screen appears.</p> Key term: Option ROM</strong> Firmware stored on an expansion card (such as a graphics card or network adapter) that extends the BIOS with device-specific initialization code and services. The BIOS discovers and executes option ROMs during POST. </div> Interrupt Vector Table</h3> The BIOS sets up the interrupt vector table (IVT) in the first 1 KB of RAM (addresses 0x0000 to 0x03FF). This table contains 256 entries, each pointing to a routine that handles a specific interrupt. Hardware interrupts from the keyboard, timer, and disk controller all route through this table.</p> The IVT is a real-mode structure, and it will be replaced entirely when the operating system switches to protected mode. But during the boot process, everything -- keyboard input, disk reads, screen output -- relies on the BIOS interrupt handlers in this table.</p> The BIOS Setup Screen</h2> You have probably seen it: press Delete or F2 during POST and you get a screen full of settings. This is the BIOS setup utility. It allows you to configure boot order, memory timings, CPU settings, and dozens of other parameters.</p> These settings are stored in a small battery-backed memory called CMOS RAM (or, on modern systems, in a section of the SPI flash chip). The battery -- the coin cell on your motherboard -- keeps these settings alive when the system is powered off.</p> The BIOS reads these settings during POST to determine how to configure the hardware. If the CMOS battery dies, the settings revert to factory defaults, which is why a dead CMOS battery often causes the system to lose its date and time.</p> Finding a Boot Device</h2> After all hardware is initialized and tested, the BIOS has one final task: find an operating system to load.</p> The BIOS checks devices in the order specified by the boot priority list (configured in the BIOS setup). For each device, it attempts to read the first sector -- 512 bytes -- from the device. If the last two bytes of that sector are 0x55 and 0xAA (the boot signature), the BIOS considers the device bootable.</p> Fig. 03 -- Boot device search sequence</div> Sector 0 (512 bytes) = Master Boot Record</text></p> Load to 0x7C00, JMP</text> </marker> </marker> </marker> </marker> </defs> </svg> The BIOS checks each boot device in priority order. When it finds a device whose first sector ends with the 0x55AA signature, it loads that sector into memory at address 0x7C00 and jumps to it.</div> </div> </figure> The BIOS loads those 512 bytes into RAM at address 0x7C00</strong> and jumps to that address. Control passes from the firmware to whatever code was in that boot sector. On a legacy BIOS system, this is the Master Boot Record (MBR), which typically contains a small bootloader that knows how to find and load a larger bootloader (like GRUB), which in turn loads the operating system kernel.</p> The address 0x7C00 is another one of those historical artifacts. It was chosen by the designers of the original IBM PC 5150 in 1981. The number has no deep significance -- it was simply a convenient location in memory that would not conflict with anything else. But because the entire PC ecosystem depends on it, it has persisted for over four decades.</p> The BIOS completes POST, initializes all hardware, enumerates buses, executes option ROMs, and then searches for a bootable device. When found, it loads the boot sector (512 bytes) to address 0x7C00 and transfers control to it. This is the moment firmware hands off to the boot process chain. </div> The BIOS Data Area</h2> During POST, the BIOS writes a collection of data to a reserved region of RAM starting at address 0x0400, known as the BIOS Data Area (BDA). This region contains information the BIOS has gathered about the system: the number and type of disk drives, the amount of conventional memory, the keyboard buffer, serial and parallel port addresses, and the current video mode.</p> The BDA is a communication channel. The BIOS writes it; early boot code and real-mode operating systems read it. It is yet another structure that only matters in real mode and will be overwritten once the operating system takes over.</p> Limitations of the Legacy BIOS</h2> The traditional BIOS, for all its importance, has significant limitations:</p> Real mode only</strong>: The BIOS runs in 16-bit real mode with a 1 MB address limit. It cannot directly use modern CPU features.</li> MBR partition limit</strong>: The MBR partition table supports a maximum of four primary partitions and disks up to 2 TB.</li> No standard security model</strong>: There is no mechanism to verify that the code being loaded is authentic.</li> No standard driver model</strong>: Each device needs its own option ROM; there is no plug-in driver architecture.</li> Slow initialization</strong>: The sequential, one-device-at-a-time nature of POST is slow on modern systems with many devices.</li> </ul> These limitations led to the development of UEFI, which replaces the legacy BIOS with a more modern firmware interface. UEFI runs in 32-bit or 64-bit mode, supports the GPT partition scheme (allowing disks larger than 2 TB and up to 128 partitions), includes a standard driver model, and provides Secure Boot for verifying boot code integrity.</p> We will cover UEFI in detail in Article 04. But first, a brief detour to explain why the rest of this series focuses on one particular operating system.</p> Next: Why Linux?</a></p> The Reset Vector 2025-01-03T00:00:00+00:00 The CPU has been released from reset. The clock is ticking. Voltages are stable. The processor is now expected to do something -- but what?</p> A CPU does not know what an operating system is. It does not know about hard drives, keyboards, or screens. It has no awareness of what it is or what it is supposed to do. It knows exactly one thing: an address. A specific, hardwired location in its address space where it must look for its first instruction.</p> That address is called the reset vector.</p> What Is an Address Space?</h2> Before we can talk about the reset vector, we need to talk about how a CPU sees memory. The CPU does not interact with physical RAM chips directly by name. Instead, it sees a flat, numbered sequence of memory locations called the address space.</p> Think of the address space as a very long row of numbered post office boxes. Box 0 is at one end. The last box is at the other. The CPU can read from or write to any box by specifying its number. That number is the address.</p> Key term: Address space</strong> The complete range of memory addresses a CPU can reference. A 32-bit CPU can address 2^32 locations (4 GB). A 64-bit CPU can theoretically address 2^64 locations, though practical implementations use fewer address lines. </div> On an x86 CPU (the architecture used in most PCs and servers), the address space at power-on is not what you might expect. Not every address maps to RAM. Some addresses map to ROM chips, some map to hardware device registers, and large regions may map to nothing at all.</p> This mapping of addresses to physical devices is called the memory map, and understanding it is essential to understanding the reset vector.</p> The Memory Map at Power-On</h2> When an x86 CPU first starts, it operates in a mode called real mode. In real mode, the processor can address 1 MB of memory (addresses 0x00000 through 0xFFFFF). This is a legacy constraint inherited from the original 8086 processor from 1978, and modern CPUs maintain backward compatibility with it.</p> But the CPU does not start fetching instructions from the bottom of that 1 MB range. It starts from near the top.</p> Fig. 01 -- x86 memory map at power-on (first 1 MB)</div> 0xFFFFF (1 MB)</text> 0xF0000</text> </p> Option ROMs</text> (Video BIOS, etc.)</text> 0xC0000</text> </p> Video RAM</text> 128 KB</text> 0xA0000</text> </p> Conventional RAM</text> 640 KB</text> (mostly empty at boot)</text> 0x00000</text> </p> ROM</text> ROM</text> MMIO</text> DRAM</text></p> Reset</text> vector</text> </marker> </defs> </svg> The first 1 MB of address space on an x86 system. The BIOS ROM occupies the top 64 KB, and the CPU's reset vector points into this region. Conventional RAM sits below, mostly empty at power-on.</div> </div> </figure> The top 64 KB (addresses 0xF0000 to 0xFFFFF) is mapped to a ROM chip -- specifically, the chip that contains the BIOS firmware. Below that are option ROMs (like the video card's built-in firmware), video memory, and then 640 KB of conventional RAM that is mostly empty at this point.</p> The Reset Vector on x86</h2> On an x86 processor, the reset vector is at address 0xFFFFFFF0</strong> -- sixteen bytes below the 4 GB boundary. Yes, you read that correctly. Even though the CPU is supposedly in real mode (which can only address 1 MB), the very first fetch after reset reaches up to the top of the 32-bit address space.</p> How is this possible? The answer is a quirk of x86 design. At reset, the CPU sets its code segment (CS) base to 0xFFFF0000, not the usual real-mode value. The instruction pointer (IP) is set to 0xFFF0. The combination gives a physical address of 0xFFFFFFF0. This trick allows the CPU to find the BIOS ROM, which the motherboard's chipset maps to the top of the 4 GB address space.</p> Key term: Reset vector</strong> A fixed memory address, hardwired into the CPU, that contains the first instruction to execute after a reset. On x86 processors, this address is 0xFFFFFFF0. The motherboard ensures that a ROM chip containing BIOS firmware is mapped at this address. </div> The first instruction at this address is almost always a jump (JMP) instruction. It jumps to the main body of the BIOS code, which might be at a completely different address. The reset vector is just the entry point -- a signpost that says "go over there to find the real startup code."</p> Fig. 02 -- The first instruction fetch</div> JMP to main</text> BIOS body</text></p> </marker> </marker> </defs> </svg> The CPU fetches its first instruction from 0xFFFFFFF0 (mapped to the BIOS ROM). That instruction is a JMP that redirects execution to the main body of the BIOS firmware.</div> </div> </figure> Why a Fixed Address?</h2> Why does the CPU need a hardwired starting address? Why not let the user configure it?</p> The answer is simple: at the moment of reset, there is no configuration. There are no settings. There is no software running to read a configuration file. The CPU needs to start somewhere, and that somewhere must be knowable without any prior setup. A fixed address baked into the silicon is the only practical solution.</p> Every CPU architecture has its own reset vector:</p> x86 (Intel/AMD)</strong>: 0xFFFFFFF0</li> ARM Cortex-A</strong>: 0x00000000 (or 0xFFFF0000 with high-vector configuration)</li> RISC-V</strong>: Implementation-defined, commonly 0x80000000</li> MIPS</strong>: 0xBFC00000</li> </ul> The address itself is arbitrary -- what matters is that the CPU designer and the board designer agree on it. The CPU will fetch from that address, and the board must ensure that a ROM containing valid code is mapped there.</p> The reset vector is a contract between the CPU and the motherboard. The CPU promises to fetch its first instruction from a specific address. The motherboard promises to have executable firmware at that address. </div> How the Chipset Makes It Work</h2> The CPU does not physically wire itself to a ROM chip. Between the CPU and any memory or I/O device sits the chipset -- a set of controller chips on the motherboard that routes addresses to the correct physical device.</p> When the CPU puts the address 0xFFFFFFF0 on its address bus, the chipset sees this address and knows it falls in the range assigned to the BIOS ROM (or, on modern systems, the SPI flash chip that contains the UEFI firmware). The chipset routes the read request to that chip, which responds with the bytes stored at that location.</p> This mapping is itself hardwired into the chipset. At power-on, the chipset has a default configuration that maps the top of the address space to the firmware flash chip. Later in the boot process, firmware might reconfigure the chipset to remap these regions, but at the instant of the first instruction fetch, the defaults are all that matter.</p> Fig. 03 -- Address routing through the chipset</div> Reset vector</text> fetch goes here</text></p> </marker> </marker> </marker> </defs> </svg> The chipset decodes the address from the CPU and routes the request to the correct physical device. At reset, the address 0xFFFFFFF0 is routed to the SPI flash chip containing the BIOS/UEFI firmware.</div> </div> </figure> The CPU's State at Reset</h2> When the reset line is deasserted and the CPU begins execution, its internal state is well-defined. The x86 architecture specifies exact values for every register after reset:</p> CS (Code Segment)</strong>: Base = 0xFFFF0000, selector = 0xF000</li> IP (Instruction Pointer)</strong>: 0xFFF0</li> All other segment registers</strong>: Selector = 0x0000</li> General purpose registers</strong>: Most set to 0x0000</li> EFLAGS</strong>: 0x00000002 (interrupts disabled)</li> CR0</strong>: 0x60000010 (real mode, cache disabled)</li> </ul> Two things stand out in this list. First, interrupts are disabled. The CPU will not respond to hardware interrupts until firmware explicitly enables them. This makes sense -- the interrupt vector table has not been set up yet, so responding to an interrupt would cause a crash.</p> Second, the cache is disabled. The CPU will read directly from the ROM chip for every instruction fetch, with no caching. This is slow, but it is safe. Caching requires configuration, and no configuration has happened yet.</p> Key term: Real mode</strong> The operating mode of an x86 CPU immediately after reset. In real mode, the CPU behaves like an 8086 from 1978: it uses 16-bit registers, can address 1 MB of memory, and has no memory protection. Modern operating systems switch to protected mode (32-bit) or long mode (64-bit) early in the boot process. </div> No RAM Yet</h2> Here is a detail that surprises many people: at the moment the CPU begins executing, RAM is not working.</p> Modern DDR memory requires initialization before it can be used. The memory controller needs to know the speed, timing, voltage, and organization of the installed DIMMs. This information is stored on a small chip on each DIMM called the SPD (Serial Presence Detect) EEPROM. The firmware must read the SPD data, configure the memory controller, and train the memory interface before a single byte of RAM is usable.</p> So the BIOS firmware, executing from the reset vector, must run its first instructions without any RAM. It cannot use a stack (which normally lives in RAM). It cannot store variables in RAM. It can only use CPU registers and execute directly from ROM.</p> This is one of the most constrained execution environments in all of computing. The firmware must bootstrap itself using only the CPU's internal registers, reading instructions directly from the ROM chip through the chipset, until it has initialized RAM and can begin using memory normally.</p> At the moment of the first instruction fetch, the CPU is running in real mode with caches disabled, interrupts off, and no usable RAM. The firmware must initialize the memory controller before it can use RAM for a stack or variables. Until then, it runs entirely from ROM using only CPU registers. </div> Other Architectures</h2> The x86 approach -- reset vector near the top of the address space, jump to BIOS -- is specific to x86. Other architectures handle the reset vector differently.</p> ARM processors typically start at address 0x00000000, where they expect to find not a single instruction but a vector table -- a series of addresses, each pointing to a handler for a different type of event (reset, undefined instruction, software interrupt, and so on). The first entry in the table is the reset vector.</p> RISC-V leaves the reset vector as an implementation choice. A RISC-V chip designer can put it anywhere in the address space, as long as the board maps executable firmware there. Common choices are 0x80000000 or 0x20000000.</p> The concept is the same everywhere: a fixed, known starting address that both the CPU and the board agree on. The specifics differ, but the principle is universal.</p> From Vector to Firmware</h2> The reset vector is nothing more than a handoff point. It is the hardware saying to the software: "You start here." What happens next -- the Power-On Self Test, hardware initialization, device enumeration -- is all firmware territory.</p> The CPU has done its part. It woke up, fetched its first instruction from the hardwired address, and began executing. From here, control passes to the code burned into the BIOS or UEFI firmware chip.</p> That firmware has a lot of work to do. It needs to test the hardware, initialize devices, configure the memory controller, set up the interrupt table, and eventually find a bootable device. All of that starts with the Power-On Self Test.</p> Next: POST and the BIOS</a></p> The Moment Power Arrives 2025-01-02T00:00:00+00:00 You press the power button. A light comes on. A fan spins. Somewhere inside the case, a cascade of events begins that will end with your operating system greeting you at a login prompt. But between the moment your finger touches that button and the moment the CPU executes its very first instruction, there is an entire world of electrical engineering that most people never think about.</p> This article covers the first few milliseconds of that journey. No software is running yet. No code has executed. This is pure physics: electrons moving through copper, capacitors charging, voltages stabilizing, and a clock beginning to tick.</p> The Power Button Is Not an On Switch</h2> The power button on a modern computer does not directly connect the power supply to the motherboard. It is a momentary switch -- press it and release it. The button sends a short signal to the motherboard's power control circuit, which then tells the power supply to turn on.</p> Think of it like a doorbell. You press the button, but you are not opening the door. You are telling someone inside to open it for you.</p> Key term: Momentary switch</strong> A switch that is only active while you are pressing it. Release it and the circuit opens again. The power button on your PC is a momentary switch connected to the motherboard's front panel header. </div> On ATX-standard motherboards, the power button connects two pins on the front panel header. When those pins are briefly shorted together, the motherboard's standby power circuit sends a signal to the power supply unit (PSU) telling it to turn on. The PSU has been in standby mode this whole time -- a small 5-volt standby rail has been active since the power cable was plugged in. That standby power keeps the motherboard's power management logic alive, waiting for your button press.</p> Fig. 01 -- From button press to PSU activation</div> The power button shorts two pins on the front panel header. The motherboard pulls PS_ON# low, telling the PSU to activate all voltage rails.</div> </div> </figure> The signal that flows from the motherboard to the PSU is called PS_ON# (the # means it is active-low -- the signal activates the PSU by being pulled to ground, not by going high). When the PSU receives this signal, it begins converting AC mains power into the DC voltages the computer needs.</p> The Voltage Rails</h2> A computer does not run on a single voltage. Different components need different voltages, and the PSU provides them simultaneously on separate "rails." The three main rails on an ATX power supply are:</p> +12V</strong> -- The heavy lifter. Powers the CPU (through a separate connector), graphics card, fans, and hard drives. Most of the power budget lives here.</li> +5V</strong> -- Used by USB ports, some motherboard logic, and legacy components.</li> +3.3V</strong> -- Powers RAM, some chipset logic, and various low-power ICs on the motherboard.</li> </ul> There is also -12V</strong> (rarely used on modern systems, historically for serial ports) and the +5V standby</strong> rail we already mentioned, which stays on whenever the PSU is plugged into a wall outlet.</p> Key term: Voltage rail</strong> A continuous conductor inside the power supply that carries a specific voltage to all connected components. Each rail is an independent regulated output. The "+12V rail" means the set of wires and traces that carry 12 volts DC throughout the system. </div> When the PSU turns on, these rails do not instantly jump to their target voltages. Capacitors inside the PSU and on the motherboard need time to charge. The voltage on each rail climbs from zero toward its target over a period of milliseconds. This is called the ramp-up period.</p> Capacitors and the Ramp-Up</h2> A capacitor is a component that stores electrical energy. Think of it as a small rechargeable battery that charges and discharges very quickly. Capacitors are everywhere in a computer -- on the motherboard, inside the PSU, on graphics cards, on RAM sticks.</p> During power-on, every capacitor in the system is discharged (assuming the machine has been off for a while). When the PSU begins outputting voltage, current flows into these capacitors, charging them. The voltage on each rail rises in a smooth curve, not a sudden jump.</p> This matters because digital logic circuits have minimum voltage requirements. A chip rated for 3.3V operation might malfunction if you try to run it at 2.1V. The system cannot begin operating until all voltages are stable and within specification.</p> Fig. 02 -- Voltage ramp-up on the +12V rail</div> 0V</text> 3.3V</text> 5V</text> 12V</text></p> +12V</text> +5V</text> +3.3V</text> 0ms</text> 5ms</text> 10ms</text> time</text></p> PWR_OK asserted here</text> </svg> All three voltage rails ramp from 0V to their target over several milliseconds. The Power Good signal is not asserted until all rails are stable and within tolerance.</div> </div> </figure> The ramp-up is not instantaneous, and different rails stabilize at different rates. The ATX specification requires the PSU to bring all rails within tolerance before asserting a specific signal that tells the rest of the system: everything is ready.</p> The Power Good Signal</h2> The PSU does not just dump voltage onto the motherboard and hope for the best. It monitors its own output. Once all voltage rails have stabilized within their specified tolerances (typically within 5% of the target voltage), the PSU asserts a signal called PWR_OK</strong> (also known as Power Good or PG).</p> Key term: Power Good (PWR_OK)</strong> A signal from the power supply to the motherboard indicating that all output voltages are stable and within specification. Until this signal goes high, the motherboard holds the entire system in reset. </div> The ATX specification says the PSU must wait between 100 and 500 milliseconds after turning on before asserting PWR_OK. This gives capacitors time to charge and voltages time to settle.</p> Until PWR_OK goes high, the motherboard holds the CPU in a reset state. The CPU literally cannot do anything. It sits there, held in reset by a signal from the motherboard's power management circuitry, waiting for permission to start.</p> This is a safety mechanism. If the CPU tried to execute instructions while voltages were still climbing, it would read garbage from memory, corrupt registers, and crash before it ever really started. The Power Good signal prevents that.</p> The CPU does not begin executing instructions the instant you press the power button. It waits -- held in reset -- until the PSU reports that all voltages are stable. This delay is typically 100 to 500 milliseconds. </div> Clock Stabilization</h2> Digital logic does not operate continuously like an analog circuit. It operates in steps, synchronized by a clock signal. Every operation the CPU performs -- reading a value, adding two numbers, writing a result -- happens on the tick of a clock.</p> The motherboard has a crystal oscillator that generates this clock signal. When power first arrives, the oscillator needs time to start vibrating at a stable frequency. The first few cycles are irregular -- the frequency wobbles, the amplitude is inconsistent. This is called the oscillator startup time, and it typically takes a few milliseconds.</p> The CPU's clock is derived from this base oscillator through a component called a Phase-Locked Loop (PLL). The PLL multiplies the base frequency up to the CPU's operating frequency. A base clock of 100 MHz might be multiplied by 35 to produce a 3.5 GHz CPU clock. The PLL also needs time to lock onto the correct frequency after power arrives.</p> Fig. 03 -- Clock stabilization timeline</div> 0ms</text> ~1ms</text> ~3ms</text></p> Crystal oscillator</text> starting up</text> PLL acquiring</text> frequency lock</text> Clean square wave</text> CPU can execute</text> </svg></p> The crystal oscillator starts with irregular vibrations, the PLL gradually locks onto the correct frequency, and only then does the CPU have a stable clock to operate from.</div> </div> </figure> No instruction can execute until the clock is stable. The CPU needs a predictable, regular tick to synchronize its internal operations. An unstable clock would cause timing violations -- signals arriving too early or too late -- which would produce unpredictable results.</p> The Sequence in Full</h2> Let us put the whole sequence together. From the moment you press the power button to the moment the CPU is released from reset, here is what happens:</p> You press the power button.</strong> The momentary switch shorts two pins on the front panel header.</li> The motherboard pulls PS_ON# low.</strong> This tells the PSU to activate.</li> The PSU begins converting AC to DC.</strong> Voltage rails start ramping up from zero.</li> Capacitors charge.</strong> Throughout the PSU and motherboard, capacitors absorb current and smooth the rising voltages.</li> Voltages stabilize.</strong> Each rail reaches its target and stays within tolerance.</li> The crystal oscillator starts.</strong> It needs a few milliseconds to reach a stable frequency.</li> The PLL locks.</strong> The CPU's clock multiplier circuit acquires lock on the base frequency.</li> The PSU asserts PWR_OK.</strong> All rails are stable. The system has permission to start.</li> The motherboard releases the CPU from reset.</strong> The reset line goes inactive.</li> The CPU fetches its first instruction.</strong> The boot process has begun.</li> </ol> This entire sequence takes somewhere between 100 milliseconds and one second. In human terms, it is the time between pressing the button and hearing the first fan spin up. In computer terms, it is an eternity -- a modern CPU could execute billions of instructions in that time, if only it were allowed to start.</p> Before any software runs, the hardware must complete a physical startup sequence: voltages must ramp and stabilize, the clock must lock, and the Power Good signal must be asserted. Only then is the CPU released from reset to fetch its first instruction. </div> What Can Go Wrong</h2> If the PSU cannot bring a voltage rail within tolerance, PWR_OK never asserts. The system stays in reset forever. From the outside, this looks like a dead computer -- you press the button, maybe a fan twitches, but nothing happens.</p> If PWR_OK is asserted too early (a defective or out-of-spec PSU), the CPU might start executing with unstable voltages. The result is unpredictable: it might boot and crash seconds later, it might corrupt data, or it might appear to work fine until the system is under heavy load and the marginal voltage sags below tolerance.</p> This is why a reliable power supply matters. The PSU is the foundation of every single thing the computer does. Every bit stored in RAM, every instruction the CPU executes, every packet sent over the network -- all of it depends on clean, stable voltage from the PSU.</p> What Comes Next</h2> The CPU is out of reset. The clock is ticking. Voltages are stable. The processor is alive and ready to execute its first instruction.</p> But where does it find that first instruction? The CPU does not know what a hard drive is. It does not know about your operating system. It has no concept of files or programs. All it knows is one thing: a hardwired address in memory where it must look for its first instruction.</p> That address is called the reset vector, and it is the subject of the next article.</p> Next: The Reset Vector</a></p> Bits, Bytes, and Voltage 2025-01-01T00:00:00+00:00 A computer does not think. It does not understand. It reacts to electricity. Every program you have ever run, every file you have ever saved, every pixel on your screen right now -- all of it reduces to electrical signals moving through circuits at close to the speed of light.</p> Before we can talk about how a computer starts up, we need to understand the language it speaks. That language has exactly two words.</p> Two Voltages, One Idea</h2> Inside a modern computer, the processor and memory chips run on a supply voltage -- typically somewhere between 0.8 and 1.2 volts for a CPU core. The exact number depends on the chip, but the principle is the same everywhere.</p> The chip treats voltage levels as one of two states. A voltage near the supply level counts as "high." A voltage near ground counts as "low." Anything in between is undefined -- the hardware is not designed to work there, and if a signal lingers in that middle zone, the circuit may behave unpredictably.</p> Key term: Logic levels</strong> The two defined voltage states in a digital circuit. "High" represents one value and "low" represents the other. The exact voltage thresholds depend on the chip technology, but every digital system draws the same hard line: there are exactly two valid states. </div> This is not an arbitrary design choice. Engineers use two states because they are easy to distinguish electrically, even in the presence of noise, temperature drift, and manufacturing variation. A system with ten voltage levels would be more information-dense per wire, but vastly harder to build reliably. Two states give you the widest possible margin for error.</p> Fig. 00 -- Voltage levels in a digital circuit</div> </p> TIME</text> VOLTAGE</text> </svg></p> A digital signal swings between HIGH and LOW zones. The yellow trace shows a clean signal. The red zone in the middle is forbidden territory -- a signal that stays there produces unpredictable results.</div> </div> </figure> The Bit</h2> We call each of these two-state signals a bit</strong>. The word is a contraction of "binary digit." A bit is the smallest possible unit of information: it distinguishes between exactly two alternatives.</p> By convention, we label these alternatives 1</strong> and 0</strong>. HIGH voltage maps to 1. LOW voltage maps to 0. But the mapping is arbitrary. Some circuits reverse it. What matters is that every conductor in a digital circuit carries exactly one bit of information at any given moment.</p> A single bit is not very expressive. It can represent on/off, yes/no, true/false. That is all. To say anything more interesting, you need more bits.</p> Key term: Bit</strong> The fundamental unit of digital information. A bit has exactly two possible values: 0 or 1. Inside the machine, each bit is a voltage level on a wire. </div> Counting in Binary</h2> When you write a number like 347 in everyday decimal notation, each digit position represents a power of ten. The 3 means "three hundreds" (3 x 10^2). The 4 means "four tens" (4 x 10^1). The 7 means "seven ones" (7 x 10^0).</p> Binary works identically, but with powers of two. Each position is worth twice as much as the one to its right.</p> Consider the binary number 1101</code>. Reading from right to left:</p> Position 0: 1</strong> x 2^0 = 1</li> Position 1: 0</strong> x 2^1 = 0</li> Position 2: 1</strong> x 2^2 = 4</li> Position 3: 1</strong> x 2^3 = 8</li> </ul> Add them up: 8 + 4 + 0 + 1 = 13</strong>.</p> That is all binary is. The same positional number system you already know, with a smaller alphabet.</p> Fig. 01 -- Binary place values</div> 4-BIT EXAMPLE FROM TEXT</text></p> 2^3</text> 2^2</text> 2^1</text> 2^0</text></p> 8</text> 4</text> 2</text> 1</text></p> 13 =</text> 1</text> 1</text> 0</text> 1</text></p> 8 + 4 + 0 + 1 = 13</text></p> 8-BIT (ONE BYTE)</text></p> 2^7</text> 2^6</text> 2^5</text> 2^4</text> 2^3</text> 2^2</text> 2^1</text> 2^0</text></p> 128</text> 64</text> 32</text> 16</text> 8</text> 4</text> 2</text> 1</text></p> 65 =</text> 0</text> 1</text> 0</text> 0</text> 0</text> 0</text> 0</text> 1</text> 64 + 1 = 65 (ASCII letter "A")</text></p> 255 =</text> 1</text> 1</text> 1</text> 1</text> 1</text> 1</text> 1</text> 1</text> </svg></p> Top: the 4-bit example from the text -- 1101 = 13. Bottom: two full bytes. 01000001 is 65, the ASCII code for the letter "A". 11111111 is 255, the largest value a single byte can hold.</div> </div> </figure> The Byte</h2> Eight bits grouped together form a byte</strong>. This is the fundamental unit of storage and data transfer in virtually every modern computer. One byte can represent 2^8 = 256 different values, numbered 0 through 255.</p> Why eight? Partly convention, partly practical engineering. Eight bits is enough to represent a single character in Western text (the ASCII standard uses 7 bits, with the eighth available for error checking or extensions). It divides neatly into halves (called nibbles</strong> -- 4 bits each), and powers of two scale cleanly: 2 bytes = 16 bits, 4 bytes = 32 bits, 8 bytes = 64 bits.</p> Key term: Byte</strong> A group of 8 bits. One byte can hold any value from 0 to 255. Almost every unit of computer storage -- kilobytes, megabytes, gigabytes -- is measured in multiples of bytes. </div> The sizes you see in everyday computing are all built from bytes:</p> Name</th> Bytes</th> Bits</th></tr></thead> Byte</td> 1</td> 8</td></tr> Kilobyte</td> 1,024</td> 8,192</td></tr> Megabyte</td> 1,048,576</td> 8,388,608</td></tr> Gigabyte</td> 1,073,741,824</td> 8,589,934,592</td></tr> </tbody></table> Notice these are powers of 1,024, not 1,000. That factor of 1,024 (which is 2^10) comes from the binary nature of memory addressing. A chip with 10 address lines can reach 1,024 locations. Marketing departments prefer to use 1,000 because it makes the number bigger, which is why your "500 GB" hard drive shows up as 465 GB in your operating system. The drive manufacturer counted in billions; the OS counts in powers of 1,024.</p> Hexadecimal: A Shorthand for Humans</h2> Binary numbers get long fast. The number 255 in binary is 11111111</code> -- eight characters to represent a fairly small value. For numbers in the millions, binary notation becomes unreadable.</p> Programmers use hexadecimal</strong> (base 16) as a compact way to write binary values. Hex uses sixteen symbols: 0-9 and A-F. Each hex digit represents exactly four bits -- one nibble.</p> This mapping is precise and mechanical:</p> Hex</th> Binary</th> Decimal</th></tr></thead> 0</td> 0000</td> 0</td></tr> 1</td> 0001</td> 1</td></tr> 2</td> 0010</td> 2</td></tr> 3</td> 0011</td> 3</td></tr> 4</td> 0100</td> 4</td></tr> 5</td> 0101</td> 5</td></tr> 6</td> 0110</td> 6</td></tr> 7</td> 0111</td> 7</td></tr> 8</td> 1000</td> 8</td></tr> 9</td> 1001</td> 9</td></tr> A</td> 1010</td> 10</td></tr> B</td> 1011</td> 11</td></tr> C</td> 1100</td> 12</td></tr> D</td> 1101</td> 13</td></tr> E</td> 1110</td> 14</td></tr> F</td> 1111</td> 15</td></tr> </tbody></table> To convert hex to binary, replace each hex digit with its four-bit equivalent. 0xFF</code> becomes 1111 1111</code>. 0x4A</code> becomes 0100 1010</code>. No arithmetic required -- just substitution.</p> Hex numbers are usually prefixed with 0x</code> to distinguish them from decimal. When you see 0xDEADBEEF</code> in a debugger, that is a 32-bit value: 1101 1110 1010 1101 1011 1110 1110 1111</code>.</p> Logic Gates: Where Meaning Begins</h2> Voltage levels become useful when you start combining them. A logic gate</strong> is a circuit that takes one or more input bits and produces an output bit according to a fixed rule.</p> The three fundamental gates are:</p> AND</strong>: The output is 1 only if both inputs are 1. Think of two switches in series -- both must be closed for current to flow.</p> OR</strong>: The output is 1 if either input (or both) is 1. Think of two switches in parallel -- either one lets current through.</p> NOT</strong>: The output is the opposite of the input. One input, one output. A 1 becomes 0, a 0 becomes 1.</p> Fig. 02 -- The three fundamental logic gates</div> AND</text></p> &</text></p> A</text> B</text> OUT</text> A B | OUT</text> 0 0 | 0</text> 0 1 | 0</text> 1 0 | 0</text> 1 1 | 1</text></p> OR</text></p> A</text> B</text> OUT</text> >=1</text> A B | OUT</text> 0 0 | 0</text> 0 1 | 1</text> 1 0 | 1</text> 1 1 | 1</text></p> NOT</text></p> A</text> OUT</text> A | OUT</text> 0 | 1</text> 1 | 0</text></p> Every circuit in a modern processor is built from combinations</text> of these three operations. AND, OR, NOT -- nothing else is needed.</text> </svg></p> The three fundamental gates with their truth tables. AND requires both inputs high. OR requires at least one. NOT inverts. Every computation a CPU performs decomposes into these three operations.</div> </div> </figure> Every computation your processor performs -- addition, comparison, memory access, encryption, video decoding -- decomposes into combinations of these three operations. A modern CPU contains billions of transistors, but each transistor is acting as a switch in a logic gate. The complexity comes from composition, not from any single component.</p> From Gates to Arithmetic</h3> You can build an adder from AND, OR, and NOT gates. To add two single-bit numbers:</p> The sum</strong> bit is the XOR of the two inputs (XOR is "exclusive or" -- true when exactly one input is 1).</li> The carry</strong> bit is the AND of the two inputs.</li> </ul> XOR itself is built from AND, OR, and NOT: A XOR B = (A OR B) AND NOT (A AND B)</code>.</p> Chain eight of these single-bit adders together, feeding each carry output into the next stage's carry input, and you have an 8-bit adder. It adds two bytes and produces a byte-sized result plus a carry flag. This is the same principle used in every ALU (arithmetic logic unit) ever built.</p> Key term: ALU (Arithmetic Logic Unit)</strong> The part of a processor that performs mathematical and logical operations. An ALU is built entirely from logic gates. It takes two inputs and an operation code, and produces a result. Every calculation your computer performs passes through an ALU. </div> How Memory Stores a Bit</h2> A logic gate computes, but it does not remember. Once the input changes, the old output is gone. To store information, you need a circuit that can hold its state.</p> The simplest storage element is a latch</strong> -- two NOT gates connected in a loop. The output of the first feeds the input of the second, and the output of the second feeds back to the input of the first. This creates a stable feedback loop that holds either a 0 or a 1 indefinitely, as long as power is supplied.</p> Add some control gates to decide when</em> the latch is allowed to change, and you have a flip-flop</strong> -- a one-bit memory cell that updates only on a clock edge. Group eight flip-flops together and you have a register</strong> that stores one byte. Stack millions of these cells into a grid with row and column addressing, and you have RAM.</p> Every byte in your computer's memory is a row of eight tiny circuits, each one holding a voltage level. When the power goes off, the voltage drains, and the data disappears. This is why RAM is called "volatile" storage -- it forgets everything the moment it loses power. </div> This will matter enormously when we talk about what happens at power-on. The processor wakes up with empty RAM. Every byte reads as garbage. The machine must find its instructions from somewhere that does not</em> forget -- and that somewhere is the subject of the next few articles.</p> Everything Is a Number</h2> Text, images, sound, video, executable programs, network packets, encryption keys -- inside the machine, all of it is bytes. There is no difference in the hardware between a byte that represents the letter "A" (0x41 in ASCII), a byte that represents the color of a pixel, and a byte that is part of a machine instruction.</p> The meaning of a byte depends entirely on context. The value 0x41 is the letter "A" if a text editor reads it, the number 65 if a calculator reads it, a shade of red if a graphics program reads it, or a processor instruction (INC ECX on x86) if the CPU executes it. Same pattern of voltages. Different interpretation.</p> This is one of the most important ideas in computing: data and instructions live in the same memory, encoded the same way</strong>. The computer does not know whether a given byte is a number, a letter, or an instruction. It does whatever the program counter tells it to do with whatever bytes it finds. If the program counter points at your JPEG, the CPU will cheerfully try to execute your vacation photo. It will not end well, but nothing in the hardware prevents it.</p> A computer does not understand data types. Every value in memory is just a pattern of bits. Meaning is imposed by the program that reads them. The same byte can be a number, a character, a color, or a machine instruction depending on context. </div> Why This Matters for Cold Boot</h2> When you press the power button, every piece of RAM in the system starts in an undefined state. The processor has no program loaded. There is no operating system. There are no files.</p> What exists is a set of circuits that can move voltages around according to fixed rules. Billions of transistors implementing AND, OR, and NOT. A bus that can carry bytes from one place to another. A small piece of non-volatile memory -- a chip that does not forget when the power goes off -- containing just enough instructions to get started.</p> The entire journey from power button to running operating system is the process of loading bytes into RAM, one layer at a time, each layer complex enough to load the next. Every step in that chain operates on the principles covered in this article: voltage levels, binary encoding, logic gates, and the fundamental interchangeability of data and instructions.</p> That is what Cold Boot is about. We start here, at the bottom, where electricity becomes information. The next article picks up the story at the exact moment you press the power button.</p> Next: The Moment Power Arrives</a></p>

The Essential Variables</h2>
Some environment variables are so common that nearly every program recognizes them. Here are the ones you need to know:</p>

USER</h3>
The name of the currently logged-in user. Programs use it for logging, file ownership, and access control decisions.</p>

SHELL</h3>
The path to the user's default login shell, as set in `/etc/passwd</code>. Note that this is the default shell, not necessarily the shell currently running. If you start zsh from inside bash, $SHELL</code> still says /bin/bash</code>.</p>`

LANG and LC_*</h3>
Language and locale settings. They control how programs display dates, numbers, currency, and sort text. `LANG=en_US.UTF-8</code> means English, United States, using the UTF-8 character encoding.</p>`

The Major Shells</h2>
The concept of a command-line shell is older than Linux. Here are the shells you are most likely to encounter:</p>

Beyond POST: Hardware Enumeration</h2>
Once POST confirms that the basic hardware is functional, the BIOS moves on to more complex initialization. This phase detects and configures all the devices attached to the system.</p>

Cold Boot

Your First Shell Script

Loops</h2>

The Environment

Pipes and Redirection

Standard I/O

What a Shell Actually Is

Users, Groups, and Permissions

Special Permission Bits</h2>
Beyond the basic rwx bits, there are three special bits that modify how files and directories behave.</p>

The TTY and Terminal

The Process Model

PID 1: init and systemd

A Brief History of Init</h2>
The concept of a first process dates back to the earliest Unix systems in the 1970s. Over the decades, several implementations have competed for the role.</p>

The Root Filesystem

Device Drivers

The Clock Signal

Kernel Init

Decompressing and Handing Off

Finding the Kernel

Stage 2: GRUB

Stage 1: The Bootloader

Finding the Boot Device

The Bus System

Memory Initialization

UEFI vs Legacy BIOS

Why Linux?

POST and the BIOS

The Reset Vector

The Moment Power Arrives

Bits, Bytes, and Voltage