Lesson 6 of 12 · 10 min · Processes and services
Processes, signals and jobs
What you will learn
- What a process is, its PID and parent, and the common process states
- Listing and finding processes with
ps,top,pgrepand/proc - Signals:
SIGTERM,SIGKILL,SIGHUP,SIGINT, and how programs handle them - Exit codes, background jobs and priorities
In simple words
A program is like a recipe in a book. A process is that recipe being cooked right now. A computer cooks hundreds at once, and each gets a number so you can check on it or stop it. This lesson shows how to see what is running and how to stop something politely, or by force when it won't listen.
Words to know
- Process: a program that is running right now.
- PID: process ID: the number Linux gives each running process.
- Signal: a short message sent to a process, such as "please stop".
- Exit code: the number a process leaves when it ends. 0 means success; anything else means a problem.
A process is a running program
A program is a file on disk. A process is that program running: it has its own memory, open files, a user it runs as, and a number, the PID. Starting a process is done by an existing process (its parent) so processes form a tree, which starts with PID 1: the first process the kernel starts (systemd on most distributions). If a parent dies, its children are adopted by PID 1.
ps -eo pid,ppid,user,stat,etime,cmd --sort=pid | head
PID PPID USER STAT ELAPSED CMD
1 0 root Ss 00:27 /sbin/init
23 1 root S<s 00:27 /usr/lib/systemd/systemd-journald
89 1 root Ss 00:27 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
90 89 www-data S 00:27 nginx: worker process
Columns: PID, parent PID (here the nginx worker's parent is the nginx master), user, state, elapsed time, command. The kernel also exposes each process under /proc/<PID>/ (command line, open files in fd/, memory, environment), which is where tools read their data.
Process states (the STAT column)
| Letter | Meaning |
|---|---|
R |
running or ready to run |
S |
sleeping, waiting for something (normal for most) |
D |
uninterruptible sleep, usually waiting on disk or network storage. Cannot be killed; many in D means an I/O problem |
T |
stopped (for example Ctrl+Z) |
Z |
zombie: finished, but the parent has not collected its exit status yet. Uses no resources except a table slot; fix the parent, you cannot kill a zombie |
Extra characters: s session leader, < high priority, N low priority, + in the foreground.
Looking at processes
ps aux # everything, BSD style: user, %CPU, %MEM, command
ps -ef --forest # everything with the parent/child tree
ps -o pid,ppid,stat,cmd -p 1234
pgrep -a nginx # find PIDs by name and show the command line
top # live view. Keys: P sort by CPU, M by memory, k kill, q quit
top shows load average and CPU/memory (lesson 9 explains how to read them). htop is a friendlier version if installed. To see which process uses a port or a file: ss -tlnp or lsof -p PID (lesson 10).
Signals
A signal is a small message the kernel or another process sends to a process. You send one with kill, which is badly named: it sends any signal.
kill -l | head -3
1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP
6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1
11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM
The ones you must know:
| Signal | Number | Meaning | Can the program handle it? |
|---|---|---|---|
SIGTERM |
15 | "please stop": the default of kill PID |
yes: it can clean up, close files, finish requests |
SIGINT |
2 | interrupt: what Ctrl+C sends | yes |
SIGHUP |
1 | terminal closed; many daemons treat it as "reload your config" | yes |
SIGKILL |
9 | terminate now, no cleanup | no: the kernel ends it |
SIGSTOP / SIGCONT |
19 / 18 | pause / resume | SIGSTOP cannot be handled |
Always try SIGTERM first and wait a few seconds. kill -9 gives no chance to flush data or remove lock files and can corrupt files being written. Use it only when TERM did not work.
kill PID # SIGTERM
kill -HUP PID # ask the daemon to reload
kill -9 PID # last resort
pkill -f 'python app.py' # by pattern on the full command line
killall nginx # by exact process name
A program can catch a signal and run its own code. In a shell script:
trap 'echo got TERM; exit 0' TERM
When a program is stopped by a signal, the shell reports exit code 128 + signal number. Tested here:
exit after TERM: 143 # 128 + 15
exit after KILL: 137 # 128 + 9
You will see 137 in container work all the time: a container "exited with 137" was killed by SIGKILL, very often by the out-of-memory killer.
Exit codes
Every process ends with a number between 0 and 255. 0 means success; anything else means failure. The shell keeps the last one in $?.
grep -q root /etc/passwd; echo $? # 0: found
grep -q nosuchuser /etc/passwd; echo $? # 1: not found
&& runs the next command only if the previous succeeded, || only if it failed. CI pipelines and scripts decide by exit codes, which is the subject of the shell scripting course.
Foreground, background and jobs
sleep 300 & # & starts it in the background
jobs # list this shell's jobs
fg %1 # bring job 1 to the foreground
# Ctrl+Z stops the foreground job; bg %1 continues it in the background
Background jobs of a shell normally end when the terminal closes (SIGHUP). To keep something running after you log out:
nohup ./long-task.sh > task.log 2>&1 &
For anything that must live on a server, do not use nohup or screen; make it a service (next lesson), so it starts at boot, restarts on failure and logs properly. tmux or screen are good for keeping an interactive session alive.
Priority
A nice value from -20 (most important) to 19 (least) tells the scheduler how generous to be. Ordinary processes run at 0. nice -n 10 command starts a polite background job, renice 10 -p PID changes a running one. Only root may lower the number.
Common mistakes
kill -9first. Try TERM, wait, then KILL.- Hunting a zombie. You cannot kill a zombie; look at its parent (
ps -o ppid= -p PID) and fix or restart that. - Killing by name too broadly.
pkill pythonstops every Python process on the box. Check withpgrep -afirst. - Running a server with
&and calling it deployed. It dies with the session. Use systemd.
Try it yourself
- Start
sleep 300 &, find it withpgrep -a sleepand withps, and stop it withkill. Then runwaiton its PID and read$?. What exit code do you see, and why? - Write a script that traps
SIGTERM, prints a message and exits 0. Run it in the background and send it TERM. Do the same withkill -9and compare. - Use
ps -ef --forestto find the parent of a process and its children. Then read/proc/<PID>/cmdlinefor one of them. - Start a CPU-busy job such as
yes > /dev/null &, find it intop, lower its priority withrenice, and stop it.
Check yourself
What is the difference between a program and a process?
A program is an executable file; a process is one running instance of it with its own PID, memory, open files and user. One program can have many processes.
Why use kill (SIGTERM) before kill -9?
SIGTERM lets the program shut down cleanly (flush buffers, close connections, remove temp files). SIGKILL cannot be caught, so the program gets no chance to clean up and data can be lost.
A command ended with exit code 137. What happened?
It was killed by signal 9 (128 + 9), usually by the OOM killer or by someone running kill -9. In containers this typically means the memory limit was exceeded.
What is a zombie process and how do you get rid of it?
A finished process whose parent has not read its exit status. It holds only a process table entry. You cannot kill it; fix the parent so it waits for its children, or restart the parent, after which PID 1 cleans up.
Interview questions
How do you find which process is using the most memory?
ps aux --sort=-%mem | head or sort by M inside top.
What does kill -HUP usually do for a daemon?
It asks it to re-read its configuration without restarting, which many servers (nginx, sshd, rsyslog) support. The exact behaviour is defined by each program.
What is PID 1?
The first process the kernel starts, typically systemd. It starts the rest of the system, adopts orphaned processes and reaps zombies. In a container, PID 1 is whatever process the container runs, which is why a container's main process must handle signals properly.
Summary
- A process is a running program with a PID and a parent;
ps,top,pgrepand/procshow them. - Signals control processes: TERM (15) asks, KILL (9) forces; exit code
128+nmeans killed by signal n. - Exit code 0 is success; everything else is failure, and
&&/||use that. - Use services (not
&ornohup) for anything that must stay up.
Going deeper
See how much of this you can explain
A free mock interview asks you new questions on Linux for DevOps for DevOps Engineer and scores every answer, with what to fix.