Python's `subprocess`: The Right Way to Talk to Your OS

Junior/Mid Engineer Asked at: Any company with DevOps, SRE, or automation needs.

Q: Write a Python script to run a shell command (like `df -h`) and print the output.

Why this matters: This question is a test of security and control. Your Python application is a clean, well-defined world. The shell is a powerful, chaotic, and dangerous place. The interviewer wants to see if you know how to build a safe, airtight gate between these two worlds, or if you just punch a hole in the wall.

Interview frequency: High. It's the foundation of any script that needs to interact with the underlying operating system.

❌ The Death Trap

The candidate reaches for the old, simple, and catastrophically insecure `os.system`. It's the answer you learn in your first programming class, and it's the one that gets you hacked in the real world.

"The most common—and wrong—answer is:"

import os

# This is simple, but it has no way to capture the output
# and is a massive security hole if the command includes user input.
os.system('df -h')

This is a trap because it seems to work, but it offers you no control. You can't capture the output to a variable, you can't check the error code easily, and most importantly, it's vulnerable to Shell Injection. If you were to build a command like `os.system(f"ping {user_input}")` and a user enters `8.8.8.8; rm -rf /`, you have just executed a command to delete your entire filesystem. This is a critical failure.

🔄 The Reframe

What they're really asking: "Show me you understand the principle of least privilege and how to safely delegate work to external processes without sacrificing control or security."

This reveals your engineering discipline. Can you be trusted to write code that interacts with the most powerful parts of a system? Your choice of `subprocess` over `os.system` is a clear signal that you can.

🧠 The Mental Model

I use the **"Hiring a Specialist Contractor"** model.

1. The Wrong Way (`os.system`): You give a stranger your master key and tell them to "fix the plumbing." They might fix it, or they might wander through your house and steal your things. You have no control and no idea what they really did.

2. The Right Way (`subprocess`): You hire a bonded, insured contractor. You give them a specific work order (`['df', '-h']`) and nothing else. You wait for them to finish, and they hand you a detailed report (`CompletedProcess`) with the results, any problems they encountered, and a final status. It's safe, predictable, and controlled.

📖 The War Story

Situation: "A company built a simple internal web app that allowed developers to check the status of a service by pinging it. The backend was a simple Python script."

Challenge: "The original developer used `os.system` to build the command, concatenating a string: `os.system('ping -c 1 ' + hostname)`. The `hostname` came directly from the web form. It worked fine for hostnames like `google.com`."

Stakes: "A security researcher discovered the app and entered `google.com; cat /etc/passwd` into the form. The backend blindly executed `ping -c 1 google.com; cat /etc/passwd`, and the server's user file was displayed on the webpage. This simple oversight exposed sensitive system information because the code mixed data (the hostname) with control (the shell commands), a classic injection vulnerability."

✅ The Answer

"The modern and correct way to do this in Python is with the `subprocess` module. It's designed to solve the security and control problems of older methods like `os.system`. Here's how I would write a robust script to get disk usage."

The Robust Solution

import subprocess
import sys

def get_disk_usage():
    """
    Runs the 'df -h' command safely using subprocess and returns its output.
    """
    # The command and its arguments are passed as a list.
    # This is the key to preventing shell injection.
    command = ['df', '-h']

    try:
        print(f"Running command: {' '.join(command)}")
        
        # subprocess.run is the modern, recommended function.
        result = subprocess.run(
            command,
            capture_output=True,  # Capture stdout and stderr
            text=True,          # Decode output as text (UTF-8 by default)
            check=True           # Raise CalledProcessError if return code is non-zero
        )

        print("Command executed successfully.")
        return result.stdout

    except FileNotFoundError:
        print(f"Error: Command '{command[0]}' not found. Is it in your PATH?", file=sys.stderr)
    except subprocess.CalledProcessError as e:
        # This block is reached if check=True and the command returns an error.
        print(f"Error: Command failed with exit code {e.returncode}", file=sys.stderr)
        print(f"Stderr:\n{e.stderr}", file=sys.stderr)
    
    return None

if __name__ == "__main__":
    disk_usage_output = get_disk_usage()
    if disk_usage_output:
        print("\n--- Disk Usage Output ---")
        print(disk_usage_output)

Key Principles:

Security: By passing the command as a list (`['df', '-h']`), we ensure that the arguments are passed directly to the system call. The shell is never involved, and injection is impossible.
Control: The `subprocess.run` function returns a `CompletedProcess` object. We have clean, programmatic access to `.stdout`, `.stderr`, and `.returncode`.
Robustness: Using `check=True` turns a silent failure (a non-zero exit code) into a loud, explicit Python exception (`CalledProcessError`) that we can catch and handle properly.

🎯 The Memorable Hook

"Your code is a safe, orderly garden. The shell is the untamed jungle outside. `subprocess` is the reinforced gate you use to interact with it safely; `os.system` is leaving the back door wide open."

This creates a powerful, visceral analogy for the security boundary between an application and the shell, showing you think in terms of risk and safety, not just functionality.

💭 Inevitable Follow-ups

Q: "What if you need to run a complex command that involves a pipe, like `ps aux | grep python`?"

Be ready: "The Pythonic way is to replicate the pipe in Python itself, not to use `shell=True`. I would create two `subprocess.Popen` objects. The `stdout` of the first process (`ps aux`) would be piped to the `stdin` of the second process (`grep python`). This maintains security and gives you full control over the pipeline."

Q: "When would you ever use `shell=True`?"

Be ready: "Very, very rarely, and only with extreme caution. It might be necessary if you need to execute a command that is a shell built-in or relies on shell features like wildcards (`*`) and you can't replicate it easily. If I absolutely had to use it, I would never pass user-provided input to it, and I'd use Python's `shlex.quote()` to sanitize any variable parts of the command string to prevent injection."

🔄 Adapt This Framework

If you're junior: Explaining why `os.system` is bad and showing a working `subprocess.run` example with the command as a list is a home run. This demonstrates a solid grasp of modern best practices.

If you're senior: You should immediately identify `os.system` as an anti-pattern. Your answer should proactively cover the key arguments to `subprocess.run` (`check`, `capture_output`, `text`) and you should be prepared to discuss advanced use cases like piping with `Popen` without being prompted.