I have a touch of a love and hate relationship with bash. I spend tons of your time within the terminal, and bash is my default “programming language“. Sometimes I tell folks that find, grep and xargs run their infrastructure, and that they laugh and laugh until they realize I’m serious.

Picking up some Python may be a perfect choice for system administrators. it is also great for anyone who has got to affect anything during a terminal but doesn’t want to use bash, or has needs that are too complex for bash. Once a task goes beyond

find $(pwd) -name "*.txt" | xargs -I {} echo "do stuff with {}"

it’s time to interrupt out the Python!

There are tons of advantages to using Python as your attend instruction language.

  1. Python has many nice libraries to assist out with just about anything. that has handling system operations, reading files, listing directories, writing for loops, checking for exit codes, and so on.
  2. Autocomplete with IDEs. Seriously. Who wants to possess to memorize anything?
  3. Robust testing suite if that’s your thing (and if it isn’t , you ought to consider making it your thing).
  4. The iPython console. It’s wonderful. It’s amazing. I LOVE IT.
  5. Python is out there on most systems, and if it isn’t you’ll catch on with Miniconda.
  6. Robust error checking with attempt to catch blocks.
  7. If you’re employed on different operating systems you’ll use Python libraries which will affect all that under the hood.
  8. Even if you’ve got no programming ability Python is a simple language to urge started with.

Let’s get Started

To get started, first you will need to either have Python installed or install it with Miniconda.

Check if you have iPython installed

which python
which ipython

If both of those are successful, you’re in business! If you’ve got Python, but not iPython, you’ll need to install it. you’ll install it as a system package, but i actually recommend that you simply just install it with Miniconda.

Install Miniconda

Grab the installer for your OS here. I suggest getting the Python3 installation.

Then it’s just an easy installation.

bash Miniconda3-latest-Linux-x86_64.sh

Follow the prompts and you will have Miniconda3 installed. Once you’ve got it installed you’ll be wanting to run an update, because this is often tech and in fact you would like to run an update. 😉

conda update conda
conda config --add channels conda-forge
conda update -y --all
conda install -y ipython

Troubleshooting

If you’ve got trouble installing any packages here are some tips.

  1. Run conda clean –all and check out again.
  2. Make sure you’re using the right channel.
  3. Run conda update -y –all
  4. Try to install as little as possible to your global conda space. Instead create environments for various tasks and projects, which we’ll get into next.

Create Environments with Conda

If you have ever used virtualenv, pipenv (is that a thing?), Rbenv, plenv, anyenv or any of the opposite various envs that have popped up over the years, this may sound very familiar to you. the thought is that different projects should have their own isolated software environments.

conda create -n my-project ipython package1 package2 package2

If you’re like me and like to have iPython readily availabe make sure you install it to any new environments!

Python Libraries for System Administration

Before we get into the examples let’s just list some handy packages along side their docs.

My attend package is that the os package. you’ll use it to list directories, check if files exist, check if symlinks exist, make directories, run system commands, get and set environmental variables, and more. It’s great!

My second package for running system commands that do not exist as handy python libraries is that the subprocess module.

The shutil has file operations that are not within the os library.

The pprint library prints out complex data structures with nice indentation.

The pytest library let’s you test your Python code, because let’s face it, nothing ever works correctly the primary (few) times.

How Do I Execute my Code?

Finally! Code!

 

When you’re using Python for system administration you’ll dive straight into the iPython console, or write scripts then execute them with python name-of-script.py.

If you favor to write down your scripts you’ve got numerous choices, and it’s truly a matter of private preference. i exploit PyCharm, which is paid, but Visual Studio Code and Atom are equally excellent free choices.

I find that it depends on what I’m performing on . Sometimes I just open up the iPython console and begin typing, and other times i want something more robust with tests and whatnot.

If you’re using either the iPython console or any of the editors I listed above, you’ll have autocomplete. Autocomplete is awesome! With iPython simply start typing your function and press tab to urge an inventory of potential functions you’ll want.

I cannot express how much I love autocomplete.

Get Help

You can go to any of the doc pages for any library, but if you know the name of either the library or the function you can bring it up in iPython.

You can bring up the help menu in most IDEs and text editors too, but that will be specific to your editor.

Examples

First you will need to import your packages

import os
import subprocess
import shutil
from pprint import pprint

Here are some examples of common file and directory operations.

# Check if a path exists
os.path.exists('/path/on/filesystem')
# List the contents of a directory
# This returns a list
dir_list = os.listdir()
for item in dir_list:
    print(item)
# Get the Absolute Path name of a file (file + current working dir)
os.path.abspath('some-file')
#Get the basename - returns file
os.path.basename('/path/to/file')
# Split a directory path - platform independent
os.path.split(os.getcwd())
# Out[17]: ('/Users', 'jillian')
# Split a directory path - platform independent
os.path.split(os.getcwd())
# Out[17]: ('/Users', 'jillian')
# Check if a path is a symlink
os.path.islink()

Move files and directories around

# Copy a directory
# cp -rf
shutil.copytree('src', 'dest')
# Copy a file
# cp -rf
shutil.copyfile('file1', 'file2'
# Move a directory
# mv
shutil.move('src', 'dest')

Not everything is going to be available through python libraries, such as installing system libraries, so run a few system commands!

# Run an arbitrary system command
command = "echo 'hello'"
result = subprocess.run(command.split(' '), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
#Print the stdout and stderr
print(result.stdout)
print(result.stderr)

Write to files!

# Write to a file (and create it if it doesn't exist)
# echo "hello" > hello.txt
f= open("hello.txt","w+")
f.write("hello!")
f.close()
# Append to a file
# echo "hello" >> hello.txt
f = open("hello.txt", "a+")
f.write("hello again!")
f.close()

 

Write some tests!

Tests mostly work by using a function called assert, which is essentially saying make sure this is true and if not die loudly.

def test_system_command():
    """Test the exit code of a system command"""
    command = "echo 'hello'"
    result = subprocess.run(command.split(' '), stdout=subprocess.PIPE)
    assert result.returncode == 0

Put this function in a file called test_my_code.py and run as pytest test_my_code.py.

Wrap Up:

That’s it for my main tips and tricks for using Python as your go-to bash replacement. subsequent time you would like to write down a loop in bash, consider breaking out the iPython console and seeing what you’ll come up with instead!

Categorized in:

PYTHON

Tagged in:

,