Back to articles list Articles
13 minutes read

10 Useful Tips for Writing Python Scripts

Author's photo
Xavier Rigoulet

Would you like some helpful tips for Python scripts? Read on to get some advice from a pro!

In this article, I will give you some Python tips and tricks alongside script examples to help you make the best out of Python.

First, let's define what a Python script is. A Python script is a set of instructions written in Python to ask the computer to perform some tasks.

Before we get started with the Python tips, make sure that you understand Python data structures first. If you’re not familiar with them, consider taking our Python Data Structures course. Its 118 interactive exercises will give you the opportunity to work with lists, nested lists, sets, tuples, and dictionaries.

For this article, we will start with a simple use case and improve our script as we implement the different tips for writing useful Python scripts.

10 Useful Tips for Better Python Scripts

For a starter, we will write a script that reads an RGB image and converts it to grayscale, a basic pre-processing step in computer vision. We’ll use OpenCV for this step. If you do not have OpenCV installed, you can do it easily with pip:

pip install opencv-python

For this example, I have downloaded 50 images of Formula One cars saved in data/formula_one with files from 000000.jpg to 000049.jpg.

First, let's create a Python file called gray.py and execute it in the terminal. It will read the images, convert them to grayscale and display them on our screen.

import cv2
import glob

image_files = sorted(glob.glob("./data/formula_one/*.jpg"))

for i in image_files:
   img = cv2.imread(i)
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
   gray_img_small = cv2.resize(gray_img, (960, 540))
   cv2.imshow('Grayscale Image', gray_img_small)
   cv2.waitKey(0)

cv2.destroyAllWindows()

Below is an example of the results:

10 Useful Tips for Writing Python Scripts

Now that we have something running, let's see how we can improve this script with the first Python script tip.

1.  Emphasize Python Code Readability

My first tip is not to write scripts for yourself, even if you are the only one using them. Instead, think about your future self – or, if you write Python code for someone else, about the person who will maintain the code after you. In other words, your code must be well-commented, readable, and understandable by anyone who’ll look at your Python script. If you decide to over-engineer your code and make it highly optimized at the expense of readability, you must be thorough in your comments.

I haven't put any comments in my previous example, so this can be the first step to improving our Python script. Let's use comments to explain what our code does:

import cv2
import glob

# Load the images in ascending order
image_files = sorted(glob.glob("./data/formula_one/*.jpg"))

# Loop through the files display them as grayscale
for i in image_files:
   img = cv2.imread(i)  # Read the image
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert the image to grayscale
   gray_img_small = cv2.resize(gray_img, (960, 540)) # Resize the image for visualization purpose
   cv2.imshow('Grayscale Image', gray_img_small) # Display the gray image in a window called 'Grayscale Image'
   cv2.waitKey(0) # Press any key to close the window
  
cv2.destroyAllWindows() # Close all the open windows

Now our code is much clearer. Later on, these comments will help you remember your thought process when writing the code. However, our comments might be a bit obvious, as this article is a tutorial on Python tips and tricks. In this context, comments often explain the “what” of the code; instead, it’s often a good practice to write the “why” of this code snippet. For example, I could’ve written the comments like this:

import cv2
import glob

# Sorting the images in order is essential when the images are recorded in sequences (i.e., videos, sensor capture with timestamps)
# This is particularly important if the images are video frames, where the order matters. 
image_files = sorted(glob.glob("./data/formula_one/*.jpg"))

# Converting to grayscale normalizes images and makes image processing faster. It helps machine learning models when color is not an important feature. 
for i in image_files:
   img = cv2.imread(i) 
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
   gray_img_small = cv2.resize(gray_img, (960, 540)) 
   cv2.imshow('Grayscale Image', gray_img_small)
   cv2.waitKey(0)  

cv2.destroyAllWindows()

2.  Write Reusable Python Functions

If you can make your code fully reusable with the help of functions, you will save yourself a lot of pain. So, instead of writing the instructions directly in your file, try to wrap them up within functions. This will force you to generalize your code and make it more reusable for future scripts.

Let's write a function to take our image file as an argument and return the non-resized gray image. We’ll put the function at the top of the script, just after the imports:

import cv2
import glob

# Color to Gray Function
def color2gray(filename):
  
   img = cv2.imread(filename)  # Read the image
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert the image to grayscale
   gray_img_small = cv2.resize(gray_img, (960, 540)) # Resize the image for visualization purposes
   cv2.imshow('Grayscale Image', gray_img_small) # Display the gray image in a window called 'Grayscale Image'
   cv2.waitKey(0) # Press any key to close the window

   return gray_img

# Load the images in ascending order
image_files = sorted(glob.glob("./data/formula_one/*.jpg"))

# Loop through the files and display the images as grayscale
for i in image_files:
  color2gray(i)
 cv2.destroyAllWindows() # Close all the open windows

3.  Write Your Own Python Package

My next tip is a follow-up for the previous one. Now that you’ve already started to make your code reusable by adding functions, you can store those functions in a Python file. You can import this file like any library and pull your functions whenever you need them.

The next step is to organize your code in classes. This can also allow you to use the dir() method to access the list of functions in your file.

Let's put our previous function in a new Python file called utils.py and import it like any Python package. Remember that your utils.py file needs to be in the same directory.

import cv2

# Color to Gray Function
def color2gray(filename):
  
   img = cv2.imread(filename)  # Read the image
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert the image to grayscale
   gray_img_small = cv2.resize(gray_img, (960, 540)) # Resize the image for visualization purposes
   cv2.imshow('Grayscale Image', gray_img_small) # Display the gray image in a window called 'Grayscale Image'
   cv2.waitKey(0) # Press any key to close the window

   return gray_img

And the updated gray.py file:

import cv2
import glob
import utils

# Load the images in ascending order
image_files = sorted(glob.glob("./data/formula_one/*.jpg"))
# Loop through the files display them as grayscale
for i in image_files:
  utils.color2gray(i)
 cv2.destroyAllWindows() # Close all the open windows

If, for some reason, your utils.py file is in another directory, you can import it into your gray.py file:

import cv2
import glob
import sys 
sys.path.append("your-path-to-utils")
import utils

# Load the images in ascending order
image_files = sorted(glob.glob("./data/formula_one/*.jpg"))
# Loop through the files display them as grayscale
for i in image_files:
  utils.color2gray(i)
 
cv2.destroyAllWindows() # Close all the open windows

There's more to know about Python packages, but this should get you started. Readability is one of the advantages of storing your functions in a dedicated Python file, but it also makes your code easier to maintain.

You can access the list of your functions with dir().

import utils

print(dir(utils))

We get the following output: 

['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'color2gray', 'cv2']

You can see color2gray, the function we defined before. But that's not all. Next, let's move on to one of my best Python tips and tricks as we look at the argparse module.

4.  Include Command-Line Arguments

Want to make your script reusable? Generalize it! What a hassle to have to dig into your previous Python script to edit it for your current use case! So, save yourself the trouble and use argparse. You will thank yourself later!

Argparse is a powerful Python library. I encourage you to read my guide to the Python argparse module to learn more about it. In the meantime, let's update our code to include it and be able to execute our Python script with command-line arguments.

import cv2
import glob
import utils
import argparse as ap

# Define an argument parser object
parser = ap.ArgumentParser()

# Define the arguments to add
parser.add_argument("-d", "--directory", type=str, required=True, help="Directory of files to load")
parser.add_argument("-e", "--ext", type=str, default=".jpg", help="Define file extension")
args = parser.parse_args()

# Load the images in ascending order
image_files = sorted(glob.glob(args.directory+"*"+args.ext))
# Loop through the files display them as grayscale
for i in image_files:
   utils.color2gray(i)

cv2.destroyAllWindows() # Close all the open windows

We can run our script as follows: 

python gray.py --directory ./data/formula_one/ --ext .jpg

Including argparse in your script makes it fully reusable. Using it, you can type the arguments you need to execute the scripts according to your needs without editing the code directly.

5.  Use docstring Instead of Single-Line Comments

Next, I recommend using docstring instead of single-line comments (i.e comments with #). There are two reasons for this. On the one hand, you can write multi-line comments effortlessly and correctly; on the other hand, the program user can have access to the comments by calling the help() function.

Strings are an essential Python data structure. Feel free to check out our course on working with strings in Python if you want to learn more.

Let's edit our utils.py file to include a docstring comment:

import cv2

# Color to Gray Function
def color2gray(filename):
   """
   This function reads an image and converts it to grayscale.
   For visualization purposes, we resize the output as a 960x540 image.

   Press any key to close the window and continue.
   """
  
   img = cv2.imread(filename) 
   gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
   gray_img_small = cv2.resize(gray_img, (960, 540))
   cv2.imshow('Grayscale Image', gray_img_small)
   cv2.waitKey(0)
   return gray_img
print(help(color2gray))

Then, we can run our utils.py in the terminal …

python utils.py

… and get the function documentation below:

Help on function color2gray in module __main__:

color2gray(filename)
    This function reads an image and converts it to grayscale.
    For visualization purposes, we resize the output as a 960x540 image.
    
    Once the image is displayed, press any key to close the window and continue.
(END)

6.  Use Version Control

Have you had your laptop die in your hands because someone spilled a cup of water on it? Or have you lost all your files because you made a typo when running the Linux rm -rf command to delete a directory? Both have happened to me, so you don’t have to repeat my mistake!

To avoid rewriting your scripts or taking your screwdriver to open your laptop and extract your hard drive, save yourself some trouble with version control. A few options are available, such as GitLab, GitHub, Beanstalk, and more. I personally use GitHub. Pushing your code regularly will ensure that you always have a project version available. It can be done in the blink of an eye:

  1. Go to your project directory and open your command-line terminal.
  2. Type $ git init
  3. Type $ git add your-project-filenames
  4. Type $ git commit
  5. Go to your GitHub account and create a new repository.
  6. In your command-line terminal, type $ git remote add origin git@github.com:your-username/your-repo
  7. Type $ git push

That’s it! You’re all set. If you want to learn more about how to get started using GitHub, now is a good time to learn!

7.  Serialize Your Python Variables for Reuse

Object serialization can also be handy. Let's say one of your functions takes a long time to run and you will have to reuse the same output in multiple files, either for research purposes (like doing multiple experiments) or for some other use case. Instead of running your code multiple times, you can serialize the output by saving it as a pickle file and reloading it in your next script.

In our previous example, our function returned the grayscale image as a NumPy array; our script runs 50 images. Maybe you have a script doing the same things for thousands of images and you need those NumPy arrays for another purpose. We can save these data structures for future reuse to avoid re-running the code. Let's import pickle and update our script:

import cv2
import glob
import utils
import argparse as ap
import pickle as pkl

# Define an argument parser object
parser = ap.ArgumentParser()

# Define the arguments to add
parser.add_argument("-d", "--directory", type=str, required=True, help="Directory of files to load")
parser.add_argument("-e", "--ext", type=str, default=".jpg", help="Define file extension")

args = parser.parse_args()

# Load the images in ascending order
image_files = sorted(glob.glob(args.directory+"*"+args.ext))
# Create an empty list
gray_arr = []
# Loop through the files display them as grayscale
for i in image_files:
   arr = utils.color2gray(i)
   gray_arr.append(arr) # Append the gray arrays to a list

cv2.destroyAllWindows() # Close all the open windows

# Save the gray images as a list of Numpy arrays in a pickle file.
with open("gray_arr.pkl", "wb") as f:
   pkl.dump(gray_arr, f)

Now, you can directly load the gray_arr.pkl file and retrieve the saved data structure. This can save you time by avoiding re-running the script multiple times to retrieve the results. It can also be a way to free up memory. Instead of storing your values in a list, you can save them as a pickle file and delete the variable to reduce the load on your memory.

8.  Use Optimized Python Libraries

My next essential tip comes directly from the previous example where we saved a bunch of images as NumPy arrays – a Python library written in C. C is a much faster language than Python. But Python, despite being slower, is straightforward to use and tends to make development faster, which explains its popularity and versatility.

By using libraries written in C, you make your Python code much easier and faster. If your hardware supports Cuda, you can also use CuPy, a version of NumPy optimized for GPUs, making your code faster still. These libraries are also very convenient and allow you to make complex mathematical operations with one function call. 

9.  Write Pythonic Code

My next tip is to use Python one-liners to make your code more concise and elegant. Our previous code stores the output of the color2gray function in a traditional list:

# Create an empty list
gray_arr = []
# Loop through the files display them as grayscale
for i in image_files:
   arr = utils.color2gray(i)
   gray_arr.append(arr)

However, we can rewrite this in Pythonic fashion as a one-liner:

gray_arr = [utils.color2gray(i) for i in image_files]

Now our code is more concise and more elegant – and, above all, more Pythonic!

10.  Take Care of Possible Errors

When your script works properly without throwing any errors, that's great. But it’s rarely the case. And sometimes errors are part of the process and should not stop your script. For example, you may want to scrape some images, but the request throws an exception error and cannot return any image. Then your script stops without executing the instructions you asked it to do. Thus, the way you end your Python script is important. To prevent your script from stopping, you can make use of try-except; in this case, exception handling will only skip the file and keep your script from stopping.    

More Helpful Tips for Writing Python Scripts

In this article, we learned some useful tips for writing Python scripts. There is much more to say, but we have already covered enough ground to get you started. If you want to explore further, feel free to read some of my other things that can help you write better Python code. And if you haven't yet, why not join our Python programming track?

Tags:

You may also like