How to execute shell command in Python and a quick note on Python Popen

This is a quick note on how to execute shell commands in Python and construct arguments for Popen in subprocess module in Python. It bothered me for a while and I just recently figured out parts of it.

The easiest way to execute a shell command in a Python script is using the

os.system(cmd)

API. The problem is that is always executing in front process group and gives you little control over the process you just created.

Following the recommendation in python’s page, I used Popen which gives you a process object, allows you to control (1)  stdin, stdout, stderr (2) start the process as a child process in the background (3) wait for the process to finish using waitpid and many other additional functionalities as documented in the subprocess module

https://docs.python.org/2/library/subprocess.html

The problem with that is I was confused with the args in Popen API. I want to be able to simply run a command like os.system(cmd).

And I figured out that the right syntax would be

subprocess.Popen([cmd], shell=True)

for example,

subprocess.Popen([‘ls -l ‘], shell=True) would work.

The documentation for this usage is

args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments.

So the trade off is that if we want Python to deal with escape characters, it would be better to use the sequence of program arguments API. However, I had a lot of issue with getting that to work. For easy commands without involving much escape characters, I would highly recommend using a single command line string with shell=True.

Advertisements
This entry was posted in Tools and tagged . Bookmark the permalink.

One Response to How to execute shell command in Python and a quick note on Python Popen

  1. Pingback: Python Notes | Yunming Zhang's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s