This is a quick note on how to execute shell commands in Python and construct arguments for Popen in subprocess module in Python. It bothered me for a while and I just recently figured out parts of it.
The easiest way to execute a shell command in a Python script is using the
API. The problem is that is always executing in front process group and gives you little control over the process you just created.
Following the recommendation in python’s page, I used Popen which gives you a process object, allows you to control (1) stdin, stdout, stderr (2) start the process as a child process in the background (3) wait for the process to finish using waitpid and many other additional functionalities as documented in the subprocess module
The problem with that is I was confused with the args in Popen API. I want to be able to simply run a command like os.system(cmd).
And I figured out that the right syntax would be
subprocess.Popen([‘ls -l ‘], shell=True) would work.
The documentation for this usage is
args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments.
So the trade off is that if we want Python to deal with escape characters, it would be better to use the sequence of program arguments API. However, I had a lot of issue with getting that to work. For easy commands without involving much escape characters, I would highly recommend using a single command line string with shell=True.