Encode configuration files in python

Python is a powerful languauge for encoding configuration information for a software program. Especially the dictionary contruct and the ability to nest data structures allows to encode complex configuration parameters. Also storing the configuration as a text file allows for easier debugging and manual editing of the configuration. For example, the unix passwd file can be encoded as a dictionary with the user name as the key. Each entry in the dictionary would be another dictionary with name, uid, etc. as keys or it could be a tuple with fixed positions for name, uid, etc.

Offcourse since python is not designed to be used for encoding configuration, it does not directly provide routines to load  and save configuration files written as python scripts. Saving the configuration file is fairly simple as the str method cleanly converts any python data structure to a string that can be directly written to a file. Note that for a string type configuration parameter, you need to explicitely add quotes while writing to the file. The python code to save a configuration file myconfig.cfg would look like
follows:

f = open(”/path/to/myconfig.cfg”, “w”)
f.write(”some_config_param = “)
f.write(str(some_config_param))
f.write(”\n”)

Loading a configuration file written as python data structures and then accessing the configuration information seemlessly is slightly more complex. Using the imp module is one possible way to load the configuration file. The imp module provides two functions, find_module to search for a module using the standard heuristics and load_module to load the file found by find_module and return a module object. The return values of find_module are to be passed to load_module as parameters. Once the module object is available, one can access the configurtion information through its attributes. There are couple of issues with using the imp module.

• The file needs to have a .py extension since findmodule searches files with only certain extentions and guesses the type of the file from the extension. This can be worked around by opening the config file instead of using find_module and passing the open file to load_module.
• The load_module method compiles the file as a .pyc file before importing it. That leaves a unrequired file behind. The compiled file is used by load_module if it is newer than the config file. In case of a race between two threads, one saving the config file and the other one loading it, the compiled file could get a timestamp same as the config file but with the old contents. Any further load_module calls load from the compiled file and hence, load the old config data.

The execfile function is a better way to load a configuration file. Again, the execfile method is not intended for loading python data structures. It’s primary use is to run an independent piece of python code. But the function allows us to specify the global and local dictionaries as parameters. Also, the effects of the executed code are reflected in the parameters. The python code to load a configuration file myconfig.cfg would look like follows:

configuration_globals = {}
configuration_locals = {}
execfile(”/path/to/myconfig.cfg”, configuration_globals, configuration_locals)
some_config_param = configuration_locals["some_config_param"]

Happy programming!

Add comment March 6th, 2008