Library

Saving internal data in a file with Python

Some automated tests failed. In this case you want to analyse what went wrong. It can be hard to analyse the cause of the problem. You can save all test data in a log file.  There is an easier way in python. Saving internal data in a file with Python is not difficult.

Suppose the following. You have some data structures in your test code. For example a dict with key value pairs. The tests finished. You want to know what values the data structure contains. Python can save the content of the data structure for you in a file.

Of course you can write a function that loops through the dict and saves then each key value pair in a file. There is an easier way to do this. Make use of the collections.abc module.

saving the internal data

The collections.abc module provides abstract base classes. You can use these classes to have a custom container class. Let’s see what you can do with it. You can create a class that is a subclass of collections.abc.MutableMapping. This is a mapping container. First you create some functions in the base class to get it working.

from collecations.abc import MutableMapping
from contextlib import suppress
import os

class FileDict(MutableMapping):
  def __init__(self, dirname, pairs=(), **kwargs):
       self.dirname = dirname
       with suppress(FileExistsError):
           os.mkdir(dirname)
       self.update(pairs, **kwargs)

  def __getitem__(self, key):
      fullname = os.path.join(self.dirname, key)
      try:
          with open(fullname) as f:
             return f.read()
       except FileNotFoundError:
           raise KeyError(key) from None

  def __setitem__(self, key, value):
    fullname = os.path.join(self.dirname, key)
    with open(fullname, "w") as f:
        f.write(value)

  def __delitem__(self, key):
      fullname = os.path.join(self.dirname, key)
      try:
          os.remove(fullname)
      except FileNotFoundError:
           raise KeyError(key) from None

  def __len__(self):
      return len(os.listdir(self.dirname))

  def __iter__(self):
       return iter(os.listdir(self.dirname))

  def __repr__(self):
        return f'FileDict(tuple(self.items()))'

You can have  pop, popitem, clear, update and setdefault for free in your class. The only thing to do is implementing some mandatory function. The mandatory functions are __getitem__, __setitem__, __delitem__, __iter__ and __len__.

The FileDict class is a kind of persistent map now. The class stores the map in a directory with a given name. The __init__ function receives that name. 

  • The __setitem__ function will store the value in a file with the key as name.
  • The __getitem__ retrieves the value in the file with the key as name. 
s = FileDict('starks')
s['test'] = 'data'
s['reply'] = 'valid']
print(s['test'])

Usage

Use the class as any other dict in python. The only difference is that behind the scenes the dict stores the values on your hard disk too. This  has two advantages. 

  1. You can see what is in the dict.
  2. You can manipulate the dict while the program or test is running

Saving internal data in a file with Python is easy with the collections.abc module. There is a lot more in that module. Check it out for yourself.

About the author

I currently work as a Test Automation Consultant at b.ignited. Here I work for different clients in different industries to help them start and speed up their testing cycles

I’ve been testing software since 2000 when I became involved in testing telephone applications and hardware. Since then, I’ve been able to expand my experience by testing a variety of embedded, web, mobile and desktop applications. I have used various development methodologies from waterfall to agile.

I consider myself to be a lifelong learner.