SupportBank

Learning goals

  • A variety of data formats (CSV, JSON, XML)
  • Exception handling
  • Logging
  • Object Oriented Design

Programs used

Problem background

You’re working for a support team who like to run social events for each other. They mostly operate on an IOU basis and keep records of who owes money to whom. Over time though, these records have gotten a bit out of hand. Your job is to write a program which reads their records and works out how much money each member of the support team owes.

Each IOU can be thought of as one person paying another… but you can perform this via an intermediary, like moving money between bank accounts. Instead of Alice paying Bob directly, Alice pays the money to a central bank, and Bob receives money from the central bank.

SupportBank Intermediary Example Diagram

Setup instructions

You’ll find the code in this repo. Follow the instructions laid out in the README to get the project up and running.

Reading CSV files

The support team keep their records in CSV format. Their records for 2014 are stored in Transactions2014.csv (which should already be in the repository that you cloned).

Download the file and open it in Excel. Note that there’s a header row, telling you what each column means. Every record has a date and represents a payment from one person to another person. There’s also a narrative field which describes what the payment was for.

Write a program which creates an account for each person, and then creates transactions between the accounts. The person in the ‘From’ column is paying money, so the amount needs to be deducted from their account. The person in the ‘To’ column is being paid, so the amount needs to be added to their account. Use a class for each type of object you want to create.

Your program should support two commands, which can be typed in on the console:

  • List All – should output the names of each person, and the total amount of money they should receive from the bank. (It could be a negative number, if they owe the bank money!)

  • List [Account] – should print a list of every transaction, with the date and narrative, for that person’s account. For example, List Jon A would list all of Jon A’s transactions.

Before you start, consider (and discuss with your trainer) what data type you should use to represent currency amounts.

Logging and exception handling

Modify your program so that it also loads all of the transactions for 2015: DodgyTransactions2015.csv

You’ll probably notice that some dodgy data is present in the file and your program fails in some interesting way. In your work as a software developer, users will try to enter any old data into your program. You need to make sure that you explain, politely but firmly, what they’ve done wrong.

Firstly, let’s add some logging using a module from the Python standard library called logging. Add the following lines of code to the start of your program:

import logging

logging.basicConfig(filename='SupportBank.log', filemode='w', level=logging.DEBUG)

This code tells Python that when we use the logging module to write logs, we would like them to be written to the file SupportBank.log, and we would like all logs at level DEBUG or higher to be saved. To actually write some logs, you can use a line of code like this:

logging.info('An informative log!')

Look at the documentation from the logging module. Notice that there are several levels of severity at which you can log errors. Try logging something at the point when your program starts up, and check that a log file has been created.

Now add logging to your program. Get to a point where you could work out what went wrong by reading your log files. (Don’t try to fix the problem yet!)

Great. You now have forensic evidence to work out why things went wrong. Now change your program so that it fails gracefully and tells the user which line of the CSV caused the problem. Think about what failing gracefully means, in this situation. Should we import the remaining transactions from the file? Should we just stop at the line that failed? Could we validate the rest of the file and tell the user up-front where all of the errors are? What would make sense if you were using the software? Discuss with your trainers and work out what to do in this situation.

An aside on logging

Logging should be like your program’s diary, recording interesting happenings, its expectations and things it receives.

Dear diary, the user started running me at 10:30am

Dear diary, they're using the transaction import screen

Dear diary, they've given me a file called DodgyTransactions2015.csv to process

Dear diary, there's something wrong on line 57! It's supposed to be a Date, but it doesn't look correct!

Dear diary, something very bad has happened, I need to stop now

Often your logs will be all that’s left when things go wrong. Make sure that they’re descriptive enough so that you know why your program failed.

JSON

So your program works great. The support team accountants can rest easy, knowing that all of their debts can be reconciled… Except for one minor detail. Back in 2013, the support team didn’t store their records in CSV format. They stored them in a different format, called JSON. Open the 2013 transaction file Transactions2013.json and take a look. Hopefully, it’s fairly obvious how this data format works, and how the transactions in JSON format correspond to the old CSV transactions. JSON is one of the most widely used data formats worldwide. It’s used on the web for servers to communicate with clients, and also with each other.

Next step – you guessed it. Modify your program to accept JSON files in addition to CSV files.

Take a look at the documentation for the json module in the Python standard library to get started.

Tip

As you work through this part of the exercise, start thinking about the modules in your program and the relationship between them. Try to keep your modules small and focused, and make their function obvious based on their name. Try not to have common functionality repeated across two or more modules!

Extend the interface of your program by adding a new command: Import File [filename] which reads the file from disk. You’ll need different behaviour for CSV and JSON files, so make sure that you do the right thing based on the type of the file supplied.

XML

This is just getting silly. The support team’s transactions for 2012 were in XML format. This stands for eXtensible Markup Language, and is another commonly-used data format. It supports lots of different features and is much more powerful than CSV or JSON, but as a result is somewhat harder to work with. No wonder they moved away to using JSON instead.

Open the 2012 transactions file Transactions2012.xml and take a look at the structure. Again, it should be fairly obvious how this corresponds to the other format.

It may not come as a surprise to you now, but there’s a module in the Python standard library that will parse XML files for you. Take a look at the documentation for the xml.etree module.

Stretch goals

Add a new command: Export File [filename] which writes your transactions out in a format of your choosing.