Since at least April, IBM has been looking at the issues surrounding the shocking TSB software rollout that cause so much banking pain for customers.
Big Blue has discovered that the main issues seem to stem from a lack of testing and control surrounding software changes. All in all it looks like the War Room’s would find a problem which was then changed without checking with other War rooms. This sort of practice tends to compound errors due to a lack of understanding around the impact of change.
Together with the over confident rollout on mass rather than several stages of testing and roll back to rectify issues. It seems shocking to this writer that any software organisation with so many customers and especially in the finance sector would not have a plan to roll back if there was a failure, never mind the simple mindedness of TSB actually believing that they had tested everything prior to putting the system “live” for the world to see (crashed and burned).
The IBM presentation document makes interesting reading with several hints for software houses on IBM’s own practices. All of the stages they mention and highlight as TSB failures along with the recommendations should help anyone write a comprehensive playbook for testing and rolling out new software.
This ex-software test engineer is please to see IBM pointing out so many errors that can be learnt from by others. I seem to recall the MD of a company I used to work for stating “We don’t need a test department, that’s what customers are for”. As shocking as that is to anyone in the sector it seems that TSB’s test team have forgotten some simple lessons and should go back to the drawing board.
At some point you have to realise that clear communication between the Software Team, the Test Team and Management are essential, in the case of TSB I would have to guess at a communications breakdown.
For TSB customers, the nightmare has been very real, for TSB it has led to more bad banking press and that has to generate trust issues. Lets remember that publishing faulty code that causes systems to grind to a halt is never great, but think of the potential for vulnerabilities in the system and suddenly I’m glad I don’t bank with TSB!