TL;DR: Its really awesome when you understand the customer’s business language and can do thorough testing for them and document it.
I’ve been working on a bug for the last few days, which involves some math, and positive vs negative signs and such stuff. Basically:
- Customer sent me a screenshot where the math was obviously not working.
- We figured out what was going on (boundary case not handled), and handled it.
Normally, we’d be done at that point. Except, this customer does not have a dedicated QA department; we are the first line QA department. So.. we decided to be thorough. Hence, this Test Document (screenshot; intentionally small so that details are illegible)
We posted the testing results of 7 types of things + 5 states of things, and highlighted any suspicious behavior (in yellow) for confirmation from the client —
And we have a brilliant customer, who gave us prompt feedback: (in blue)
Which we then fixed, in purple.
Time Creep
This 2-3 hour bug has now become a 2-3 day bug. I feel a bit guilty about that, yet – if these situations had been discovered in the future, that would have been more angst, spread amongst more people; I’m glad we could figure this out now rather than later.
Also, by doing this testing and documenting it, I’m generating a snapshot-in-time of how the app is supposed to work.
Unit Tests?
The only thing that would make this even more awesome would be to do these as unit tests. But that would have been 3-4 days instead; also, we’re trying to wrap this project up. Also, the customer doesn’t speak “unit tests”, but the customer definitely speaks “screenshots of the app”, so we’re choosing to converse in the latter language.
But how did it get Borged up to begin with?
I had shared the content of this post with a friend, and he asked the obvious question – why wasn’t this caught sooner? The app has been used in production for over a year now.
Reason Part 1: this code (recalculating estimated cash balance in an account) was recently rewritten for performance optimization. The old code (before my time on the project) was huge, complex, not documented, and slow – it was actually used for generating charts forecasting cash balance out into the future, and was designed to handle one portfolio at a time; the new code could do things in batches of portfolios, maximizing lookups of shared stocks and options, etc. When we did the optimized calculations, we tested everything that we (and the client) knew of at the time. No regrets.
Reason Part 2: My client is also small enough that they don’t have a dedicated business guy who documents every rule around every calculation. There’s a wealth of “common shared knowledge” amongst several people. While we, as developers, try to understand that as much as possible, there’s a reason they do what they do very well – that stuff is complex. Instead, our goal is to get 80%-90% close, “usable”, and then if there’s gaps remaining, those will get revealed, and patched later. Reference concept Minimum Viable Product. (Only doable with in-house applications; externally facing applications in a marketplace need more attention on polish and completeness.)
