Day at the Office: A Tale of Testing

imageTL;DR:  Its really awesome when you understand the customer’s business language and can do thorough testing for them and document it.

I’ve been working on a bug for the last few days, which involves some math, and positive vs negative signs and such stuff.   Basically:

  • Customer sent me a screenshot where the math was obviously not working.
  • We figured out what was going on (boundary case not handled), and handled it.  

Normally, we’d be done at that point.  Except, this customer does not have a dedicated QA department; we are the first line QA department.  So.. we decided to be thorough.  Hence, this Test Document (screenshot; intentionally small so that details are illegible)

We posted the testing results of 7 types of things + 5 states of things, and highlighted any suspicious behavior (in yellow) for confirmation from the client —

And we have a brilliant customer, who gave us prompt feedback: (in blue)

Which we then fixed, in purple.

Time Creep

This 2-3 hour bug has now become a 2-3 day bug.   I feel a bit guilty about that, yet – if these situations had been discovered in the future, that would have been more angst, spread amongst more people; I’m glad we could figure this out now rather than later.

Also, by doing this testing and documenting it, I’m generating a snapshot-in-time of how the app is supposed to work. 

Unit Tests?

The only thing that would make this even more awesome would be to do these as unit tests.   But that would have been 3-4 days instead; also, we’re trying to wrap this project up.   Also, the customer doesn’t speak “unit tests”, but the customer definitely speaks “screenshots of the app”, so we’re choosing to converse in the latter language.

But how did it get Borged up to begin with?

I had shared the content of this post with a friend, and he asked the obvious question – why wasn’t this caught sooner?   The app has been used in production for over a year now. 

Reason Part 1:  this code (recalculating estimated cash balance in an account) was recently rewritten for performance optimization.  The old code (before my time on the project) was huge, complex, not documented, and slow – it was actually used for generating charts forecasting cash balance out into the future, and was designed to handle one portfolio at a time; the new code could do things in batches of portfolios, maximizing lookups of shared stocks and options, etc.    When we did the optimized calculations, we tested everything that we (and the client) knew of at the time.   No regrets.

Reason Part 2:  My client is also small enough that they don’t have a dedicated business guy who documents every rule around every calculation.   There’s a wealth of “common shared knowledge” amongst several people.  While we, as developers, try to understand that as much as possible, there’s a reason they do what they do very well – that stuff is complex.   Instead, our goal is to get 80%-90% close, “usable”, and then if there’s gaps remaining, those will get revealed, and patched later.  Reference concept Minimum Viable Product.   (Only doable with in-house applications; externally facing applications in a marketplace need more attention on polish and completeness.)

Tagged with:
Posted in Code

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

.net 3d 3d-printing 4k abc15 algorithms ames android anonymous types audio editing aws backup basecamp beatunes biorhythm bittorrent blender blog boston marathon bpm c# caffeine campfire candycrush car carmax certification charity chiropractor cities-skylines clog clone codelouisville codepalousa coding coffee collaboration color run ComputerElbow ComputerVision configuration consulting cooking crash course crashplan crestwood cryptocurrency cycling dabda dan dapper DataSet ddl diabetes dictation dotnetcore dotnetmud downtown e-cycling elite excel exercise expiration facebook fashion feature-branching firefall flipflops Flow FL Studio focus food forecastle fortresscraft franklinplanner gadgets game-design games git github google docs google maps gopro gps grandpa greenshot hack half marathon headless health heart rate hiren ignew integration testing interop inventory ios ipad itunes javascript jobs karma kdf keyboards keys kittens lamont laptop lavalamp leaf lego life lifehack linq linqtotwitter linux los angeles louisville mandelbulber massage therapy mastery-teaching maths merge metformin Minecraft miniature modeling monitor mud muhammad ali institute music mvc mycartracks netfabb nexus10 node nomanssky nostalgia nutrition nwipe oldham county grand slam opal openjscad openscad owin pacedj paper mockup pepakura performance photoscan politics pomodoro postgresql powershell premiere priorities prius process product-management project-management qa ReFS resharper review rmi roman road 5k RSI rubiks running samsung 700t sandals schedule scooter scribblelive selenium service shapeways sleep slic3r sneakersync snot software software-engineering solidoodle soylent spacegame speaking sql sqlite SSDT SSIS standing state-machine stayfocusd stonehearth studying sunset tablet teaching team teamcity teamtreehouse terraform tesla testing tfs time timelapse torque touch tracks trs80 Tuple tutor twitter ubuntu unit testing utilities video video editing visual studio vscode vsvim warp stabilizer windows 8 windows home server wordpress wpf xml zozo
%d bloggers like this: