I was given the task of “testing everything” regarding an integration for my client.
This is a screenshot of a document of screenshots; appropriately zoomed out so that no detail is visible.
- The source system is in yellow; those are portrait screenshots of a multi-page web-form.
- The target system is in gray; its several screens of a Windows app.
- It used to be that gray was loaded from PDF’s, and yellow generated PDF’s. Human error, lots of time, etc.
- Now there is an intermediate system, lets call it blue, that is used to load the gray system. It has a code-generated API document that is 467 pages long, that mostly tells you that X field is stored at Y location in the database that gray displays fields from.
You might think that this could be straight forward, but its not:
- Some fields in yellow are old, and do not belong in gray
- Some fields in gray are manual entry, cannot be loaded via blue.
- Some fields in gray will in the future be loaded by blue, but not yet
- Some fields in yellow actually go to another system, purple.
- Furthermore, yellow is customizable. Not everything is displayed for everybody; some of the fields had never been used before to my knowledge, and might just be old stale stuff.
No problem! Just have the business persons spec out which fields go where!
- There’s a business person who knows yellow.
- There’s another person who knows blue.
- And another person who knows how blue loads gray, and knows more about gray.
- Everybody knows about the existence of purple. But, I didn’t find a person who knew purple. And blue doesn’t talk to purple, and we’re only talking to blue.
- These people are hard to pin down and have like 15 minutes to spare, tops.
- Even if I could get them all together, how to document it?
- The solution would be to find the person who was interested in having yellow go to gray who knew the fields on both sides. Unfortunately, there was no such person available. This is all speculative work that someday, in the future, would be good to have.
So, sensing somebody will have to feel the pain… I volunteered (billable) to Brute Force Test it. Where do I start?
- I could get an alphabetical list of all the fields in yellow.
- I mostly knew how yellow was put together – approximately which page had which fields.
- I knew almost nothing about gray.
My Solution
- Put on my QA Hat.
- Turn on everything possible in yellow
- Create a full application (ever possible field filled out), send it through blue to gray.
- Take screenshots of every screen so there is no doubt what I put in or what I got.
- Generate an Excel with the list of all fields from yellow, and the values I filled them out with.
- Because I don’t know gray, I’ll iterate through it one field at a time.
- The other ones, I can much more quickly find things, so less seek time.
- One page, one field at a time, find what I think is the ma
tch in yellow. - I did this against the excel spreadsheet that I had generated from yellow.
- Using pen and paper, cross stuff off from my screenshots of gray.
- Using Excel, update the field to field to field spreadsheet with success or failure
- Use Excel Filters to the group the fields into which groups need to be asked to which person.
- Ask the questions, get answers, in the 15 minute window I could get everybody together during.
- Address individual fields with detected problems one at a time, with screenshots of yellow blue and gray as needed.
This has been my life for the last week and a half. I’m almost through my second pass of fixing fields, thanks to feedback from the person who knows gray. And we’re a lot smarter about what in yellow doesn’t map with gray.
Mr. Pink was not harmed during this process. He can definitely say that he wasn’t because he knows when he was and when he wasn’t. But he cannot definitely say that about anybody else, because he doesn’t definitely know.
