Unit Testing vs Integration Testing

Unit Testing

mock everything external to your code.
mock everything up to the thing to be tested
do some stuff
assert one thing.
Don’t continue a test – instead, mock up the environment to match the new thing to test, and write that in a new test, with a new assert.
Its cheap to run lots of tests
Its cheap to mock things.

Integration Testing

Its expensive to set up data
- That’s a talk to itself, on how to do that in a sane repeatable way.
There is no mocking. Its real stuff, all the time.
Thus, when you achieve one milestone, your test (more like a “flow”) continues to the second milestone. Examples:
- Upload, Pause, Continue Upload, Download, Pause, Continue Download
- Upload, kill upload, Upload again
- Upload, Download, kill Download, Download again.
- Create item, edit item, rename/move item, delete item.
- Its too expensive to try to get to a middle state, unlike mock-land.
Along the way, you print out “assessments” (to go with a AAA-style term) of where your test is at and what data its seeing.
- ie, Arrange, and then
- Act Assess Assert
- Act Assess Assert
- Act Assess Assert
In case of failure, you compare the log of the failed test with the log of a successful previous test to see what’s different.
The test can be VERY complicated – and LONG – and that’s fine. You only know the detail of the test while you are building it.
- Once it goes green in a CI system, you forget the detail, until it fails.
- If it does fail, you debug through the test to re-understand the test and inspect data along the way.
Expect flakiness
- Sometimes things just fail. Example: locks placed on tables in PostGres for Unique in their UAT environment by some other process.
- Sometimes things fail because of a reason
  - Somebody changed a schema.
  - Somebody deleted some key data
  - Some process crashed
  - Previous other test left behind data that FK locks the data you want to work with.
  - All these need human care and feeding and verification, its not “mathematically sound” like unit tests are.
- Its a good idea to put an Assert.Ignore() if any failures happen during Arrange() section (ie, databases are down, file system full, etc – no longer a valid test. Not failed, but not valid, so ignored.
  - Can postpone this till a test starts to be flaky.
But when it works
- you know that all the stuff is CORRECT in that environment.
- And when it works in CI day after day after day, any failures = “somebody changed something unexpected” and needs to be looked at.
  - Fairly often its a shared DB, and somebody else changed schema in a way that you’re not yet accounting for.
  - Or somebody changed the defaults of something, and your test data that you’re hinging a test on has not been updated to match.

Which Ones To Use Where

Use Unit Tests to explore the boundaries and paths inherent in a piece of code.
- Faster
- Many of them
Use a single integration test (or just a few) to run through the code with all dependent systems
- try to hit every SQL statement / external service at least once
- If it worked, combined with all the unit tests, you’re probably good

Mocking Time at 1000x speed

I had a complicated bit of logic that involved multiple threads, waiting for the right time, checking for time expiration, etc. I wanted to unit test it, which meant mocking Time. And not in a simple way, but in a complex, works-across-threads way.

This is what I ended up with:

That’s pretty simple. The fun stuff happens in the Stub version:

Thus my logic component, which spawns multiple threads, each of which take a section of work to do, and each unit of work has “cannot start before” and “must not end after” – when it runs it against unit tests, it runs it at 100x or 1000x normal speed – but the simulation is accurate across all the threads.

Day at the Office: Syncing Systems

I have been on a coding adventure for the last two days.

Customer has a system, which involves holdings, and then proposed trades on those holdings. However, there’s also a master system that sends a new list of holdings on a daily basis. The master system does not know about the proposed trades, it says “here’s the new list”.

The algorithm when updating the list of holdings to the new holdings went like this:

delete anything that doesn’t have a proposed trade on it
copy in the stuff from master, creating new ones, except…
if an existing holding is found, instead of creating a new one, update the holding to look like what the master says it should (with various keys to try to grab the right one)

The Problem

There was a holding of Quantity = 20 that had proposed trades of –10 against it. The incoming holdings were of quantity 3 and 15. It updated 20 to 3, causing a math imbalance in the system.

Further problem on more investigation

It was also iterating the incoming list, so if I had trades against a holding H1, and H1 (or something like it) was not included in the new list, then the trades would get left behind against a stale holding.

My Solution Take 2

Delete any holding without a proposed trade on it
Figure out our maximum id so far
Always insert all data from master as new. (Now we have duplicate copies)
Walk through all the old holdings and proposed trades; try to match the proposed trades up against the new holdings

If no match found, log it and delete the proposed trade

Having moved all the proposed stuff over to a new parent, now delete all old holdings <= maximum id (no more duplicates)

Why I thought it was Clever

Suppose the load fails.

Since I’m only either updating or deleting proposed trades, I will not have any duplication of proposed trades.

Since I’m deleting all holdings prior to where I started inserting the new holdings, as long as I successfully complete a load, I will delete any old bad data from a previously failed load.

The Matching Algorithm

The general idea is: Sort “available buckets”, sort “Proposed Trades”, and then start matching trades to buckets.

I tested this out on paper – as in, I cut out little pieces of paper, and tried the different options; I determined sorting low to high on buckets and high to low on trades was my best bet.
I used a fit option of f(x1,y1,x2,y2):decimal to give me, for any trade, which bucket it would best fit in (after sizing was taken care of). (the X’s and Y’s are acquisition dates and execution prices, both of which can be “touched up” on the master system, making an exact match every time non-feasible)

To Translate This To Code

Because the matching stuff could get complicated, I made its own little command for it (see command-query separation).

I was not shy about creating more DTO’s for the specific inputs and outputs needed by this command.

I wrote enough tests to verify that it was doing the right thing. When I do unit tests, Iassume only developers will read them, so I don’t spend a lot of time making the names super-descriptive. Some-descriptive, yes, not super-descriptive.

Embedded image permalink

And then I wired it up. Turns out, I wired it into location #1, and that turned out to not work – transactions, entity framework, new transactions, missing ID’s, etc. So I ended up wiring it into location #2 (after the fact), and that worked a lot better. Good thing I had black-boxed its meaning, it was easy surgery.