Unit Testing

• mock everything external to your code.
• mock everything up to the thing to be tested
• do some stuff
• assert one thing.
• Don’t continue a test – instead, mock up the environment to match the new thing to test, and write that in a new test, with a new assert.
• Its cheap to run lots of tests
• Its cheap to mock things.

Integration Testing

• Its expensive to set up data
  • That’s a talk to itself, on how to do that in a sane repeatable way.
• There is no mocking. Its real stuff, all the time.
• Thus, when you achieve one milestone, your test (more like a “flow”) continues to the second milestone. Examples:
  • Upload, Pause, Continue Upload, Download, Pause, Continue Download
  • Upload, kill upload, Upload again
  • Upload, Download, kill Download, Download again.
  • Create item, edit item, rename/move item, delete item.
  • Its too expensive to try to get to a middle state, unlike mock-land.
• Along the way, you print out “assessments” (to go with a AAA-style term) of where your test is at and what data its seeing.
  • ie, Arrange, and then
  • Act Assess Assert
  • Act Assess Assert
  • Act Assess Assert
• In case of failure, you compare the log of the failed test with the log of a successful previous test to see what’s different.
• The test can be VERY complicated – and LONG – and that’s fine. You only know the detail of the test while you are building it.
  • Once it goes green in a CI system, you forget the detail, until it fails.
  • If it does fail, you debug through the test to re-understand the test and inspect data along the way.
• Expect flakiness
  • Sometimes things just fail. Example: locks placed on tables in PostGres for Unique in their UAT environment by some other process.
  • Sometimes things fail because of a reason
    • Somebody changed a schema.
    • Somebody deleted some key data
    • Some process crashed
    • Previous other test left behind data that FK locks the data you want to work with.
    • All these need human care and feeding and verification, its not “mathematically sound” like unit tests are.
  • Its a good idea to put an Assert.Ignore() if any failures happen during Arrange() section (ie, databases are down, file system full, etc – no longer a valid test. Not failed, but not valid, so ignored.
    • Can postpone this till a test starts to be flaky.
• But when it works
  • you know that all the stuff is CORRECT in that environment.
  • And when it works in CI day after day after day, any failures = “somebody changed something unexpected” and needs to be looked at.
    • Fairly often its a shared DB, and somebody else changed schema in a way that you’re not yet accounting for.
    • Or somebody changed the defaults of something, and your test data that you’re hinging a test on has not been updated to match.

Which Ones To Use Where

• Use Unit Tests to explore the boundaries and paths inherent in a piece of code.
  • Faster
  • Many of them
• Use a single integration test (or just a few) to run through the code with all dependent systems
  • try to hit every SQL statement / external service at least once
  • If it worked, combined with all the unit tests, you’re probably good