Tag: integration testing

Unit Testing vs Integration Testing

Unit Testing

  • mock everything external to your code.
  • mock everything up to the thing to be tested
  • do some stuff
  • assert one thing.
  • Don’t continue a test – instead, mock up the environment to match the new thing to test, and write that in a new test, with a new assert.
  • Its cheap to run lots of tests
  • Its cheap to mock things.

Integration Testing

  • Its expensive to set up data
    • That’s a talk to itself, on how to do that in a sane repeatable way.
  • There is no mocking. Its real stuff, all the time.
  • Thus, when you achieve one milestone, your test (more like a “flow”) continues to the second milestone. Examples:
    • Upload, Pause, Continue Upload, Download, Pause, Continue Download
    • Upload, kill upload, Upload again
    • Upload, Download, kill Download, Download again.
    • Create item, edit item, rename/move item, delete item.
    • Its too expensive to try to get to a middle state, unlike mock-land.
  • Along the way, you print out “assessments” (to go with a AAA-style term) of where your test is at and what data its seeing.
    • ie, Arrange, and then
    • Act Assess Assert
    • Act Assess Assert
    • Act Assess Assert
  • In case of failure, you compare the log of the failed test with the log of a successful previous test to see what’s different.
  • The test can be VERY complicated – and LONG – and that’s fine. You only know the detail of the test while you are building it.
    • Once it goes green in a CI system, you forget the detail, until it fails.
    • If it does fail, you debug through the test to re-understand the test and inspect data along the way.
  • Expect flakiness
    • Sometimes things just fail. Example: locks placed on tables in PostGres for Unique in their UAT environment by some other process.
    • Sometimes things fail because of a reason
      • Somebody changed a schema.
      • Somebody deleted some key data
      • Some process crashed
      • Previous other test left behind data that FK locks the data you want to work with.
      • All these need human care and feeding and verification, its not “mathematically sound” like unit tests are.
    • Its a good idea to put an Assert.Ignore() if any failures happen during Arrange() section (ie, databases are down, file system full, etc – no longer a valid test. Not failed, but not valid, so ignored.
      • Can postpone this till a test starts to be flaky.
  • But when it works
    • you know that all the stuff is CORRECT in that environment.
    • And when it works in CI day after day after day, any failures = “somebody changed something unexpected” and needs to be looked at.
      • Fairly often its a shared DB, and somebody else changed schema in a way that you’re not yet accounting for.
      • Or somebody changed the defaults of something, and your test data that you’re hinging a test on has not been updated to match.

Which Ones To Use Where

  • Use Unit Tests to explore the boundaries and paths inherent in a piece of code.
    • Faster
    • Many of them
  • Use a single integration test (or just a few) to run through the code with all dependent systems
    • try to hit every SQL statement / external service at least once
    • If it worked, combined with all the unit tests, you’re probably good