Unit Testing vs Integration Testing

Unit Testing

  • mock everything external to your code.
  • mock everything up to the thing to be tested
  • do some stuff
  • assert one thing.
  • Don’t continue a test – instead, mock up the environment to match the new thing to test, and write that in a new test, with a new assert.
  • Its cheap to run lots of tests
  • Its cheap to mock things.

Integration Testing

  • Its expensive to set up data
    • That’s a talk to itself, on how to do that in a sane repeatable way.
  • There is no mocking. Its real stuff, all the time.
  • Thus, when you achieve one milestone, your test (more like a “flow”) continues to the second milestone. Examples:
    • Upload, Pause, Continue Upload, Download, Pause, Continue Download
    • Upload, kill upload, Upload again
    • Upload, Download, kill Download, Download again.
    • Create item, edit item, rename/move item, delete item.
    • Its too expensive to try to get to a middle state, unlike mock-land.
  • Along the way, you print out “assessments” (to go with a AAA-style term) of where your test is at and what data its seeing.
    • ie, Arrange, and then
    • Act Assess Assert
    • Act Assess Assert
    • Act Assess Assert
  • In case of failure, you compare the log of the failed test with the log of a successful previous test to see what’s different.
  • The test can be VERY complicated – and LONG – and that’s fine. You only know the detail of the test while you are building it.
    • Once it goes green in a CI system, you forget the detail, until it fails.
    • If it does fail, you debug through the test to re-understand the test and inspect data along the way.
  • Expect flakiness
    • Sometimes things just fail. Example: locks placed on tables in PostGres for Unique in their UAT environment by some other process.
    • Sometimes things fail because of a reason
      • Somebody changed a schema.
      • Somebody deleted some key data
      • Some process crashed
      • Previous other test left behind data that FK locks the data you want to work with.
      • All these need human care and feeding and verification, its not “mathematically sound” like unit tests are.
    • Its a good idea to put an Assert.Ignore() if any failures happen during Arrange() section (ie, databases are down, file system full, etc – no longer a valid test. Not failed, but not valid, so ignored.
      • Can postpone this till a test starts to be flaky.
  • But when it works
    • you know that all the stuff is CORRECT in that environment.
    • And when it works in CI day after day after day, any failures = “somebody changed something unexpected” and needs to be looked at.
      • Fairly often its a shared DB, and somebody else changed schema in a way that you’re not yet accounting for.
      • Or somebody changed the defaults of something, and your test data that you’re hinging a test on has not been updated to match.

Which Ones To Use Where

  • Use Unit Tests to explore the boundaries and paths inherent in a piece of code.
    • Faster
    • Many of them
  • Use a single integration test (or just a few) to run through the code with all dependent systems
    • try to hit every SQL statement / external service at least once
    • If it worked, combined with all the unit tests, you’re probably good
Tagged with: ,
Posted in Code

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories
Tags
.net 3d 3d-printing 4k abc15 algorithms ames android anonymous types asp.net audio editing aws backup basecamp beatunes biorhythm bitcoin bittorrent blender blog boston marathon bpm c# caffeine campfire candycrush car carmax charity chiropractor cities-skylines clog clone codelouisville codepalousa coding coffee collaboration color run ComputerElbow ComputerVision configuration consulting cooking crash course crashplan crestwood cycling dabda dan dapper DataSet ddl diabetes dictation dotnetcore dotnetmud downtown e-cycling elite excel exercise expiration facebook feature-branching firefall flipflops Flow FL Studio focus food forecastle fortresscraft franklinplanner gadgets game-design games git github google docs google maps gopro gps grandpa greenshot hack half marathon headless health heart rate hiren ignew integration testing interop inventory ios ipad itunes javascript jobs karma kdf keyboards keys kittens lamont laptop lavalamp lego life lifehack linq linqtotwitter linux los angeles louisville mandelbulber massage therapy mastery-teaching maths merge metformin Minecraft miniature modeling monitor mud muhammad ali institute music mvc mycartracks netfabb nexus10 node nostalgia nutrition nwipe oldham county grand slam opal openjscad openscad owin pacedj paper mockup pepakura performance photoscan politics pomodoro postgresql powershell premiere prius process product-management project-management qa ReFS resharper review rmi roman road 5k RSI rubiks running samsung 700t sandals schedule scooter scribblelive selenium service shapeways sleep slic3r sneakersync snot software software-engineering solidoodle soylent spacegame speaking sql sqlite SSDT SSIS standing state-machine stayfocusd stonehearth sunset tablet teaching team teamcity teamtreehouse terraform tesla testing tfs time timelapse torque touch tracks trs80 Tuple tutor twitter ubuntu unit testing utilities video video editing visual studio vscode vsvim warp stabilizer windows 8 windows home server wordpress wpf xml
Archives
%d bloggers like this: