Invental/Review lessons/Tests that can never fail

4 lessons · 17 checks

Tests that can never fail

A green suite only means something if each test could have gone red. These lessons cover tests that pass for the wrong reason, and the checklist below is the quickest way to find them in your own suite.

Major

Mocks must return the real shapes

A blanket “mock it everywhere” fix can quietly replace the test doubles that make tests meaningful. Then the suite turns green, or red, for the wrong reason. A mock has to match the real module's exports and return shapes, not just exist.

Read the lesson → Major

Fix the SQL dialect bug in every dialect

When one SQL dialect breaks on user input, check every dialect that builds the same string by hand: the bug is in the shared pattern, not the one driver someone tested.

Read the lesson → Major

Test the worst input a limit still allows

When you add a limit to stop malicious input, test the worst input the limit still allows, not only the one it now rejects.

Read the lesson → Major

Negative tests must create what they exclude

A negative test (“X is never included”) has to create X first. Otherwise it passes because X doesn't exist, and it proves nothing about the guard it is named after.

Read the lesson →

The checks, with the lesson behind each

For each module you mock in tests, does the mock return the same shape the real function does?+
A blanket "mock it everywhere" fix can quietly replace the test doubles that make tests meaningful. Then the suite turns green, or red, for the wrong reason. A mock has to match the real module's exports and return shapes, not just exist.
If your app supports more than one database, does every dialect run the same input-normalisation code, with tests that run on each one?+
When one SQL dialect breaks on user input, check every dialect that builds the same string by hand: the bug is in the shared pattern, not the one driver someone tested.
For each input limit you added for safety, do you have a test that feeds the largest input the limit accepts and measures the cost?+
When you add a limit to stop malicious input, test the worst input the limit still allows, not only the one it now rejects.
For each "must never include / never leak" test in your suite, does the test first create the thing that must not appear?+
A negative test ("X is never included") has to create X first. Otherwise it passes because X doesn't exist, and it proves nothing about the guard it is named after.

More checks from the same reviews

Yes/no questions from other reviews on this theme. Each one is quick to answer for your own app.

  1. Do you have a test that feeds your API a real-shaped upstream error (e.g. a Google 403 rate-limit) and checks the status your client gets?
  2. Do your “should not throw” tests fail on any error, not only on one specific message?
  3. For every change in your PR, can you point to a reproduction or a test that failed before it?
  4. Do your fakes and stubs answer every method from the same seeded data, or does one of them return a hard-coded default?
  5. Does your daily report count the whole end day when the end date is today, and is there a test with a frozen clock that proves it?
  6. Do your tests check the content of a conversion (formats, levels), or only that the call didn't throw?
  7. When you add a new package to a monorepo, is there a test that fails if it's missing from the release list?
  8. For each limit in your API (body size, page size, upload size), is there a test just under and just over it?
  9. For every test file type in your repo (.test.ts, .spec.ts, .browser.test.ts, .e2e.ts), can you point to the CI step that runs it?
  10. Could your app load a package at runtime that works on your laptop but is missing from your production container?
  11. If someone runs your test suite while logged in to your production registry or cloud account, can any test publish or delete something real?
  12. Do your type tests assert the exact expected type, or only that it is “not string”?
  13. How many of your end-to-end tests were skipped on the last CI run, and do you know why each one was?

Want a senior reviewer to run these checks on your code? AI-assisted development that doesn't ship hallucinations →

Want these checked on your code?

We run reviews like these on real codebases. Tell us what you are building and we confirm scope before anything starts.

— Get in touch
hi@invental.co ↗
— Or
— What to include

A link or short description of the code or system, what you want checked, your stack, and when you need the answer. No repository access is needed until scope is agreed.

— Or leave a note
We reply within one business day.