Back to blog
EngineeringDate unavailable· min read

M2 Complete: When Security Tests Catch What Mocks Hide

730 lines of security and performance tests complete the HTTP layer for Unified Lists—and the audit found what the mocks were hiding.

We just shipped 730 lines of tests that complete Milestone 2—the HTTP layer for Unified Lists. It's the boring, unglamorous work that makes the difference between "it works on my machine" and "it works in production when someone tries to break it."

In our implementation plan, Milestone 2 is simple: take the ListService CRUD layer (M1, already complete) and expose it through 12 FastAPI endpoints. Create the router, wire up the service, register it in the app. Done, right?

Not quite. M2 isn't complete until we know the HTTP layer is secure and fast. That meant two test suites: security (T-93 through T-100) and performance (T-101 through T-103).

Security Tests: The Boring Checklist

438 lines covering the obvious-but-necessary properties: user isolation (user B can't GET user A's lists → 404), SQL injection resistance (list names and query keywords can't escape to raw SQL), XSS payload handling (JSON responses sanitize HTML), API key enforcement on all 12 endpoints, oversized content rejection (1MB payloads fail at the Pydantic layer), and batch size limits.

Here's where it got interesting. The first draft of T-99 (oversized content rejection) used a mock side_effect to simulate validation failure. The test passed. Ship it.

Except the TDD audit caught it. Finding F2: "Remove mock side_effect that masked real Pydantic validation." The mock was hiding the fact that ListItemCreate.content has max_length=2000 and would reject a 1MB payload before the service was ever called. The test verified nothing.

We removed the mock, sent the real oversized payload, and watched Pydantic reject it at the request boundary. That's the test. The system already had the guard; we just needed to verify it was there.

Finding F3 was worse: T-100 (batch size limit) also had a mock side_effect. When we removed it to test the real validation, we discovered there was no batch size limit anywhere in the code. BatchUpdateRequest.updates was min_length=1 with no ceiling. A client could theoretically send 10,000 updates in a single request.

We added max_length=200 to the Field definition, removed the mock, and ran the test again. Now it verifies a real constraint. The bug would have shipped if we'd trusted the mocked test.

Performance Tests: Contract Enforcement

292 lines validating that the HTTP layer doesn't add unacceptable overhead to the service layer we already know is fast. We established two NFRs (non-functional requirements): NFR-1 says CRUD operations (create, get, update, delete) must complete in < 500ms at p95. NFR-2 says query and batch operations must complete in < 1 second at p95.

The tests use mocked ListService responses (so we measure HTTP-layer overhead, not database latency) and run 30 iterations per endpoint to get a stable p95 estimate. T-101 validates NFR-1 (CRUD endpoints), T-102 validates NFR-2 (query and batch), and T-103 is a canary test that fails if the p95 calculation itself is broken.

All three passed on the first run. The HTTP layer is thin—FastAPI routing, Pydantic validation, and JSON serialization add negligible latency. But now we have the contract test to prove it, and if a future change breaks that assumption, the CI build will fail.

The TDD audit also caught three smaller issues: an unused statistics import (F1), a missing wrong-API-key test case (F4—T-98 only tested missing keys, not incorrect ones), and unclear docstrings in T-95/T-96 that didn't specify they only test router-layer behavior, not end-to-end security (F5).

What's Next

With M2 complete, we move to Milestone 3: agent tools. The HTTP layer exists so the frontend can talk to the service. The tools exist so the agent can talk to the service. That's the 75% use case—most list interactions will happen through natural language chat, not by clicking buttons.

M3-T1 is seven tool definitions (create_list, add_items, query_items, promote_to_reminder, etc.) that wrap the HTTP endpoints in conversational interfaces. M3-T2 registers them in the tool set. M3-T3 integrates the nudge evaluator into the conversation-start flow. M3-T4 is the end-to-end chat test: create a list, add items, query them, promote one to a reminder—all through natural language.

After that, Milestone 4 is the frontend—the 25% use case for when you want the visual dashboard. And then M5 is migration: move existing Tasks and Reminders into the new unified schema and deprecate the old endpoints.

But for now, the HTTP layer is solid. The tests prove it's secure, fast, and won't break when someone tries something stupid. And the audit proved the tests are actually testing something real.