- [[#The Test Pyramid|The Test Pyramid]] - [[#Unit Tests|Unit Tests]] - [[#Service Tests (Integration Tests)|Service Tests (Integration Tests)]] - [[#End-to-End (E2E) Tests|End-to-End (E2E) Tests]] - [[#Consumer-Driven Contracts (CDCs)|Consumer-Driven Contracts (CDCs)]] - [[#Consumer-Driven Contracts (CDCs)#Why CDCs?|Why CDCs?]] - [[#Consumer-Driven Contracts (CDCs)#How It Works|How It Works]] - [[#Consumer-Driven Contracts (CDCs)#Tools|Tools]] - [[#Testing in Production|Testing in Production]] - [[#Testing in Production#Smoke Tests|Smoke Tests]] - [[#Testing in Production#Synthetic Monitoring|Synthetic Monitoring]] - [[#Testing in Production#Feature Flags|Feature Flags]] - [[#Key Testing Principles|Key Testing Principles]] - [[#Key Testing Principles#1. Favor Fast Feedback|1. Favor Fast Feedback]] - [[#Key Testing Principles#2. Tests Should Be Deterministic|2. Tests Should Be Deterministic]] - [[#Key Testing Principles#3. Own the Test Pyramid|3. Own the Test Pyramid]] - [[#Key Testing Principles#4. Testing Is Risk Management|4. Testing Is Risk Management]] - [[#How MusicCorp Compares to Chapter 9 Recommendations|How MusicCorp Compares to Chapter 9 Recommendations]] - [[#Action Items for MusicCorp|Action Items for MusicCorp]] - [[#Action Items for MusicCorp#High Priority|High Priority]] - [[#Action Items for MusicCorp#Medium Priority|Medium Priority]] - [[#Action Items for MusicCorp#Lower Priority|Lower Priority]] - [[#Discussion Questions|Discussion Questions]] - [[#Key Quotes|Key Quotes]] - [[#Recommended Reading|Recommended Reading]] ## The Test Pyramid ``` /\ / \ E2E Tests (few, slow, brittle) /----\ / \ Service/Integration Tests /--------\ / \ Unit Tests (many, fast, reliable) -------------- ``` - More tests at the bottom, fewer at the top - Lower levels are faster and more reliable - Higher levels catch more integration issues but are slower and flakier --- ## Unit Tests Test single functions/classes in isolation. **Characteristics:** - Fast, deterministic, easy to debug - Mock external dependencies - Run on every commit **In MusicCorp:** - Go: `go test -v ./...` in services/order-go - Python: `pytest` for service logic ```go // Example: Order state machine test func TestOrderCanTransitionFromPlacedToPaid(t *testing.T) { order := NewOrder("test-id", "CD001", 1) err := order.TransitionTo(StatusPaid) if err != nil { t.Errorf("expected transition to succeed, got %v", err) } } ``` --- ## Service Tests (Integration Tests) Test a single microservice's behavior. **Characteristics:** - May involve real databases, message queues - Slower than unit tests - Stub/mock external microservices **In MusicCorp:** - Test Order service with real PostgreSQL - Mock Catalog/Inventory HTTP calls - Use test Kafka topics ```python # Example: Integration test with real database def test_create_order_persists_to_database(test_db): response = client.post("/orders", json={"sku": "CD001", "quantity": 1}) assert response.status_code == 201 # Verify in database order = test_db.query(Order).filter_by(sku="CD001").first() assert order is not None assert order.status == "PLACED" ``` --- ## End-to-End (E2E) Tests Test entire system behavior across multiple microservices. **Characteristics:** - Slowest, most brittle - Cross multiple microservices - Use sparingly; supplement with other techniques **When to Use:** - Critical user journeys - Smoke tests after deployment - Regression testing for major features ```bash # E2E test: Full order flow curl -X POST http://localhost/api/orders/orders \ -d '{"sku": "CD001", "quantity": 1}' # Wait for saga completion sleep 5 # Verify order completed curl http://localhost/api/orders/orders/{order_id} # Expected: status = "COMPLETED" ``` --- ## Consumer-Driven Contracts (CDCs) Consumers define expectations as contracts; providers verify they meet contracts. ### Why CDCs? | Traditional Approach | CDC Approach | |---------------------|--------------| | Provider defines API | Consumer defines expectations | | Consumer adapts to provider | Provider adapts to consumer needs | | Breaking changes discovered late | Breaking changes caught early | ### How It Works 1. **Consumer** writes a contract: "I expect GET /albums/CD001 to return {name, price}" 2. **Contract** is shared with provider team 3. **Provider** runs contract tests as part of CI 4. **Break detected** if provider changes break consumer expectations ### Tools - **Pact**: Popular CDC testing framework - **Spring Cloud Contract**: For Spring Boot services - **Contract stored centrally**: Pact Broker, Git repo ```json // Example Pact contract { "consumer": { "name": "order-service" }, "provider": { "name": "catalog-service" }, "interactions": [{ "description": "get album by SKU", "request": { "method": "GET", "path": "/albums/CD001" }, "response": { "status": 200, "body": { "sku": "CD001", "name": "string", "price": 14.99 } } }] } ``` --- ## Testing in Production ### Smoke Tests Quick verification that deployment succeeded: ```bash # After deployment curl -f http://localhost/api/orders/health || exit 1 curl -f http://localhost/api/catalog/health || exit 1 curl -f http://localhost/api/inventory/health || exit 1 ``` ### Synthetic Monitoring Fake requests to verify behavior: - Scheduled requests that mimic real users - Alert on failures or degraded performance - Don't pollute real metrics (tag as synthetic) ### Feature Flags Limit blast radius of new features: ```python # Example: Feature flag for new payment provider if feature_flags.is_enabled("new_payment_provider", user_id): process_with_new_provider(payment) else: process_with_old_provider(payment) ``` --- ## Key Testing Principles ### 1. Favor Fast Feedback ``` Fail fast → Fail often → Learn quickly ``` Run fast tests first; save slow tests for later stages. ### 2. Tests Should Be Deterministic No flakiness. A test should always pass or always fail for the same code. **Red flags:** - "Just run it again" - "Works on my machine" - Random sleeps to wait for async operations ### 3. Own the Test Pyramid Balance coverage across levels: | Level | Quantity | Speed | Reliability | |-------|----------|-------|-------------| | Unit | Many | Fast | High | | Integration | Some | Medium | Medium | | E2E | Few | Slow | Lower | ### 4. Testing Is Risk Management You can't test everything. Focus on: - High-risk areas (payment, security) - Frequently changing code - Customer-facing features - Integration points --- ## How MusicCorp Compares to Chapter 9 Recommendations | Book Recommendation | Our Implementation | Status | |---------------------|-------------------|--------| | **Unit tests** | Go tests exist, Python tests minimal | Partial | | **Integration tests** | Saga compensation test | Partial | | **E2E tests** | Manual via demo scripts | Gap | | **Contract testing** | Not implemented | Gap | | **Test pyramid** | Bottom-heavy (unit) would be good | Gap | | **CI runs tests** | GitHub Actions runs tests | Done | | **Smoke tests** | Health checks exist | Partial | --- ## Action Items for MusicCorp ### High Priority 1. **Add Python unit tests** - Test state machine transitions - Test event handling logic - Mock database and Kafka 2. **Add integration tests** - Test each service with real PostgreSQL - Use testcontainers for isolated DB ### Medium Priority 1. **Add E2E test suite** - Test full order flow - Run in CI against Kind cluster 2. **Consider contract testing** - Pact for REST API contracts - Event schema validation ### Lower Priority 1. **Synthetic monitoring** - Scheduled health checks - Alert on failures --- ## Discussion Questions 1. **Test pyramid balance**: We have Go unit tests but minimal Python tests. How do we prioritize adding tests? 2. **Contract testing value**: Is Pact worth the overhead for 5 services? When would it become valuable? 3. **E2E test environment**: Should E2E tests run against Kind cluster in CI, or is that too slow/complex? 4. **Flaky tests**: How do we handle tests that depend on timing (Kafka event delivery, async operations)? 5. **Testing sagas**: How do you test a choreographed saga where the flow spans multiple services? --- ## Key Quotes > "Tests should be deterministic. They should pass or fail for the same code every time." > "The test pyramid exists because the higher you go, the slower and more brittle tests become." > "Consumer-driven contracts catch breaking changes early, at the provider's CI pipeline." > "Testing is a risk management exercise, not a guarantee of correctness." --- ## Recommended Reading - *Growing Object-Oriented Software, Guided by Tests* by Steve Freeman and Nat Pryce - *Continuous Delivery* by Jez Humble and David Farley - Pact documentation: <https://docs.pact.io/>