magnus919_agent-skills/backend-engineering/references/database-testing.md

Backend Engineering Methodology Reference

Database access patterns & service-level testing — a comprehensive reference for backend engineering teams. Compiled: 2026-06-05

---

Table of Contents

1. Connection Pooling Configuration & Sizing
2. Query Optimization — Index Usage, Query Plans, EXPLAIN
3. N+1 Detection & Mitigation
4. Pagination Strategies — Cursor vs Offset vs Keyset
5. Transaction Boundary Design
6. Read/Write Splitting
7. Replication Lag Handling
8. Service-Level Testing Overview
9. Unit Testing Business Logic
10. Integration Testing — API Contracts, Testcontainers, WireMock
11. Contract Testing — Pact
12. Test Fixtures
13. CI Integration

---

1. Connection Pooling Configuration & Sizing

The Problem

Creating a new TCP connection per request does not scale. At 10K+ RPS, the database is overwhelmed. PostgreSQL defaults to 100 simultaneous connections; exceeding that produces "sorry, too many clients already." Each new connection setup adds 20-50 ms of latency.

The Solution

Connection pooling pre-establishes a fixed set of connections at application startup. Threads borrow a connection, execute queries, and return it to the pool.

┌──────────────┐     borrow     ┌──────────────────┐
│  App Thread  │ ─────────────→ │   Connection     │
│  (request)   │                │     Pool         │
│              │ ←───────────── │  (HikariCP/      │
│              │    return      │   pgBouncer)     │
└──────────────┘                └────────┬─────────┘
                                        │
                              ┌─────────▼─────────┐
                              │  Database Server   │
                              │  (PostgreSQL/MySQL)│
                              └───────────────────┘

Pool Sizing Formula

The most commonly cited rule of thumb: pool size = 2x (number of CPU cores).

However, the correct approach is empirical:

1. Start small — 20-30 connections for most services.
2. Run load tests with real traffic patterns. Monitor DB CPU, memory, connection wait times, and query latency.
3. Add a 15-20% buffer above measured peak usage.
4. Consider multiple pools for distinct workload patterns (e.g., small pool for admin queries, larger for user-facing traffic).

Key Configuration Parameters

Parameter	Description	Common Default
maximumPoolSize	Max connections in the pool	10-30
minimumIdle	Min idle connections to maintain	same as maxPoolSize
connectionTimeout	Max wait time for a connection (ms)	30000
idleTimeout	Max time a connection stays idle (ms)	600000 (10 min)
maxLifetime	Max lifetime of a connection in pool (ms)	1800000 (30 min)

Recommended Libraries

Language	Library	Notes
Java/Kotlin	HikariCP	Industry standard — fastest, lightest
Python	psycopg2.pool / SQLAlchemy pool	Built-in; tune pool_size and max_overflow
Node.js	pg-pool	Default pool for node-postgres
Go	pgxpool (/jackc/pgx)	High-performance Postgres driver
Ruby	connection_pool	Used by ActiveRecord internally
Rust	deadpool-postgres	Async pool for tokio-postgres
.NET	Npgsql pooling (built-in)	Connection pooling enabled by default

Proxy-Based Pooling (pgBouncer / PgCat)

For microservices or serverless, use a database proxy instead of app-level pooling:

[databases]
mydb = host=localhost port=5432 dbname=mydb

[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
pool_mode = transaction    # transaction-level pooling
max_client_conn = 100
default_pool_size = 20

• Transaction pooling — connections are returned to pool after each transaction (most common).
• Session pooling — connection held for entire session (useful for prepared statements).
• Statement pooling — connection returned after each statement (rarest).

Serverless Considerations

Serverless functions are short-lived and cannot maintain persistent pools. Use proxy-based solutions:

• AWS RDS Proxy (managed, IAM auth)
• Cloudflare Hyperdrive
• Supabase Supavisor
• PgCat (open-source proxy)

---

2. Query Optimization — Index Usage, Query Plans, EXPLAIN

Index Types (PostgreSQL-focused)

Index Type	Best For	Considerations
B-Tree (default)	Equality & range queries, ORDER BY, foreign keys	General-purpose; works for most cases
Hash	Equality lookups only	Single-column; not WAL-logged in older versions
GIN (Generalized Inverted Index)	Full-text search, arrays, JSONB containment	Larger than B-tree; slower to build
GiST (Generalized Search Tree)	Geometric data, full-text search (ranking)	Lossy; supports nearest-neighbor
BRIN (Block Range INdex)	Very large tables with naturally ordered data (time-series, logs)	Extremely compact; only good for correlated data
SP-GiST	Space-partitioned data (maps, network trees)	Niche; for clustered data
Covering Index (INCLUDE columns)	Index-only scans	Adds payload columns without affecting key sort order

Composite Index Guidelines

• Order matters: column(s) for equality first, then range/ORDER BY columns.
• Leftmost prefix rule: a query must use the leftmost columns in the index to benefit from it.
• Example: CREATE INDEX idx_users_org_status ON users (organization_id, status, created_at);
  • Helps WHERE org_id = ? AND status = ? ORDER BY created_at
  • Does NOT help WHERE status = ? alone.

EXPLAIN Fundamentals

EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) SELECT * FROM orders WHERE user_id = 42;

Key plan node types:

Node	Meaning
Seq Scan	Full table scan — expensive on large tables
Index Scan	Index lookup + heap fetch
Index Only Scan	All needed columns in the index itself (fastest)
Bitmap Heap Scan	Multiple index matches combined into a bitmap
Nested Loop	For each row in outer, scan inner (good for small joins)
Hash Join	Build hash table on one side, probe with other
Merge Join	Sort both sides, merge (good for large sorted sets)

What to Look For in a Query Plan

1. Sequential scans on large tables — missing index.
2. High rows vs actual rows discrepancy — planner has stale statistics; run ANALYZE.
3. Sort nodes with large memory — consider pre-sorted index or increased work_mem.
4. Nested Loop joining large row sets — might need a different join strategy.
5. Bitmap Heap Scan with many row versions — vacuum might be needed.
6. Filter after index scan — index is missing a column used in WHERE.

Index Maintenance

-- Check index usage
SELECT schemaname, tablename, indexname, idx_scan, idx_tup_read, idx_tup_fetch
FROM pg_stat_user_indexes
WHERE idx_scan = 0;  -- unused indexes (candidates for removal)

-- Rebuild bloated indexes
REINDEX INDEX CONCURRENTLY idx_name;   -- non-blocking in PG 12+

Common Anti-Patterns

• Indexing low-cardinality columns (e.g., boolean) alone — not selective enough.
• Over-indexing — each index adds write overhead (INSERT/UPDATE/DELETE slower).
• Missing composite indexes for common query patterns.
• SELECT * pulling columns not covered by the index, forcing heap lookups.
• Function calls on indexed columns (WHERE LOWER(email) = 'x') unless using a functional index.

---

3. N+1 Detection & Mitigation

What Is N+1?

The N+1 selects problem occurs when an application issues 1 query to fetch N parent rows, then issues N additional queries to fetch related data for each parent — N+1 total queries instead of a single efficient query.

Example (ORM-level Pseudocode)

# N+1: 1 query for users + N queries for orders
users = User.query.all()              # 1 query → 100 users
for user in users:
    orders = user.orders               # 100 queries!
    ...

-- Queries generated:
SELECT * FROM users;                                       -- 1
SELECT * FROM orders WHERE user_id = 1;                    -- 2
SELECT * FROM orders WHERE user_id = 2;                    -- ...
SELECT * FROM orders WHERE user_id = 100;                  -- 101

Detection Techniques

1. ORM query logging — enable SQL logging and watch for repeated similar queries.
2. APM tools — Scout, New Relic, Datadog highlight N+1 patterns automatically.
3. Manual EXPLAIN — detect many identical queries in a short window.
4. Static analysis — Rails' bullet gem, Django's nplusone, Java's jpa-nplusone.
5. Database-side analysis — pg_stat_statements showing high call counts.

Mitigation Strategies

Strategy	ORM	How
Eager loading (JOIN)	Django select_related / Rails includes / Hibernate JOIN FETCH	Single query with JOIN
Batch loading	Django prefetch_related / Rails preload / Hibernate @BatchSize	Separate query per table, batched with WHERE IN
GraphQL DataLoader	Any GraphQL stack	Per-request batching & deduplication
Lazy + batch	Common in ORMs	Delay execution until accessed, then batch

# Fix with eager loading (Django)
users = User.objects.select_related('profile').prefetch_related('orders').all()

# Fix with DataLoader (GraphQL)
from promise import Promise
from promise.dataloader import DataLoader

class OrderLoader(DataLoader):
    def batch_load_fn(self, user_ids):
        orders = Order.objects.filter(user_id__in=user_ids)
        return Promise.resolve([list(orders.filter(user_id=uid)) for uid in user_ids])

When N+1 Is Acceptable

• Small, fixed N (e.g., < 10 related items).
• Admin panels or reports where latency is not critical.
• Cached results with low cache-miss volume.

---

4. Pagination Strategies — Cursor vs Offset vs Keyset

Offset/Limit (Most Common, Least Scalable)

SELECT * FROM orders ORDER BY created_at DESC LIMIT 20 OFFSET 0;
SELECT * FROM orders ORDER BY created_at DESC LIMIT 20 OFFSET 20;

Pros:

• Simple to implement.
• Supports arbitrary page jumps (page 1, page 5, page 100).
• Intuitive for developers.

Cons:

• Performance degrades with page depth — OFFSET 100000 must scan/skip 100K rows.
• Phantom reads / missing rows — if rows are inserted/deleted between requests, items may appear on multiple pages or be skipped entirely.
• Inconsistent under write load — OFFSET changes meaning as data shifts.

Cursor-Based Pagination (Most Scalable, API-First)

-- First page: no cursor
SELECT * FROM orders ORDER BY created_at DESC LIMIT 20;

-- Next page: use the last item's cursor value
SELECT * FROM orders
WHERE created_at < '2026-06-04T12:00:00Z'  -- cursor value
ORDER BY created_at DESC LIMIT 20;

// API response shape
{
  "data": [...],
  "pagination": {
    "next_cursor": "eyJpZCI6MTIzNDUsImNyZWF0ZWRfYXQiOiIyMDI2LTA2LTA0VDEyOjAwOjAwWiJ9",
    "has_more": true
  }
}

Pros:

• O(1) performance at any depth — uses index seek, not scan+skip.
• Consistent — no phantom reads or missed rows; cursor marks a fixed position.
• Resilient to write load — insertion/deletion doesn't shift cursor position.

Cons:

• No arbitrary page jumping (only next/prev).
• Requires a unique, sortable column (usually an ID or timestamp).
• Cursor encoding/decoding overhead (base64, opaque tokens).

Keyset Pagination (Seek Method)

-- Composite pagination on (created_at, id)
SELECT * FROM orders
WHERE (created_at, id) < ('2026-06-04T12:00:00Z', 12345)
ORDER BY created_at DESC, id DESC
LIMIT 20;

• Uses a composite index on (created_at, id).
• Similar performance to cursor-based — index seek on the tuple.
• Requires a compound comparison and a multi-column index.

Comparison Table

Aspect	Offset/Limit	Cursor	Keyset
Performance at depth	O(n) — degrades	O(1) — constant	O(1) — constant
Random page access	Yes	No	No
Phantom reads	Yes	No	No
Consistency	Unstable	Stable	Stable
Implementation complexity	Trivial	Medium	Low-Medium
Requires sortable unique column	No	Yes	Yes
Write-aware	No	Yes	Yes

Recommendation

Use Case	Strategy
Admin panels, small datasets	Offset/Limit (fine for < 10K rows)
Public APIs, infinite scroll	Cursor (REST/GraphQL best practice)
Time-series, logs, audit trails	Cursor or Keyset on timestamp + ID
Internal tools with DB pagination	Keyset (lowest complexity, no cursor encoding)

---

5. Transaction Boundary Design

ACID Properties

Property	Meaning
Atomicity	All or nothing — transaction either completes fully or has no effect
Consistency	Transaction leaves DB in a valid state (constraints preserved)
Isolation	Concurrent transactions do not interfere with each other
Durability	Committed changes persist through failures

Isolation Levels

Level	Dirty Read	Non-Repeatable Read	Phantom Read	Serialization Anomaly
Read Uncommitted	Possible	Possible	Possible	Possible
Read Committed (default in PostgreSQL, SQL Server, Oracle)	Safe	Possible	Possible	Possible
Repeatable Read	Safe	Safe	Possible (PG: safe)	Possible
Serializable	Safe	Safe	Safe	Safe

PostgreSQL specifics:

• Default is Read Committed.
• Repeatable Read also prevents phantom reads (uses snapshot isolation).
• Serializable uses Serializable Snapshot Isolation (SSI) — detects serialization conflicts and aborts.

Choosing an Isolation Level

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
-- or for the session:
SET default_transaction_isolation = 'repeatable read';

Level	When to Use
Read Committed	Default for most workloads. Good balance of consistency and performance.
Repeatable Read	Financial calculations, reporting — when you need consistent snapshots.
Serializable	Critical data integrity (ledgers, inventory allocation). Higher abort rate.

Transaction Retry Patterns

Optimistic retry (for Serializable / Repeatable Read conflicts):

RETRY_COUNT = 0
MAX_RETRIES = 3
BACKOFF = [50ms, 150ms, 500ms]

WHILE RETRY_COUNT <= MAX_RETRIES:
    BEGIN TRANSACTION
    TRY:
        -- business logic
        COMMIT
        BREAK
    CATCH serialization_failure:
        ROLLBACK
        SLEEP(BACKOFF[RETRY_COUNT])
        RETRY_COUNT += 1
    CATCH deadlock:
        ROLLBACK
        SLEEP(random 0-100ms)
        RETRY_COUNT += 1

IF RETRY_COUNT > MAX_RETRIES:
    RAISE "Transaction failed after retries"

Best practices:

• Use exponential backoff with jitter to avoid thundering herd.
• Keep transactions short — minimize lock duration.
• Read before write — detect conflicts early inside the transaction.
• Use optimistic locking (version column) for entity-level concurrency instead of pessimistic locks when possible.

Distributed Transactions

Pattern	Description	When to Use
Two-Phase Commit (2PC)	Coordinator prepares all participants, then commits	Within a single database system only
Saga (Choreography)	Each service publishes events; compensating actions roll back	Microservices, async boundaries
Saga (Orchestration)	Central orchestrator sends commands and handles compensation	Complex multi-service workflows
Outbox Pattern	Write events to an outbox table in the same DB transaction, then async publish	Event-driven architecture with exactly-once guarantees
Idempotency Keys	Unique key per operation prevents duplicate processing	Payment handling, any external API call

Transaction Anti-Patterns

• Long-running transactions that hold locks — split into smaller units.
• Nested transactions across service boundaries — use Sagas instead.
• Transaction inside a loop — batch the work into a single transaction.
• Mixing heavy I/O inside a transaction — external API calls should happen before or after.
• Not handling retries for serialization failures — every Serializable workload needs retry logic.

---

6. Read/Write Splitting

Architecture

                    ┌─────────────────┐
                    │   Application    │
                    │  (ORM / Client)  │
                    └────┬────────┬───┘
                         │        │
                    Writes     Reads
                         │        │
                    ┌────▼──┐ ┌──▼────┐
                    │Primary│ │Replica│ ──→ (more replicas)
                    │(Write)│ │(Read) │
                    └───────┘ └───────┘
                        │          ↑
                        │  Async   │
                        │  Repl.   │
                        └──────────┘

Implementation Approaches

Approach	Mechanism	Pros	Cons
ORM-level (read_from=replica)	Config in ORM (Django DATABASES, Rails config)	Simple; no infra change	Every service must configure manually
Database Proxy (ProxySQL, PgBouncer, PgCat)	Route based on query type	Centralized; no app changes	Extra hop; proxy becomes SPOF
Middleware (e.g., Spring @Transactional(readOnly=true))	Annotation-driven routing	Fine-grained control; declarative	Framework-specific
Client-side (multi-DB driver config)	Connection string per role	Minimal infra	Deploy-time configuration

Query Routing Rules

Writes → Primary:
  - INSERT, UPDATE, DELETE, MERGE
  - DDL (CREATE TABLE, ALTER)
  - SELECT ... FOR UPDATE (needs primary)
  - SELECT inside a read-write transaction

Reads → Replica:
  - SELECT (no locking)
  - Read-only transactions (@Transactional(readOnly=true))
  - Reporting queries, analytics

When NOT to Read from Replicas

• Read-after-write queries — data may not have replicated yet.
• Strong consistency requirements (ledgers, inventory).
• Tightly coupled workflows where the next read depends on the previous write.

Spring Boot Example (ReadWriteSplit Routing)

@Transactional(readOnly = true)
public OrderDTO getOrder(Long id) { ... }  // routed to replica

@Transactional
public OrderDTO createOrder(OrderDTO dto) { ... }  // routed to primary

Configure AbstractRoutingDataSource with a @ReadOnlyRepository annotation or AOP advice to switch between primary and replica DataSource.

---

7. Replication Lag Handling

The Problem

Asynchronously replicated databases always have some lag between write on the primary and visibility on replicas. This causes:

• Read-after-write inconsistency — user creates a resource, then immediately gets a 404 reading from a stale replica.
• Monotonic read violation — user sees a newer version of data, then an older version (from a different replica).
• Causality violations — entity A's state depends on entity B, but B's update hasn't arrived yet.

Handling Strategies

Strategy	Description	Complexity
Read-your-writes (RYW)	Route reads for recently-written data to the primary	Low
Monotonic reads	Route a session's reads to the same replica	Low
Bounded staleness	Reject reads from replicas lagging beyond a threshold	Medium
Causal consistency (GTID)	Track which transaction IDs the client has seen; ensure replica applies those before serving reads	Medium
Wait-for-replication	After write, wait for replica to catch up before serving reads	Medium
Synchronous replication	Primary waits for N replicas before committing	High (latency cost)

Read-Your-Writes (RYW) Pattern

class DatabaseRouter:
    def __init__(self):
        self.recent_writes = {}  # user_id → timestamp

    def execute_write(self, user_id, query, params):
        result = primary.execute(query, params)
        self.recent_writes[user_id] = time.now()
        return result

    def execute_read(self, user_id, query, params):
        last_write = self.recent_writes.get(user_id, 0)
        if time.now() - last_write < 5:  # 5-second window
            return primary.execute(query, params)  # use primary
        else:
            return replica.execute(query, params)  # use replica

Monotonic Read Consistency (Shopify Pattern)

Route all related reads to the same replica using a hash-based sticky selection:

/* consistent_read_id:user_42 */ SELECT * FROM orders WHERE user_id = 42;

Hash("user_42") % NUM_REPLICAS = replica_index → always hits the same server

Trade-off: Simple and low-overhead; occasional inconsistency if that replica goes down.

Wait-for-Replication

def write_and_wait(data):
    primary.execute("INSERT INTO ...", data)
    # Wait for the write to arrive on at least one replica
    primary.execute("SELECT pg_current_wal_lsn()")  # Postgres
    # or use pg_stat_replication

def read_with_consistency(key):
    # Check that replica has caught up to a known LSN
    replica_lsn = replica.execute("SELECT pg_last_wal_replay_lsn()")
    if replica_lsn >= required_lsn:
        return replica.read(key)
    else:
        return primary.read(key)  # fallback to primary

Strategies by Use Case

Use Case	Recommended Strategy
User-facing web app after form submit	Read-your-writes (route to primary for 5-30s)
Social feed, timeline	Monotonic reads (sessions stick to one replica)
Analytics, reporting	Bounded staleness acceptable; lag of minutes is fine
Inventory, financial ledger	Always read from primary (strong consistency)
Notifications	Accept eventual consistency; timestamp-driven dedup

---

8. Service-Level Testing Overview

                     Coverage ▲
                              │
                    ┌─────────┤
                    │  E2E    │   Few, slow, expensive
                ┌───┤  Tests  │
                │   └─────────┤
            ┌───┤            │
            │   │   Service  │   Medium count, medium speed
        ┌───┤   │  (Integ.) │
        │   │   └───────────┤
    ┌───┤   │              │
    │   │   │    Unit      │   Many, fast, cheap
    │   │   │    Tests     │
    └───┴───┴──────────────┘

The Test Pyramid recommends:

• Unit tests: ~70% — fast, deterministic, test business logic in isolation.
• Integration tests: ~20% — test boundaries (DB, external APIs).
• Contract tests: ~5% — verify API agreements between services.
• E2E tests: ~5% — happy-path critical flows.

---

9. Unit Testing Business Logic

Principles

• Test in isolation — mock/stub all collaborators (DB, file system, network).
• Focus on logic — test business rules, transformations, validations, and state changes.
• Deterministic — no flaky tests. No external dependencies.
• Fast — individual tests complete in milliseconds.

What to Unit Test

# GOOD: Pure business logic — test this
class OrderService:
    def calculate_discount(self, order_total, customer_tier):
        if customer_tier == 'vip':
            return order_total * 0.20
        elif order_total > 1000:
            return order_total * 0.10
        else:
            return 0

# BAD: Impure — involves I/O, mock the boundary instead
class OrderController:
    def create_order(self, request):
        order = Order(...)
        db.save(order)          # this is an integration concern
        notification.send(order) # mock this in unit tests
        return order

Repository/Data Layer Abstraction

Use the Repository Pattern to make business logic testable:

// Business logic — unit testable with mock repository
public class OrderFulfillmentService {
    private final OrderRepository orderRepo;
    private final InventoryClient inventoryClient;

    public FulfillmentResult fulfillOrder(String orderId) {
        Order order = orderRepo.findById(orderId);
        if (order == null) return FulfillmentResult.notFound();

        boolean inStock = inventoryClient.checkStock(order.getSku(), order.getQuantity());
        if (!inStock) return FulfillmentResult.outOfStock();

        order.setStatus(OrderStatus.FULFILLED);
        orderRepo.save(order);
        return FulfillmentResult.success();
    }
}
// Unit test: Mock orderRepo and inventoryClient, test all branches

Testing Patterns

Pattern	Description
Given-When-Then	Arrange → Act → Assert structure
Parameterized tests	Test many input combinations with one test method
Property-based testing	Generate random inputs, assert invariants hold
State-based vs Interaction-based	Prefer state assertions over verifying mock interactions

---

10. Integration Testing — API Contracts, Testcontainers, WireMock

Testcontainers

What: Library that provides lightweight, disposable containers for testing (PostgreSQL, Redis, Kafka, etc.) as JUnit @Rule / @Container.

Why real containers instead of in-memory:

Approach	Issues
H2 (in-memory)	Different SQL dialect, missing features, different behavior under load
SQLite	No JSONB, no PostGIS, no full-text search, different type coercion
Testcontainers	Real PostgreSQL/MySQL — 100% behavior match

Example (Java / Spring Boot + Testcontainers):

@SpringBootTest
@Testcontainers
class UserRepositoryIntegrationTest {

    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16")
        .withDatabaseName("testdb")
        .withUsername("test")
        .withPassword("test");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    private UserRepository userRepository;

    @Test
    void shouldPersistAndRetrieveUser() {
        User user = new User("alice@example.com", "Alice");
        User saved = userRepository.save(user);

        Optional<User> found = userRepository.findByEmail("alice@example.com");
        assertThat(found).isPresent();
        assertThat(found.get().getName()).isEqualTo("Alice");
    }
}

Testcontainers in other languages:

Language	Library
Python	testcontainers (pip)
Node.js	testcontainers (npm)
Go	testcontainers-go
.NET	Testcontainers for .NET
Rust	testcontainers (crate)

WireMock

What: HTTP-based API mock server. Stub external HTTP services during integration tests.

@SpringBootTest
@WireMockTest(httpPort = 8089)
class PaymentServiceIntegrationTest {

    @Test
    void shouldProcessPaymentWhenGatewayRespondsSuccess() {
        // Arrange: stub the external payment gateway
        stubFor(post(urlEqualTo("/gateway/charge"))
            .willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/json")
                .withBody("""
                    { "status": "success", "transaction_id": "txn_123" }
                """)));

        // Act
        PaymentResult result = paymentService.charge(new Payment("user_1", 50.00));

        // Assert
        assertThat(result.isSuccess()).isTrue();
        assertThat(result.getTransactionId()).isEqualTo("txn_123");
    }

    @Test
    void shouldHandleGatewayTimeoutGracefully() {
        stubFor(post(urlEqualTo("/gateway/charge"))
            .willReturn(aResponse()
                .withStatus(504)));

        assertThrows(PaymentGatewayTimeoutException.class, () -> {
            paymentService.charge(new Payment("user_1", 50.00));
        });
    }
}

WireMock capabilities:

• Stub based on URL, HTTP method, headers, body.
• Simulate delays, timeouts, and network failures.
• Record/playback (proxying real APIs during development).
• Verify requests were made (assert on expected interactions).
• Fault injection (malformed responses, connection resets).

Integration Test Best Practices

1. Test the boundary — Repository tests with Testcontainers, external API tests with WireMock.
2. Keep tests independent — each test gets its own transaction or container state.
3. Clean up between tests — truncate tables or use transactional rollback.
4. Use realistic data — edge cases that trigger unique constraints, nulls, long strings.
5. Don't test the framework — you don't need to test that Hibernate/JPA/ActiveRecord works.
6. Name tests by behavior — shouldRejectOrderWhenInventoryExhausted(), never testOrder1().

---

11. Contract Testing — Pact

What Is Contract Testing?

Contract testing verifies that two services (consumer and provider) can communicate correctly by testing each side independently against a shared contract — without deploying both services.

Pact Workflow

1. Consumer writes expectations (Pact file)
   ┌──────────┐                 ┌──────────┐
   │ Consumer │ ── generates ──→│ Pact     │
   │  Tests   │                 │ File     │
   └──────────┘                 └────┬─────┘
                                     │
2. Provider verifies against Pact   │
   ┌──────────┐                      │
   │ Provider │ ←── verifies ────────│
   │  Tests   │                      │
   └──────────┘                      │
                                     │
3. Pact Broker stores & diff         │
   ┌──────────────┐                  │
   │ Pact Broker  │ ←── stores ──────│
   │ (versioned)  │                  │
   └──────┬───────┘                  │
          │                          │
4. Can-I-Deploy checks versions      │
   ┌──────────┐                      │
   │ CI/CD    │ ←── compatibility ───│
   └──────────┘                      │

Consumer-Side Test (Pact)

@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "PaymentProvider", port = "8080")
class OrderServiceConsumerPactTest {

    @Pact(consumer = "OrderService")
    public V4Pact createPact(PactDslWithProvider builder) {
        return builder
            .given("a payment method exists with ID 'pm_1'")
            .uponReceiving("a request to charge a payment")
                .path("/gateway/charge")
                .method("POST")
                .headers("Content-Type", "application/json")
                .body(new PactDslJsonBody()
                    .stringType("payment_method_id", "pm_1")
                    .decimalType("amount", 49.99)
                )
            .willRespondWith()
                .status(200)
                .headers("Content-Type", "application/json")
                .body(new PactDslJsonBody()
                    .stringType("status", "success")
                    .stringType("transaction_id", "txn_abc123")
                )
            .toPact();
    }

    @Test
    @PactTestFor(pactMethod = "createPact")
    void shouldChargePaymentSuccessfully(MockServer mockServer) {
        PaymentClient client = new PaymentClient(mockServer.getUrl());
        PaymentResponse response = client.charge("pm_1", 49.99);
        assertThat(response.getStatus()).isEqualTo("success");
    }
}

Provider-Side Verification

@Provider("PaymentProvider")
@PactBroker(url = "${pactbroker.url}")
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class PaymentProviderPactVerificationTest {

    @LocalServerPort
    int port;

    @BeforeEach
    void setup(PactVerificationContext context) {
        context.setTarget(new HttpTestTarget("localhost", port));
    }

    @TestTemplate
    @ExtendWith(PactVerificationInvocationContextProvider.class)
    void pactVerificationTestTemplate(PactVerificationContext context) {
        context.verifyInteraction();
    }

    @State("a payment method exists with ID 'pm_1'")
    void setupPaymentMethod() {
        // Set up test data — this runs before the provider is called
        paymentMethodRepository.save(new PaymentMethod("pm_1", ...));
    }
}

Pact Best Practices

• Version both consumer and provider — Pact Broker tracks compatibility matrix.
• Use can-i-deploy — the pact-broker can-i-deploy command checks if two versions are compatible before deploying.
• Don't over-specify — use matchers (stringType, decimalType) instead of exact values for most fields. Exact values should only be for fields where the value matters (e.g., status enums).
• Tag pacts by environment — tag pact versions with "prod", "staging" to gate deployments.
• Run provider verification in CI — not just locally. Break the build if a provider change breaks a consumer contract.

---

12. Test Fixtures

What Are Test Fixtures?

Test fixtures are predefined data setups that provide a known baseline state before tests run. They reduce duplication and make tests readable.

Fixture Strategies

Strategy	Description	Best For
Inline (test-local)	Create data directly in the test method	Simple, focused tests
Factory methods	Helper functions that create objects with sensible defaults	Most cases — flexible, composable
Factory Boy / build()	Use a library to generate test objects	Complex object graphs
Seed SQL files	Pre-populated SQL inserts loaded before test suite	Integration + E2E tests
JSON/YAML snapshots	Load test data from fixture files	When data is complex and nested

Example: Factory Pattern (Python)

# factories.py
class UserFactory:
    @staticmethod
    def create(
        email="test@example.com",
        name="Test User",
        tier="standard",
        balance=Decimal("100.00")
    ):
        return User(
            email=email,
            name=name,
            tier=tier,
            balance=balance
        )

# test_discount.py
def test_vip_discount():
    vip = UserFactory.create(tier="vip", balance=Decimal("500.00"))
    result = discount_service.calculate(vip, 200)
    assert result == Decimal("40.00")  # 20% VIP discount

Factory Boy (Python) / Builders (Java)

import factory

class OrderFactory(factory.Factory):
    class Meta:
        model = Order

    id = factory.Sequence(lambda n: n)
    user = factory.SubFactory(UserFactory)
    total = Decimal("100.00")
    status = OrderStatus.PENDING
    created_at = factory.Faker("date_time_this_year")

# Usage — only override what matters
order = OrderFactory.create(status=OrderStatus.FULFILLED)
assert order.user.email == "test@example.com"  # default from UserFactory

Fixture Anti-Patterns

• Shared mutable fixtures — tests that mutate shared state cause flaky ordering dependencies.
• Too much data — loading 1000 rows for every test is slow; use the minimum needed.
• Copy-paste fixtures — leads to drift; use factories with default values.
• Magic numbers — use named constants: UNIT_PRICE = Decimal("10.00") instead of bare 10.00.

---

13. CI Integration

Test Execution in CI

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌──────────────┐
│   Lint &    │     │   Unit      │     │ Integration │     │   Contract   │
│   Static    │ ──→ │   Tests     │ ──→ │   Tests     │ ──→ │   Tests /    │
│   Analysis  │     │ (fast, par) │     │ (slower)    │     │   E2E Tests  │
└─────────────┘     └─────────────┘     └─────────────┘     └──────────────┘
     < 2 min           < 5 min            < 15 min            < 30 min

Parallelization

• Unit tests — run in parallel across CPU cores (pytest-xdist, JUnit parallel).
• Integration tests — parallel by service/module; isolate with Testcontainers per test class.
• Contract tests — consumer tests in parallel; provider tests sequentially per pact file.

CI Pipeline Example (GitHub Actions)

name: CI
on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with: { java-version: '21', distribution: 'temurin' }
      - run: ./gradlew test --parallel     # unit tests only

  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - run: ./gradlew integrationTest --tests *IntegrationTest
    # Alternatively, use Testcontainers which starts containers in-test

  contract-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./gradlew pactVerify          # provider-side verification
      - run: ./gradlew pactPublish         # publish to Pact Broker

  pact-can-i-deploy:
    needs: contract-tests
    runs-on: ubuntu-latest
    steps:
      - run: pact-broker can-i-deploy
          --pacticipant OrderService
          --version $(cat version.txt)
          --to-environment production

  e2e-tests:
    needs: [integration-tests, pact-can-i-deploy]
    runs-on: ubuntu-latest
    steps:
      - run: docker compose -f docker-compose.e2e.yml up --abort-on-container-exit

CI Best Practices

Practice	Rationale
Fail fast	Run fastest tests first (unit → integration → E2E).
Cache dependencies	Maven/Gradle/npm/pip caches speed up repeat builds.
Cache Docker layers	Testcontainers pulls — pre-warm image caches.
Isolate flaky tests	Quarantine flaky tests; don't let them block the pipeline.
Test against production-like databases	Use Testcontainers with the same DB version as production.
Run pact verification as a required check	Never deploy a provider that breaks a consumer contract.
Use test reports as artifacts	Publish JUnit XML / HTML reports for debugging.

Test Run Optimization

• Selective test execution — only run tests for changed modules (gradle --changed-latest, pytest --last-failed).
• Test splitting — split integration tests across multiple CI runners (--shard flags).
• Docker layer reuse — Dockerfile changes cause full rebuilds; keep rarely-changed layers early.
• Database migrations in CI — run migrations once, snapshot the DB, restore for each test runner.

---

References & Further Reading

• PostgreSQL Documentation — EXPLAIN, Transaction Isolation
• HikariCP — GitHub (connection pool sizing)
• PgBouncer — Official docs (transaction pooling)
• Pact — Documentation (contract testing)
• Testcontainers — Official site (integration testing)
• WireMock — Official site (API mocking)
• Shopify Engineering — Read Consistency with Database Replicas
• Crunchy Data — Postgres Indexes for Newbies
• Scout APM — Understanding N+1 Database Queries
• AWS — RDS Proxy (serverless connection pooling)