magnus919_agent-skills/qa-methodology/references/sdet-engineering.md

SDET Engineering

Definition and Distinction

An SDET (Software Development Engineer in Test) is an engineer whose product is test infrastructure — frameworks, tooling, pipelines, and platforms that enable the entire organization to verify software quality efficiently.

What an SDET is NOT:

• Not "a tester who codes": A manual tester who learned Selenium is not an SDET. Writing scripts that automate existing manual steps produces brittle, low-value automation.
• Not "a developer who tests": A developer who writes unit tests for their own code is practicing good development hygiene, not building test infrastructure for others.

The SDET's customers are other engineers. Success is measured by how effectively the organization can detect defects, not by how many tests the SDET personally writes.

Competency Model: 7 Habits (Angie Jones)

#	Habit	Core Practice
1	Be intentional	Automate selectively aligned to goals, not "all the things"
2	Enhance development skills	OOP, design patterns, clean code — not just API syntax
3	Enhance testing skills	Balance developer and tester mindsets; verify behavior, not just execution
4	Explore new tools	Match tools to contexts; never force one tool on every problem
5	Automate throughout the tech stack	Use seams at unit/service/API layers; UI automation sparingly
6	Collaborate	Strategy requires input from exploratory testers, developers, and product
7	Automate beyond the tests	Data generation, environment setup, log parsing — remove repetitive toil

Source: Angie Jones, "7 Habits of Highly Effective SDETs" (angiejones.tech, 2018).

gTAA / TAF Layered Architecture

The generic Test Automation Architecture (gTAA) organizes test infrastructure as layers with clear responsibilities:

┌─────────────────────────────────────────┐
│  Layer 5: Test Reporting & Analytics    │  Dashboards, trend analysis, flake metrics
├─────────────────────────────────────────┤
│  Layer 4: Test Execution & Orchestration│  CI runners, parallelism, sharding, retry policy
├─────────────────────────────────────────┤
│  Layer 3: Test Scripts / Scenarios      │  Business-readable test cases, data-driven flows
├─────────────────────────────────────────┤
│  Layer 2: Test Services / Utilities     │  API clients, page objects, data factories, auth helpers
├─────────────────────────────────────────┤
│  Layer 1: Core Framework / Adapters     │  Driver management, config, logging, plugin system
└─────────────────────────────────────────┘

Key principle: Higher layers depend on lower layers, never the reverse. Test scripts (L3) never import driver internals (L1) directly.

Design Patterns for Test Code

Page Object Model (POM)

Encapsulates UI structure behind a semantic interface. Tests interact with page meaning, not selectors.

class CheckoutPage:
    def __init__(self, page):
        self._page = page
        self._submit = page.locator('[data-testid="checkout-submit"]')

    def complete_order(self, card: str) -> OrderConfirmation:
        self._page.fill('[name="card"]', card)
        self._submit.click()
        return OrderConfirmation(self._page)

Flow Model

Models multi-page user journeys as composable transitions. Each step returns the next page state, enabling type-safe navigation chains.

confirmation = (
    HomePage(page)
    .search("widget")
    .select_result(0)
    .add_to_cart()
    .checkout(card="4111...")
)
assert confirmation.is_successful()

Flow Model complements POM: POM encapsulates single pages; Flow Model composes them into end-to-end journeys.

SOLID Applied to Test Code

Principle	Test Code Application
Single Responsibility	One test class per feature area; one assertion concept per test
Open/Closed	Extend test data via fixtures, not by modifying shared helpers
Liskov Substitution	Any test-double must be swappable for the real dependency without test changes
Interface Segregation	Page objects expose only methods relevant to their page (no god-objects)
Dependency Inversion	Tests depend on abstractions (interfaces), not concrete driver implementations

Build vs. Buy Decision Framework

Decision Criteria

Criterion	Favors BUILD	Favors BUY/ADOPT
Team size	> 5 SDETs maintaining infra	< 3 engineers
Longevity	Custom needs persist 2+ years	Needs may shift within 12 months
Integration surface	Deep internal system hooks (custom protocols)	Standard web/API/mobile
Maintenance cost tolerance	Org can fund ongoing maintenance	Prefer vendor/community maintenance
TCO (3-year)	Custom amortizes below commercial	Commercial license < build + maintain

Decision Table

Scenario	Recommendation
Standard web E2E, < 5 engineers	Adopt Playwright/Cypress
Custom protocol (IoT, proprietary binary)	Build adapter layer atop open framework
High-scale parallelism + custom reporting at > 20 teams	Build orchestration; adopt execution engines
Mobile-only, small team	Adopt Appium/Detox
Uncertain requirements, < 12 months horizon	Adopt; revisit when needs stabilize

Gotcha — NIH syndrome: Building a custom framework "because none fit perfectly" when an existing tool covers 90% of needs wastes months. Extend, don't replace.

Testability Engineering

Designing Systems for Testability

Technique	Mechanism	Example
Dependency Injection	Inject collaborators via constructor/parameter	OrderService(repo: Repository, clock: Clock)
Architectural seams	Boundaries where behavior can be altered without editing	Interface between service and external gateway
Observability hooks	Expose internal state for verification	Health endpoints, debug headers, structured logs

Test-Double Selection Criteria

Double Type	Use When	Avoid When
Stub	You need canned return values; no interaction verification	You need to verify call sequences
Mock	Verifying interactions (was X called N times?)	Over-mocking creates brittle coupling to implementation
Fake	You need working behavior without external cost (in-memory DB, local SMTP)	Behavior diverges from production over time

Selection rule: Default to stubs for data, fakes for stateful collaborators, mocks only for interaction-critical boundaries. Never mock value objects.

CI/CD Integration

Multi-Level Pipeline Architecture

Level	Trigger	Contents	Time Budget
L1: Pre-merge	Every PR	Unit + fast integration + lint + type-check	< 5 min
L2: Post-merge	Merge to main	Full integration + contract tests + E2E smoke	< 15 min
L3: Scheduled	Nightly / weekly	Full E2E + performance + soak + mutation	< 60 min

Configuration Management

Test configuration (URLs, credentials, feature flags) lives in environment-specific config, never hardcoded. Use layered config: defaults → environment overrides → CI secrets.

Contract Testing

Consumer-driven contracts (e.g., Pact) verify service boundaries independently of full integration:

• Consumer defines expectations → publishes contract
• Provider verifies against contract in its own CI
• Breaks are caught before deployment, not during integration testing

Test Data and Environment Self-Service

Capability	Implementation Pattern
Data on demand	Factory/builder functions generating valid entities per test
Environment provisioning	Ephemeral environments via containers (Docker Compose, k8s namespaces)
State isolation	Each test owns its data; no shared mutable database state
Self-service portal	Engineers spin up test environments without SRE ticket

Observability and Shift-Right

Technique	QA Application
Correlation IDs	Trace a user journey across microservices; reproduce failures from production traces
Canary releases	Deploy to 1–5% traffic; monitor error rates before full rollout; auto-rollback on SLO breach
Feature flags	Gate risky features; enable targeted regression testing in production; kill-switch without deploy

Shift-right does not replace shift-left. It validates that pre-merge testing caught what matters, and feeds escaped-defect data back into suite evolution.

Flakiness and Reliability Engineering

Core principle: Test code is production code. It deserves the same review, ownership, and SLA expectations as application code.

Flakiness Triage (Flakinator-Style)

Step	Action
1. Detect	Statistical flake scoring: Bayesian analysis of pass/fail patterns over N runs
2. Classify	Root cause category: timing/race, resource contention, test-order dependency, external service
3. Quarantine	Remove from blocking path; track in dashboard with owner and SLA
4. Fix or delete	Owner resolves within 5 business days or deletes the test
5. Burn-in	20+ consecutive green runs before re-enabling as blocking

The ~18-Month Decay Rule

Test suites without active maintenance decay: flake rates climb, false confidence accumulates, and developer trust erodes. Budget ~20% of test infrastructure capacity for ongoing maintenance. If an organization cannot sustain this, adopt fewer, higher-value tests rather than a large neglected suite.

Source: Google Testing Blog, "Flaky Tests at Google and How We Mitigate Them" (2016); Atlassian Engineering, "Taming Test Flakiness with Flakinator" (2025).

Career Progression

Stage	Focus	Scope
Junior SDET	Learn framework, write tests under guidance	Task
Senior SDET	Own a product area's test infrastructure	Project
Staff SDET	Set test architecture standards across teams	Product
Principal SDET	Multi-year QE vision; industry contribution	Org

For detailed leveling mechanics, promotion packets, and archetypes, see qa-career-levels.md.

Emerging AI Dimensions

Dimension	Current State (2025–2026)	QA Implication
Self-healing tests	Tools auto-update selectors on UI changes (Healenium, Applitools)	Reduces maintenance burden but masks real UI regressions if unchecked
AI log analysis	LLM-assisted root-cause analysis of CI failures	Accelerates triage; requires validation against deterministic signals
Agentic testing pyramids	AI agents generate and execute test scenarios autonomously	QA role shifts to strategy, oracle design, and verifying agent-generated test quality

Gotcha — AI-generated tests without oracle verification: An agent that generates 500 tests asserting nothing is worse than 50 well-designed tests. Always verify that AI-generated tests have meaningful assertions and kill mutants.

Decision Table: SDET Scope Choices

Question	If YES	If NO
Is the team > 5 engineers maintaining test infra?	Invest in layered gTAA	Adopt existing framework directly
Does the system have custom protocols?	Build adapter layer	Use standard tool
Are flake rates > 5%?	Prioritize reliability engineering over new tests	Continue balanced investment
Is AI-generated code > 50% of PRs?	Add mutation testing + independent verification gates	Standard review sufficient

Exit condition: You are done applying this reference when you can identify the appropriate gTAA layers for your system, make a build-vs-buy recommendation with documented criteria, and establish a flake-management SLA for your team's test suite.

Worked Example: Build vs. Buy for an API-First Startup

Context: 8-person startup, 3 backend services, REST + gRPC, no dedicated QA. Team needs E2E confidence.

Criterion	Assessment
Team size	3 engineers touching tests → favors BUY
Integration surface	Standard REST + gRPC → no custom protocol
Longevity	Product-market fit uncertain; needs may pivot in 12 months
TCO	Playwright + pytest adoption: 2 weeks. Custom framework: 3 months + ongoing

Recommendation: Adopt Playwright (API testing) + pytest (unit/integration). Add contract testing (Pact) at service boundaries when team reaches 12+ engineers. Revisit build-vs-buy at 2-year mark if custom orchestration needs emerge.

Composition Links

• Career levels, promotion packets, archetypes: qa-career-levels.md
• Test automation patterns (parallelism, ML selection): test-automation.md
• Flaky quarantine workflow details: test-automation.md
• Quality metrics and gate design: quality-gates-and-metrics.md

---

Sources: Angie Jones, "7 Habits of Highly Effective SDETs" (angiejones.tech, 2018); Google Testing Blog, "Flaky Tests at Google" (testing.googleblog.com, 2016); Atlassian Engineering, "Taming Test Flakiness with Flakinator" (2025); Lisa Crispin & Janet Gregory, Agile Testing (2009); Gerard Meszaros, xUnit Test Patterns (2007); Pact Foundation (docs.pact.io); Will Larson, Staff Engineer (2020).