magnus919_agent-skills/data-architect/references/discovery-framework.md

# Architecture Discovery Framework

A structured approach to understanding a client's context before making recommendations. Use this when starting an architecture consulting session.

## Phase 1: Problem & Context

Establish what we're actually solving:

**Business:**
- What business problem are you solving? What's the measurable outcome?
- Who is asking for this? What's their pain point?
- What happens if we do nothing?
- What does success look like in 3 months, 12 months?

**Organizational:**
- Who are the stakeholders? Who has decision authority?
- What's the team structure — who builds, who operates, who consumes?
- What's the current team capability (size, seniority, key skills)?
- Is there executive sponsorship for cross-team changes?

**Temporal:**
- What's the timeline? What's driving the urgency?
- What's frozen (can't change) vs flexible?
- Are there existing commitments (vendor contracts, hiring freezes, compliance deadlines)?

## Phase 2: Current State

Understand what exists today:

**Data Profile:**
- What data sources exist? (Count, types, systems of record)
- Current data volume, velocity, variety — and projected 18-month growth
- Where is data quality acceptable vs problematic?
- What's the schema evolution frequency of source systems?

**Architecture:**
- Diagram the current data flow — source to consumption
- What's the current stack? (warehouse, pipeline tooling, BI layer, catalog)
- What's working well (don't fix what isn't broken)?
- What's the biggest operational pain point?

**Consumption:**
- Who consumes data and how? (analysts, data scientists, operational apps, ML models)
- What are their actual latency, freshness, and accuracy requirements?
- What do they complain about most?

**Governance:**
- What data governance exists currently? (ownership, quality, lineage, access control)
- What compliance/regulatory frameworks apply? (GDPR, HIPAA, SOX, CCPA, BCBS 239)
- Are there existing data sharing agreements or contractual constraints?

**Cost:**
- Current data infrastructure spend (compute, storage, tools, people)
- Where is cost growing fastest?
- Are there wasteful patterns (orphaned tables, unused pipelines, over-provisioned compute)?

## Phase 3: Constraints

Identify the hard boundaries:

**Non-negotiable:**
- Regulatory requirements
- Budget ceilings
- Hiring timeline
- Existing vendor contracts

**Tradeable:**
- Time-to-market vs robustness
- Feature scope vs timeline
- Central control vs team autonomy
- Managed service premium vs operational flexibility

**Unknowns to validate:**
- What assumptions are we making that should be tested?
- What POCs should we run before committing to an architecture?

## Phase 4: Recommendation Structure

Present findings as:

**Quick wins (this month)**
High impact, low effort — do these immediately regardless of long-term direction.
*(e.g., add data quality checks at ingestion, document current lineage, consolidate redundant pipelines)*

**Foundation (next quarter)**
Medium-term investments that enable future phases.
*(e.g., implement data catalog, establish governance board, standardize naming conventions)*

**Transformation (next 12 months)**
Major architectural shifts that require sequence and dependency management.
*(e.g., migrate from on-prem EDW to cloud, adopt Data Mesh, implement streaming platform)*

**Deferred / Not Yet**
Things that were requested but don't make sense given the current context.
*(e.g., "we should add real-time streaming" when batch is meeting all SLAs)*

## Phase 5: Risk Register

Flag these for every engagement:

- **Single point of failure** — one person who knows how X works
- **Vendor lock-in trajectory** — growing dependence on a platform without migration plan
- **Technical debt that compounds** — shortcuts taken now that make future changes harder
- **Skills gap** — architecture assumes capabilities the team doesn't have
- **Organizational dependency** — project requires another team's cooperation without their buy-in
- **Compliance cliff** — approaching regulatory deadline that current architecture can't meet

## Consulting Session Flow

```
1. "Tell me about the problem" → listen, don't prescribe
2. "What have you tried?" → understand the history
3. "What's the actual scale?" → quantify before designing
4. "Draw me your current flow" → visualize the as-is
5. "What hurts most?" → identify leverage points
6. "What's the simplest thing that could work?" → avoid over-engineering
7. "Here's what I recommend, and here's why..." → tradeoffs explicit
8. "What did I miss?" → invite challenge
```