Defending Your Resume: The Deep-Dive Round

Be ready with the inventory-and-prioritise story, plus at least one honest failure.

• Sequencing — inventory with consumers and cost, retire the dead jobs, rehost the risky Spark workloads, replatform the SQL-shaped ones to BigQuery-native
• Foundation first — landing zone, IAM, CI/CD, orchestration and monitoring before any pipeline moved
• Parallel run — legacy and cloud producing output side by side with automated daily reconciliation, then consumer sign-off before cutover
• Honest failure — undocumented logic in legacy scripts, or an unknown consumer discovered only after cutover. Say what it cost and what process changed as a result
• Different next time — invest in lineage/discovery earlier; almost everyone underestimates discovery and overestimates how well the estate is understood

Likely follow-up: How did you validate correctness at scale, table by table?

The trap is answering 'we used Kafka.' The real answer identifies *what dominated* the old latency.

• Where the time went — usually scheduled batch waiting: files landed hourly, a job ran, results appeared later. The dominant cost was *waiting for the next run*, not compute
• The change — event-driven ingestion, streaming micro-batches with a short trigger, writing directly to partitioned serving tables
• Measurement — an explicit end-to-end metric: event timestamp → queryable timestamp, tracked on a dashboard, plus consumer lag. Quote before and after
• Correctness — event-time windows with watermarks; idempotent sink so retries do not duplicate
• Trade-off — always-on cost and higher operational complexity, justified by the alerting use case

Name concrete failure modes that only appear at volume — this proves the number is lived experience, not a résumé flourish.

• Skew — one hot key (a busy site, a null id, an unknown default) stalls a stage at 199/200 tasks
• Small files — millions of tiny files make listing and scheduling dominate actual compute
• Cost non-linearity — a missing partition filter turns a cheap query into a full-history scan and a very visible bill
• Backfill cost — you cannot simply --full-refresh a billion-row table; backfills must be chunked and resumable
• Retry storms — a downstream outage plus aggressive retries creates a thundering herd
• Schema drift at volume — a single upstream type change can poison a whole day's partition before anyone notices

Distinguish technical leadership from people management, and be concrete about the artefacts you produced.

• Architecture standards — the layering convention, naming, testing and review rules you defined and enforced
• Work breakdown — splitting a migration into ownable pieces with clear interfaces so six people were not blocking each other
• Review and mentoring — design review before implementation; specific growth for named individuals
• Stakeholder alignment — translating between network engineers, business analysts and the data team
• A hard call you made — a technology choice, a scope cut, a decision to stop or redesign something. Leadership questions want a *decision*, not a description of ceremonies

This is really a governance question wearing a technical costume.

• Conformed dimensions owned centrally — one dim_cell_site, one dim_subscriber, shared by every domain
• Convention enforced in CI — folder structure, naming, materialisation defaults, required tests and descriptions
• Layer rules — staging is 1:1 with a source; cross-domain joins only happen in core
• Review gate — a new core model or metric definition requires review by the owning team
• Documentation and lineage published so consumers self-serve rather than re-deriving their own version
• Result — quantify: reduced duplication, one agreed definition per metric, faster onboarding of new domains