Overview

Role purpose. Take an approved design to a deployed and validated bundle, operating the multi-agent system and owning the quality of everything that goes into dev. This person does not write all the code; they are accountable for it.

Responsibilities

• Data engineering or ETL development.
• Python and SQL in production environments.
• Hands-on experience with Apache Spark
• Databricks (nice to have: Asset Bundles).
• Agent Operation and Auditing: Interact with the multi-agent system, put the plan.md under scrutiny, and thoroughly review the generated code bundle (detecting bottlenecks such as data skew, excessive shuffles, or incorrect data typing).
• Parity Testing and Reconciliation: Design and run data parity validations certifying that the Databricks output matches the legacy source load 100% (volume, values, and schemas).
• Exception Management and Escalation: Independently resolve minor code warnings and promptly escalate NEEDS-HUMAN events, pattern changes, or semantic inconsistencies to the Architect.
• Code Quality and Security: Remediate findings from SAST tools (e.g., Fluid Attacks), dependency vulnerabilities, and non-conformities before opening the PR.
• Delivery Cycle (Git Flow): Manage feat/TICKET-slug branches, ensure strict compliance with local pre-checks, and lead the resolution of REJECTED tickets or defects raised in QA/UAT.
• Transition Stability: Sustain operations, rapid anomaly remediation, and pipeline telemetry during the post-deployment hypercare window until formal handover to support.