Data Engineer (Azure Data Platform)

Job Description

Must Have Technical/Functional Skills:

Cosmos; PL/SQL; MSBI; Digital : Microsoft Power BI; Digital : Big Data Platforms

Roles & Responsibilities:

• Automated Incident Detection & Remediation
• Design and implement automated incident detection and remediation frameworks for data platforms and pipelines.
• Shift incident response from reactive manual processes to proactive policy-driven workflows.
• Improve mean time to detect (MTTD) and mean time to remediate (MTTR) through automation.
• DataOps Automation & AIOps Frameworks
• Build and operationalize DataOps automation frameworks incorporating AIOps where applicable for anomaly detection and predictive alerting.
• Reduce operational noise and repetitive manual tasks through intelligent automation.
• Establish standardized operational runbooks and automated recovery patterns.
• Platform Observability & Telemetry Standardization
• Define and implement platform-wide observability and telemetry standards for data workloads.
• Enable consistent metrics logging alerting and health signals across platforms and pipelines.
• Ensure operational visibility is actionable and supports automated response.
• Self-Service Operational Tooling
• Build self-service operational tooling that enables engineering teams to debug monitor and resolve issues independently.
• Reduce operational dependencies on centralized teams and vendors.
• Empower teams with standardized dashboards insights and tooling.
• Automated Cost Governance & Optimization
• Establish automated cost governance and optimization frameworks for data platforms.
• Improve cost visibility through telemetry alerts and policy-based controls.
• Drive proactive cost management rather than reactive spend reviews.

Required Skills & Experience

• Strong experience in Data Operations Data Platform Operations or SRE/DataOps roles.
• Proven experience reducing operational toil through automation-first approaches.
• Hands-on experience with incident management observability and operational tooling.
• Experience operating within or transforming managed services or vendor-led operating models.
• Strong understanding of cost governance and optimization in data platforms.
• Hands-on experience with Microsoft Azure data platform services such as Azure Data Factory Azure Databricks Azure Synapse Analytics Microsoft Fabric
• Experience implementing CI/CD and release automation using Azure DevOps including Azure Pipelines repository management deployment workflows and environment-based promotion controls.
• Experience with Microsoft Entra ID Azure Key Vault and Microsoft Purview for identity secrets management governance data lineage and compliance controls.
• Strong scripting and engineering skills in Python SQL and PowerShell to build automation operational runbooks telemetry pipelines and remediation workflows.
• 8 years of relevant experience.