Job Title

Data Platform Engineering Lead - Enterprise Data Office (Asset Management)

Job Summary

We are seeking a highly experienced Data Platform Engineering Lead to architect build and manage enterprise-scale cloud-native data platforms supporting Asset Management business functions. This leadership role is responsible for driving platform engineering strategy cloud infrastructure automation DevOps enablement platform reliability security governance and operational excellence across the Enterprise Data Office (EDO).

The ideal candidate will have deep expertise in Databricks dbt Apache Airflow Enterprise Batch Tools (EBT) AWS Cloud Terraform Python Kubernetes DevOps and Platform Engineering with proven experience supporting enterprise data platforms in Asset Management Investment Management or Financial Services organizations.

Experience Required

• 10-15 Years of experience in Data Engineering Platform Engineering or Cloud Engineering
• 5 Years leading enterprise Data Platform or Cloud Platform teams
• Experience building enterprise-scale cloud-native data platforms
• Experience in Asset Management Investment Management or Financial Services

Key Responsibilities

Platform Engineering & Architecture

• Define and execute the Enterprise Data Platform engineering strategy aligned with EDO and enterprise architecture standards.
• Design build and manage scalable secure and highly available cloud-native data platforms.
• Establish reusable platform frameworks engineering standards and accelerators.
• Lead cloud modernization initiatives across enterprise data and analytics ecosystems.
• Manage the complete platform lifecycle including scalability availability reliability and cost optimization.

Databricks Platform Engineering

• Architect and administer enterprise Databricks environments.
• Design and implement Lakehouse Architecture using Delta Lake.
• Manage and optimize:
  • Databricks Workflows
  • Delta Live Tables (DLT)
  • Unity Catalog
  • MLflow
  • Structured Streaming
  • Cluster Policies
  • Security & Access Controls
• Optimize platform performance reliability and operational costs.
• Implement governance across Development QA UAT and Production environments.

Cloud Engineering (AWS)

Design and manage cloud-native data platform solutions using:

• Amazon S3
• AWS ECS
• AWS EKS
• AWS Lambda
• AWS Glue
• Amazon CloudWatch

Implement secure scalable and highly available cloud infrastructure.

Infrastructure as Code (IaC)

• Develop reusable Terraform modules.
• Automate infrastructure provisioning and configuration.
• Implement Infrastructure-as-Code (IaC) best practices.
• Standardize infrastructure deployment and governance.
• Maintain infrastructure version control.
• Ensure compliance through automated infrastructure management.

DevOps & CI/CD

Design and implement enterprise DevOps practices using:

• GitHub Actions
• GitLab CI/CD
• Jenkins
• Azure DevOps
• Docker
• Kubernetes
• Terraform

Key responsibilities include:

• CI/CD pipeline design
• Automated deployments
• GitOps implementation
• Release automation
• Self-service engineering capabilities
• Deployment reliability improvements

Platform Operations & Reliability

• Establish Site Reliability Engineering (SRE) practices.
• Define platform SLAs SLOs and operational KPIs.
• Implement platform monitoring observability and alerting.
• Lead incident management root cause analysis (RCA) and preventive actions.
• Improve platform resilience and operational efficiency.

Data Platform Operations

Support enterprise data platforms including:

• Databricks
• Data Lake
• Enterprise Batch Tools (EBT)
• Apache Airflow
• Data Integration Platforms
• Analytics Platforms

Responsibilities include:

• Platform monitoring
• Workflow orchestration
• Batch processing
• Data pipeline reliability
• Operational support
• Performance tuning

Stakeholder Collaboration

Collaborate with:

• Enterprise Data Office (EDO)
• Chief Data Office (CDO)
• Portfolio Management Teams
• Investment Operations
• Risk & Compliance Teams
• Data Governance Teams
• Enterprise Architecture
• Infrastructure & Cloud Engineering
• Application Development Teams

Translate business requirements into enterprise platform capabilities and drive strategic technology initiatives.

Mandatory Technical Skills

Data Platform

• Databricks
• Delta Lake
• Delta Live Tables (DLT)
• Unity Catalog
• MLflow
• Structured Streaming
• Apache Airflow
• Enterprise Batch Tools (EBT)
• dbt

Cloud

• AWS
• Amazon S3
• AWS ECS
• AWS EKS
• AWS Lambda
• AWS Glue
• Amazon CloudWatch

Infrastructure & DevOps

• Terraform
• Docker
• Kubernetes
• GitHub Actions
• GitLab CI/CD
• Jenkins
• Azure DevOps
• GitOps
• CI/CD

Programming

• Python
• SQL
• Shell Scripting (preferred)

Operations

• Site Reliability Engineering (SRE)
• Monitoring & Observability
• Incident Management
• Root Cause Analysis
• Performance Optimization
• Capacity Planning
• Security & Governance

Preferred Skills

• Asset Management
• Investment Management
• Wealth Management
• Data Governance
• Regulatory Reporting
• Lakehouse Architecture
• DataOps
• Platform Automation
• Enterprise Data Warehouse
• API Integration

Leadership Skills

• Platform Strategy
• Technical Leadership
• Architecture Governance
• Stakeholder Management
• Team Mentoring
• Delivery Management
• Cross-functional Collaboration
• Continuous Improvement

Education

• Bachelors or Masters degree in Computer Science Information Technology Engineering or a related field.