Lead Data Engineer
Job Summary
We are seeking an experienced Senior Data Engineer to drive the performance governance and AI-native maturity of our enterprise Data Platform in Databricks. This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices designing scalable pipelines tuning workloads for performance and cost and operationalizing modern data and AI capabilities on Lakehouse.
The ideal candidate has deep hands-on Databricks expertise a strong performance-engineering instinct and a builders mindset for AI-assisted operations. Youll own the Databricks performance and governance standards for the platform mentor engineers and shape the direction for AI-native operations.
- Design and develop scalable data pipelines and Lakehouse solutions on Databricks.
- Tune Databricks workloads for performance and cost including cluster sizing query optimization and Delta Lake table design.
- Establish and enforce best practices for partitioning clustering and workload isolation.
- Track performance trends identify high-cost queries and partner with source teams and end users to resolve long-running loads.
- Design and operationalize Unity Catalog for data governance access control lineage and security.
- Build monitoring and self-healing automation using Databricks-native AI and agentic capabilities.
- Drive CI/CD workflows for Databricks assets setting DevOps best practices for deployment and release management.
- Lead design reviews and mentor Data Engineers on Databricks best practices and AI-native features.
- Own Databricks vendor coordination case management escalations and release adoption strategy.
- Within 612 months youll define the platforms tuning and governance standards lead design reviews mentor junior engineers and shape the AI-native operations roadmap.
- Bachelors or Masters degree in Computer Science Information Technology or equivalent relevant experience.
- 6 years of data engineering experience with 2 years hands-on Databricks in enterprise settings.
- Deep understanding of Databricks Lakehouse architecture Delta Lake Unity Catalog and Workflow orchestration.
- Proven ability to tune Spark workloads for cost and performance at production scale.
- Advanced Python (PySpark) and SQL skills.
- Working knowledge of CI/CD practices and DevOps principles applied to data workloads.
- Experience with observability tooling for Databricks.
- Experience with Databricks-native AI capabilities and agentic frameworks.
- Familiarity with Databricks Serverless Compute and DBSQL performance tuning.
- A Databricks Certified Professional.
- Exposure to Infrastructure-as-Code is a plus.
- Performance-engineering mindset measures tunes and re-measures.
- Curiosity for AI-native operations and continuous automation.
- Strong sense of platform ownership quality cost and reliability.
- Effective communication with engineering peers vendors and business stakeholders.
- Influence outcomes across source teams vendors and business stakeholders without direct authority.
Required Experience:
Senior IC
About Company
NXP is a global semiconductor company creating solutions that enable secure connections for a smarter world.