Data Engineer with Databricks
Job Location:
Monthly Salary:
Posted:
Application Deadline:
Vacancies:
Job Summary
Job Description:
-
Design develop and maintain scalable batch data pipelines using Apache Spark on Azure Databricks.
-
Work on Spark repositories implementing new features and enhancements.
-
Use Git for day-to-day development (branching committing merging pull requests).
-
Build Spark executable JARs using Maven and manage dependencies.
-
Execute and validate Spark jobs using Spark Submit on Azure Databricks with custom runtime arguments and I/O paths.
-
Configure orchestrate and monitor Databricks Workflows Jobs and Pipelines for reliable batch execution.
-
Implement and manage Delta Lake components (Delta Tables Delta Live Tables).
-
Ensure code quality stability and best practices in version control build and deployment.
-
Collaborate with architects data modelers and business stakeholders to translate requirements into technical solutions.
