Data Engineer with Databricks

Acme Services Private Limited

Job Location:

Bengaluru - India

Monthly Salary: Not Disclosed

Posted on: 5 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Description:

Design develop and maintain scalable batch data pipelines using Apache Spark on Azure Databricks.
Work on Spark repositories implementing new features and enhancements.
Use Git for day-to-day development (branching committing merging pull requests).
Build Spark executable JARs using Maven and manage dependencies.
Execute and validate Spark jobs using Spark Submit on Azure Databricks with custom runtime arguments and I/O paths.
Configure orchestrate and monitor Databricks Workflows Jobs and Pipelines for reliable batch execution.
Implement and manage Delta Lake components (Delta Tables Delta Live Tables).
Ensure code quality stability and best practices in version control build and deployment.
Collaborate with architects data modelers and business stakeholders to translate requirements into technical solutions.

Job Description: Design develop and maintain scalable batch data pipelines using Apache Spark on Azure Databricks. Work on Spark repositories implementing new features and enhancements. Use Git for day-to-day development (branching committing merging pull requests). Build Spark execut...