Data Engineer Databricks & Treasury Data Lake

Location: Remote (Freelance)

Project Duration: 6-12 months

THE COMPANY

Our client is a recruitment consultancy specialising in IT Finance HR and Digital Technology. Our client is seeking an experienced Data Engineer specializing in Databricks and Treasury Data Lake environments to join a fully remote project.

The successful candidate will be responsible for designing developing maintaining and optimizing end-to-end data pipelines within a modern Lakehouse architecture ensuring reliable scalable and high-performance data solutions. This role requires strong hands-on experience with Databricks DBT advanced SQL and enterprise data integration alongside a solid understanding of data governance automation and CI/CD practices.

RESPONSIBILITIES

Design build and maintain end-to-end data ingestion pipelines integrating multiple source systems.

Develop and optimize data transformations across bronze silver and gold layers within a Databricks Lakehouse architecture.

Implement automated data quality controls monitoring frameworks and alerting mechanisms to ensure data integrity and reliability.

Design and maintain data models and transformation logic using DBT and advanced SQL.

Leverage Databricks Lakehouse Federation and Lakeflow Connect to support efficient data integration and accessibility.

Develop and maintain CI/CD pipelines using GitHub and Azure DevOps ensuring controlled deployments testing and version management.

Identify and resolve performance bottlenecks improving scalability maintainability and efficiency.

Collaborate with business and technical stakeholders to translate requirements into robust data engineering solutions.

Support data governance standards documentation troubleshooting and release management activities.

Contribute to analytics and reporting capabilities by delivering reliable high-quality data products.

REQUIREMENTS

Proven professional experience as a Data Engineer in enterprise data environments.

Strong hands-on expertise with Databricks and modern Data Lake/Lakehouse architecture.

Practical experience developing data transformations using DBT (Data Build Tool).

Advanced SQL skills including query optimization and complex data transformations.

Experience designing and managing data pipelines across bronze silver and gold layers.

Knowledge of Databricks Lakehouse Federation and Lakeflow Connect.

Experience implementing data quality frameworks monitoring and automated alerting.

Familiarity with GitHub Azure DevOps and CI/CD processes.

Understanding of data governance data modeling and enterprise integration practices.

Knowledge of artificial intelligence-related concepts and their application within modern data ecosystems.

Strong analytical troubleshooting documentation and communication skills.

Ability to work independently and collaborate effectively with distributed teams.