Data Integration Engineer

Data Integration Engineer (Azure & Databricks)

Location: Canada (Hybrid/Remote)
Experience: 58 Years

Position Overview

We are seeking a hands-on Data Integration Engineer to join our growing data and analytics team. This role is focused on the design development enhancement and support of enterprise data integration solutions within an Azure and Databricks ecosystem.

The ideal candidate is an execution-oriented professional who enjoys building and supporting data pipelines integrating data from multiple source systems and implementing scalable modern data platform solutions. While the role requires participation in solution discussions the primary focus is on delivery implementation and operational support rather than enterprise architecture or strategic consulting.

Key Responsibilities

Data Integration & Engineering

• Design develop and maintain scalable data integration pipelines using Azure and Databricks.
• Build and support batch and near real-time data ingestion processes from multiple internal and external source systems.
• Develop data transformation logic to support analytics reporting and business consumption requirements.
• Implement data quality validation reconciliation and monitoring processes.
• Optimize pipeline performance and troubleshoot production issues.

Databricks Development

• Develop and maintain Databricks notebooks workflows and processing pipelines.
• Build transformation frameworks using Spark and Databricks best practices.
• Support data ingestion cleansing enrichment and aggregation activities.
• Work with large and complex datasets across multiple domains.

Azure Data Platform Delivery

• Develop solutions using Azure data services including:
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure Databricks
• Azure SQL
• Azure Synapse (preferred)
• Support deployment monitoring and operational activities across the data platform.

Data Architecture Implementation

• Implement and support Medallion Architecture (Bronze Silver Gold layers).
• Ensure data lineage governance and consistency across the platform.
• Contribute to data modeling and solution design discussions.
• Translate architectural direction into technical implementation and delivery.

Team Collaboration

• Collaborate closely with Data Engineers Solution Architects Product Owners and Business Stakeholders.
• Support ongoing initiatives and enhancements within an established delivery team.
• Participate in Agile ceremonies sprint planning estimation and backlog refinement.
• Assist in production support troubleshooting and continuous improvement initiatives.

Required Qualifications

• 58 years of experience in Data Engineering Data Integration or Data Platform Development.
• Strong hands-on experience with Azure Databricks.
• Proven experience building and supporting enterprise-scale data pipelines.
• Strong understanding of data ingestion transformation and integration patterns.
• Experience integrating data from multiple source systems and platforms.
• Solid understanding of modern data lake and lakehouse architectures.
• Experience implementing Medallion Architecture concepts.
• Strong SQL development and data analysis skills.
• Experience working within Agile delivery teams.

Technical Skills

Required

• Azure Databricks
• Apache Spark / PySpark
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• SQL
• Python
• Data Integration & ETL/ELT Development
• Data Quality & Reconciliation

Preferred

• Azure Synapse Analytics
• Delta Lake
• CI/CD for Data Pipelines
• Azure DevOps
• Git
• Kafka or Event-Driven Architectures
• Power BI

Preferred Experience

• Experience supporting cloud-based analytics and reporting platforms.
• Experience working with complex enterprise data ecosystems.
• Exposure to financial services insurance banking or regulated industries.
• Experience supporting production environments and ongoing operational initiatives.
• Familiarity with data governance and data management best practices.