Data Engineer MS Fabrics

Role: MS Fabric Data Engineer

Location: Sydney

Job Type: Permanent

Experience: 5 yrs

Job Description:

Roles and Responsibilities:

• Design build and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and One Lake architectures for structured and unstructured data.
• Ingest data from multiple sources (databases APIs files streaming sources) into Fabric.
• Implement ETL/ELT processes using Spark (Pyspark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality reliability and consistency through validation and monitoring.
• Optimize pipeline performance storage and capacity utilization in Fabric.
• Implement security access control (RBAC) and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor troubleshoot and resolve data pipeline and performance issues.
• Implement data governance lineage and cataloguing using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects analysts and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement:

• Design enterprise data architecture leveraging Fabric components (Lakehouse Warehouse Dataflows Gen2 Pipelines OneLake).
• Build and optimise Fabric Lakehouse tables Warehouses Pipelines and Dataflows Gen2.
• Define data modelling standards governance frameworks and security patterns.
• Develop reusable ETL/ELT frameworks using PySpark SQL and Fabric notebooks.
• Manage CI/CD workflows via Azure DevOps and Git.
• Architect scalable cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
• Apply best practices for performance tuning cost optimisation and data quality.
• Provide technical leadership and guidance to engineering teams.
• Expertise in Delta Lake Fabric Data Engineering ADLS PySpark/SQL.
• Strong understanding of data modelling and integration with Power BI datasets.
• Strong understanding of metadata management Purview access control and FinOps.
• Certifications: DP-203 Fabric Analytics Engineer (preferred).

Design build and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.

Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.

Ingest data from multiple sources (databases APIs files streaming sources) into Fabric.

Implement ETL/ELT processes using Spark (PySpark) and SQL.

Design and optimize data models to support analytics and reporting workloads.

Develop and maintain Fabric Data Warehouse solutions using T-SQL.

Ensure data quality reliability and consistency through validation and monitoring.

Optimize pipeline performance storage and capacity utilization in Fabric.

Implement security access control (RBAC) and data sensitivity labels.

Enable and support Power BI semantic models and downstream reporting use cases.

Monitor troubleshoot and resolve data pipeline and performance issues.

Implement data governance lineage and cataloging using Microsoft Purview.

Support CI/CD and DevOps practices for analytics and data engineering workloads.

Collaborate with architects analysts and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement :

Design enterprise data architecture leveraging Fabric components (Lakehouse Warehouse Dataflows Gen2 Pipelines OneLake).

Build and optimise Fabric Lakehouse tables Warehouses Pipelines and Dataflows Gen2.

Define data modelling standards governance frameworks and security patterns.

Develop reusable ETL/ELT frameworks using PySpark SQL and Fabric notebooks.

Manage CI/CD workflows via Azure DevOps and Git.

Architect scalable cost-efficient data solutions integrating Fabric with Azure/enterprise systems.

Apply best practices for performance tuning cost optimisation and data quality.

Provide technical leadership and guidance to engineering teams.

Expertise in Delta Lake Fabric Data Engineering ADLS PySpark/SQL.

Strong understanding of data modelling and integration with Power BI datasets.

Strong understanding of metadata management Purview access control and FinOps.

Certifications: DP-203 Fabric Analytics Engineer (preferred).