Databricks Lead – Python, PySpark, SQL & Supply Chain Analytics

Databricks Lead Python PySpark SQL & Supply Chain Analytics

Experience Required: 6-8 Years

Location: Toronto / Edmonton / Calgary

Work Model: Hybrid (3 days per week in office)

Role Summary

Seeking a Databricks Lead with strong expertise in Databricks to design build and operationalize scalable Supply Chain data products that deliver measurable business value. This role combines advanced analytics data pipeline development and product delivery to transform complex Supply Chain Management (SCM) data into trusted governed and actionable insights.

Key Responsibilities

• Develop analytics solutions that address Supply Chain business challenges leveraging knowledge of oil & gas operations procurement supplier performance spend analytics and enterprise value drivers.
  

• Design build test and optimize scalable ETL/ELT pipelines in Databricks using Python PySpark SQL notebooks jobs and orchestration patterns to support reliable Silver and Gold data products.
  

• Transform and model data from ERP and enterprise platforms including Oracle Cloud SAP ADW Microsoft Dataflow Dataverse and other approved data sources.
  

• Build and evaluate machine learning statistical and AI-assisted solutions including:
  

  • Classification
    

  • Forecasting
    

  • Anomaly Detection
    

  • Text Analytics
    

  • Natural Language Processing (NLP)
    

• Utilize Databricks platform capabilities such as:
  

  • Notebooks
    

  • Jobs
    

  • Workflows
    

  • Unity Catalog
    

  • SQL Warehouses
    

  • Genie Code
    

  • Lakeflow / Lakeflow Designer
    

• Maintain project documentation including idea assessments requirements design documents and weekly status reports.
  

• Communicate technical concepts requirements solution designs and product details effectively to business stakeholders in clear non-technical language.
  

Required Skills

• Databricks
  

• Python
  

• PySpark
  

• SQL / Oracle SQL
  

• Data Architecture & Data Modeling
  

• ETL / ELT Pipeline Development
  

• Machine Learning
  

• Supply Chain Analytics
  

• Data Integration and Data Transformation
  

Preferred Domain Experience

• Supply Chain Management (SCM)
  

• Oil & Gas Industry
  

• Procurement Analytics
  

• Supplier Performance Analytics
  

• Spend Analytics