Systems Analyst III Databricks Data Engineer

Systems Analyst III / Databricks Data Engineer

Client: Texas Health and Human Services Commission (HHSC)
Location: Austin TX (Hybrid - 2 Days Onsite 3 Days Remote)
Duration: 12 Months (High possibility of extension)
Job Type: Contract (W2/1099/C2C)
Interview: Microsoft Teams / In-Person
Local Candidates Only

Job Summary

Texas Health and Human Services Commission (HHSC) is seeking an experienced Systems Analyst III / Databricks Data Engineer to design develop and optimize enterprise-scale data solutions using the Databricks platform. The ideal candidate will have extensive experience building scalable ETL/ELT pipelines implementing modern Lakehouse architecture and developing analytical dashboards that enable business decision-making.

This role requires deep expertise in Databricks Apache Spark Delta Lake Lakeflow Declarative Pipelines (DLT) SQL Python/Scala data warehousing and cloud-based data engineering practices. The selected consultant will collaborate with business stakeholders architects and technical teams to deliver secure scalable and high-performance data solutions.

Key Responsibilities

• Design develop and maintain scalable enterprise data solutions using the Databricks platform.
• Develop and optimize ETL/ELT pipelines using Apache Spark (PySpark or Scala).
• Build and maintain Lakehouse architectures utilizing Delta Lake and Medallion Architecture (Bronze Silver Gold).
• Implement Lakeflow Declarative Pipelines (formerly Delta Live Tables) for production-grade data processing.
• Develop and schedule workflows using Lakeflow Jobs (formerly Databricks Workflows) or equivalent orchestration tools such as Apache Airflow.
• Design and maintain dimensional data models including Star and Snowflake schemas.
• Develop high-performance SQL queries for analytical workloads.
• Create interactive dashboards and applications using Databricks SQL Dashboards and Databricks Apps.
• Implement enterprise data governance metadata management data quality validation and security best practices.
• Monitor pipeline performance and optimize workloads for scalability reliability and cost efficiency.
• Analyze business requirements and translate them into technical solutions.
• Perform system analysis troubleshooting and production support activities.
• Collaborate with business users project managers developers and architects throughout the software development lifecycle.
• Document technical designs workflows data models and operational procedures.
• Present technical findings and business insights to both technical and non-technical stakeholders.

Required Qualifications

• Bachelors degree in Computer Science Information Systems Engineering or a related field (or equivalent experience).
• Minimum 8 years of IT experience supporting the design development deployment or delivery of technology solutions.
• Minimum 8 years of experience with Databricks.
• Strong expertise in Apache Spark using PySpark or Scala.
• Extensive experience building enterprise ETL/ELT pipelines.
• Strong knowledge of Delta Lake.
• Experience implementing Lakeflow Declarative Pipelines (Delta Live Tables).
• Experience with Medallion Architecture (Bronze Silver Gold).
• Expertise in Lakehouse architecture.
• Strong experience with SQL.
• Strong programming skills in Python (preferred) or Scala.
• Experience designing dimensional data models (Star/Snowflake schema).
• Experience developing Databricks SQL Dashboards and Databricks Apps.
• Experience implementing data governance data quality security and validation frameworks.
• Experience scheduling and orchestrating jobs using Lakeflow Jobs or Apache Airflow.
• Excellent verbal and written communication skills.
• Ability to work independently while collaborating across cross-functional teams.

Preferred Qualifications

• Databricks Certified Data Engineer (Associate or Professional).
• Experience working with Texas State agencies or other public sector organizations.
• Experience with Git-based version control.
• Experience implementing CI/CD pipelines for data engineering projects.
• Familiarity with DevOps best practices.
• Experience working in Agile/Scrum environments.

Technical Skills

Data Engineering

• Databricks
• Apache Spark
• PySpark
• Scala
• Delta Lake
• Delta Live Tables (DLT)
• Lakeflow Declarative Pipelines
• Lakeflow Jobs
• Databricks Workflows
• Data Lakehouse
• Medallion Architecture

Programming

• Python
• SQL
• Scala

Data Warehousing

• Dimensional Modeling
• Star Schema
• Snowflake Schema
• ETL
• ELT

Reporting & Analytics

• Databricks SQL
• Databricks Dashboards
• Databricks Apps

Data Management

• Data Governance
• Data Quality
• Data Validation
• Data Security
• Metadata Management

DevOps & Tools

• Git
• CI/CD
• Azure DevOps (preferred)
• Apache Airflow

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and presentation abilities
• Ability to interact with business and technical stakeholders
• Strong documentation skills
• Self-motivated and detail-oriented
• Ability to manage multiple priorities in a fast-paced environment

Required Skills:

data bricks