Lead Databricks Integration Architect

I hope you are doing well.

Please find the job details below:

Job Title: Lead Databricks Integration Architect
Location: Washington DC

Requirement:

Lead Solutions Architect specializing in Databricks and Application Integration on behalf of our client in Washington D.C.
This practitioner-level leader will create and execute the technical blueprints that link the enterprise Databricks platform directly to downstream operational applications microservices business intelligence tools and external APIs.
Acting as a hands-on mentor and technical authority this architect will build reusable platform libraries lead engineering code reviews and optimize real-time streaming data pipelines.
This position operates under a SAFe (Scaled Agile) framework with a hybrid schedule requiring on-site attendance for critical ceremonies like PI planning in Washington D.C.

Key Responsibilities
Hands-on Integration Blueprinting: Design document and actively build end-to-end integration patterns utilizing Databricks Delta Live Tables (DLT) structured streaming and core APIs to connect data with enterprise software.

Framework & Code Library Development:
Partner with Core Platform and Cloud Infrastructure teams to write reusable platform libraries and abstraction layers that speed up development cycles.

Data Quality Mechanics:
Architect and enforce technical data quality frameworks including automated DLT Expectations templates and granular data lineage tracking.

Mentorship & Code Reviews:
Lead architectural review boards oversee peer code/design reviews and provide direct technical mentorship to mid-and-senior data engineers.

Performance Tuning (FinOps):
Implement technical optimization strategies (Z-Ordering Liquid Partitioning Photon compute) to balance data throughput with cloud processing costs.

CI/CD Lifecycle Governance:
Design and govern multi-tier code deployment strategies (Dev Test Stage Prod) for Databricks assets and automated Infrastructure as Code (IaC).

Required Qualifications

Professional Experience:

• 8 years of experience in IT solution architecture application design or senior-level data engineering with a history of building integrated cloud data ecosystems.
• Hands-on Databricks & Language Mastery: Authoritative hands-on engineering experience using the Databricks platform PySpark SQL and core Delta Lake concepts.
• Advanced DLT & Streaming Expertise: Deep technical proficiency building and blueprinting complex Delta Live Tables (DLT) architectures and real-time structured streaming solutions.
• Enterprise Data Modeling: Expert-level understanding of designing relational dimensional and Medallion (Bronze/Silver/Gold) configurations optimized for multi-application consumption.
• Automation & CI/CD Architecture: Strong experience designing enterprise deployment strategies and automated infrastructure provisioning using Git Azure DevOps GitHub Actions or Terraform.
• Collaborative Leadership: Exceptional capability to lead technical squads drive consensus across engineering teams and clearly explain technical roadmaps to non-technical stakeholders.
• Agile Integration: Ability to thrive in a SAFe (Scaled Agile) environment including full commitment to iteration planning and on-site PI planning in Washington D.C.