DE&A Databricks Data Engineer, Platform Admin and Operations Support

We are seeking a skilled Databricks Engineer with strong expertise in cluster performance optimization storage tuning and platform reliability. The ideal candidate will be responsible for ensuring efficient stable and scalable Databricks operations especially during mission-critical processing cycles while supporting integrated SnowflakeDatabricks ecosystems.

• Cluster Performance Optimization
  • Configure and optimize Databricks clusters including cluster sizing autoscaling strategies and runtime tuning
  • Manage driver and executor configurations to improve workload performance
  • Optimize cost and performance through effective use of spot and on-demand instances
• Storage & Data Optimization
  • Implement Delta Lake optimizations (e.g. file compaction partitioning Z-ordering)
  • Design and manage efficient data layouts for large-scale analytical workloads
  • Ensure optimal storage performance and query efficiency
• Operational Excellence & Support
  • Provide support during mission-critical data processing cycles ensuring system stability and job reliability
  • Monitor clusters and workflows proactively identifying and resolving performance bottlenecks
  • Ensure high availability and reliability of Databricks workloads
• Incident Management & Stability
  • Manage high-severity incidents with timely resolution
  • Perform root cause analysis (RCA) and implement preventive measures
  • Ensure overall platform stability across integrated SnowflakeDatabricks environments

Required Skills & Qualifications

• Hands-on experience withDatabricks cluster management and performance tuning
• Strong understanding ofDelta Lake and data optimization techniques
• Experience inmonitoring troubleshooting and supporting production workloads
• Proven ability to handlecritical incidents and perform RCA
• Familiarity withSnowflakeDatabricks integration ecosystems
• Proficiency inSQL and Spark (PySpark/Scala)

Good to Have

• Experience with orchestration tools (e.g. Airflow)
• Exposure to cloud platforms such as AWS Azure or GCP
• Knowledge of cost optimization strategies in cloud-based data platforms

Soft Skills

• Strong problem-solving and analytical abilities
• Ability to work in high-pressure mission-critical environments
• Effective communication with stakeholders and cross-functional teams