Sr Databricks Certified Data Engineer

Position: Sr Databricks Certified Data Engineer

Location: Remote

Duration: Full Time

Job Description

• Sr Databricks Certified Data Engineer
• Must Have: Databricks Certified Data Engineer Professional Certification.
• Strongly preferred: Prior experience working in the Databricks Partner Program Databricks Partner Champion Certificate

• 10 years of experience in data engineering data platforms & analytics.
• Comfortable writing code in either Python or Scala.
• Extensive Working knowledge of Databricks is required.
• Experience with advanced Databricks concepts such as Genie Spaces Agents and Genie Code.
• Experience handling or leading large scale projects/customers.
• Working knowledge of two or more common Cloud ecosystems (AWS Azure GCP) with expertise in at least one.
• Deep experience with distributed computing with Apache Spark and knowledge of Apache Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps Capable of design and deployment of highly performant end-to-end data architectures Experience with technical project delivery managing scope and timelines Documentation and white-boarding skills Experience working with clients and managing conflicts Experience in building scalable streaming and batch solutions using cloud-native components

Here is some feedback I got for one of the consultant that took the interview:

1) Not strong in Data engineering and advanced concepts.

2) No Experience with advanced Databricks concepts such as Genie Spaces Agents and Genie Code.

3) No experience handling or leading large scale projects/customers.

Key Skills:

• They want someone with. Spark fundamentals Spark architecture DataFrames partitions shuffles joins and performance behavior.
• Delta Lake ACID tables schema evolution merges optimization and reliable lakehouse storage patterns.
• ETL / pipeline design batch and incremental pipelines medallion-style thinking and production pipeline design.
• Lakeflow /SDP / Jobs / workflows orchestration declarative pipelines and workload deployment on Databricks.
• Ingestion patterns landing data from databases files streams and connectors into bronze/silver/gold pipelines.
• Performance tuning cluster sizing file layout pipeline optimization and Spark tuning best practices.
• Data modeling and warehousing dimensional modeling warehouse migration patterns and serving BI/reporting use cases.
• CI/CD and deployment asset bundles deployment workflows

Regards

Manoj Goud

Derex Technologies INC

Contact : Ext 206

Additional Information :

All your information will be kept confidential according to EEO guidelines.

Remote Work :

Yes

Employment Type :

Full-time