Senior Snowflake Data Engineer

Role Description

• 12 years of experience as a Data Engineer.
• Design develop and maintain scalable resilient data engineering solutions.
• Strong expertise in Snowflake Python PySpark DBT Qlik Replicate and Airflow.
• Experience building enterprise-grade data pipelines CDC solutions and cloud data platforms.
• Strong understanding of data governance security and DevOps practices.

Must-Have Technical Skills

• 12 years of hands-on Data Engineering experience.
• Deep expertise in Snowflake including:
  • Snowpipe
  • Tasks
  • Streams
  • Dynamic Tables
  • Advanced SQL
  • Data Masking Policies
  • RBAC
  • Performance tuning
• Strong experience building and maintaining DBT Cloud models with:
  • Testing
  • Documentation
  • Data lineage
• Excellent proficiency in:
  • Python
  • PySpark
• Experience developing ingestion frameworks for:
  • Files
  • APIs
  • Schema validation
  • Retry mechanisms
  • Metadata capture
• Experience ingesting data from:
  • CSV
  • Fixed-width files
  • Multi-record layouts
  • JSON
  • XML
  • Excel
  • Semi-structured data
• Experience designing ingestion patterns for:
  • Mainframe VSAM
  • EBCDIC data formats
• Strong expertise in:
  • Schema drift detection
  • Schema evolution
  • Metadata-driven architecture
• Hands-on experience with:
  • Qlik Replicate
  • Change Data Capture (CDC)
  • Full-load database replication
• Experience replicating data from:
  • Oracle
  • SQL Server
  • DB2
• Strong understanding of:
  • Data governance
  • RBAC
  • Data access controls
  • Snowflake security
• Experience implementing:
  • Dynamic data masking
  • Conditional masking
  • Role-based masking
• Experience with Protegrity or similar tokenization/data protection platforms.
• Hands-on experience with:
  • Astronomer Airflow
  • Workflow orchestration
  • Dependency management
  • Scheduling
  • Retry logic
  • SLA monitoring
• Experience integrating CI/CD pipelines using:
  • GitLab
  • Azure DevOps
• Strong knowledge of:
  • Git branching strategies
  • Merge requests
  • Code reviews
  • Repository management
• Experience implementing:
  • Monitoring
  • Alerting
  • Pipeline observability
• Expertise in optimizing:
  • Snowflake compute
  • Storage
  • Query performance
  • Pipeline scalability

Roles & Responsibilities

• Design and develop scalable resilient data pipelines using Snowflake.
• Build and maintain DBT models with testing documentation and lineage.
• Develop Python/PySpark ingestion frameworks for APIs and file-based integrations.
• Design ingestion solutions for structured and semi-structured data sources.
• Implement Mainframe VSAM ingestion for complex EBCDIC data.
• Detect analyze and manage schema drift across multiple data sources.
• Develop metadata-driven schema evolution strategies.
• Coordinate schema changes through CI/CD pipelines.
• Configure and maintain Qlik Replicate for CDC and full-load replication.
• Build reliable auditable and recoverable replication pipelines.
• Implement Snowflake data masking RBAC and governance policies.
• Apply Protegrity tokenization to protect sensitive data.
• Build and schedule workflows using Astronomer Airflow.
• Integrate data pipelines with GitLab and Azure DevOps CI/CD processes.
• Manage source code repositories and perform code reviews.
• Implement monitoring and alerting for ingestion replication and transformation pipelines.
• Optimize Snowflake performance storage utilization and query execution.
• Scale data pipelines to support increasing data volumes and low-latency processing.

Generic Managerial Skills

• Excellent analytical and problem-solving skills.
• Strong verbal and written communication skills.
• Ability to collaborate with cross-functional engineering and business teams.
• Strong stakeholder management and coordination skills.
• Ability to lead technical discussions and mentor junior engineers.
• Experience working in Agile/Scrum environments.
• Strong ownership accountability and commitment to quality delivery.
• Ability to manage multiple priorities in a fast-paced environment.