Data Engineer

Job Title : Data Engineer

Experience : 5 TO 10 Years

Location : Bangalore

Roles & Responsibilities

• Oversee the entire data infrastructure to ensure scalability operation efficiency and resiliency
  
• Mentor junior data engineers within the organization
  
• Design develop and maintain data pipelines and ETL processes using Microsoft Azure services (e.g. Azure Data Factory Azure Synapse Azure Databricks Azure Fabric)
  
• Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g. Azure Data Lake Storage Gen 2 & Azure Blob storage)
  
• Collaborate with data scientists data analysts data architects and other stakeholders to understand data requirements and deliver high-quality data solutions
  
• Optimize data pipelines in the Azure environment for performance scalability and reliability
  
• Ensure data quality and integrity through data validation techniques and frameworks
  
• Develop and maintain documentation for data processes configurations and best practices
• Monitor and troubleshoot data pipeline issues to ensure timely resolution
  
• Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting-edge
  
• Manage the CI/CD process for deploying and maintaining data solutions

Requisites

• Bachelors degree in Computer Science Engineering or a related field (or equivalent experience) and able to demonstrate high proficiency in programming fundamentals
  
• Relevant years of proven experience as a Data Engineer or similar role dealing with data and ETL processes. 5 - 10 years of experience
  
• Strong knowledge of Microsoft Azure services including Azure Data Factory Azure Synapse Azure Databricks Azure Blob Storage and Azure Data Lake Gen 2
  
• Experience utilizing SQL DML to query modern RDBMS in an efficient manner (e.g. SQL Server PostgreSQL)
  
• Strong understanding of Software Engineering principles and how they apply to Data Engineering (e.g. CI/CD version control testing)
  
• Experience with big data technologies (e.g. Spark)
  
• Strong problem-solving skills and attention to detail Excellent communication and collaboration skills

Required Skills:

Data EngineeringPythonAWSETLInfrastructure