Data Engineering
Job Location:
Monthly Salary:
Posted on:
26 days ago
Posted on:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Roles & Responsibilities:
- Expected to be an SME collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5 years of overall IT experience which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design build and deployment of Python-based applications.
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR API Gateway RDS instance and Lambda.
- Understand complex data sets and ETL processes and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
- Experience with cloud-based data platforms and distributed computing frameworks.
- Ability to optimize data workflows for performance and scalability.
- Familiarity with database management systems and data warehousing concepts.
- Competence in scripting and automation to streamline data processing tasks.
- Expected to be an SME collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5 years of overall IT experience which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design build and deployment of Python-based applications.
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR API Gateway RDS instance and Lambda.
- Understand complex data sets and ETL processes and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
- Experience with cloud-based data platforms and distributed computing frameworks.
- Ability to optimize data workflows for performance and scalability.
- Familiarity with database management systems and data warehousing concepts.
- Competence in scripting and automation to streamline data processing tasks.
