Data Engineer-Customer Analytics AIML

What awaits you/ Job Profile

• Analyze Organize data and Build data systems with multiple data sources

• Working closely with Stakeholders and Cross functional teams for the Data analysis.

What should you bring along

• Build data systems and pipelines
• Standardize the data assets which maximize data reusability
• Evaluate business needs and objectives
• Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS Cloud based and On-prep databases
• Conduct complex data analysis and report on results
• Combine raw information from different sources
• Explore ways to enhance data quality and reliability
• Implementing functional requirements including incremental loads data snapshots and identity resolution
• Optimizing data pipelines to improve performance and cost while ensuring a high quality of data within the data lake:
• Monitoring services and jobs for cost and performance ensuring continual operations of data pipelines and fixing of defects.
• Constantly looking for opportunities to optimize data pipelines to improve performance.

Must have technical skill

• Must have coding skills in Spark/Pyspark Python and SQL
• Must have Knowledge of AWS tools Glue Athena step functions and S3

Good to have technical skills

• Good knowledge of CI/CD (Github / Github Actions Terraform)

• Knowledge on Agile methodologies.