Senior Data Engineer

About the Opportunity

Our client is a forward-thinking financial institution committed to redefining banking through a tech-first data-driven approach. They are building a digital bank from the ground up applying lessons from some of the worlds most innovative technology companies to deliver exceptional products and customer experiences.

They are seeking an all-round Senior Data Engineer to join their Data Services this role you will design maintain and enhance the critical data infrastructure that powers analytics and operations across the organization. This includes managing the data lake operational databases high-volume batch and real-time processing systems and metadata repositoriesall working in concert to deliver accurate timely and actionable insights.

You will collaborate closely with data scientists to structure schemas and design data models work with product teams to integrate new data sources and partner with fellow engineers to bring cutting-edge data technologies to life.

The Ideal Candidate

Our client values logical thinkers who balance respect for best practices with independent critical thinking. They seek professionals who are adaptable capable of owning projects from end-to-end communicate effectively in English and thrive in collaborative high-performing team environments. While deep experience in specific technologies is valued they prioritize candidates who are eager to learn grow and fill any gaps on the job.

Key Responsibilities

• Develop and Maintain Data Infrastructure: Build optimize and manage the data lake and its associated processing frameworks ensuring reliable and scalable data ingestion from diverse sources.
• Design and Orchestrate ETL Pipelines: Architect and implement robust data workflows using orchestration tools moving data from source systems through various processing stages into the lake.
• Enable Analytics and Data Science: Structure data schemas and design data models to support business intelligence reporting and machine learning initiatives.
• Integrate New Data Sources: Work with product and engineering teams to onboard new data streams ensuring seamless integration and data quality.
• Champion Technology Evolution: Evaluate and introduce emerging tools and frameworks to continuously improve the data platforms performance and capabilities.

Required Experience & Skills

• Core Data Engineering Expertise: Solid understanding of data lake architectures and experience working with columnar big data databases (e.g. Athena Redshift Vertica Hive/Hadoop). Familiarity with Iceberg is a significant advantage.
• ETL & Workflow Orchestration: Proven ability to design and implement ETL pipelines and manage workflows using tools such as Apache Airflow Luigi or AWS Batch.
• Programming Proficiency: Strong Python skills with hands-on experience using relevant libraries (e.g. boto3 pandas pytest). PySpark experience is highly desirable.
• Cloud & AWS Services: Practical experience with cloud environmentsparticularly AWS (Glue EMR EC2 S3 Lambda IAM CloudWatch) or equivalent platforms.
• Containerization & Orchestration: Familiarity with Docker and Kubernetes (or AWS ECS/EKS) for deployment and scaling.
• CI/CD & Version Control: Experience with CI/CD tools (e.g. CircleCI Jenkins AWS CodePipeline) and solid git practices (branching strategies collaboration workflows).
• Agile Methodologies: Comfortable working within Agile/Lean frameworks such as Scrum or Kanban.
• Bonus Skills (Highly Valued):
  • Distributed messaging and streaming systems (Kafka Pulsar RabbitMQ)
  • Streaming processing frameworks (Spark Streaming Apache Beam Apache Flink)
  • Metadata catalogue and lineage tools (Amundsen Apache Atlas Alation)
  • JVM languages (Java Scala Kotlin) and related frameworks
  • RDBMS/NoSQL databases (PostgreSQL MySQL DynamoDB Redis)
  • BI tools (Tableau Looker PowerBI QuickSight)
  • Logging and monitoring stacks (ELK Datadog Prometheus Grafana)
  • Data privacy and security concepts (encryption tokenization Apache Ranger)

Experience Level

This is a senior position (L3L4 level). Candidates with approximately 3 years of relevant data engineering experience are encouraged to apply. What matters most is the quality of experienceour client values professionals who have worked in environments that foster continuous learning and skill development. Length of tenure alone is not a differentiator; they seek individuals who can demonstrate meaningful growth and technical progression throughout their careers.