Data Platform Engineer

Senior Data Platform Engineer with strong Spark and Spark Connect expertise focused on building and optimizing large-scale data platforms in Kubernetes environments.

Mission background

In the context of an ambitious transformation program the mission will consist in developing new IT services (API applications...) supporting existing around our central Data Hub platform.
The successful candidate will have proven expertise in the full cycle of software development cloud and related open source technologies such as python presto spark spark connect parquet Delta postgreSQL docker kubernetes ... are common language for you.Youll be joining a team of enthusiastic developers.Together you develop applications for a fast-changing IT landscape: an exciting and versatile challenge!

List of tasks expected within the mission

Responsibilities:

• Design develop and maintain high-value reusable and reliable software applications and services.
• Identify and correct any sticking points and iron out any bugs
• Launch new functionalities
• Stand guard over code quality organisation and automation.
• DevOps practices CI/CD pipelines and cloud-based backend deployments
• Handle complete application lifecycle management including requirements gathering from stakeholders and regular demos.
• Ability to mentor junior developers.

Qualifications:

• Proven Expert in backend development using Python Spark Spark Connect Flask and FastAPI.
• Knowledge about parquet files & Delta tables
• Expert knowledge of cloud infrastructure components like kubernetes docker open-shift
• Proficiency in database management and optimisation with PostgreSQL.
• Strong knowledge of message queuing systems like Kafka.
• Strong skills in backend architecture design and deployment.
• Strong knowledge of network- cloud- infrastructure like vault illumio ngnix ...
• Experience working on data-centric backend applications.
• Familiarity with Agile methodologies.
• Spoken and written proficiency in languages English and French. Dutch is a plus.

Required Skills:

APIPYTHONPOSTGRESQLDOCKERSPARKKUBERNETES