Software Engineer – Data Platform

Job Description

This is a software engineering role focused on building production-grade data platform components.

You will design and operate services APIs and data pipelines in a CI/CD-driven environment.

This is NOT a notebook-driven or analytics-focused role.

Job Description:

• Design implement and maintain secure scalable and fault-tolerant data ingestion pipelines from a wide variety of source systems including SAP Salesforce SharePoint APIs and (legacy) manufacturing platforms.
• Build and enhance production-grade metadata-driven data handling components and inbound-/outbound-interfaces.
• Own production data interfaces end-to-end.
• Develop reusable version-controlled platform components APIs or data interfaces following software engineering best practices (CI/CD automated testing version control monitoring).
• Contributes to a global data engineering team delivering to all major business domains.
• Establish and maintain holistic data quality management monitoring and reporting.
• Collaborates across business IT cybersecurity infrastructure and architecture teams to ensure secure and sustainable delivery.

Main Tasks:

• Design and implement fault-tolerant observable data pipelines including logging alerting and failure handling strategies.
• Build and maintain Python- or Scala-based extraction services (e.g. Debezium Server custom APIs rclone)
• Implement CDC delta and event-based patterns.
• Build and enhance push-based HTTP Kerberos-authenticated custom data lake transfer delivery component.
• Integrate with systems such as Salesforce SharePoint and other API- or file-based endpoints.  
• Establish operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal.
• Establish and maintain a web-accessible data catalog application as part of the data platform focusing on backend services metadata management and integration with platform components.
• Integrate dataset discoverability preview/exploration options and lineage information using Unity Catalog as a backend metadata system.
• Design and implement structured access request workflows including request submission approval chains audit trail and enablement triggers.
• Perform design reviews with Cybersecurity.
• Ensure documentation and compliance for all interfaces and data ingress points.
• Manage audit and traceability requirements.
• Collaborate closely with IT and business users to translate requirements into scalable technical patterns.
• Serve as technical escalation point for complex source integration. 
• Define and implement a multi-layered data quality framework including unit-level integration-level and cross-pipeline validation rules.
• Establish centralized and version-controlled storage of DQ rules with integration into orchestration and CI/CD pipelines.
• Implement automatic DQ monitoring with severity levels (Critical High Medium Low) and enable flagging filtering and quarantining logic at relevant stages of the pipeline.
• Collaborate with source system owners and business stakeholders to define meaningful and actionable DQ thresholds.

Qualifications :

• Degree in Computer Science Data Engineering or a related field.
• 58 years in software development with hands-on experience with inbound- and outbound data exchange interfaces using a variety of technologies such as APIs CDC managing endpoints.
• Hands-on experience with Scala / Java Python and advanced SQL using Git for collaborative development and CI/CD methodologies.
• Experience building robust ingestion pipelines and ETL-jobs in a software engineering context.
• Track record with Azure or other hyperscaler resource management.
• Familiarity with data lake housing and data warehousing Databricks and Azure knowledge is a plus.
• Comfortable working across geographies and time zones; collaborates effectively with global teams and enterprise stakeholders.
• Candidates with a strong backend/software engineering background (e.g. Java Scala distributed systems) who want to move into data platform engineering are highly encouraged to apply.

NOT a fit if your experience is primarily:

Notebook-based data processing (e.g. Jupyter ad-hoc PySpark) without production deployment

Dashboarding / analytics / BI tools

SQL-only roles without software engineering experience

Additional Information :

The well-being of our employees is important to us. Thats why we offer exciting career prospects and support you in achieving a good work-life balance with additional benefits such as:

• Training opportunities
• Mobile and flexible working models
• Sabbaticals

and much more...

Sounds interesting for you Click here to find out more.

Diversity Inclusion & Belonging are important to us and make our company strong and successful. We offer equal opportunities to everyone - regardless of age gender nationality cultural background disability religion ideology or sexual orientation.

Ready to drive with Continental Take the first step and fill in the online application.

Remote Work :

No

Employment Type :

Full-time