Senior Data Engineer

Maybe you call yourself a Senior Data Engineer a pipeline architect a why-is-this-TOS-data-a-mess specialist or maybe you just call yourself Sam. Whatever you call yourself youll join All Your BIs AI-native data platform for container terminal operators and build the core of it: the pipelines and data model that turn fragmented hard-to-trust terminal data into something operators run their day on.

• A salary between per year inclusive of holiday allowance based on your experience.

• 5000 a year for your own growth with a real say in how you spend it.

• 25 vacation days the option to buy up to 10 more local public holidays the option of a sabbatical up to 4 months.

• Two trips abroad a year with the whole team occasional drinks and team outings.

• 500 home-office budget and a great laptop.

• A well-earned Hey well done Buddy! so now and then.

• Fully remote within Europe.

Benefits across the EU: the perks above are AYBIs and go to everyone everywhere. Statutory benefits (pension health social security leave sick pay notice) follow the hires country via their local contract or EOR.

All Your BI is a remote-first data agency based in Rotterdam. Since 2019 weve grown from a boutique consultancy into an international data team of more than 60 data heroes spread across Europe. We deliver managed BI services for logistics and industrial companies helping reduce data waste and move businesses forward. We live for the WOW moment that sparkle in users eyes when they gain new insights and discover opportunities. We work according to Holacracy: little hierarchy lots of ownership. Serious work. Serious fun.

is All Your BIs SaaS platform for container terminals. It unifies fragmented data TOS maintenance gate sensors into one governed model so every site works from the same numbers instead of rebuilding its own reporting from scratch. The platform is growing fast. Youll help shape the engineering foundations that need to scale next: multi-tenant data architecture near-real-time pipelines from 24/7 operations and a standardized data model that works across terminals worldwide.

You will design and build the pipelines that move data from terminal operating systems (such as Navis N4) into our platform and the transformation layer that turns raw operational data into curated documented datasets.

Concretely you will work on:

• ELT pipelines from on-premises terminal databases into our Azure-based platform with near-real-time refresh (3-minute microbatch) for operational reporting and daily batch for analytics. Extraction uses watermark-based incremental queries over Kafka (Azure Event Hubs) to PostgreSQL staging.

• A multi-tenant data architecture on Azure PostgreSQL with database-per-tenant isolation and a standardized STG ODS DataMart layer structure driven by our internal declarative transformation framework (dewi)

• Our internal connector tooling written in Go which you will help develop and extend

• Data quality monitoring: gap detection freshness alerting and reconciliation

• The standardized terminal data model: mapping source data from different TOS versions and configurations onto shared definitions of moves calls and equipment events

You will work directly with terminal operations staff without intermediate layers.

• Attitude and experience:curiosity and a genuine appetite for architectural ownership and 5 years of data engineering experience including end-to-end ownership of production data platforms. This is a product under active development we need you comfortable deciding not just executing.

• Core stack: strong SQL and Python; Go experience is a significant plus (our connectors and consumers are written in Go). Solid experience with SQL Server as a source system and PostgreSQL as a target store.

• Pipelines and streaming:experience building and operating ELT pipelines against operational source systems including watermark/incremental or CDC/replica-based extraction and with event-driven or streaming architectures we use Azure Event Hubs with a Kafka-compatible endpoint.

• Cloud and observability:cloud data platform experience on Azure familiarity with AKS Azure DevOps CI/CD and GitOps (Argo CD) is a plus alongside a strong focus on data correctness and instrumentation we use OpenTelemetry.

• Nice to have:knowledge of the container terminal or port logistics industry direct experience with Navis N4 or another TOS multi-tenant SaaS data architecture experience and familiarity with declarative YAML-driven transformation frameworks. Industry knowledge is a strong plus not a hard requirement well teach you the terminal domain.
  

Were a growing team across Europe and we hire to widen that mix not narrow it. What matters is what you can do and the perspective you bring not your age gender ethnicity orientation disability background or where you studied. If the role excites you and you bring most of it apply: the people who add the most rarely tick every box.