Data Pipeline Engineer Home-Based

• Data Pipeline Engineer / Data Pipeline Architect
• 3-month contract (initially)
• Remote Working (UK-based)
• Market Rates
• Outside IR35

We are supporting a central government client who are seeking an experienced Data Pipeline Engineer and Architect to help design and build an open-source Data Transformation Layer for a national data exchange platform.

This is a hands-on engineering and architecture role combining the design of reusable reliable data pipelines with practical development using Apache NiFi RDF SPARQL Python and Java.

The platform will use open-source cloud-agnostic technologies to ingest transform validate and exchange data securely between systems.

• Proven experience designing and delivering data integration or ETL solutions using Apache NiFi.
• Practical project experience with RDF triples SPARQL and semantic data modelling.
• Hands-on development experience in both Python and Java.
• Experience designing reusable pipelines across APIs databases files and other data sources.
• Understanding of reliable data processing validation automated testing and operational monitoring.
• Experience with version control CI/CD secure integrations and repeatable deployments.
• Ability to translate high-level requirements into technical designs and production-ready solutions.

Desirable Experience

• Apache Jena and Jena Fuseki.
• Apache Kafka and event-driven data integration.
• SHACL or other RDF validation approaches.
• Custom NiFi processor development.
• Docker Kubernetes and portable deployment architectures.
• Large RDF datasets SPARQL optimisation or secure data exchange between organisations.

Apache NiFi RDF SPARQL Python Java Apache Jena Fuseki Kafka ETL Data Integration Open Source

• Design and evolve the architecture of a reusable cloud-agnostic data transformation platform.
• Build test and maintain Apache NiFi flows including integrations with APIs databases files and messaging systems.
• Develop pipelines to ingest transform validate and publish RDF triples using SPARQL for querying and updates.
• Develop supporting components and integrations using Python and Java.
• Define data mappings interface contracts technical specifications and architectural decisions.
• Implement robust processing including error handling retries duplicate management and recovery.
• Establish automated testing monitoring version control and repeatable deployment practices.
• Collaborate with architects developers and stakeholders to translate requirements into working solutions.
• Produce clear technical documentation and support the delivery of reusable platform capabilities.

NB: Candidates must be eligible for BPSS security clearance which will processed following successful interviews (5-7 days).

To learn more about this newly created opportunity consulting as a Data Pipeline Engineer / Data Pipeline Architect on an initial 3-month engagement operating on a remote-working basis for a prominent UK Government Agency; please call and speak with Edward Laing here at ISR Recruitment on 0or please send through a copy of your latest online profile and/or CV () for an immediate call back in the strictest confidence.

Please contact Edward here at ISR Recruitment to learn more about our client and how they are leading the way in developing the next generation of technical solutions through innovation and transformational technology