Platform Architect

Anticipated Contract End Date/Length: Approximately 4.5 months
Work Set Up: Hybrid (60% office 40% remote)
Clearance Required: BPSS

Our client in the Information Technology and Services industry is looking for a Platform Architect to enhance and optimise an enterprise Data Platform through hands-on Azure Databricks architecture engineering operational support and platform optimisation. The role will focus on enabling Databricks Serverless capabilities strengthening FinOps practices and platform governance and enhancing the Databricks Discovery Zone to support migration and consolidation of POSIT/RStudio workloads.

What you will do:

• Assess existing workloads and select appropriate compute models including Serverless Jobs Compute Interactive Compute and Classic Clusters based on workload SLA performance utilisation and cost.
• Configure and implement Databricks Serverless capabilities across notebooks jobs SQL workloads analytical processing and data pipelines.
• Develop workload placement standards and guidance identifying scenarios where Serverless may not provide the most cost-effective solution.
• Implement compute policies autoscaling quotas budget controls and operational guardrails.
• Monitor platform performance and costs identifying oversized underutilised idle or inefficient resources.
• Define and implement a practical FinOps operating model covering ownership accountability environments projects applications cost centres and teams.
• Establish mandatory tagging standards and integrate automated validation into CI/CD pipelines.
• Deliver granular cost attribution and reporting across workspaces projects applications workloads and business teams.
• Configure budgets spend thresholds alerts and usage monitoring to proactively manage platform costs.
• Analyse usage and billing data to identify cost anomalies inefficient workloads excessive storage unnecessary data movement and underutilised resources.
• Enhance the Databricks Discovery Zone to support migration and modernisation of analytics and data science workloads currently running on POSIT/RStudio.
• Enable secure API and external system integrations external data ingestion BI and reporting connectivity scheduling and orchestration local IDE-based development and LLM and AI integration.
• Define reusable onboarding migration and delivery patterns that reduce technology sprawl while improving platform security supportability and delivery speed.
• Design build and optimise scalable data ingestion and transformation solutions using Python PySpark SQL and Delta Lake.
• Implement batch and incremental processing patterns including CDC schema evolution reconciliation data quality controls and error handling.
• Develop reusable integration frameworks for REST APIs SaaS platforms databases files object storage document repositories enterprise systems and external data sources.
• Implement secure authentication secrets management and credential handling practices.
• Deliver end-to-end data flows from source ingestion through governed and curated data layers supporting analytics BI machine learning and application consumption.
• Support production platforms and drive continuous optimisation across architecture engineering implementation and operational support.
• Document standards patterns operational procedures and architectural decisions for technical and business stakeholders.

Qualifications :

• Possess deep hands-on experience implementing administering and troubleshooting enterprise-scale Azure Databricks platforms in production.
• Have strong expertise in Databricks Serverless architecture workload placement compute optimisation Databricks SQL Delta Lake and query and workload performance optimisation.
• Have strong knowledge of Azure networking identity security monitoring secrets management and private connectivity.
• Possess advanced Python PySpark and SQL development skills within enterprise-scale data engineering environments.
• Have experience with jobs workflows orchestration incremental processing CDC data quality reconciliation and operational monitoring.
• Have experience implementing FinOps practices tagging strategies cost attribution monitoring budget management and cost optimisation.
• Have experience integrating REST APIs and external data sources within enterprise data platforms.
• Demonstrate strong understanding of platform governance security compliance operational support and controlled delivery within complex or regulated organisations.
• Collaborate effectively with data engineers data scientists architects security teams platform teams and business stakeholders.
• Demonstrate the ability to work across architecture engineering implementation optimisation and operational support without dependence on specialist teams.
• Communicate effectively and produce clear technical documentation standards operational procedures and architectural decisions.
• Have experience migrating analytics and data science workloads from POSIT/RStudio to Databricks; highly desirable.
• Have experience converting R-based workloads and libraries to modern data platform solutions; highly desirable.
• Possess AI/ML experience including LLM integration RAG/vector retrieval model serving and model lifecycle management; highly desirable.
• Have experience delivering large-scale platform transformation programmes; highly desirable.
• Have experience operating within highly regulated or complex enterprise environments; highly desirable.

Additional Information :

Candidates must be legally authorized to live and work in the country where the position is based without requiring employer sponsorship.

HelloKindred is committed to fair transparent and inclusive hiring practices. We assess candidates based on skills experience and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully only candidates selected for an interview will be contacted.

HelloKindred is an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race colour religion sex gender identity or expression sexual orientation age national origin disability veteran status or any other protected characteristic under applicable law.

Remote Work :

No

Employment Type :

Contract