Data Platform Engineer

The Role

We are hiring a Senior Data Platform Engineer to help build and scale a cloud-based data platform in a highly regulated environment.

The organisation is partway through its migration to Azure, with production workloads already in place. The focus now is not strategy or architecture it's execution: building reliable, production-grade data platforms that will underpin future data and AI capabilities.

This role is for engineers who have actually built platforms within azure databricks, not just contributed to them or designed them at a high level.

You will be working in a low-risk environment, where data security, governance, and cost control are critical. The expectation is that you can deliver within those constraints, not work around them.

Responsibilities

* Building and operating Azure Databricks-based data platforms used in production
* Developing end-to-end data pipelines using Python and SQL
* Configuring and optimising Databricks environments, including:

* Cluster setup, tuning, and troubleshooting
* Job orchestration and scheduling
* Managing performance vs cost trade-offs

* Implementing data transformation and modelling workflows (dbt or similar)
* Owning platform components from initial build through to production support
* Ensuring data platforms meet strict security, governance, and risk requirements
* Contributing to ongoing cloud migration and platform modernisation work

Required Experience

* Strong, hands-on experience building Azure Databricks data platforms from scratch
* Proven track record delivering and supporting production-grade data pipelines
* Solid experience with:

* Azure
* Azure Databricks
* Python
* SQL

* Experience working with data at scale (not just small or isolated datasets)
* Background in regulated or enterprise environments with governance constraints
* Comfortable working in cost-sensitive environments where efficiency matters

Nice to Have

* Hands-on experience with dbt in production
* Experience in financial services or similar regulated industries
* Exposure to cloud migration programmes
* Experience optimising Databricks for performance and cost

Why This Role

* Opportunity to build real, production data platforms, not just prototypes
* Direct impact on a platform that will enable future AI and data capabilities
* Work in an environment where engineering quality, security, and reliability actually matter
* Join a team that values delivery over theory