Lead Databricks Engineer - Single Customer View
Lead Databricks Engineer - Single Customer View Location: Bournemouth (Hybrid) Contract: 12-month FTC Salary: £85,000 + BenefitsLead Databricks Engineer: About the RoleJoin our Single Customer View (SCV) Programme, a strategic initiative within Financial Services aimed at delivering a unified, trusted view of customer data.We''re seeking a highly skilled Lead Databricks Engineer to design and implement scalable data pipelines that form the backbone of our Lakehouse platform, enabling accurate analytics, reporting, and regulatory compliance.You''ll work with cutting-edge technologies including Databricks, PySpark, and Azure Data Factory, applying best practices in data engineering and governance to support this critical programme.Lead Databricks Engineer: Key ResponsibilitiesBuild and maintain Databricks pipelines (batch and incremental) using PySpark and SQL.Orchestrate end-to-end workflows with Azure Data Factory.Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming).Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets.Apply robust monitoring, logging, and error-handling frameworks.Integrate pipelines with downstream systems such as Power BI.Collaborate with analysts, business teams, and engineers to deliver consistent, well-documented datasets.Support deployments and automation via Azure DevOps CI/CD.Gather and refine requirements from business stakeholders.Lead Databricks Engineer: About
Perform a fresh search...
-
Create your ideal job search criteria by
completing our quick and simple form and
receive daily job alerts tailored to you!