Overview

We’re seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform. This permanent position focuses on building architecture for long-term evolution, not just migration.Responsibilities

Architect solutions using modern data engineering patterns to enable long-term evolution of the platform.Establish data pipelines and CI/CD practices; build and maintain scalable, adaptable systems.Mentor team members and collaborate with product analysts, engineers, and stakeholders to align data capabilities with business objectives.Lead migration efforts (initial focus: 6 months) and drive platform evolution and capability expansion.Contribute to engineering best practices for data governance, operations, and observability.What You''ll Do

Architect a modern data platform for long-term business evolution.Build robust batch and real-time processing pipelines, integrating data from diverse sources (web app database, external APIs, analytics, CRM, finance, observability tools) into a unified lakehouse.Create comprehensive documentation and training materials.Translate business requirements into maintainable technical solutions.Requirements and Technical Qualifications

Azure Data Platform Architecture

— Microsoft Azure data stack expertise (including self-hosting non-Microsoft solutions).Database Experience

— PostgreSQL and SQL Server, with data replication/integration patterns between them.Data Modeling

— Dimensional and relational data modeling expertise.Data Engineering and Pipelines

— Python and SQL for ELT/ETL; batch, streaming, and micro-batching architectures; CDC patterns and incremental load strategies at scale; experience ingesting data from diverse external APIs to consolidate data into a unified lakehouse.Infrastructure and Networking

— Azure networking and security best practices for enterprise data platforms; experience with event-driven architectures and API integration patterns.Governance, Compliance and Security

— Understanding of information security requirements; experience with Microsoft Purview or similar governance frameworks; GDPR/data protection implementation including right-to-be-forgotten; RBAC, encryption, audit logging; data lineage and quality enforcement.DevOps and Quality

— Test automation for data pipelines (unit, integration, regression); monitoring of pipeline runs and errors; performance tuning and cost optimization; experience with Git source control.Non-Technical

— Strong communication and documentation skills; technical writing with examples; ability to present to both technical and non-technical audiences; create architecture diagrams and data flows, runbooks.Why This Matters

You’ll establish the foundation for our data platform’s next decade. The platform will power internal and customer products and enable real-time insights for faster, smarter decision-making.Team and Location

Reports to: Product organisation – alignment of data capabilities with business objectivesLocation: London office 3 days per week; remote work the rest of the timeTeam: Senior technical lead and engineer for data platform initiatives, collaborating with business analysts, engineers, and product stakeholdersTimeline: Initial migration focus (6 months), followed by platform evolution and capability expansionEmployment details

Seniority level: Mid-Senior levelEmployment type: Full-timeJob function: Information TechnologyIndustries: Software Development

#J-18808-Ljbffr