Location
London

Hours
Flexible hybrid working model balancing in-person and remote work. Standard one day per week remote working, plus 44 additional remote working days per year. Core in-person hours from 10am until 4pm.

Salary
Competitive salary package

About the Role
The Data Engineer sits within the Data Innovation team and works day-to-day on the CF lakehouse, our Databricks platform that holds routine healthcare data behind our analytical and product work. This hands-on, delivery-focused role involves writing SQL and Python, building and maintaining automated pipelines, and transforming raw data into structured, production-ready material that consultants and clients rely on. You will learn actively from senior engineers, contribute to the team’s collective knowledge, and take on greater scope as you grow. The work includes data cleansing and validation for client engagements, pipeline development and maintenance, and contributing to CF’s technical products through hackathons. This role is ideal for an engineer with a couple of years of experience seeking to build a broad technical foundation in a data-rich healthcare consultancy, work close to real client problems, and see how technical work delivers client value.

Responsibilities
- Build and maintain automated data pipelines using PySpark and Spark with appropriate monitoring
- Develop data models and pipelines under senior guidance, producing production-ready or client-ready code
- Implement data quality, validation and consistency checks, and perform data cleansing
- Write unit, functional and integration tests, following Git and GitHub workflows
- Participate in agile processes including updating user stories, stand-ups, retrospectives and demos
- Flag deviations early and help identify and deliver mitigations
- Build understanding of healthcare data and the CF lakehouse structure
- Collaborate with teams to bring analytical insights to client problems
- Communicate technical solutions effectively to non-technical colleagues
- Learn best-practice development processes and seek new tools and techniques
- Use AI tools to improve code quality and efficiency
- Support business development, bid writing, product development and thought leadership activities

Experience
- Around 2 to 3 years in data engineering or a comparable technical data role
- Proficient in SQL, used daily on the lakehouse
- Intermediate Python for data transformation and automation
- Experience building automated data pipelines using PySpark or Spark with monitoring
- Ability to write unit, functional and integration tests
- Comfortable working with Git and GitHub workflows
- Curiosity about healthcare data and its relation to care delivery and policy
- Clear communicator who knows when to ask for support and works well in a team

About You
Motivated to build a broad technical foundation in healthcare data engineering, eager to learn from senior colleagues, and keen to contribute to real client problems. You thrive in a collaborative environment and communicate effectively across technical and non-technical teams.

Desirable (Not expected on day one)
- Experience with Databricks lakehouse: notebooks, Workflows and Jobs, Delta Lake
- Basic knowledge of Unity Catalog (training provided)
- Awareness of cloud basics such as storage, access control, and medallion architecture (bronze to silver to gold)
- Exposure to dbt and basic dimensional modelling

Qualifications
No formal qualifications specified; emphasis on relevant experience, technical skills, and a willingness to learn.