Job overview The Lead Data Engineer is a crucial role within the London Secure Data Environment (SDE) Programme, responsible for architecting and engineering solutions to support a new data and AI ecosystem across the London region. It is intended that these solutions will have direct and measurable impacts on both life sciences functions and data-driven population health pathways across London. As a senior technical lead, you will co-design the technical roadmap and deliverables. Our mode of working is focused on product design and delivery (rather than providing an on-demand/ticketed service to stakeholders). You may be responsible for supervising, coordinating, and upskilling more junior technical staff across multiple NHS organisations, including Trusts and Integrated Care Boards. Main duties of the job The Lead Clinical Data Analytics Engineer is a senior technical role that will: Design and lead on technical objectives for the SDE programme. Lead development of ELT/ETL and cohort creation pipelines for primary and secondary care data across London. Work with a dedicated London AI Centre team to deploy language AI technologies for extracting and standardising unstructured clinical records. Provide expert technical support to the London Data Service for the standardisation of London data into the OMOP common data model. Lead collaborations with NHS teams from Integrated Care Boards and Hospital Trusts to produce pan-London data analytics and visualisations. Co-ordinate, support, and upskill local analysts in cross-London collaborations to ensure alignment on projects and timelines. Lead collaborations with NHS teams to develop and deploy machine learning models, in areas such as risk stratification, case-finding, anticipatory care, decision support, and forecasting. Build responsible technical solutions for creation of decision intelligence from both descriptive and predictive analytics, including systems for monitoring, and for impact evaluation. Produce striking and visual reports that will influence regional healthcare policy and guide population health programmes. Working for our organisation The AI, Data & Digital Innovation team is made up of data science, technology and digital transformation experts, operating as part of the wider King Health partner’s (KHP) group – based in GSTT but working closely as a team with KCH and KCL. The team regularly contributes to externally funded programmes – including development and implementation of data platforms and AI technologies, aiming to achieve better outcomes for patients’ diagnosis and treatment, improve clinical and operational workflows, and drive innovative healthcare research. This includes the London Artificial Intelligence Centre for Value-Based Healthcare (London AI Centre): a consortium of NHS, academic and industry partners led by Guy’s & St Thomas’, King’s College London and KCH, in partnership with 10 NHS Trusts and 4 universities. Guy’s and St Thomas’ NHS Foundation Trust comprises five of the UK’s best known hospitals – Guy’s, St Thomas’, Evelina London Children’s Hospital, Royal Brompton and Harefield – as well as community services in Lambeth and Southwark, all with a long history of high quality care, clinical excellence, research and innovation. We have around 22,700 staff, making us one of the largest NHS Trusts in the country and one of the biggest employers locally. We aim to reflect the diversity of the communities we serve and continue to develop new and existing partnerships with local people, patients, neighbouring NHS organisations, local authorities and charitable bodies and GPs Detailed job description and main responsibilities Workstream 1: Hospital Trust Data Standardisation / Federation This workstream includes the ingestion and standardisation of legacy data, unstructured data (through language AI), and live data from EHR systems, into common data models, and building use-cases on top. This is primarily to support research, life sciences, and federated analytics; and secondarily to support internal hospital clinical analytics. This workstream also includes loading standardised EHR data and radiology metadata into FLIP. Over the course of the programme, the Lead Data Engineer will have the following responsibilities: Designing technical objectives for data ingestion and standardisation in hospital Trusts, in support of the London SDE programme. Our target Trusts are GSTT and KCH (hands-on), as well as UCLH and Imperial College Hospital (support), with possible extension to other sites depending on programme resource. Architecting, implementing, and maintaining infrastructure and ETL/ELT for data ingestion and standardisation into an NHS-compatible Common Data Model, into the OMOP Common Data Model, and into FLIP. Working with CogStack team to deploy language AI for extracting and standardising unstructured clinical records, and integrate these into the data ingestion pipeline (with model registration and data lineage tracking). Leading collaborations across NHS and commercial partners in the technical design and build of use-cases and products (including cohort discovery, self-service analytics, and predictive models) that may sit on top of the resulting common data model, or that may make use of FLIP capabilities. Workstream 2: London Population Health / MLOps This workstream includes working in Integrated Care Board and pan-London Snowflake/Azure data environments to deploy and maintain reproducible pipelines (from cohort creation to deployment) for analytics, data science, and machine learning. This role will help to pioneer how we modernise population health predictive analytics in the NHS, by introducing industry standard MLOps practices to monitor data, predictions, and impacts. The Lead Data Engineer will have the following responsibilities: Owning the building of SQL/Python pipelines (primarily in Snowflake and DBT) to extract data from different databases and raw sources, using the London terminology server to generate reproducible cohort features for analysis and machine learning. Working with NHS teams to develop and deploy machine learning models and MLOps for priority areas, which may include risk stratification, case-finding, anticipatory care, decision support, and forecasting. Working with clinical data science teams to ensure that solutions are impactful, safe, and fair, and translate responsible AI concepts into technical MLOps solutions. Working with clinical and AI ethics teams to develop responsible AI management frameworks for health organisations, including leading on the integration of technical frameworks for model management and change control. Helping to develop the skills of NHS analysts within the London SDE team, as well as within regional and local teams. Maintaining a central repository of reproducible code and machine learning models, based on a common data model shared across London regions. Leading engagement with technical teams across London and the NHS, including at events or conferences, to showcase technical solutions and promote horizontal scaling of responsible AI. Person specification Qualifications / Education Essential criteria Degree qualification in health informatics / computer or data science / health AI, or medical degree with equivalent experience Desirable criteria Medical degree or clinical experience Knowledge & experience Essential criteria Experience in technical roles in health data engineering and health data analytics Extensive technical understanding of NHS data, including from structured and multi-modal sources. Strong background in health informatics, including design of information solutions and data models Experience in data science and machine learning projects in the healthcare sector. Highly developed understanding of clinical care pathways and clinical pain-points. Able to communicate effectively with clinical teams. Experience of working in a product delivery setting, including ability to conduct end user-research, and set performance targets for evaluation. Strong track record of effective written and spoken communication, including in presentations, across all levels of seniority and technical ability. Experience working in large complex organisations, including in a health data or health technology environment. Proven track record and demonstrated ability of working collaboratively with a variety of organisational levels to deliver programme goals. Knowledge and understanding of risk management and mitigation strategies in technical projects. Knowledge of clinical and administrative IT systems and processes, as well as of administrative roles and processes within healthcare Desirable criteria Previous experience working in, or collaborating with, academia, in high impact and multi-institutional settings. Previous experience of working in, or collaborating with, commercial organisations, including in health-tech or life sciences. Previous experience or building or working with OMOP Common Data Model Knowledge & experience Skills & Ability Essential criteria Proficiency in SQL Proficiency in Python Ability to work in secure research environments for NHS data Exceptional analytical, problem-solving, and judgement skills. Understanding of reproducible code and CI/CD practices Advanced understanding and application of data science and statistical techniques to healthcare data. Strong leadership skills, with ability to manage and develop a diverse team Ability to manage a technical team using project methodologies such as agile Ability to work under pressure and to specified deadlines Ability to manage people and lead a number of work streams within the team Ability to manage a technical product lifecycle, including through end-user feedback and testing Applies excellent communication skills, both written and verbal, including advanced formal presentation skills, to audiences that may be highly specialist professionals, or non-technical, or may be hostile. In line with above, is able to demonstrate the ability to communicate the most complex information, or sensitive / contentious information, clearly, confidently, and concisely. Credibility to get others on board with the proposed approach, to engage different staff groups so that they are motivated and clear on what they need to do, and to maintain confidence through delivery challenges Gathers and incorporates the views of service users and carers into service change, including in circumstances where such change is likely to be unpopular or challenged by senior scrutiny or regulatory bodies. Ability to make clear rational decisions, exercising independent judgement in a professional and competent manner. Works flexibly, prioritises workload and resources in response to changing demands and requirements. Ability to plan, organise and present workshops to NHS staff across different organisations Knowledge & awareness of diversity and human rights as appropriate to role Ability to plan and organise own workload without direct supervision but to seek guidance as appropriate Ability to assimilate new information quickly and adapt to a flexible and rapidly changing work environment Desirable criteria Experience in R Skills in advanced data visualisation, presentation, frontend tools Proficiency using DBT +/- Snowflake +/- Azure Further details / informal visits contact Name Joe Zhang Job title Head of Data Science Email address [email protected]