Location
London - Shell Centre

Hours
Full Time - Regular

Salary
Negotiable

About the Role
Are you a Senior Data Engineer passionate about building scalable data solutions and turning data into business value? In this role, you'll work on data engineering projects within IT – T&S, delivering complex data management solutions by leveraging industry best practices. You will collaborate closely with project teams to design, build, and optimize efficient data pipelines and data platforms that enable the business to unlock insights and maximize the value of its data. If you thrive in solving complex data challenges and enjoy creating high-quality, reliable data solutions, then this role is for you.

You will be responsible for understanding the business vision for data solutions and working with all project stakeholders to enable that vision. Collaborating with multiple project team members, you will help business analysts create data mapping documents that define end-to-end pipelines. You will build and deliver data pipelines that ingest, transform, and load data (both structured and unstructured) from multiple sources into target data lakes or databases.

Additionally, you will build the data management layer in line with business needs and standards, triage and resolve pipeline or data layer issues based on priority, automate data loading during initial or incremental loads, and support the operations team during critical issues to facilitate faster resolution. You will leverage agentic workflows to generate data pipelines and perform deep query analysis and optimization using Azure SQL Databases.

This role offers the opportunity to work at the forefront of technology and collaborate with experienced colleagues in a values-led culture that encourages you to be the best version of yourself. Shell provides flexible working hours, the possibility of remote/mobile working, competitive salary and benefits, and a commitment to diversity and inclusion.

Experience
- IT industry experience with strong exposure to Big Data technologies.
- Proven ability to handle large volumes, velocity, and variety of data.
- Experience with infrastructure concepts and working across multiple cloud tenants.
- Proficient in data warehousing and traditional ETL technologies.
- Experience building cloud-native data pipelines with modularization, containerization, and unit test frameworks.
- Strong SQL skills including writing stored procedures, functions, and triggers.
- Extensive experience with the Azure ecosystem including Databricks, Delta Lake, Azure Data Factory (ADF), Blob Storage, and ADLS.
- Strong coding skills in Apache Spark, primarily PySpark, and Polars dataframe technology.
- Solid programming skills in Python.
- Proficient with agentic workflows using GitHub Copilot.
- Experience implementing complex data processing algorithms in real-time using Spark and other technologies.
- Experience working with both structured and unstructured data.
- Knowledge of data visualization and modeling tools.
- Excellent analytical and problem-solving skills with a willingness to take ownership and resolve technical challenges.
- Excellent communication and stakeholder management skills.
- Industry certification in Data Engineering is an advantage.
- Knowledge of latest database technologies such as MongoDB, Cosmos DB, Cassandra is a plus.

About you
You are passionate about Big Data technologies and appreciate the value effective data management solutions bring to business. You thrive in solving complex data challenges and enjoy creating high-quality, reliable data solutions. You are a strong communicator, able to collaborate effectively with diverse teams and stakeholders. You take ownership of your work and are motivated to continuously learn and grow.

Qualifications
Relevant industry certifications in Data Engineering or related fields are advantageous but not mandatory. Strong technical skills and proven experience in the required technologies and tools are essential.