Lead Data Engineer

ago

Location

Waterloo, London

Hybrid

Salary

Competitive

Employment Type

Contractor

Experience Level

Entry

Junior

Mid

Senior

Expert

Our Client

Global Energy Company

SPECIALTY

Upstream/Downstream, Bio-Fuels, Integrated Gas, New Energies, Chemicals, Energy and Trading

INDUSTRY

Energy

Company Size

80000+ Employees

Aubay's Take

Our client is one of the Super Major global energy companies with who are working to power progress through cleaner energy solutions. You will have the opportunity to work in a challenging but rewarding environment that is fast paced and changing fundamentally, and work towards transforming the business of a Super Major energy company to meet the ambition to be a net-zero emissions energy business by 2050, whilst delivering a world class business case that has a strong societal license to operate. In your role you will be expected to enact change and deliver value globally across business lines and geographies.

Image of the recruiter for this specific role

Have Questions?

Contact Prerita

Email - pvaswani@aubay.com

LinkedIn - Prerita's Profile

Role Summary

Aubay UK is seeking an experienced Principal AI Data Engineer to join a major transformation programme within the energy sector. This is a 6-month contract engagement (Inside IR35) with possible extension, focused on designing, developing, and deploying enterprise-scale GenAI and Agentic AI solutions. The successful candidate will combine advanced AI engineering expertise, Databricks capabilities, and stakeholder engagement skills to deliver innovative AI products from rapid prototype through to production.

Required Skills and Experience

  • Bachelor’s or Master’s degree in Statistics, Mathematics, Econometrics, Computer Science, or a related discipline.
  • 8-12+ years of experience delivering Data Science, AI, or Machine Learning solutions.
  • Deep understanding of LLMs including GPT, Llama, Claude, and Mistral.
  • Strong experience with Databricks, including Delta Lake, DLT, and Unity Catalog governance.
  • Experience with Spark Structured Streaming and Autoloader.
  • Strong programming skills in Python, SQL, and/or Scala.
  • Experience in data modelling, ETL/ELT processes, and enterprise data architectures.
  • Hands-on experience with Azure Cloud services.
  • Experience with Azure OpenAI, AI Foundry, Databricks Mosaic AI, MLflow, Agent Bricks, and Genie.
  • Strong understanding of GenAI evaluation frameworks, testing methodologies, and benchmarking.
  • Experience with Git-based development and collaborative coding practices.
  • Excellent communication, stakeholder management, and problem-solving abilities.

Desired Skills and Experience

  • Experience with LangChain, LangGraph, AutoGen, and smolagents.
  • Knowledge of Docker and CI/CD pipelines.
  • Experience with Azure Data Factory (ADF) and Qlik Replicate.
  • Familiarity with enterprise AI governance frameworks.
  • Experience leading delivery backlogs and shaping technical roadmaps.
  • Experience working within trading, supply chain, or energy environments.

Role Responsibilities

  • Develop and evaluate AI, GenAI, and Agentic AI prototypes using modern AI platforms and frameworks.
  • Build, optimise, and evaluate Retrieval-Augmented Generation (RAG) solutions.
  • Design and deploy AI agents and scalable AI workflows.
  • Deliver production-ready GenAI solutions used directly by commercial and business teams.
  • Manage and process large-scale structured and unstructured datasets using Databricks.
  • Implement testing, benchmarking, reproducibility, and model evaluation frameworks.
  • Deploy AI solutions using Azure AI services, Databricks infrastructure, and Docker.
  • Apply DevOps best practices including CI/CD, testing, version control, and GitHub workflows.
  • Collaborate with technical teams and business stakeholders to ensure alignment between business objectives and AI solutions.
  • Contribute to architectural design, sprint planning, backlog development, and long-term platform strategy.

This site uses cookies 🍪

We use cookies and similar technologies to give you a personalised browsing experience, personalised advertising and to analyse our web traffic. Click 'Accept' if you'd like to allow all cookies. Alternatively, you can choose which types of cookies you'd like to accept or disable by clicking 'Reject' below. For more information, please see our Cookie Policy.