Senior Data Engineer (Trading space)
9-month contract
ago
Location
Gulf Region, London
Hybrid
Salary
Inside IR35
Employment Type
Contractor
Experience Level
Entry
Junior
Mid
Senior
Expert
Technologies Used
Databricks
Azure
Python
SQL
Our Client
International Energy Company
SPECIALTY
Oil, Gas, LNG, Shipping
INDUSTRY
Energy
Company Size
10000+ Employees
Aubay's Take
Our client is an international energy company who covers the full spectrum of the oil and gas value chain. Our client is dedicated to attracting and hiring the highest calibre employees, from all around the world. As they scale their operations, they are committed to operating in a socially responsible manner, supporting, nurturing, and empowering their people, investing their resources to responsibly and safely contribute to the communities of which they are a part.
Role Summary
Aubay is seeking an experienced Senior Data Engineer with great interpersonal skills to join our team. Our client has been built on a foundation of innovative technology and skilled people. By providing energy to sustain people's lives over many years, we have become one of the world's leading companies. The IT Engineering team offers you the chance to make an impact that matters – be it in complex predictive maintenance and inventory management solutions to optimise the uptime of our assets or innovative solutions that satisfy customers worldwide. Your growth is our future in IT Engineering. Information technology supports all aspects of our businesses, and we have some of the world's most complex problems to solve. You will not only have the opportunity to work with the latest technology, but with our scale, you will also have the chance to shape the future of digital technology by working with industry leaders. With open-source technology being important to us, you will also have the opportunity to contribute back to the software community. The diversity of positions on offer will help you grow in your career. The Senior Data Engineer will play a critical role in designing, building, and maintaining scalable data pipelines and analytics solutions using Azure and Databricks ecosystem. Collaborate closely with business stakeholders to ensure the data is accessible, reliable, and optimised for performance.
Key Accountabilities:
• Design and implement robust data pipelines using Azure Data Factory (ADF) and Databricks.
• Develop scalable ETL/ELT processes to ingest, transform, and load data from various sources.
• Write clean, efficient, and reusable Python code for data processing and automation.
• Optimise and manage SQL queries for data extraction, transformation, and reporting.
• Collaborate with business stakeholders to understand data requirements and deliver solutions aligned with
business goals.
• Ensure data quality, integrity, and security across all data platforms.
• Monitor and troubleshoot data pipelines to ensure high availability and performance.
• Implement logging, alerting, and error-handling mechanisms for proactive issue resolution
• Apply domain knowledge of trading and financial markets to enhance data models and analytics capabilities.
Required Skills and Experience
• 5+ years of experience in data engineering or related roles.
• Proven expertise in Azure Data Factory and Azure Databricks.
• Strong proficiency in Python for data engineering tasks.
• Solid understanding of SQL and relational databases.
• Exposure to real-time data processing frameworks (e.g., Spark Streaming, Kafka).
• Experience in maintaining and supporting production-grade data pipelines.
• Familiarity with trading business domain, including market data, trade lifecycle, and financial instruments.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration abilities