Job Description – Data Engineer
Data Engineer (Azure | Databricks | PySpark | SQL)
Contract | Remote | UK Public Sector Project
Overview
:We are seeking a Data Engineer to support delivery of key data components within our evolving data ecosystem. This role will work closely with architects, analysts, engineers, and wider delivery teams to implement pipeline builds, data models, and integration patterns aligned to the organisation’s new Target Operating Model for Data
.You will contribute to the development of scalable data flows, optimise ingestion & transformation activities, and ensure solutions meet technical standards, performance expectations, and security controls. This is a hands-on engineering role ideal for someone who enjoys building, shaping, and improving modern data platforms
.
Who We’re Looking Fo
r:A highly capable Data Engineer with hands-on experience building ETL pipelines, implementing Data Lake architectures, and working across Azure platform services. You will need strong problem-solving ability, a solid engineering mindset, and the ability to work collaboratively within a multi-disciplinary tea
m.If you’re confident writing production-grade code, applying design patterns, and building scalable, high-performing data products, this role offers technical autonomy, ownership, and challeng
e.
Key Responsibiliti
es:Build and maintain ETL pipelines and code in cloud data platfor
ms.Support ingestion activity and onboarding of new data sourc
es.Work with the Databricks Data Intelligence Platform and develop Data Engineering workloa
ds.Construct raw, refined and curated data layers; catalogue assets appropriate
ly.Validate solutions against functional and non-functional requiremen
ts.Deliver datasets, transformations and performance-optimised data produc
ts.Improve processes, engineering patterns, and reusable tooli
ng.Monitor and measure pipeline performance; support incident resoluti
on.Ensure documentation meets acceptance standards and is approved central
ly.Actively engage in Agile ceremonies and governance foru
ms.
Mandatory Requireme
nts:Strong experience with Python, PySpark & SQL for data engineer
ing.Hands-on experience with Azure Databri
cks.Strong knowledge of Azure Data Factory & Fab
ric.Experience with Microsoft Purv
iew.Familiarity with Event-driven data ingestion (Event Grid / Pub-S
ub).Understanding of SOLID principles, Async programming, Mediator/Factory patte
rns.Experience delivering unit + integration testing in Databri
cks.Knowledge of Secure ETL design with Entra IMID/SCIM integrat
ion.Understanding of Azure best practice, APIM, and platform governa
nce.Ability to build and serve Power BI models via Databricks data sour
ces.
Desir
able:Prior experience working within UK Public Sector environm
ents.
Soft S
kills:Strong stakeholder communication and cross-team collabor
ation.Analytical and solution-focused mi
ndset.Able to work independently, take ownership and drive pro
gress.Commitment to clean, scalable, well-documented engine
ering.Adaptable, proactive, and comfortable working in dynamic delivery environ
ments.
Contract D
etails:Contract Type: Freelance / C
ontractLocation: Remote (UK-based candidates pre
ferred)Start Dat
e: ASAPClearance: Candidates must be eligible to work with UK Government departments
/ BPSS.
Discovering Direct IT Contract Opportunities for Contract Spy members.