£450 - £550 per day
Contract Spy
London, UK
Senior Data Engineer – Databricks & Big Data Pipelines (SC Cleared is required for this role)
Rate: £450-550 per day inside IR35
We're looking for a Senior Data Engineer – Databricks & Big Data Pipelines to join our team, working in our UK remote working division. This position requires active UK Security Check (SC) clearance and offers the opportunity to build advanced, large-scale data engineering solutions for an innovative analytics platform. You will design, develop and optimize distributed data pipelines on Databricks and cloud ecosystems while ensuring performance, scalability, and compliance. The role combines technical depth with client-facing delivery as part of an agile and collaborative environment.
Responsibilities
Design, develop and optimize end-to-end data pipelines using Databricks, Spark/PySpark and Python
Lead the full pipeline lifecycle: analysis, architecture, development,...