Senior Data Engineer
Key Responsibilities
- Design, develop, and support our Big Data platforms, including Kafka, Hadoop, Dremio, and related technologies.
- Build, deploy, and oversee robust data processing pipelines using Java, Python, Spark, and Flink.
- Collaborate with development teams on data modeling, ingestion approaches, and capacity planning.
- Work closely with users to maintain data accuracy, consistency, and availability across systems.
- Act as a Big Data subject-matter expert, providing guidance on technical questions and best practices.
Required Skills & Experience
- 5+ years of experience in a mature data engineering environment.
- 3+ years of hands‑on experience developing Kafka streaming applications or managing Kafka clusters.
- 2+ years building applications or pipelines on Big Data architectures (S3, HDFS, Databricks, Iceberg, etc.).
- Proficiency with Apache Spark, Apache Flink, or comparable data processing tools.
- Strong programming capabilities in Java, Python, and SQL.
- Experience with widely used Python-based data science libraries.
- Practical knowledge of Kubernetes and Docker.
- Familiarity with monitoring tools such as Prometheus/Grafana, AlertManager, Alerta, and OpsGenie.
- Strong background in statistical analysis.
- Demonstrated ability to troubleshoot and perform root‑cause analysis.
- Experience writing Unix scripts (bash, Python, etc.).
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your CV and details on file so when we see similar roles or see skillsets that drive growth in organisations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your CV so you can be considered for roles that have yet to be created.
Yes, we help with CV and interview preparation. From customised support on how to optimise your CV to interview preparation and compensation negotiations, we advocate for you throughout your next career move.