Lead Data Engineer
A top-tier trading firm is looking for an experienced Data Engineer to strengthen the data stack supporting its Chicago research group and help evolve a modern Lakehouse-style architecture. You'll work with tools like Databricks, Spark, and Delta Lake to turn experimental research workflows into stable, scalable production systems.
As a key engineering presence in Chicago, you'll shape local data strategy while partnering with global research and platform teams. Your work will power real-time analysis, predictive modeling, and critical decisions across the trading lifecycle.
What You'll Do
- Build and maintain reliable ETL/ELT pipelines using Spark, Structured Streaming, Databricks, and internal high‑performance tooling
- Convert research prototypes into durable, high‑performance production workflows
- Create reusable templates, patterns, and documentation for power users
- Define and champion data engineering best practices
- Mentor junior engineers and promote strong DataOps habits
What You'll Get
- Direct influence on data systems that drive elite research and real-time trading
- A central role in shaping U.S. Lakehouse engineering and contributing to global architecture
- Ownership over complex workflows and cross-team templates
- Close collaboration with quants and traders to unlock data-driven insights
- Partnership with exceptional engineers across Chicago, Amsterdam, and Sydney
Additional benefits include:
- Working with colleagues from 40+ nationalities
- 401(k) with 50% employer match
- Comprehensive medical, dental, vision, mental health, disability, and life coverage
- 25 vacation days plus market holidays
- Office perks including daily meals, snacks, events, clubs, and sports leagues
Who You Are
- 5+ years of experience delivering scalable data pipelines
- Expert in Python, Spark, and Databricks
- Familiar with Delta Lake, streaming platforms like Kafka, and distributed compute
- Comfortable with cloud-native data design (preferably AWS) and cost-aware architectures
- Proficient with relational databases and modern orchestration tools
- Bonus: experience in C++ or Rust, and familiarity with MLOps or MLflow
- Strong communicator who leads projects independently and mentors others
- Bachelor's or Master's in Computer Science, Engineering, or a related field
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your CV and details on file so when we see similar roles or see skillsets that drive growth in organisations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your CV so you can be considered for roles that have yet to be created.
Yes, we help with CV and interview preparation. From customised support on how to optimise your CV to interview preparation and compensation negotiations, we advocate for you throughout your next career move.