Databricks Data Engineer
A leading trading firm is looking to bring on a strong data engineer with Databricks expertise. As markets become increasingly data‑driven and competitive, our ability to ingest, process, and deliver high‑quality data at scale is essential. In this role, you will own and operate a business‑critical Databricks platform used across our global trading organization.
You'll drive platform standards, introduce new capabilities, and help shape the long‑term technical direction of Databricks. This is a high‑impact engineering position with significant autonomy, strong cross‑team visibility, and the opportunity to directly influence how our research and trading teams consume data.
Core Focus
- Design, build, and operate Databricks workspaces and multi‑environment deployments at scale
- Manage workspace configuration, cluster policies, job settings, access controls, and platform governance
- Lead the rollout of new platform features (e.g., Unity Catalog, UC Volumes, cluster policy enhancements, security updates)
- Establish and enforce standards for reliability, security, data governance, and cost controls
- Continuously improve the platform's performance, scalability, automation, and developer experience
- Build and maintain Databricks and cloud infrastructure using Terraform, including reusable, version‑controlled modules
- Create automated operational tooling in Python and shell for workspace, cluster, and job lifecycle management
- Develop repeatable deployment patterns across dev/test/prod environments following IaC best practices
- Design and maintain CI/CD pipelines for infrastructure and Databricks configuration
- Implement automated checks, policy validations, and safe promotion of changes across environments
- Support Git‑based workflows, peer review processes, and high‑governance deployment workflows
- Ensure that all platform changes meet standards for reliability, auditability, and risk tolerance
- Troubleshoot platform‑level issues across compute, jobs, networking, access, and integration points
- Improve observability, metrics, alerting, and operational dashboards for Databricks
- Manage core system configuration, cloud networking, IAM, and access boundary controls
- Partner with security, infrastructure, research, and trading teams to provide a stable, performant platform
- Implement and manage RBAC, SCIM/IdP integration, secrets management, and zero‑trust principles
- Ensure platform compliance with internal controls, operational standards, and audit requirements
- Participate in incident response, post‑mortems, and reliability improvement initiatives
- Balance rapid delivery withcarefulgovernance-this role interacts with systems where incorrect changes can have downstream trading impact
- Strong understanding of risk tolerance, peer review requirements, and deployment safeguards is essential
- You will directly own a mission‑critical platform central to research, analytics, and trading operations
- Opportunity to shape the platform roadmap, introduce new Databricks capabilities, and drive foundational architectural decisions
- Growth path into broader platform leadership, technical ownership, or architect‑level responsibilities
- Work in a world‑class engineering environment with exceptionally strong peers, rigorous technical standards, and support for continuous improvement
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your resume and details on file so when we see similar roles or see skillsets that drive growth in organizations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your resume so you can be considered for roles that have yet to be created.
Yes, we help with resume and interview preparation. From customized support on how to optimize your resume to interview preparation and compensation negotiations, we advocate for you throughout your next career move.