Weather Data Engineer
Weather Data Engineer - AI-Driven Forecasting & Analytics
Location (On-Site): NYC or London
We're seeking a highly skilled and experienced Weather Data Engineer to help build a proprietary, differentiated weather analytics and infrastructure platform. Weather data and AI-driven modeling are central to our strategy to deliver a forecasting edge and drive Sharpe improvement across our trading business.
As a key member of the team, you'll design, implement, and maintain cloud-native (AWS) data pipelines and infrastructure for ingesting, processing, and serving real-time and historical weather data-including raw observations, climate simulation data, and AI model outputs. You'll play a critical role in supporting AI efforts across our Commodity teams, enabling the development and deployment of proprietary, decorrelated forecasting models and advanced analytics.
You'll collaborate closely with meteorologists, data scientists, technologists, and AI researchers to integrate new weather datasets, support signal postprocessing and debiasing, and drive innovation in weather analytics.
Responsibilities:
- Build scalable, cloud-native (AWS) data pipelines for ingesting and processing weather data (e.g., satellite, radar, sensor networks, CMIP6).
- Develop APIs and data services to support analytics, modeling, and visualization.
- Support centralization and optimization of AI model infrastructure (e.g., blending, debiasing, finetuning).
- Collaborate with cross-functional teams to onboard and optimize new datasets and support research initiatives.
- Implement automated data validation, monitoring, and alerting for reliability and uptime.
- Continuously improve infrastructure to accelerate analytics and enhance operational scale.
- Promote best practices in collaborative software development (CI/CD, testing, code review).
- Maintain clear documentation and foster knowledge sharing.
Requirements:
Essential:
- Degree in Computer Science, Atmospheric Science, Engineering, or related field.
- 5+ years of experience building and supporting production data systems.
- Strong Python skills and familiarity with multiple programming paradigms.
- Deep understanding of system architecture and full technology stack.
- Experience with collaborative development workflows and tools.
- Proficiency with databases (e.g., Postgres, Redshift, Snowflake).
- Solid grasp of time-series and geospatial data.
- Experience with Linux and scripting.
- Familiarity with weather/climate datasets (GRIB, NetCDF, HDF5, CSV, JSON).
- Knowledge of weather data sources (NOAA, ECMWF, GFS, satellite, radar).
Beneficial:
- Proficiency in OO languages (e.g., Java, C#).
- Experience with distributed computing (Spark, Dask, Slurm).
- Familiarity with event-driven architectures (Kafka, RabbitMQ).
- Cloud platform experience (AWS, GCP, Azure).
- Orchestration and containerization (Airflow, Kubernetes, Docker).
- Monitoring and alerting tools (CloudWatch, Prometheus, Grafana).
- Background in weather modeling or analytics workflows.
- Experience supporting research analytics and dashboarding.
If you're passionate about building world-class infrastructure for weather analytics and want to work at the frontier of data engineering, meteorology, and AI, we'd love to hear from you.
*No contractors*
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your CV and details on file so when we see similar roles or see skillsets that drive growth in organisations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your CV so you can be considered for roles that have yet to be created.
Yes, we help with CV and interview preparation. From customised support on how to optimise your CV to interview preparation and compensation negotiations, we advocate for you throughout your next career move.