Weather Data Engineer

Weather Data Engineer - AI-Driven Forecasting & Analytics

Location (On-Site): NYC or London

We're seeking a highly skilled and experienced Weather Data Engineer to help build a proprietary, differentiated weather analytics and infrastructure platform. Weather data and AI-driven modeling are central to our strategy to deliver a forecasting edge and drive Sharpe improvement across our trading business.

As a key member of the team, you'll design, implement, and maintain cloud-native (AWS) data pipelines and infrastructure for ingesting, processing, and serving real-time and historical weather data-including raw observations, climate simulation data, and AI model outputs. You'll play a critical role in supporting AI efforts across our Commodity teams, enabling the development and deployment of proprietary, decorrelated forecasting models and advanced analytics.

You'll collaborate closely with meteorologists, data scientists, technologists, and AI researchers to integrate new weather datasets, support signal postprocessing and debiasing, and drive innovation in weather analytics.

Responsibilities:

• Build scalable, cloud-native (AWS) data pipelines for ingesting and processing weather data (e.g., satellite, radar, sensor networks, CMIP6).
• Develop APIs and data services to support analytics, modeling, and visualization.
• Support centralization and optimization of AI model infrastructure (e.g., blending, debiasing, finetuning).
• Collaborate with cross-functional teams to onboard and optimize new datasets and support research initiatives.
• Implement automated data validation, monitoring, and alerting for reliability and uptime.
• Continuously improve infrastructure to accelerate analytics and enhance operational scale.
• Promote best practices in collaborative software development (CI/CD, testing, code review).
• Maintain clear documentation and foster knowledge sharing.

Requirements:

Essential:

• Degree in Computer Science, Atmospheric Science, Engineering, or related field.
• 5+ years of experience building and supporting production data systems.
• Strong Python skills and familiarity with multiple programming paradigms.
• Deep understanding of system architecture and full technology stack.
• Experience with collaborative development workflows and tools.
• Proficiency with databases (e.g., Postgres, Redshift, Snowflake).
• Solid grasp of time-series and geospatial data.
• Experience with Linux and scripting.
• Familiarity with weather/climate datasets (GRIB, NetCDF, HDF5, CSV, JSON).
• Knowledge of weather data sources (NOAA, ECMWF, GFS, satellite, radar).

Beneficial:

• Proficiency in OO languages (e.g., Java, C#).
• Experience with distributed computing (Spark, Dask, Slurm).
• Familiarity with event-driven architectures (Kafka, RabbitMQ).
• Cloud platform experience (AWS, GCP, Azure).
• Orchestration and containerization (Airflow, Kubernetes, Docker).
• Monitoring and alerting tools (CloudWatch, Prometheus, Grafana).
• Background in weather modeling or analytics workflows.
• Experience supporting research analytics and dashboarding.

If you're passionate about building world-class infrastructure for weather analytics and want to work at the frontier of data engineering, meteorology, and AI, we'd love to hear from you.

*No contractors*