Skip to content
windborne

Initial thoughts

Learning the data first

Section titled “Learning the data first”
  • Read the paper and the weather basics before writing code.
  • The sample is the atmosphere at 7 March 2025, 00:00 UTC, on a 0.25 degree grid (721 x 1,440): geopotential, temperature, two wind components, and specific humidity per level.
  • Two inputs for the same atmosphere: neogfs (GFS, 25 levels) and neohres (HRES, 20 levels), both mapped to the model’s 28 internal levels.

Approach

Section titled “Approach”
  • Tools: uv for Python, Claude Code for implementation.
  • Rebuild the model inputs from the sample, confirm the fields look like weather, and get a six-hour forecast from the released weights.
  • Inference runs on a CUDA GPU on AWS. Data notebooks are in the PR, https://github.com/windborne/WeatherMesh-3/pull/5.

Pipeline

Section titled “Pipeline”
GFS fetch -> preprocess -> inference -> validation -> outputs -> S3
                                                   -> CloudWatch and SNS
  • Runtime split into modules under server/wm3pipe.
  • Verified against the sample: both 157-channel tensors assemble with sensible ranges, and 500 hPa height sits near 5,568 gpm.
  • The fetcher reads the GFS .idx and pulls only about 140 byte ranges, so a fetch is ~7 s.

Runner

Section titled “Runner”
  • EventBridge, Step Functions, and SageMaker Processing.
  • The GPU exists only while a forecast runs, so there is no idle cost.
  • Step Functions handles retries and routes failures to SNS.