Skip to content
windborne

Architecture

WeatherMesh-3 runs every six hours as a SageMaker Processing job.

flowchart LR
  EB[EventBridge rule<br/>00, 06, 12, 18 UTC] --> SF[Step Functions<br/>wm3-cycle]
  SF -->|createProcessingJob.sync| PJ[SageMaker Processing<br/>ml.g5.xlarge, ECR image]
  GFS[(NOAA GFS<br/>AWS Open Data)] -->|f000 GRIB ranges| PJ
  WT[(S3 weights<br/>wm3-gpu)] -->|WeatherMesh3.pt| PJ
  PJ --> V{validation passes?}
  V -->|yes| F[(S3 forecasts)]
  F --> L[latest.json]
  V -->|no| Q[(S3 quarantine)]
  PJ -->|metrics| CW[CloudWatch dashboard and alarms]
  SF -->|job failure| SNS[SNS email]
  PJ -.->|invalid output| SNS

How it runs

Section titled “How it runs”
  • EventBridge triggers the wm3-cycle state machine, which starts one Processing job on an ml.g5.xlarge GPU from the ECR image.
  • The job fetches the latest GFS, runs the model, validates, and writes to S3: valid forecasts to forecasts/ and latest.json, invalid ones to quarantine/.
  • CloudWatch records timings and diagnostics; SNS alerts on failure, invalid output, or a missing run.