Forecast Command Specification | GitHub Agentic Workflows
Skip to main content Forecast Command Specification Version : 1.0.0 Status : Draft Latest Version : forecast-specification Editor : GitHub Agentic Workflows Team Abstract Section titled “Abstract” This specification defines the gh aw forecast command for the GitHub Agentic Workflows project. The command performs historical sampling of completed agentic workflow runs and applies a Monte Carlo simulation engine to project future AI Credits (AIC) consumption over a configurable time horizon.
The specification covers workflow discovery (local and remote modes), data sampling via the GitHub Actions API, the Poisson–bootstrap Monte Carlo projection algorithm, and both console-table and machine-readable JSON output formats. Forecasts are run-level usage projections, not episode or throughput reports. Implementations conforming to this specification provide operators with probabilistic cost forecasts suitable for capacity planning, cost estimation, and budget governance.
Status of This Document Section titled “Status of This Document” This section describes the status of this document at the time of publication. This is a Draft specification. The gh aw forecast command is a stable, generally available command; forecasts it produces are probabilistic estimates and may be inaccurate. This document is governed by the GitHub Agentic Workflows project specifications process. Feedback should be filed as GitHub issues against the github/gh-aw repository.
Table of Contents Section titled “Table of Contents” Introduction Conformance Terminology Command Interface Workflow Discovery Data Sampling Monte Carlo Projection Engine Retired Episode Analysis Output Formats Error Handling Implementation Requirements Compliance Testing Norms Sync Notes Appendices References Change Log 1. Introduction Section titled “1. Introduction” 1.1 Purpose Section titled “1.1 Purpose” The gh aw forecast command addresses the operational need to predict future Large Language Model (LLM) token expenditure for agentic workflows managed by gh-aw.
Token consumption is a primary cost driver for agentic systems; the ability to project future usage from historical observations enables: Capacity Planning : Anticipating token demand before budget thresholds are reached. Cost Governance : Providing P10/P50/P90 confidence intervals for financial planning. Workflow Comparison : Ranking workflows by projected token cost across a shared time period. Experiment Evaluation : Measuring the token impact of A/B experiment variants.
The command combines empirical bootstrapping of historical token observations with a Poisson-distributed run-count model to produce statistically sound projections without requiring parametric distribution assumptions on token usage. 1.2 Scope Section titled “1.2 Scope” This specification covers: Command-line interface: flags, positional arguments, and invocation modes Workflow discovery in local ( .github/workflows/ ) and remote ( --repo ) modes Historical run sampling and per-run metric derivation The Monte Carlo simulation algorithm producing P10, P50, P90 percentile estimates Console table output format Machine-readable JSON output schema ( --json ) Error conditions and graceful-degradation behavior This specification does NOT cover: The AI Credits (AIC) computation algorithm (defined in the AI Credits Specification ) The aw_info.json artifact schema A/B experiment frontmatter schema (defined in the A/B Experiments Specification ) Billing, pricing, or financial modeling beyond token projections Streaming or real-time token consumption reporting Episode grouping, episode-level metrics, and throughput ( yield ) reporting, removed from forecast in #34750 1.3 Design Goals Section titled “1.3 Design Goals” A conforming gh aw forecast implementation MUST be designed for: Empirical Accuracy : Projections derived from observed historical data rather than assumed distributions.
Probabilistic Reporting : P10/P50/P90 uncertainty bounds communicated to callers. Graceful Degradation : Missing data (no runs, no artifacts, no frontmatter) MUST produce partial results rather than failures. Dual Modes : Both local-repository and remote-repository operation without requiring a checkout. Interoperability : JSON output schema stable enough for machine consumption by downstream tooling. 2. Conformance Section titled “2.
Conformance” 2.1 Conformance Classes Section titled “2.1 Conformance Classes” A conforming forecast implementation is one that satisfies all MUST, REQUIRED, and SHALL requirements in this specification. A partially conforming forecast implementation is one that satisfies all MUST requirements in Sections 4, 5, 6, and 7 but MAY lack support for optional features such as experiment variant reporting or verbose diagnostics.
Retired episode and yield features are not conformance requirements at any level. 2.2 Requirements Notation Section titled “2.2 Requirements Notation” The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119 . 2.3 Compliance Levels Section titled “2.3 Compliance Levels” Implementations MUST support: Level 1 (Required) : Command invocation, workflow discovery, historical data sampling, and Monte Carlo projection with console output.
Level 2 (Standard) : JSON output ( --json ), remote-repository mode ( --repo ), and experiment variant reporting. Level 3 (Complete) : All optional features including --verbose diagnostics, concurrency limit reporting, and frontmatter metadata enrichment. 3. Terminology Section titled “3. Terminology” 3.1 AI Credits (AIC) Section titled “3.1 AI Credits (AIC)” A normalized unit of LLM token consumption defined in the AI Credits Specification .
AIC accounts for token class weights and model multipliers to produce a single comparable scalar across heterogeneous LLM invocations. 3.2 Workflow Run Section titled “3.2 Workflow Run” A single execution of a GitHub Actions workflow. A run has a unique numeric run ID, an event type, a status ( completed , in_progress , queued ), a conclusion ( success , failure , cancelled , etc.), and a head commit SHA. 3.3 Historical Window Section titled “3.3 Historical Window” The time interval [now − days, now] used to bound the set of completed and in-progress runs eligible for sampling.
Controlled by the --days flag. 3.4 Sample Section titled “3.4 Sample” The subset of completed and in-progress workflow runs within the historical window selected for metric derivation. The maximum sample size per workflow is controlled by the --sample flag. 3.5 Monte Carlo Trial Section titled “3.5 Monte Carlo Trial” A single independent simulation that draws stochastic values for run count, per-run token usage, and per-run success, combining them to produce one projected AI Credit total for the projection period.
3.6 Projection Period Section titled “3.6 Projection Period” The future time interval for which token consumption is projected. Controlled by the --period flag; either one calendar week ( week ) or one calendar month ( month ). 3.7 Observed Runs Per Period Section titled “3.7 Observed Runs Per Period” The rate of workflow runs observed in the historical window, extrapolated to the projection period length: observed_runs_per_period = (sampled_run_count / history_days) × period_days Where period_days is 7 for week and 30 for month .
3.8 Bootstrap Resampling Section titled “3.8 Bootstrap Resampling” An empirical resampling technique where individual observations are drawn with replacement from the observed sample. Used in Section 7 to model per-run token usage without parametric distribution assumptions. 3.9 Lock File Section titled “3.9 Lock File” A .lock.yml file located in .github/workflows/ that declares a compiled agentic workflow
The rest of this page is on the live site.