AGENTS instructions

Naming

Write Dag (title case) in all prose. Keep the all-caps or lowercase spelling only when reproducing a literal code token — never rewrite these, even inside fenced code blocks:

• Python: the SDK class DAG (from airflow.sdk import DAG, dag = DAG("my_dag", ...)); identifiers like dag_id, dag, my_dag.
• CLI: airflow dags list, airflow dags test, etc.
• Paths and config keys: dag_processing/, dagprocessor, get_dag, etc.
• Anti-pattern quotes that show the wrong form to teach the rule itself (e.g., Use "DAG" — always write "Dag").

Don't spell out Directed Acyclic Graph except for historical context.

Environment Setup

• Install prek: uv tool install prek
• Enable commit hooks: prek install
• Install breeze shim (one-time, per machine): scripts/tools/setup_breeze — installs ~/.local/bin/breeze that runs breeze via uvx from the current git worktree's dev/breeze (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See ADR 0017.
• Never run pytest, python, or airflow commands directly on the host — always use breeze.
• Place temporary scripts in dev/ (mounted as /opt/airflow/dev/ inside Breeze).

Commands

<PROJECT> is folder where pyproject.toml of the package you want to test is located. For example, airflow-core or providers/amazon. <target_branch> is the branch the PR will be merged into — usually main, but could be v3-1-test when creating a PR for the 3.1 branch.

• Run a single test: uv run --project <PROJECT> pytest path/to/test.py::TestClass::test_method -xvs
• Run a test file: uv run --project <PROJECT> pytest path/to/test.py -xvs
• Run all tests in package: uv run --project <PROJECT> pytest path/to/package -xvs
• If uv tests fail with missing system dependencies, run the tests with breeze: breeze run pytest <tests> -xvs
• Run a Python script: uv run --project <PROJECT> python dev/my_script.py
• Run core or provider tests suite in parallel: breeze testing <test_group> --run-in-parallel (test groups: core-tests, providers-tests)
• Run core or provider db tests suite in parallel: breeze testing <test_group> --run-db-tests-only --run-in-parallel (test groups: core-tests, providers-tests)
• Run core or provider non-db tests suite in parallel: breeze testing <test_group> --skip-db-tests --use-xdist (test groups: core-tests, providers-tests)
• Run single provider complete test suite: breeze testing providers-tests --test-type "Providers[PROVIDERS_LIST]" (e.g., Providers[google] or Providers[amazon] or "Providers[amazon,google]")
• Run Helm tests in parallel with xdist breeze testing helm-tests --use-xdist
• Run Helm tests with specific K8s version: breeze testing helm-tests --use-xdist --kubernetes-version 1.35.0
• Run specific Helm test type: breeze testing helm-tests --use-xdist --test-type <type> (types: airflow_aux, airflow_core, apiserver, dagprocessor, other, redis, security, statsd, webserver)
• Run other suites of tests breeze testing <test_group> (test groups: airflow-ctl-tests, docker-compose-tests, task-sdk-tests)
• Run scripts tests: uv run --project scripts pytest scripts/tests/ -xvs
• Run Airflow CLI: breeze run airflow dags list
• Type-check (non-providers): run the prek hook — prek run mypy-<project> --all-files (e.g. mypy-airflow-core, mypy-task-sdk, mypy-shared-logging; each shared/<dist> workspace member has its own mypy-shared-<dist> hook). The hook uses a dedicated virtualenv and mypy cache under .build/mypy-venvs/<hook>/ and .build/mypy-caches/<hook>/; mypy itself is installed from uv.lock via the mypy dependency group (uv sync --group mypy), so it never mutates your project .venv. The hook prefers uv from the project's main .venv/bin/uv (installed by uv sync — uv is part of the dev dependency group via the all extras) for a project-pinned uv version; it falls back to uv on $PATH with a warning if that binary is missing. Clear with breeze down --cleanup-mypy-cache.
• Type-check (providers): breeze run mypy path/to/code
• Lint with ruff only: prek run ruff --from-ref <target_branch>
• Format with ruff only: prek run ruff-format --from-ref <target_branch>
• Run regular (fast) static checks: prek run --from-ref <target_branch> --stage pre-commit
• Run manual (slower) checks: prek run --from-ref <target_branch> --stage manual
• Build docs: breeze build-docs
• Determine which tests to run based on changed files: breeze selective-checks --commit-ref <commit_with_squashed_changes>

SQLite is the default backend. Use --backend postgres or --backend mysql for integration tests that need those databases. If Docker networking fails, run docker network prune.

Repository Structure

UV workspace monorepo. Key paths:

• airflow-core/src/airflow/ — core scheduler, API, CLI, models
  • models/ — SQLAlchemy models (DagModel, TaskInstance, DagRun, Asset, etc.)
  • jobs/ — scheduler, triggerer, Dag processor runners
  • api_fastapi/core_api/ — public REST API v2, UI endpoints
  • api_fastapi/execution_api/ — task execution communication API
  • dag_processing/ — Dag parsing and validation
  • cli/ — command-line interface
  • ui/ — React/TypeScript web interface (Vite)
• task-sdk/ — lightweight SDK for Dag authoring and task execution runtime
  • src/airflow/sdk/execution_time/ — task runner, supervisor
• providers/ — 100+ provider packages, each with its own pyproject.toml
• airflow-ctl/ — management CLI tool
• chart/ — Helm chart for Kubernetes deployment
• dev/ — development utilities and scripts used to bootstrap the environment, releases, breeze dev env
• scripts/ — utility scripts for CI, Docker, and prek hooks (workspace distribution apache-airflow-scripts)
  • ci/prek/ — prek (pre-commit) hook scripts; shared utilities in common_prek_utils.py
  • tests/ — pytest tests for the scripts; run with uv run --project scripts pytest scripts/tests/

The uv.lock file is generated by uv lock, uv sync and is committed to the repo - it contains snapshot of versions of all dependencies used for development of Airflow. If at any point in time you have a conflict with uv.lock, simply delete it and run uv lock to regenerate it.

Architecture Boundaries

1. Users author Dags with the Task SDK (airflow.sdk).
2. Dag File Processor parses Dag files in separate processes and stores serialized Dags in the metadata DB. Software guards prevent individual parsing processes from accessing the database directly and enforce use of the Execution API, but these guards do not protect against intentional bypassing by malicious or misconfigured code.
3. Scheduler reads serialized Dags — never runs user code — and creates Dag runs / task instances.
4. Workers execute tasks via Task SDK and communicate with the API server through the Execution API — never access the metadata DB directly. Each task receives a short-lived JWT token scoped to its task instance ID.
5. API Server serves the React UI and handles all client-database interactions.
6. Triggerer evaluates deferred tasks/sensors in separate processes. Like the Dag File Processor, software guards steer it through the Execution API rather than direct database access, but these guards do not protect against intentional bypassing by malicious or misconfigured code.
7. Shared libraries that are symbolically linked to different Python distributions are in shared folder.
8. Airflow uses uv workspace feature to keep all the distributions sharing dependencies and venv
9. Each of the distributions should declare other needed distributions: uv --project <FOLDER> sync command acts on the selected project in the monorepo with only dependencies that it has

Security Model

When reviewing code, writing security documentation, or performing security research, keep in mind the following aspects of Airflow's security model. The authoritative reference is airflow-core/docs/security/security_model.rst and airflow-core/docs/security/jwt_token_authentication.rst.

Particularly, the intentional design choices, that are not security vulnerabilities and should not be reported as such are described in "What is NOT considered a security vulnerability" chapter of the security model.

When flagging security concerns, distinguish between:

1. Actual vulnerabilities — code that violates the documented security model (e.g., a worker gaining database access it shouldn't have, a Scheduler executing user code, an unauthenticated user accessing protected endpoints).
2. Known limitations — documented gaps where the current implementation doesn't provide full isolation (e.g., DFP/Triggerer database access, shared Execution API resources, multi-team not enforcing task-level isolation). These are tracked for improvement in future versions and should not be reported as new findings.
3. Deployment hardening opportunities — measures a Deployment Manager can take to improve isolation beyond what Airflow enforces natively (e.g., per-component configuration, asymmetric JWT keys, network policies). These belong in deployment guidance, not as code-level issues.

Shared libraries

• shared libraries provide implementation of some common utilities like logging, configuration where the code should be reused in different distributions (potentially in different versions)
• we have a number of shared libraries that are separate, small Python distributions located under shared folder
• each of the libraries has it's own src, tests, pyproject.toml and dependencies
• sources of those libraries are symbolically linked to the distributions that are using them (airflow-core, task-sdk for example)
• tests for the libraries (internal) are in the shared distribution's test and can be run from the shared distributions
• tests of the consumers using the shared libraries are present in the distributions that use the libraries and can be run from there

Coding Standards

• Always format and check Python files with ruff immediately after writing or editing them: uv run ruff format <file_path> and uv run ruff check --fix <file_path>. Do this for every Python file you create or modify, before moving on to the next step.
• No assert in production code.
• Comment sparingly — code says what, comments say why. Add a comment only when the reasoning is non-obvious and cannot be carried by a clear name or the code itself. Do not write narrating comments that restate the next line, do not pad logic with multi-line prose, and do not repeat the same rationale at several sites — put one concise note at the source of truth and let the others stand on their own. Tests whose names already describe intent need no explanatory comment. Reserve longer explanation for genuinely complex or non-obvious logic (e.g. a security check whose threat model isn't apparent), and keep even that as tight as it can be. Over-commenting is noise that ages badly and obscures the code it wraps.
• time.monotonic() for durations, not time.time().
• In airflow-core, functions with a session parameter must not call session.commit(). Use keyword-only session parameters.
• Imports at top of file. Valid exceptions: circular imports, lazy loading for worker isolation, TYPE_CHECKING blocks.
• Guard heavy type-only imports (e.g., kubernetes.client) with TYPE_CHECKING in multi-process code paths.
• Define dedicated exception classes or use existing exceptions such as ValueError instead of raising the broad AirflowException directly. Each error case should have a specific exception type that conveys what went wrong. Never add new direct raise AirflowException(...) usages — the community is actively reducing them, not adding more, and the check-no-new-airflow-exceptions prek hook enforces this across airflow-core, airflow-ctl, task-sdk, providers, and shared. Prefer a Python built-in (ValueError, TypeError, OSError, …) or a dedicated class in the appropriate exceptions.py. The only acceptable way an AirflowException line may move is relocating an already-existing one verbatim during a refactor (e.g. moving code between files) — that is not a new usage. When you touch code that already raises AirflowException, prefer narrowing it to a more specific exception rather than leaving or duplicating it.
• Translate domain-layer exceptions to HTTPException at FastAPI route boundaries. In airflow-core/src/airflow/core_api/ route handlers, catch errors raised by domain code (e.g., ValueError from airflow.state.metastore.MetastoreStateBackend for a missing row or invalid input) and re-raise as HTTPException with the right status (404 for not-found, 400 for invalid input). Otherwise they propagate as 500 Internal Server Error, leaking internals and misleading clients.
• Bulk DELETE/UPDATE in the scheduler loop or any synchronous interval task (e.g. call_regular_interval callbacks) must be batched with LIMIT and committed between batches — never issue a single unbounded bulk write against a user-driven table. Unbounded bulk writes hold row locks for the entire transaction (blocking concurrent writers) and stall the scheduler main loop. Filter columns used by the cleanup must be indexed. Follow the batching pattern in airflow-core/src/airflow/utils/db_cleanup.py.
• Name functions and methods with action verbs: get_, extract_, find_, compute_, build_, etc. Avoid noun-only names like _serialize_keys or _base_names — they read as attributes, not callables. Predicates (is_, has_) are the one exception.
• Apache License header on all new files (prek enforces this).
• Keep selective-checks behaviour and its documentation in sync. The CI optimisation logic lives in dev/breeze/src/airflow_breeze/utils/selective_checks.py (run-mode decisions, file-group matching, test-type selection, prek-hook skipping). Whenever you change a rule there — add/rename a file group, change what forces full_tests_needed/all_versions, alter how providers or test types are selected, or change which prek hooks are skipped — update dev/breeze/doc/ci/04_selective_checks.md in the same PR (the decision-rules list, the diagrams, the outputs table, and the worked examples as applicable) and add/adjust tests in dev/breeze/tests/test_selective_checks.py. The doc is the human-facing explanation of that file; letting them drift makes CI behaviour impossible to reason about.
• Newsfragments are only used by distributions whose release process consumes them via towncrier — currently airflow-core/newsfragments/, chart/newsfragments/, and dev/mypy/newsfragments/ — and only for major or breaking changes. Golden rule: never create a newsfragment unless you are certain the change is user-facing. If you are not sure the change is visible to users — build/release tooling, CI, packaging, internal refactors with no behavior change, dev-only scripts, and test-only changes are not user-facing — do not add one. Default to omitting it; a maintainer will ask for a newsfragment during review if the change warrants one. Adding a spurious newsfragment for a non-user-facing change is a defect, not a safe default. Never add newsfragments for providers/ or airflow-ctl/ — those distributions are released from main and their release managers regenerate the changelog from git log, so per-PR newsfragments are not consumed (see dev/README_RELEASE_PROVIDERS.md and dev/README_RELEASE_AIRFLOWCTL.md). For a user-visible note in those distributions, edit the changelog directly: providers/<provider>/docs/changelog.rst for providers, airflow-ctl/RELEASE_NOTES.rst for airflow-ctl. Changes to task-sdk/ ship in airflow-core — use airflow-core/newsfragments/.

Testing Standards

• Target exactly 100% coverage of what the PR changes — no more, no less. Every changed or added behaviour must have a test; every test must fail without the PR's change. Do not add tests for pre-existing logic that was already present before the PR, and do not test standard-library or third-party functions. The exception is deliberate behaviour or integration tests, which may cross those boundaries by design.
• Use pytest patterns, not unittest.TestCase.
• Use spec/autospec when mocking.
• Prefer @mock.patch decorators over with mock.patch(...) context managers for patching. Use conf_vars (from tests_common.test_utils.config) for Airflow config overrides — as a decorator when the value is fixed, as a context manager when it varies via @pytest.mark.parametrize.
• Use time_machine for time-dependent tests. Do not use datetime.now()
• Use @pytest.mark.parametrize for multiple similar inputs — consolidate tests that only differ in input/expected values into a single parametrized test.
• Use @pytest.mark.db_test for tests that require database access.
• Test fixtures: devel-common/src/tests_common/pytest_plugin.py.
• Test location mirrors source: airflow/cli/cli_parser.py → tests/cli/test_cli_parser.py.
• Do not use caplog in tests, prefer checking logic and not log output.

Commits and PRs

Write commit messages focused on user impact, not implementation details.

• Good: Fix airflow dags test command failure without serialized Dags
• Good: UI: Fix Grid view not refreshing after task actions
• Bad: Initialize Dag bundles in CLI get_dag function
• Bad: fix(cli): dags test failure — Airflow does not use Conventional Commits (feat:, fix:, chore: …). Write the subject as plain prose. A commit-msg prek hook (check-no-conventional-commit-message) rejects these, and CI checks every commit of the PR.

Always run prek install before committing any code. It installs the commit-msg hook (in addition to pre-commit) so the Conventional Commits guard runs locally; a clone that ran prek install before this hook existed must re-run it to pick up the new hook type.

Use the imperative mood and a plain message — do not use Conventional Commits prefixes (fix:, feat:, chore:, docs:, refactor:, …). apache/airflow does not follow that convention. (Area tags the project already uses, like UI: / API: / Helm:, are fine; Conventional-Commit type: tokens are not.) The same rule applies to PR titles.

The commit message body should describe why the change is made — the motivation and context — and never what the change is. The diff already shows what changed; restating it in prose adds noise.

For airflow-core (and chart/, dev/mypy/) user-facing changes, add a newsfragment in that distribution's newsfragments/ directory. Golden rule: only add a newsfragment when you are certain the change is visible to users; when in doubt, do not add one — a maintainer will request one in review if it is needed. Build/release tooling, CI, packaging, internal refactors, and dev-only scripts are not user-facing and must not get a newsfragment: echo "Brief description" > airflow-core/newsfragments/{PR_NUMBER}.{bugfix|feature|improvement|doc|misc|significant}.rst

Do not add newsfragments for providers/ or airflow-ctl/ — their release managers regenerate the changelog from git log and do not consume newsfragments. Update the changelog directly when needed: providers/<provider>/docs/changelog.rst (see providers/AGENTS.md) or airflow-ctl/RELEASE_NOTES.rst. Changes to task-sdk/ use airflow-core/newsfragments/ since task-sdk ships in airflow-core.

• NEVER add Co-Authored-By with yourself as co-author of the commit. Agents cannot be authors, humans can be, Agents are assistants.

Git remote naming conventions

Airflow standardises on two git remote names, and the rest of this file, the contributing docs, and the release docs all assume them:

• upstream — the canonical apache/airflow repository (fetch from here).
• origin — the contributor's fork of apache/airflow (push PR branches here).

Always push branches to origin. Never push directly to upstream (and never push directly to main on either remote).

Before running any remote-based command, run git remote -v and verify the names match this convention. If they do not — for example, the upstream remote is called apache, or origin points at apache/airflow with the fork under a different name like fork — do not silently go along with the existing names. Surface the mismatch to the user and propose the exact rename commands to bring the checkout in line with the convention, then ask the user to confirm before running them. Examples:

• Upstream is named apache, fork is origin (common legacy layout):

  git remote rename apache upstream
  

• origin points at apache/airflow and the fork is named fork (release-manager / "cloned upstream directly" layout):

  git remote rename origin upstream
  git remote rename fork origin
  

• Upstream is missing entirely:

  git remote add upstream https://github.com/apache/airflow.git
  # or, for SSH:
  git remote add upstream git@github.com:apache/airflow.git
  

• Fork is missing entirely:

  gh repo fork apache/airflow --remote --remote-name origin
  

After any rename/add, re-run git remote -v to confirm the new state before continuing with commands that assume upstream / origin.

If a doc, script, or command you're about to run uses the old apache name (or any other variant), translate it to the upstream convention in what you propose to the user, rather than perpetuating the old name. Flag the stale documentation so it can be fixed in a follow-up.

Before starting: check for an existing PR

Before working on an issue, check for open PRs already addressing it (gh pr list --search "<issue number or keywords>", and look for closes: / fixes: references). Airflow allows parallel work — "better PR wins" (see contributing-docs/04_how_to_contribute.rst) — but it is not the default: prefer reviewing and building on an existing PR. Open a separate one only if your approach is genuinely different. Do not blindly open another near-identical PR for an issue that already has one (or several) — that just adds reviewer noise.

Creating Pull Requests

Always push to the user's fork (origin), not to upstream (apache/airflow). Never push directly to main.

Before pushing, confirm the remote setup matches the conventions above (upstream → apache/airflow, origin → your fork). Run git remote -v and, if the names don't match, propose renames as described in "Git remote naming conventions" — ask the user to confirm before running them.

If the fork remote does not exist at all, create one:

gh repo fork apache/airflow --remote --remote-name origin

Before pushing, perform a self-review of your changes following the Gen-AI review guidelines in contributing-docs/05_pull_requests.rst and the code review checklist in .github/instructions/code-review.instructions.md:

1. Review the full diff (git diff main...HEAD) and verify every change is intentional and related to the task — remove any unrelated changes.
2. Read .github/instructions/code-review.instructions.md and check your diff against every rule — architecture boundaries, database correctness, code quality, testing requirements, API correctness, and AI-generated code signals. Fix any violations before pushing.
3. Confirm the code follows the project's coding standards and architecture boundaries described in this file.
4. Run regular (fast) static checks (prek run --from-ref <target_branch> --stage pre-commit) and fix any failures. This includes mypy checks for non-provider projects (airflow-core, task-sdk, airflow-ctl, dev, scripts, devel-common).
5. Run manual (slower) checks (prek run --from-ref <target_branch> --stage manual) and fix any failures.
6. Run relevant individual tests and confirm they pass.
7. Find which tests to run for the changes with selective-checks and run those tests in parallel to confirm they pass and check for CI-specific issues.
8. Check for security issues — no secrets, no injection vulnerabilities, no unsafe patterns.

Before pushing, always rebase your branch onto the latest target branch (usually main) to avoid merge conflicts and ensure CI runs against up-to-date code:

git fetch upstream <target_branch>
git rebase upstream/<target_branch>

If there are conflicts, resolve them and continue the rebase. If the rebase is too complex, ask the user for guidance.

Then push the branch to your fork (origin) and open the PR creation page in the browser with the body pre-filled (including the generative AI disclosure already checked):

git push -u origin <branch-name>
gh pr create --web --title "Short title (under 70 chars)" --body "$(cat <<'EOF'
Brief description of the changes.

closes: #ISSUE  (if applicable)

---

##### Was generative AI tooling used to co-author this PR?

- [X] Yes — <Agent Name and Version>

Generated-by: <Agent Name and Version> following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)

EOF
)"

The --web flag opens the browser so the user can review and submit. The --body flag pre-fills the PR template with the generative AI disclosure already completed.

Remind the user to:

1. Review the PR title — keep it short (under 70 chars), in the imperative mood, and focused on user impact. Do not use Conventional Commits prefixes (fix:, feat:, chore:, …).
2. Add a brief description of the changes at the top of the body.
3. Reference related issues when applicable (closes: #ISSUE or related: #ISSUE).

Golden rule: when a fix is imminent, open the PR, not an issue

If you already know how to fix the problem and you (or the user) are going to open the PR shortly, do not file a GitHub issue first. Go straight to the PR.

• Airflow does not use issues as a changelog, as a parallel bug database, or as a duplicate record of in-flight work. The PR itself is the canonical record — title, description, diff, discussion, and merge all live in one place. An issue that gets closed by a PR a day later is double accounting that carries no information the PR does not already carry.
• Open issues attract drive-by submissions, often from other agents, that haven't seen the in-flight work. That produces duplicate fixes, low-quality PRs that have to be closed, and wasted reviewer time. Not opening the issue avoids creating that bait in the first place.
• If you catch yourself drafting an issue body that reads like the PR description you are about to write, that is the signal — skip the issue and open the PR.

The one exception is the case covered by the next section: the PR ships a workaround, mitigation, or partial fix and the real follow-up work is genuinely deferred to a later PR. There, the issue captures work that will outlive the PR, so the issue is load-bearing rather than duplicate.

Tracking issues for deferred work

When a PR applies a workaround, version cap, mitigation, or partial fix rather than solving the underlying problem (for example: upper-binding a dependency to avoid a breaking upstream release, disabling a feature behind a flag, reverting a change that needs a better replacement, or papering over a bug so a release can ship), the deferred work must be captured in a GitHub tracking issue and the tracking issue URL must appear as a comment at the workaround site in the code.

1. Open the tracking issue first, before finalising the PR body.

2. Reference it in the PR body by number — e.g. "full migration is tracked in #65609" — so anyone reviewing the PR can see what was deferred and why.

3. Add a link to the tracking issue as a comment at the workaround itself, so the reference survives after the PR merges and anyone reading the source later can click straight through to the follow-up work. Use the full issue URL, not bare #NNNNN — bare references do not auto-link outside GitHub's web UI (e.g. when grepping in an editor, browsing a checkout, or reading the file in a terminal). For example:

   # pyproject.toml
   # Remove the <1.0 cap after migrating to httpx 1.x;
   # tracked at https://github.com/apache/airflow/issues/65609
   "httpx>=0.27.0,<1.0",
   

   # some_module.py
   # Delete this fallback once the new client is on all workers;
   # tracked at https://github.com/apache/airflow/issues/65609
   if old_client:
       ...
   

4. Do not write vague forward-looking phrases like "will open a tracking issue" or "to be filed later" in the PR body or in code comments. Open the issue, link it in both places, then submit the PR.

5. The tracking issue should describe: what the workaround is, why it was chosen, the concrete follow-up work needed, and any acceptance criteria for removing the workaround.

If a PR you already opened has such forward-looking language, open the tracking issue, add a PR comment referencing the issue URL, and push a follow-up commit that adds the tracking-issue URL as a comment at the workaround site in the code.

GitHub messages drafted by agents

Anything an agent drafts that ends up posted to GitHub on the user's account — PR / issue comments, PR-level reviews, line-level review comments, discussion replies — must end with an attribution footer. The footer is required whether or not a human reviewed the draft first; what changes between the two cases is the wording.

Place the footer on its own paragraph at the end of the message, separated from the body by a blank line and a horizontal rule. Use the same agent name string used in Generated-by: on PR bodies (for example, Claude Code (Opus 4.7)).

• Agent draft, posted without prior human review (autonomous / routine work, scheduled triage, etc.):

  ---
  Drafted-by: <Agent Name and Version> (no human review before posting)
  

• Agent draft, reviewed and approved by a human maintainer before posting:

  ---
  Drafted-by: <Agent Name and Version>; reviewed by @<github-handle> before posting
  

  The @<github-handle> is the human who actually read the draft and said "post it as-is" (or similar). It is not the user the agent is "running on behalf of" if no review took place — that case is the first form, not this one.

This footer is in addition to, not a replacement for, any per-tool disclosure rules (the PR body still keeps its own Generated-by: block under the AI-disclosure checkbox; commit messages still follow the no-self-as-co-author rule above). Do not skip the footer to shorten a message — attribution applies regardless of message length.

Do not tag individuals

AI agents MUST NOT mention or tag individual contributors, committers, PMC members, or maintainers using GitHub usernames (e.g. @user) unless explicitly instructed by a human reviewer. When suggesting who might be relevant to a discussion, refer to roles, teams, code ownership information, labels, or components instead of individuals. This keeps notification noise down and avoids pulling people into threads they have not chosen to join.

The only exceptions are mentions a human has explicitly authorized — including the @<github-handle> in the Drafted-by: … reviewed by @<handle> footer above, which names the reviewer who approved the message — and replying within a thread to people already actively participating in that same PR/issue discussion.

apache-steward framework

This repo adopts the apache/airflow-steward framework via the snapshot mechanism. The framework provides the pr-management-* skills (triage, code-review, stats, mentor); they are gitignored symlinks into the .apache-magpie/ snapshot directory.

A fresh clone needs the snapshot populated before any framework skill is invocable. Run /magpie-setup (or follow .claude/skills/magpie-setup/) to fetch it per the committed .apache-magpie.lock. The contributor-facing summary of the adoption + setup flow lives in the Agent-assisted contribution section of README.md.

Adopter-specific modifications to framework-skill workflows live in .apache-magpie-overrides/ — never edit the snapshot directly. Framework changes go via PR to apache/airflow-steward.

Boundaries

• Ask first
  • Large cross-package refactors.
  • New dependencies with broad impact.
  • Destructive data or migration changes.
• Never
  • Commit secrets, credentials, or tokens.
  • Edit generated files by hand when a generation workflow exists.
  • Use destructive git operations unless explicitly requested.

References

• contributing-docs/03a_contributors_quick_start_beginners.rst
• contributing-docs/05_pull_requests.rst
• contributing-docs/07_local_virtualenv.rst
• contributing-docs/08_static_code_checks.rst
• contributing-docs/12_provider_distributions.rst
• contributing-docs/19_execution_api_versioning.rst

Awesome Agent Conventions

A curated field guide to the convention files AI agents read, write, and act on.

22 conventions across 11 categories. From common project instruction files to newer agent-web discovery and trust formats.

Agent tools increasingly rely on plain files in a repository or website root: instructions, memory, rules, tool connections, prompt assets, discovery metadata, and protocol hints. The names are easy to mix up, and the adoption levels vary a lot.

This repo keeps the map practical:

• Know what a file is for. Each entry names the convention, usual filename, primary readers, and spec or source.
• Study real examples. Examples are fetched from public repositories by script, with provenance kept at the top of each file.
• Separate practice from proposal. Maturity labels show what is widely used, what is early, and what is still only proposed.

Contents

• Instruction & context · category page
  • 🟢 AGENTS.md
  • 🟢 CLAUDE.md
  • 🟢 Tool-specific instruction files
  • 🟠 OKF (Open Knowledge Format)
• Memory & state · category page
  • 🟢 MEMORY.md
  • 🟢 Memory Bank
• Spec-driven development · category page
  • 🟢 Spec Kit
  • 🟢 Kiro steering files
• Skills & prompt assets · category page
  • 🟢 SKILL.md
  • 🟢 Prompt asset files
  • 🟢 Claude Code commands
  • 🟢 Copilot prompt & instruction files
• Tooling & connections · category page
  • 🟢 MCP server config
• Rules & ignore files · category page
  • 🟢 Rules files
  • 🟢 AI ignore files
• Design · category page
  • 🟢 DESIGN.md
• Web & discoverability · category page
  • 🟢 llms.txt
  • 🟢 pricing.md
• Agent-web trust · category page
  • 🟠 auth.md
  • 🔵 ai.txt
• Identity & protocols · category page
  • 🟠 Agent Cards (A2A)
• Proposed namespace · category page
  • 🔵 The protocols.md namespace

What counts

This list is intentionally narrow. A file belongs here when it is a convention for agent behavior or agent-readable metadata: instructions, memory, skills, rules, tool config, prompt assets, or web-discovery hints.

Any file type can qualify - .md, .txt, .prompty, .json, dotfiles, or a directory pattern. Human-first project docs such as README.md, CONTRIBUTING.md, SECURITY.md, and CHANGELOG.md stay out unless the file has become an agent convention in its own right.

Maturity tiers

The badge is a claim about adoption, not quality. It keeps a proven convention from being presented the same way as a new idea.

Badge	Tier	Meaning
🟢	Adopted	Used in production by multiple tools, projects, or teams.
🟠	Emerging	Published by a real organization, but still early or limited in adoption.
🔵	Proposed	Publicly described, but without clear adoption beyond the proposal.

Instruction & context

Standalone page: categories/instruction-context.md

	Convention	Files	Read by	Spec
🟢	AGENTS.md	AGENTS.md	Most coding agents - OpenAI Codex, Cursor, Jules, Aider, Gemini CLI, Zed, and others	spec ↗
🟢	CLAUDE.md	CLAUDE.md	Claude Code, and tools that read the Claude memory convention	spec ↗
🟢	Tool-specific instruction files	GEMINI.md AGENT.md QWEN.md WARP.md CONVENTIONS.md copilot-instructions.md	Each file is read by its namesake tool - Gemini CLI, Amp, Qwen Code, Warp, Aider, GitHub Copilot - often alongside or as a bridge to AGENTS.md	spec ↗
🟠	OKF (Open Knowledge Format)	.md	Agents over MCP (okfy, openknowledge, superops okf CLIs); Google's knowledge-catalog ingests bundles	spec ↗

• AGENTS.md - A plain-Markdown "README for agents" - build/test commands, conventions, and gotchas an agent needs before touching the code. The most widely adopted cross-tool instruction file.
• CLAUDE.md - Anthropic's memory file for Claude Code - loaded automatically at session start to carry project commands, style rules, and standing instructions across turns.
• Tool-specific instruction files - Per-tool instruction files that predate or coexist with AGENTS.md. Some tools now default to AGENTS.md while keeping legacy filenames alive, so these variants still matter when auditing real repositories.
• OKF (Open Knowledge Format) - A machine-first organizational knowledge base: a version-controlled folder of typed Markdown files (one concept per file) that any agent reads as ground-truth context. Open-sourced by Google Cloud in 2026 as the content layer to MCP's transport.

Memory & state

Standalone page: categories/memory-state.md

	Convention	Files	Read by	Spec
🟢	MEMORY.md	MEMORY.md	Claude Code's auto-memory - the per-project MEMORY.md index it writes and re-reads each session	spec ↗
🟢	Memory Bank	projectbrief.md productContext.md activeContext.md systemPatterns.md techContext.md progress.md	Cline, Roo Code, and Cursor (via the Memory Bank custom-instructions pattern)	spec ↗

• MEMORY.md - A persistent, agent-maintained index of durable facts - written and re-read across sessions so an agent accumulates project memory instead of relearning each time.
• Memory Bank - Cline's structured memory system - a set of Markdown files an agent reads at the start of every task to reconstruct full project context after its session memory resets. The six files shown are Cline's set; tools like Roo Code use an overlapping but different variant.

Spec-driven development

Standalone page: categories/spec-driven-development.md

	Convention	Files	Read by	Spec
🟢	Spec Kit	constitution.md spec.md plan.md tasks.md	GitHub Spec Kit's slash-command agents (Copilot, Claude, Gemini, Cursor, and more)	spec ↗
🟢	Kiro steering files	product.md structure.md tech.md	AWS Kiro (steering files are largely Kiro-specific)	spec ↗

• Spec Kit - GitHub's spec-driven workflow - a constitution plus per-feature spec → plan → tasks files that drive an agent through structured, reviewable implementation.
• Kiro steering files - Kiro's always-on steering docs - product, structure, and tech files that give the agent persistent project context outside of any single spec.

Skills & prompt assets

Standalone page: categories/skills-prompt-assets.md

	Convention	Files	Read by	Spec
🟢	SKILL.md	SKILL.md	Claude Agent Skills, Claude Code, Amp, Agent Skills-compatible tools	spec ↗
🟢	Prompt asset files	.prompty .prompt system_prompt.txt	Prompty tooling, Azure AI / Semantic Kernel, and apps that load externalized prompts	spec ↗
🟢	Claude Code commands	.md	Claude Code - project .claude/commands/ and user ~/.claude/commands/	spec ↗
🟢	Copilot prompt & instruction files	.prompt.md .instructions.md	GitHub Copilot in VS Code / Copilot CLI	spec ↗

• SKILL.md - A self-contained, model-invoked capability file that tells an agent when to load a reusable procedure and how to execute it.
• Prompt asset files - Externalized prompt files - Prompty's YAML-front-mattered .prompty, plain .prompt templates, and system_prompt.txt - that pull the prompt out of source code so it can be versioned and edited on its own. Only .prompty has a formal spec (prompty.ai); .prompt and system_prompt.txt are ad-hoc externalized-prompt filenames.
• Claude Code commands - A Markdown file Claude Code exposes as a /slash-command - a reusable, version-controlled prompt workflow, with optional frontmatter (allowed-tools, model, argument-hint) and $ARGUMENTS and shell placeholders (@file references are a general Claude Code prompt feature, not command-specific). Now converging with Agent Skills, but still widely committed in its own right.
• Copilot prompt & instruction files - Modular, path-scoped Copilot context: *.instructions.md auto-attach to matching files via an applyTo glob, while *.prompt.md are reusable prompts you invoke by name - the granular cousins of a single .github/copilot-instructions.md.

Tooling & connections

Standalone page: categories/tooling-connections.md

	Convention	Files	Read by	Spec
🟢	MCP server config	.mcp.json	Claude Code, Cursor, VS Code / Copilot, and Claude Desktop - every MCP host reads the same mcpServers schema, though the filename and path differ per tool	spec ↗

• MCP server config - A JSON file that tells an agent which Model Context Protocol servers to launch and how (command, args, env) - making a project's tool and data integrations portable, shareable, and version-controlled across every MCP-capable client.

Rules & ignore files

Standalone page: categories/rules-ignore-files.md

	Convention	Files	Read by	Spec
🟢	Rules files	.cursorrules .mdc .clinerules .clinerules/ (pattern) .windsurfrules	Cursor (.cursorrules / .mdc), Cline (.clinerules/ and legacy .clinerules), Windsurf (.windsurfrules)	spec ↗
🟢	AI ignore files	.aiignore .cursorignore .codeiumignore .aiexclude	JetBrains Junie (.aiignore), Cursor (.cursorignore), Codeium/Windsurf (.codeiumignore)	spec ↗

• Rules files - Per-tool rule files that scope agent behavior - older single-file forms (.cursorrules, .clinerules, .windsurfrules) and newer directory-based, glob-scoped forms (.cursor/rules/.mdc, .clinerules/, .windsurf/rules/.md).
• AI ignore files - gitignore-syntax files that fence an AI agent out of paths - secrets, vendored code, generated output - so they're never sent to the model as context.

Design

Standalone page: categories/design.md

	Convention	Files	Read by	Spec
🟢	DESIGN.md	DESIGN.md	Google Stitch natively; and coding agents (e.g. Claude Code) when pointed at it as design context	spec ↗

• DESIGN.md - A structured, machine-readable design specification - tokens, components, and layout intent - that an agent reads to generate or keep UI consistent with an established system. Open-sourced by Google Labs in 2026 as a cross-tool draft spec.

Web & discoverability

Standalone page: categories/web-discoverability.md

	Convention	Files	Read by	Spec
🟢	llms.txt	llms.txt llms-full.txt (pattern)	Docs sites publish it for LLM tools and crawlers - though no major provider has confirmed reading it	spec ↗
🟢	pricing.md	pricing.md	Agents and LLM browsers fetching a clean, parse-able pricing page	spec ↗

• llms.txt - A proposed-turned-widely-published standard: a root-level Markdown file giving LLMs a curated, link-rich map of a site's docs. Published across hundreds of developer-docs sites - though whether the major LLM providers actually read it remains unproven.
• pricing.md - The Markdown twin of a pricing page - same URL with a .md suffix - so an agent gets structured plans and numbers instead of scraping marketing HTML. A concrete, shipping instance of the page.md pattern.

Agent-web trust

Standalone page: categories/agent-web-trust.md

	Convention	Files	Read by	Spec
🟠	auth.md	auth.md	Agents discovering how to authenticate to a service (early adopters)	spec ↗
🔵	ai.txt	ai.txt	AI training/data-mining crawlers that voluntarily honor AI usage preferences; crawler support is not yet reliable	spec ↗

• auth.md - A Markdown file that tells an agent how to authenticate with a service - discovery of auth endpoints and flows. Shipped by WorkOS as a real, working convention, but adoption beyond it is still early.
• ai.txt - A text file declaring machine-readable consent, licensing, or policy preferences for AI training and data-mining. Spawning popularized the deployed root-file pattern, and a 2026 Internet-Draft now proposes a well-known URI; adoption and crawler obedience are still thin, so it stays 🔵.

Identity & protocols

Standalone page: categories/identity-protocols.md

	Convention	Files	Read by	Spec
🟠	Agent Cards (A2A)	agent-card.json agent.json (pattern)	A2A-compatible agents discovering another agent's capabilities	spec ↗

• Agent Cards (A2A) - The Agent2Agent (A2A) capability card - a JSON document at a well-known path advertising an agent's skills, endpoints, and auth so other agents can discover and call it. Now a Linux Foundation project at v1.0; adoption is growing but early.

Proposed namespace

Standalone page: categories/proposed-namespace.md

	Convention	Files	Read by	Spec
🔵	The protocols.md namespace	proof.md	- (no demonstrated readers; aspirational)	spec ↗

• The protocols.md namespace - A single maintainer's pre-registered namespace of ~74 aspirational .md "protocols" (proof.md, signature.md, reputation.md, …) staked as Schelling points for a future agent web. Published concept, no demonstrated adoption - see the page for the audited, honest caveats.

Maintaining examples

Example files are fetched, not invented. The extractor pulls them from public sources, stores them under conventions/<slug>/examples/<source>/<filename>, and adds a line-1 provenance comment. The examples remain under their upstream owners' licenses and terms; see THIRD_PARTY_EXAMPLES.md before reusing them.

To refresh everything:

pip install -r scripts/requirements.txt
python scripts/extract.py          # fetch real files + rebuild each convention's README
python scripts/build_readme.py     # rebuild this README from scripts/targets.json

Re-running is idempotent. A missing target prints a miss and is skipped. Examples are representative samples: any file over 256 KB (for example, a multi-MB llms-full.txt) is truncated with a marker pointing back to the full source. scripts/targets.json remains the source of truth for conventions that have not been migrated yet; the skill-md pilot uses local convention metadata instead. Edit the relevant source and re-run both scripts.

Shortcut targets are available in the Makefile:

make verify          # schema + generated files + example provenance + links
make extract         # refetch public examples and rebuild generated docs
make license-report  # summarize upstream licenses for vendored examples

CI keeps the generated files and links honest. The verify workflow checks that generated docs match catalog metadata and migrated local metadata, and that every spec, example, and instance URL still resolves on each pull request and weekly. Run the same link check locally with python scripts/check_links.py.

Contributing

Read CONTRIBUTING.md. In short: an entry must pass the filter above and carry evidence for its maturity tier. Add sources to scripts/targets.json for non-migrated conventions, run the scripts, and open a PR. The skill-md pilot uses local convention metadata instead. Do not hand-write example files.

Before proposing adjacent standards, check WATCHLIST.md. Project direction lives in ROADMAP.md.

License

The curation, scripts, and original prose in this repository are MIT. Vendored example files remain under their upstream owners' licenses and terms; see THIRD_PARTY_EXAMPLES.md.