CI/CD Workflows

Overview

The workflows/ directory contains GitHub Actions workflows that automate the continuous integration and delivery pipeline for the Research Project Template. These workflows ensure code quality, test reliability, and compatibility across environments.

Directory Structure

flowchart LR
    W[.github/workflows/]
    W --> META[AGENTS.md · README.md]
    W --> CI[ci.yml<br/>16 jobs — 2 conditional via detect-job outputs — fep-lean, setup-hook-windows-smoke — plus 1 scheduled-only — public-matrix-receipt]
    W --> STALE[stale.yml<br/>Auto-label/close stale issues/PRs]
    W --> REL[release.yml<br/>Create GitHub Releases on version tags]
    W --> DA[dependabot-automerge.yml<br/>Auto-merge safe Dependabot PRs]

    classDef d fill:#0f172a,stroke:#0f172a,color:#fff
    classDef code fill:#1e3a8a,stroke:#0f172a,color:#fff
    classDef doc fill:#0f766e,stroke:#0f172a,color:#fff
    class W d
    class CI,STALE,REL code
    class META doc

CI Pipeline (ci.yml)

Triggers

Trigger	Condition
push	Commits to main
pull_request	PRs targeting main
schedule	Weekly Sunday midnight UTC (CVE catch-up)
workflow_dispatch	Manual trigger (no inputs)

Concurrency: cancel-in-progress: true — stale runs are cancelled automatically when a new commit is pushed.

Global env: UV_FROZEN=true, MPLBACKEND=Agg (non-interactive matplotlib backend).

Job Graph

health depends on lint only and is blocking. validate, security, and docs-lint depend on lint only (parallel with the verify-no-mocks subtree). setup-hook-windows-smoke depends on verify-no-mocks and detect and is skipped unless needs.detect.outputs.setup_hook == 'true'. test-infra, test-regression, test-project, and fep-lean depend on verify-no-mocks.

flowchart TB
    DET[detect<br/>optional-project outputs]
    DETP[detect-projects<br/>validated capability matrix output]
    ACTLINT[actionlint<br/>lints workflow YAML · standalone]
    LINT[lint] --> HEALTH[health<br/>unified JSON artefact]
    VNM[verify-no-mocks<br/>parallel with lint · no needs:]
    LINT --> VAL[validate]
    LINT --> SEC[security]
    LINT --> DL[docs-lint<br/>mermaid + cross-links + consistency<br/>installs mmdc + chrome-headless-shell]
    VNM --> SHW[setup-hook-windows-smoke<br/>skipped if no setup_hook.py]
    VNM --> TI[test-infra<br/>matrix: ubuntu × 3.10/3.11/3.12/3.13 + macOS × 3.12<br/>codecov on 3.12/ubuntu only]
    VNM --> TR[test-regression<br/>claim-binding pins · tests/regression/]
    VNM --> TP[test-project<br/>capability manifest roster × canonical Python versions<br/>stage_01_test.py --project per cell]
    VNM --> FL[fep-lean<br/>ubuntu-only · skipped if no lean-toolchain]
    DET --> SHW
    DET --> FL
    DETP --> TP
    TI --> PERF[performance]
    TP --> PERF

    classDef gate fill:#1e3a8a,stroke:#0f172a,color:#fff
    classDef matrix fill:#0f766e,stroke:#0f172a,color:#fff
    classDef terminal fill:#7c2d12,stroke:#0f172a,color:#fff
    classDef info fill:#334155,stroke:#0f172a,color:#fff
    class DET,DETP,LINT,VNM gate
    class TI,TR,TP,FL,SHW matrix
    class VAL,SEC,DL,PERF,ACTLINT terminal
    class HEALTH info

Shared setup — local composite actions

Jobs that need Python share one local composite action, .github/actions/setup-python-env, which runs astral-sh/setup-uv (with uv.lock cache) + actions/setup-python. The pinned action SHAs and cache config live there once instead of being copy-pasted per job. Usage (checkout must come first — a local action's files only exist after checkout):

steps:
  - uses: actions/checkout@<sha>
  - uses: ./.github/actions/setup-python-env       # defaults to Python 3.12
    # with: { python-version: ${{ matrix.python-version }} }  # matrix jobs only
  - run: uv sync                                    # per-job groups stay explicit

detect and actionlint are checkout-only and do not use it. Each job keeps its own uv sync … line so optional groups (rendering/monitoring/discopy) remain visible per job. When bumping a setup action's SHA, edit the composite action — not each job.

The health and docs-lint jobs additionally share .github/actions/setup-docs-lint, which provisions the pinned Node runtime/cache plus real mmdc and chrome-headless-shell. This keeps unified health's docs-lint constituent behaviorally equivalent to the dedicated documentation job.

Job Details

1. Lint & Type Check (lint)

• Runner: ubuntu-latest / Python 3.12
• Tools: uv run ruff check, uv run ruff format --check, uv run mypy, uv run python -m infrastructure.skills check-all-exports, uv run python scripts/audit/check_template_drift.py --strict
• Scope: Ruff uses public lint paths from infrastructure.project.public_scope lint-paths; mypy uses its narrower source-paths output.

2. Static Health Report (health)

• Runner: ubuntu-latest / Python 3.12
• Depends on: lint
• Purpose: Runs uv run python -m infrastructure.core.health --json --quiet → health-report.json; every represented static gate blocks, while behavioral and platform matrices remain separate jobs.

3. Verify No Mocks Policy (verify-no-mocks)

• Runner: ubuntu-latest / Python 3.12
• Script: scripts/audit/verify_no_mocks.py (repository root)
• Enforced policy: no configured prohibited mock-framework imports/calls (MagicMock, mocker.patch, unittest.mock, and related lexical forms) in test files.
• Boundary: --inventory separately classifies permitted environment isolation and semantic dependency replacement. CI enforces a zero ceiling for dependency replacements; environment isolation remains permitted.

3b. Setup hook — Windows smoke (setup-hook-windows-smoke)

• Runner: windows-latest / Python 3.12
• Depends on: verify-no-mocks
• Conditional: if: needs.detect.outputs.setup_hook == 'true' — no-op skip when no project ships infrastructure.project.setup_hook. The detect job computes this because job-level hashFiles() is invalid in GitHub Actions.
• Step: uv run pytest tests/infra_tests/project/test_setup_hook.py with PYTHONUTF8=1

4. Infrastructure Tests (test-infra)

• Matrix: ubuntu-latest × 3.10, 3.11, 3.12, 3.13, plus an include: of macos-latest × 3.12 (5 cells). macOS legs are ~10x cost and rarely surface OS-specific breakage beyond the 3.12 cell, so only the 3.12 smoke runs there.
• Coverage threshold: 60% (--cov-fail-under=60)
• Coverage file: .coverage.infra (isolated from project coverage)
• Exclusions: Tests marked requires_ollama are skipped (-m "not requires_ollama")
• Codecov upload: On Python 3.12 / ubuntu-latest only to avoid duplicate reports

4b. Regression Tier — claim-binding pins (test-regression)

• Depends on: verify-no-mocks, timeout-minutes: 20, ubuntu-only.
• Sync: uv sync --group public-exemplars.
• What it runs: uv run pytest tests/regression/ -q --no-cov --timeout=120, serial (no -n auto) — see docs/maintenance/regression-testing.md for why (exemplars ship colliding top-level src packages resolved via per-project aliases + temporary sys.meta_path finders whose isolation is collection-order-sensitive).
• Exit-code tolerance: exit 5 (no tests collected on a clean scaffold) is treated as success so a future empty tier doesn't hard-fail the build; any real failure (exit 1) still fails the job. A separate "Assert regression tier is not empty" step fails the job when fewer than 3 tests collect, so the claim-binding pins cannot silently vanish behind the tolerance.

5. Project Tests (test-project)

• Sync: uv sync --group public-exemplars — the same deterministic dependency union as a fresh local uv sync, including the DisCoPy, monitoring, scientific, LLM-client, and PPTX groups used by the public roster. Hypothesis comes from the dev group (see root pyproject.toml [dependency-groups] and default-groups).
• Matrix: Per-project split — the detect-projects job runs scripts/gates/public_capabilities.py --ci-matrix-json, which validates unique normalized package identities, full-minor Python compatibility, source/test syntax, format declarations, compiled/confined direct hydration, analysis declarations, reason-bearing skips, exact roster membership, and exact matrix parity before emitting the canonical project × Python include list. The current source of truth yields 24 exemplars × Python 3.10/3.12 = 48 matrix cells on ubuntu-latest; no project or Python literal is duplicated in workflow YAML. Both matrix jobs set UV_PYTHON and assert the selected runtime minor so the repository .python-version cannot override a matrix cell. Job timeout-minutes: 60.
• Coverage threshold: Each job enforces that project's own ≥ 90% floor on its src/ (per CLAUDE.md). There is no longer a combined-union run or --cov-append — every project is isolated in its own job, which also removes the old code_project/fep_lean conftest plugin-name collision.
• Coverage file: .coverage.project (isolated; removed at the start of each job before the run)
• Scope: scripts/pipeline/stage_01_test.py --project <name> --project-only --include-slow (one invocation per matrix cell), then coverage xml -o coverage-project.xml. Rotating local projects are not part of this public-repo gate; dedicated project jobs own their own toolchains.
• Codecov upload: On Python 3.12 only

6. fep_lean — real Open Gauss + Lake (fep-lean)

• Conditional: Job is skipped unless projects/fep_lean/lean/lean-toolchain exists and the detect job emits fep_lean == 'true'. When fep_lean lives under projects/working/, detect reports false and the job is skipped. Promote with mv projects/working/fep_lean projects/fep_lean to activate.
• Runner: ubuntu-latest / Python 3.12 only; job timeout-minutes: 60
• Depends on: verify-no-mocks
• Working directory (when present): projects/fep_lean for pytest; projects/fep_lean/lean for Lake warm-up
• Toolchain: SHA-pinned elan installer (with checksum verification) + pinned lean-toolchain, lake build warm-up
• Open Gauss: clone math-inc/OpenGauss, ./scripts/install.sh --plain --noninteractive --skip-system-packages, gauss doctor
• Tests: uv run pytest tests/ --timeout=1200 --cov=src --cov-fail-under=89 with COVERAGE_FILE: ../../.coverage.fep_lean
• Scaling: Full catalogue × Lean is expensive; if runtime grows past the job budget, split slow integration tests behind a pytest marker or shard topics in a follow-up workflow.

7. Validate Manuscripts (validate)

• Runner: ubuntu-latest / Python 3.12
• Steps:
  1. infrastructure.validation.cli markdown projects/*/manuscript/ — validates all active project manuscripts
  2. scripts/docgen/api_reference.py --check — API reference drift gate
  3. Dynamic project import check — imports the public project source paths from infrastructure.project.public_scope

8. Security Scan (security)

• Runner: ubuntu-latest / Python 3.12
• pip-audit: blocking; builds --ignore-vuln args from .github/pip-audit-ignore.txt; retries up to 3 times with backoff on failure (transient OSV/network issues)
• bandit: bandit -c bandit.yaml -r -ll, covers infrastructure/, scripts/, projects/; path exclusions are in bandit.yaml (exclude_dirs, including archive/WIP roots and .venv / site-packages so local trees are not scanned)

9. Documentation Lint (docs-lint)

• Runner: ubuntu-latest / Python 3.12 / Node 20-compatible actions
• Depends on: lint
• Timeout: 15 minutes
• External tools (real, not mocked):
  • mmdc (mermaid-cli) — pinned in the root package.json; run npm ci
  • chrome-headless-shell — npx --no-install puppeteer browsers install chrome-headless-shell, exported via CHROME_EXECUTABLE_PATH
• Linters (thin orchestrator scripts/audit/lint_docs.py):
  1. Mermaid — every fenced ```mermaid block in docs/, infrastructure/, .github/, scripts/, and root *.md is rendered with the real mmdc binary. Failure exits non-zero.
  2. Cross-links — every relative Markdown link must resolve on disk; fenced and inline-code spans are skipped.
  3. Consistency — N Python (sub)packages claims must match the live count under infrastructure/; rotating project names (fep_lean, cogant, …) must be conditionally framed in long-lived docs.
  4. Doc pairs — permanent-template content folders must carry paired AGENTS.md and README.md; generated/local paths and rotating projects are excluded.
• Escape hatch: append <!-- noqa: docs-lint --> to a Markdown line to suppress consistency or broken-link warnings on that line.
• Scope guarantees: the linter skips generated/local paths such as output/, .venv/, .claude/, projects/archive/, projects/working/, htmlcov/, and node_modules/.
• Module: infrastructure/validation/docs/ — mermaid_lint.py, cross_link_lint.py, consistency_lint.py, doc_pair_lint.py.

10. Performance Check (performance)

• Runner: ubuntu-latest / Python 3.12
• Depends on: test-infra + test-project
• Threshold: each infrastructure.core or public project src cold import from infrastructure.project.public_scope must complete in ≤ 5 seconds
• Per-module timing and the roster-dependent total are reported to stdout for trend analysis

Quality Gates

Gate	Threshold	Enforced by
Ruff linting	zero violations	lint job
Ruff formatting	zero diffs	lint job
mypy strict gate	zero errors across the generated public source scope	lint job
Mock-framework lexical gate	zero prohibited imports/calls	verify-no-mocks job
Public capability parity	exact roster/matrix; compatible Python floors; valid package/render/hydration/analysis declarations	detect-projects + unified health
Infrastructure coverage	≥ 60%	test-infra job
Per-project coverage (standalone)	≥ 90%	each project's own pytest gate
Combined-union public-project coverage	≥ 75%	test-project job (DEFAULT_FAIL_UNDER)
fep_lean coverage	≥ 89%	fep-lean job (skipped if projects/fep_lean/lean/lean-toolchain absent)
pip-audit	no unignored vulnerabilities	security job
Bandit MEDIUM+ (bandit.yaml)	zero findings	security job
Import time	≤ 5 seconds total	performance job

---

Stale Workflow (stale.yml)

Runs daily at 01:00 UTC using actions/stale@v10.3.0.

Item	Stale after	Closed after
Issues	60 days inactive	+ 14 days
Pull Requests	30 days inactive	+ 14 days

Exempt labels: pinned, security, in-progress, blocked, do-not-close

---

Release Workflow (release.yml)

Triggers on v*.*.* tag push or workflow_dispatch (with tag input).

1. Resolves the requested tag before checkout and checks out that exact ref
2. Proves HEAD equals the dereferenced tag commit and runs the root release contract
3. Runs the bounded pipeline-smoke infrastructure test lane + no-mocks gate on the tagged SHA (executable test evidence, without the per-commit full matrix)
4. Runs the fail-closed public capability manifest
5. Clean-exports, installs, and import-smokes every canonical public exemplar without credentials
6. Runs the strict rendered publication audit across every canonical public exemplar before building
7. Generates a commit-based changelog excerpt since the previous tag
8. Creates a GitHub Release using softprops/action-gh-release@v3.0.2 with generate_release_notes: false so the body is the git-log excerpt only (no duplicate auto-generated section)
9. Auto-marks as pre-release if tag contains -rc, -beta, or -alpha

Current pinned GitHub Actions use the Node 20 action runtime. GitHub-hosted runners satisfy this; self-hosted runners must be Actions runner v2.327.1 or newer.

Dependabot Automerge Workflow (dependabot-automerge.yml)

Triggers on pull_request_target (opened, reopened, synchronize, ready_for_review) — but only acts when github.actor == 'dependabot[bot]'.

1. Fetches Dependabot update metadata via dependabot/fetch-metadata@v3.1.0
2. Enables GitHub native auto-merge (gh pr merge --auto --squash) for semver-minor and semver-patch updates only; major bumps are left for human review
3. Never checks out or executes PR HEAD code — only reads metadata and calls the GitHub API — so the elevated pull_request_target trigger does not expose secrets to untrusted code
4. Auto-merge must also be enabled in repository settings (Settings → General → "Allow auto-merge")

---

Local CI Simulation

# Reproduce lint locally
uv sync
uv run python -m infrastructure.project.public_scope lint-paths | xargs uv run ruff check
uv run python -m infrastructure.project.public_scope lint-paths | xargs uv run ruff format --check
uv run python -m infrastructure.project.public_scope source-paths | xargs uv run mypy

# Reproduce infrastructure tests locally
COVERAGE_FILE=.coverage.infra uv run pytest tests/infra_tests/ \
  --cov=infrastructure \
  --cov-fail-under=60 \
  -m "not requires_ollama"

# Reproduce project tests locally (matrix job ignores fep_lean). The root
# default groups include the deterministic public-exemplar dependency union.
uv sync
COVERAGE_FILE=.coverage.project uv run python scripts/pipeline/stage_01_test.py --project-only --all-projects --public-projects --non-strict --include-slow
uv run coverage xml -o coverage-project.xml

# fep_lean only — requires gauss, lake, lean on PATH (see that project's tests/AGENTS.md when present)
(cd projects/fep_lean && COVERAGE_FILE=../../.coverage.fep_lean uv run pytest tests/ \
  --timeout=900 \
  --cov=src \
  --cov-fail-under=89 \
  -m "not requires_ollama")

# Reproduce security scan locally (mirror CI — build ignores from file)
IGNORE_ARGS=()
while IFS= read -r raw || [ -n "$raw" ]; do
  [[ "$raw" =~ ^[[:space:]]*# ]] && continue
  line="${raw%%#*}"
  line="$(echo "$line" | xargs)"
  [ -z "$line" ] && continue
  IGNORE_ARGS+=(--ignore-vuln "$line")
done < .github/pip-audit-ignore.txt
uv run pip-audit "${IGNORE_ARGS[@]}"
uv run bandit -c bandit.yaml -r -ll infrastructure/ scripts/ projects/

---

Troubleshooting

Linting failures

uv run python -m infrastructure.project.public_scope lint-paths | xargs uv run ruff check --fix
uv run python -m infrastructure.project.public_scope lint-paths | xargs uv run ruff format

Test failures

# Infrastructure
uv run pytest tests/infra_tests/ -v --tb=long -s

# Project tests — prefer the orchestrator (runs one pytest per project; avoids conftest/package collisions):
uv run python scripts/pipeline/stage_01_test.py --project template_code_project

# Advanced / blanket globs — running **all** `projects/*/tests/` in **one** pytest process can fail when multiple projects ship `tests/conftest` packages with identical names; use per-project directories instead.

uv run pytest projects/*/tests/ -v --tb=long -s

# Single test
uv run pytest tests/infra_tests/test_foo.py::TestClass::test_method -s --pdb

Coverage below threshold

# Infrastructure report
COVERAGE_FILE=.coverage.infra uv run pytest tests/infra_tests/ \
  --cov=infrastructure --cov-report=html
open htmlcov/index.html

# Project report — replace SRC_PATH with `projects/<name>/src` from _generated/active_projects.md when benchmarking coverage manually:
COVERAGE_FILE=.coverage.project uv run pytest projects/templates/template_code_project/tests/ \
  --cov=projects/templates/template_code_project/src --cov-report=html
open htmlcov/index.html

Performance check slow

# Profile imports
uv run python -c "
import cProfile, pstats, io
pr = cProfile.Profile()
pr.enable()
import infrastructure.core
pr.disable()
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(20)
print(s.getvalue())
"

---

See Also

• ../README.md — .github/ human guide (workflows, Dependabot, templates)
• ../AGENTS.md — GitHub integration overview
• ../../AGENTS.md — Root system overview and pipeline stages
• GitHub Actions Documentation
• uv Documentation
• Codecov Documentation

🚀 Research Project Template

📄 Published: A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication — DOI: 10.5281/zenodo.19139090

Template Repository - Click "Use this template" to create a research project with this structure

Quickstart

Just cloned the repo? Do this:

1. git clone <this-repo> && cd template
2. uv sync (installs the root environment, including deterministic dependencies for all public template exemplars)
3. ./run.sh (interactive menu) or ./run.sh --pipeline --project templates/template_code_project --core-only (non-interactive, no LLM)
4. PDFs land in output/templates/<project>/pdf/. Logs in output/templates/<project>/logs/.
5. Run ./run.sh --help for all flags. The always-present roster is generated from PUBLIC_PROJECT_NAMES in docs/_generated/active_projects.md.

Repurposing for your research? See docs/repurposing-architectures.md — maps every reusable architecture (DAG pipeline, two-layer separation, evidence registry, multi-format rendering, MCP server, publishing stack) to its module and adoption path.

For deeper guidance see docs/guides/getting-started.md and docs/RUN_GUIDE.md.

Thin-orchestrator gates: uv run python scripts/audit/check_template_drift.py --strict, uv run python scripts/gates/module_line_count_check.py, uv run python -m infrastructure.core.health — details in docs/architecture/thin-orchestrator-summary.md.

Assistants and editors: .cursorrules summarizes architecture and tooling for Cursor; CLAUDE.md is the command cheat sheet; AGENTS.md is the full system manual (pipeline, validation, configuration). For routable agent workflows, start at docs/prompts/SKILL.md and the generated skill index docs/_generated/skills_index.md.

Contributors and CI: GitHub Actions, Dependabot, and PR/issue templates live under .github/README.md (agent entry point, doc map, CI inventory) and .github/AGENTS.md (job names, thresholds, troubleshooting).

Local hooks: After uv sync, run pre-commit install and pre-commit install --hook-type pre-push to mirror Ruff, mypy, Bandit, and smoke tests locally (see .pre-commit-config.yaml).

Strict Mermaid/PDF checks: Run npm ci at the repository root. The Python documentation and rendering gates automatically resolve the pinned node_modules/.bin/mmdc; add that directory to PATH only when invoking mmdc directly. The existing Chrome resolver supplies the browser executable.

A system for research and development projects. This template provides a test-driven structure with automated PDF generation, professional documentation, and validated build pipelines.

🧭 Positioning (honest framing)

This is primarily Daniel Ari Friedman's research operating system, made public and Apache 2.0-licensed so other researchers can fork it if helpful. It is not a one-size-fits-all template — it is opinionated, Python+pytest+LaTeX-flavored, and tuned to the kind of work Daniel does (Active Inference, computational biology, cognitive security). Honest framing ages better than wishful adoption metrics.

If your workflow looks similar (TDD-on-research-code, Markdown→PDF, multi-project monorepo, optional local-LLM draft assistance, deterministic + watermarked outputs, Zenodo DOI publishing), the template will probably save you time. If it doesn't look similar, a lighter alternative (Quarto, MyST, Cookiecutter-data-science) may serve you better. See MAINTAINERS.md for ownership and STATUS.md for per-subsystem freshness so you can judge what's actively maintained vs dormant.

Long-horizon viability guides — toolchain migration, regression testing, archival redundancy, local CI, and the design for a future executable-bundle stage — live in docs/maintenance/.

🎯 What This Template Provides

This is a GitHub Template Repository that gives you:

• ✅ Multi-project support - Run multiple projects in one repository
• ✅ Project structure with clear separation of concerns
• ✅ Test-driven development setup with coverage requirements
• ✅ Automated PDF generation from markdown sources
• ✅ Thin orchestrator pattern for maintainable code
• ✅ Methods orchestration linking pipeline contracts, methods prose, artifacts, and evidence
• ✅ Executable methods contracts with DAG, script, artifact, and verification validation
• ✅ Bounded parallel quick testing with isolated project processes and serial oracles
• ✅ Public exemplar capability inventory covering structure and declared skip reasons
• ✅ Ready-to-use utilities for any research project
• ✅ Professional documentation structure (full inventory: docs/documentation-index.md)
• ✅ Advanced quality analysis and document metrics
• ✅ Reproducibility tools for scientific workflows
• ✅ Integrity verification and validation
• ✅ Publishing tools for academic dissemination
• ✅ Scientific development best practices
• ✅ Reporting with error aggregation and performance metrics
• ✅ Local Ollama workflow documented in infrastructure/llm/README.md and docs/operational/troubleshooting/llm-review.md

🗺️ Choose Your Path

Pick the entry point that matches your goal:

• Cold start, human or agent (install prerequisites, run the canonical exemplar once): START_HERE.md.
• New users (write docs, generate PDFs): start with Quickstart, then docs/guides/getting-started.md and docs/reference/quick-start-cheatsheet.md.
• Developers (figures, data analysis, automation): docs/core/architecture.md, docs/architecture/thin-orchestrator-summary.md, docs/guides/methods-orchestration.md, docs/core/workflow.md, docs/guides/figures-and-analysis.md.
• Contributors (improve the template): docs/development/contributing.md, docs/development/code-of-conduct.md, docs/development/roadmap.md, docs/development/security.md.
• Advanced (system internals, modules, deep technical dive): AGENTS.md, docs/RUN_GUIDE.md, docs/modules/modules-guide.md, docs/documentation-index.md.

🧭 Documentation Hub

📚 Documentation Index | 📖 Documentation Guide | 🔍 Quick Reference

The template ships with a large documentation corpus under docs/. The full hierarchical map (with mermaid diagram) lives in docs/AGENTS.md; the authoritative per-file index lives in docs/documentation-index.md (rely on that index, not a hard-coded file count, which drifts). Top-level layout:

• docs/core/ — essential reading: how-to-use, architecture, workflow
• docs/guides/ — progressive walkthroughs by skill level (1–12)
• docs/operational/ — build, configuration, troubleshooting, performance
• docs/reference/ — FAQ, cheatsheet, common workflows, API reference
• docs/architecture/ — two-layer architecture, thin orchestrator, decision tree
• docs/usage/ — examples, showcase, markdown writing guide
• docs/modules/ — module-by-module guides
• docs/development/ — contributing, testing, roadmap

🤖 Agentic operation and SKILLS

Agents should load the smallest applicable workflow before editing. The routing surface is first-class and available via two skill discovery channels:

Hermes / agentskills.io project-local skills

Every canonical exemplar under projects/templates/ now ships its own .agents/skills/<name>/SKILL.md with YAML frontmatter discoverable by Hermes and agentskills.io runtimes. Each skill captures when to use, quick reference (pytest / analysis / render commands), pitfalls, and cross-refs for that template. Load the skill by its name (e.g. template-code-project, template-active-inference) when working inside that exemplar.

Skill	Template	When to load
template-active-inference	template_active_inference	Use this template when several independent research tracks must compose into one manuscript whose claims stay consistent where the tracks overlap — here: a closed-form analytical oracle, a pymdp simulation harness, a Lean formalization boundary, and shared GNN/ontology notation.
template-advanced-literature-review	template_advanced_literature_review	Use this template when one review question needs distinct retrieval phases, phase-specific filters, and explicit cross-phase provenance.
template-autoresearch-project	template_autoresearch_project	Use this template when you need a bounded, offline AutoResearch loop: deterministic ML candidate evaluation over a fixed local dataset, with evidence-linked claims, machine-readable ledgers, artifact-integrity manifests, and deferred human-review gates.
template-autopoiesis	template_autopoiesis	Use this template when you need to generate runnable project trees deterministically — not a manuscript, but a whole child project (its own src/, tests/, scripts/, and manuscript/) selected by a seed from a combinatoric grammar, with recompute-based provenance verification and a falsifiable honesty manifest against green-by-construction test theater.
template-code-project	template_code_project	Use this template for code-driven computational research: algorithms in src/, numerical experiments with deterministic seeds, automated publication-quality figures, and a manuscript that reports the computed results.
template-data-descriptor	template_data_descriptor	Use this template when the contribution is a published dataset or data paper: a schema contract, file inventory, data dictionary, provenance chain, license boundary, quality checks, and machine-readable descriptor must all stay consistent before publication.
template-eda-notebook	template_eda_notebook	Use this template for exploratory data analysis on tabular data: load a dataset, surface missingness, compute descriptive statistics and per-group means, rank features by correlation, and produce a few diagnostic figures.
template-formal	template_formal	Use this template when the research subject is the type architecture itself: illegal-state-unrepresentable design, session-typed protocols, affine/linear resource-handle discipline, or a decentralized (no-shared-global-state) multiagent simulation that needs its own local storage and local networking per agent.
template-literature-meta-analysis	template_literature_meta_analysis	Use it whenever the research object is a body of literature about a topic and you want every reported number to trace to committed, regenerable artifacts.
template-madlib	template_madlib	Use this template when you need configuration-driven manuscript generation with auditable token provenance, conditional section structure, explicit/default field visibility, failure-boundary reporting, and a reviewer-visible authoring contract: lexicon categories, section titles, narrative moves, method steps, design principles, pipeline phases, evaluation criteria, QA probes, failure modes, authoring obligations, visualization controls, audit rules, and slots are declared in YAML; src/ deterministically expands those declarations; and the manuscript receives large-grain {{TOKEN}} bodies only after the source code has generated the supporting artifacts.
template-newspaper	template_newspaper	Use this template when you need data-driven, large-format print layout: multi-page broadsheets/tabloids with precise column geometry, typography-first constraints, and strict content/engine separation (YAML editions in content/, pure-Python ReportLab engine in src/).
template-pools-rules-tools	template_pools_rules_tools	Use this project when you need to: - Demonstrate how a research project integrates multiple resource directories (fonds, tools, rules) in a single pipeline - Validate that your fonds, tools, and rule infrastructure modules are correctly wired and discoverable - Onboard new teams to the three-resource architecture with a concrete, runnable example - Test cross-cutting concern integration where fonds supply data, rules govern validation, and tools execute transforms - Extend the architecture by adding new resource types; copy this project as a starting point for integration testing This template is ideal for platform teams, CI engineers, and infrastructure maintainers who need a self-validating meta-project that exercises all three resource layers without coupling to a specific domain science.
template-prose-project	template_prose_project	Use this template for manuscript-focused editorial pipelines: readability gates (Flesch-Kincaid bands), structural checking, citation/BibTeX validation, and prose-quality review workflows where the document itself is the artifact under test.
template-autoscientists	template_autoscientists	Use this template when you need to isolate and measure agent-coordination mechanisms: coordinated teams vs single-thread baselines under matched budgets, deterministic ablation studies, and honest per-mechanism effect reporting (including nulls).
template-gold-refinement	template_gold_refinement	Use this template for analogical manuscript composition research: projects that map a scientific domain onto a refinement pipeline and generate the manuscript through deterministic mega-madlib token injection.
template-methods-paper	template_methods_paper	Use this template when the paper you are writing describes a methodology rather than reports results: a procedure, protocol, or specification language, where the contribution is the controlled vocabulary and its guarantees (dimensional safety, staged validation, deterministic compilation) rather than a numeric outcome.
template-redacted-report	template_redacted_report	Use this template when a report needs formal redaction before release: classification ceilings, source-control markings, redaction decisions, authority review, reviewer approvals, source-safe hash ledgers, residual-risk checks, mosaic-risk checks, and a public audit packet must be validated before any sanitized narrative is published.
template-registered-report	template_registered_report	Use this template when the core method is a registered report or preregistered replication: hypotheses, outcomes, exclusion rules, analysis plan, power or sensitivity rationale, deviation ledger, and confirmatory-versus-exploratory claim boundaries must be locked before results are interpreted.
template-search-project	template_search_project	Use this template for literature-review and evidence-synthesis pipelines: multi-backend search (arXiv, Crossref, local corpora, optional Paperclip), deduplication, BibTeX generation, and LLM-assisted per-paper and corpus-level synthesis.
template-sia	template_sia	Use this template when you need a self-improvement-agent evaluation harness: Meta → Target → Feedback generation loops, public/private task splits to detect overfitting, deterministic fixture replay for testability, and fail-closed loop validation.
template-storybook	template_storybook	Use this template when you need full-page illustrated PDF storytelling: picture-book pages, symbolic scenes, character-generation methods, page-level orchestrators, and deterministic raster art assembled into a print-ready PDF.
template-template	template_template	Use this template when your research subject is the repository itself — programmatic introspection of architecture, pipeline DAGs, module inventories, and security layers, rendered as a manuscript whose every metric is computed live (autopoietic: the paper regenerates itself from the code it describes).
template-textbook	template_textbook	Use this template for book-length manuscripts: parts → chapters → labs → question banks declared in a single config.yaml, with auto-numbering, deterministic figure/diagram generation, and structural-contract tests that keep hundreds of pages from drifting.
template-pitch-deck	template_pitch_deck	Use this template when you need a pitch, grant report, or recurring stakeholder update treated as a build artifact — bound to live repository facts, validated for unresolved tokens and pitch-deck cliché, and regenerable byte-for-byte from source rather than hand-maintained in a proprietary slide tool.

Generated infrastructure skills

In addition to the project-local skills, the repo generates infrastructure-level SKILL.md files from live discovery:

• Shared context-engineering skills: .agents/skills/ carries a pinned, provenance-checked Agent Skills collection discoverable by Codex/OpenAI and Hermes-compatible runtimes. Use uv run python -m infrastructure.skills runtime-status to audit Codex, Claude Code, and Hermes parity, or ... runtime-install to create reversible user-level links from the pinned shared store.
• Workflow router: docs/prompts/SKILL.md (template-workflows) routes broad requests such as full audits, pipeline debugging, code changes, tests, validation, manuscript work, and release checks to exactly one child workflow.
• Agentic-use hardening: docs/prompts/agentic-use/SKILL.md covers skill inventory, routing checks, .cursor/skill_manifest.json, and generated skill-index maintenance.
• Infrastructure module skills: infrastructure/SKILL.md is the Layer-1 hub; pair the relevant infrastructure/<module>/SKILL.md with that module's AGENTS.md before editing code.
• Script and resource-pool skills: scripts/*/SKILL.md files and public fonds/templates/, rules/templates/, and tools/templates/ skills are included when present, without scanning private lifecycle roots.
• Human skill index: docs/_generated/skills_index.md lists all discovered skills. Regenerate after skill changes with uv run python -m infrastructure.skills write-index; refresh the editor manifest with uv run python -m infrastructure.skills write; verify both with uv run python -m infrastructure.skills check and uv run python -m infrastructure.skills check-contracts. The same inventory is returned by the opt-in stdio MCP server's list_skills tool via uv run python -m infrastructure.mcp_server.

🔀 Multi-Project Support

The repo can host multiple research projects in parallel. Each project owns its own src/, tests/, manuscript/, scripts/, and output/ directory under projects/<name>/. Layer-1 infrastructure is shared.

Permanent canonical exemplars — always present and tracked in git:

Exemplar	When to use
template_active_inference	Use this template when several independent research tracks must compose into one manuscript whose claims stay consistent where the tracks overlap — here: a closed-form analytical oracle, a pymdp simulation harness, a Lean formalization boundary, and shared GNN/ontology notation.
template_advanced_literature_review	Use this template when one review question needs distinct retrieval phases, phase-specific filters, and explicit cross-phase provenance.
template_autoresearch_project	Use this template when you need a bounded, offline AutoResearch loop: deterministic ML candidate evaluation over a fixed local dataset, with evidence-linked claims, machine-readable ledgers, artifact-integrity manifests, and deferred human-review gates.
template_autopoiesis	Use this template when you need to generate runnable project trees deterministically — not a manuscript, but a whole child project (its own src/, tests/, scripts/, and manuscript/) selected by a seed from a combinatoric grammar, with recompute-based provenance verification and a falsifiable honesty manifest against green-by-construction test theater.
template_code_project	Use this template for code-driven computational research: algorithms in src/, numerical experiments with deterministic seeds, automated publication-quality figures, and a manuscript that reports the computed results.
template_data_descriptor	Use this template when the contribution is a published dataset or data paper: a schema contract, file inventory, data dictionary, provenance chain, license boundary, quality checks, and machine-readable descriptor must all stay consistent before publication.
template_eda_notebook	Use this template for exploratory data analysis on tabular data: load a dataset, surface missingness, compute descriptive statistics and per-group means, rank features by correlation, and produce a few diagnostic figures.
template_formal	Use this template when the research subject is the type architecture itself: illegal-state-unrepresentable design, session-typed protocols, affine/linear resource-handle discipline, or a decentralized (no-shared-global-state) multiagent simulation that needs its own local storage and local networking per agent.
template_literature_meta_analysis	Use it whenever the research object is a body of literature about a topic and you want every reported number to trace to committed, regenerable artifacts.
template_madlib	Use this template when you need configuration-driven manuscript generation with auditable token provenance, conditional section structure, explicit/default field visibility, failure-boundary reporting, and a reviewer-visible authoring contract: lexicon categories, section titles, narrative moves, method steps, design principles, pipeline phases, evaluation criteria, QA probes, failure modes, authoring obligations, visualization controls, audit rules, and slots are declared in YAML; src/ deterministically expands those declarations; and the manuscript receives large-grain {{TOKEN}} bodies only after the source code has generated the supporting artifacts.
template_newspaper	Use this template when you need data-driven, large-format print layout: multi-page broadsheets/tabloids with precise column geometry, typography-first constraints, and strict content/engine separation (YAML editions in content/, pure-Python ReportLab engine in src/).
template_pools_rules_tools	Use this project when you need to: - Demonstrate how a research project integrates multiple resource directories (fonds, tools, rules) in a single pipeline - Validate that your fonds, tools, and rule infrastructure modules are correctly wired and discoverable - Onboard new teams to the three-resource architecture with a concrete, runnable example - Test cross-cutting concern integration where fonds supply data, rules govern validation, and tools execute transforms - Extend the architecture by adding new resource types; copy this project as a starting point for integration testing This template is ideal for platform teams, CI engineers, and infrastructure maintainers who need a self-validating meta-project that exercises all three resource layers without coupling to a specific domain science.
template_prose_project	Use this template for manuscript-focused editorial pipelines: readability gates (Flesch-Kincaid bands), structural checking, citation/BibTeX validation, and prose-quality review workflows where the document itself is the artifact under test.
template_autoscientists	Use this template when you need to isolate and measure agent-coordination mechanisms: coordinated teams vs single-thread baselines under matched budgets, deterministic ablation studies, and honest per-mechanism effect reporting (including nulls).
template_gold_refinement	Use this template for analogical manuscript composition research: projects that map a scientific domain onto a refinement pipeline and generate the manuscript through deterministic mega-madlib token injection.
template_methods_paper	Use this template when the paper you are writing describes a methodology rather than reports results: a procedure, protocol, or specification language, where the contribution is the controlled vocabulary and its guarantees (dimensional safety, staged validation, deterministic compilation) rather than a numeric outcome.
template_redacted_report	Use this template when a report needs formal redaction before release: classification ceilings, source-control markings, redaction decisions, authority review, reviewer approvals, source-safe hash ledgers, residual-risk checks, mosaic-risk checks, and a public audit packet must be validated before any sanitized narrative is published.
template_registered_report	Use this template when the core method is a registered report or preregistered replication: hypotheses, outcomes, exclusion rules, analysis plan, power or sensitivity rationale, deviation ledger, and confirmatory-versus-exploratory claim boundaries must be locked before results are interpreted.
template_search_project	Use this template for literature-review and evidence-synthesis pipelines: multi-backend search (arXiv, Crossref, local corpora, optional Paperclip), deduplication, BibTeX generation, and LLM-assisted per-paper and corpus-level synthesis.
template_sia	Use this template when you need a self-improvement-agent evaluation harness: Meta → Target → Feedback generation loops, public/private task splits to detect overfitting, deterministic fixture replay for testability, and fail-closed loop validation.
template_storybook	Use this template when you need full-page illustrated PDF storytelling: picture-book pages, symbolic scenes, character-generation methods, page-level orchestrators, and deterministic raster art assembled into a print-ready PDF.
template_template	Use this template when your research subject is the repository itself — programmatic introspection of architecture, pipeline DAGs, module inventories, and security layers, rendered as a manuscript whose every metric is computed live (autopoietic: the paper regenerates itself from the code it describes).
template_textbook	Use this template for book-length manuscripts: parts → chapters → labs → question banks declared in a single config.yaml, with auto-numbering, deterministic figure/diagram generation, and structural-contract tests that keep hundreds of pages from drifting.
template_pitch_deck	Use this template when you need a pitch, grant report, or recurring stakeholder update treated as a build artifact — bound to live repository facts, validated for unresolved tokens and pitch-deck cliché, and regenerable byte-for-byte from source rather than hand-maintained in a proprietary slide tool.

Test and coverage figures are representative; confirm against docs/_generated/COUNTS.md after substantive changes.

Choosing an exemplar: every exemplar README opens with a ## When to use this template section, and the generated differentiation map in docs/_generated/exemplar_roster.md collects them into one "copy THIS when…" table (regenerate with uv run python scripts/docgen/exemplar_roster.py; sync is test-enforced).

The permanent exemplars share the same core layout and verification checklist. The code/prose exemplars also carry the 12-file project docs/ hub (agent_instructions.md, style_guide.md, syntax_guide.md, testing_philosophy.md, rendering_pipeline.md, faq.md, quickstart.md, output_conventions.md, troubleshooting.md, architecture.md, AGENTS.md, README.md). New projects copy whichever exemplar is closest in shape and adjust from there. See projects/AGENTS.md for the full comparison.

Publication metadata for every public exemplar is generated from project config and sidecars into docs/_generated/publication_records.md, and the GitHub-facing table in .github/README.md is auto-injected from that same source. To publish a project modularly, start with the Publication runbook: it covers the standalone public GitHub mirror, first real Zenodo DOI, new-version releases, optional mirrors, status blocks, and archival handoff.

Public Exemplar Outputs And Mirrors

Every canonical exemplar under projects/templates/ is tracked in this monorepo, including its project-local output/ tree. The copied release artifacts under output/templates/<name>/ are tracked as well, so a clone of docxology/template contains both the source and the latest rendered public artifacts. Public output files above 50 MB remain excluded by the generated-artifact guard; private or rotating project outputs remain blocked.

Each exemplar also has a standalone docxology/template_* GitHub repository linked to its Zenodo concept and latest version DOI. The current matrix is docs/_generated/publication_records.md. The standalone repository must exist before scripts/publish/publish_project_release.py can create a release there; the release script publishes the GitHub release asset and Zenodo deposit, but it does not create the repository itself. To regenerate any exemplar from the monorepo:

git clone https://github.com/docxology/template
cd template
uv sync
./run.sh --project templates/template_code_project --pipeline --core-only
uv run python scripts/pipeline/stage_04_validate.py --project templates/template_code_project
uv run python scripts/pipeline/stage_05_copy.py --project templates/template_code_project

Replace template_code_project with any public exemplar name from the table above. The standalone repositories are publication mirrors; use this monorepo when you need the shared infrastructure, full render pipeline, or cross-template validation.

The canonical exemplars also ship project-local composability overlays: domain_profile.yaml declares review gates, source policy, artifact expectations, and benchmark rubric preferences; experiment_plan.yaml declares design-validation conditions, primary metric direction, expected figures/tables, baselines, and ablations. These files are declarative inputs for validation and benchmark tooling; they do not generate experiments or run autonomous agents.

🔒 Confidentiality. This is a public template repo. Only the canonical exemplars above (under projects/templates/) are git-tracked/pushed — .gitignore ignores projects/* and negates only projects/templates/. Any other project you add under projects/ (research, client, or confidential work) stays local-only and is never committed; scripts/audit/check_tracked_all.py blocks any accidental commit in the pre-push hook and CI.

Private lifecycle projects. In Daniel's working checkout, confidential projects live outside this public repo at $TEMPLATE_PRIVATE_PROJECTS_ROOT. The simplified sidecar uses working/ and archive/; optional ongoing/ (long-lived projects with no publication target) plus legacy active/, published/, and other/ folders are still supported when present. run.sh and python -m infrastructure.orchestration auto-sync existing folders into matching typed subfolders under projects/: working/* into projects/working/*, ongoing/* into projects/ongoing/*, archive/* into projects/archive/*, and optional active/* into projects/active/*. templates/ and optional active/ links behave like native rendered entries; working/, ongoing/, and archive/ links are visible for explicit targeted work but are not default-rendered. Inspect without changing the tree: uv run python -m infrastructure.orchestration link-projects --dry-run. Override the sibling path with TEMPLATE_PRIVATE_PROJECTS_ROOT or .private_projects_root; disable auto-sync with TEMPLATE_SKIP_LINK_SYNC=1. The symlinked project keeps working outputs at projects/<subfolder>/<name>/output/ (the private target), while final deliverables still copy to output/<subfolder>/<name>/ in this template checkout.

Other entries rotate between projects/working/ and projects/archive/ as work progresses. Never hard-code their paths in long-lived docs — consult docs/_generated/active_projects.md (authoritative public scope, regenerated from infrastructure.project.public_scope) and docs/_generated/COUNTS.md instead.

Common commands:

./run.sh                                     # Interactive project selection
./run.sh --project templates/template_code_project --pipeline
./run.sh --all-projects --pipeline           # All discovered projects sequentially
./secure_run.sh --steganography-only --project templates/template_code_project  # Re-watermark PDFs
mkdir -p projects/my_research/{src,tests,manuscript,scripts}  # Scaffold new project

Lifecycle: rendered = projects/templates/ plus optional projects/active/ (discovered, executed). The simplified private sidecar normally uses working/ and archive/ (plus optional ongoing/ for long-lived work with no publication target); render sidecar projects explicitly with a qualified name such as working/{name} or ongoing/{name}. See projects/PROJECTS_PARADIGM.md for lifecycle, slug rules, and discovery semantics.

🚀 Quick Start {#quick-start}

See the Quickstart at the top of this file for the canonical clone-to-PDF flow. For headless cloud deployment use docs/CLOUD_DEPLOY.md (uv is installed automatically when you run ./run.sh --pipeline). Beginner walkthrough: docs/guides/getting-started.md. One-page command reference: docs/reference/quick-start-cheatsheet.md. Twelve-level usage guide: docs/core/how-to-use.md.

System Status

Current state is captured in docs/_generated/COUNTS.md (updated from discovery, test runs, and CI configuration).

Key elements:

• Active projects listed via discover_projects()
• Coverage enforced at 60% (infrastructure) and 90% (projects)
• Tests run with real data and computations
• Commands standardized to uv run
• Outputs organized per project under output/{name}/

See docs/_generated/COUNTS.md and docs/development/testing/testing-guide.md for details.

🎓 Skill-Based Learning Paths

Twelve progressive levels — Document Creation (1–3), Figures & Automation (4–6), Test-Driven Development (7–9), System Architecture (10–12) — are documented end-to-end in docs/core/how-to-use.md. Per-band walkthroughs: docs/guides/getting-started.md, docs/guides/figures-and-analysis.md, docs/guides/testing-and-reproducibility.md, docs/guides/extending-and-automation.md.

🏗️ Project Structure

Two-layer architecture:

• Layer 1 — infrastructure/ (generic, reusable): build, validation, rendering, LLM, publishing, etc. Plus scripts/ (entry-point orchestrators) and tests/ (infrastructure tests, ≥60 % coverage).
• Layer 2 — projects/<name>/ (project-specific, customizable): src/ (algorithms, ≥90 % coverage), tests/, scripts/ (thin orchestrators), manuscript/ (markdown sections + config.yaml).
• Output is generated under each exemplar's project-local projects/templates/<name>/output/ tree. Final deterministic evidence (publication PDFs, figures, analysis data, hydrated manuscripts, and release/validation registries) may be tracked for public exemplars; checkpoints, logs, telemetry, pipeline snapshots, and LaTeX/slide build intermediates are disposable, ignored, and regeneratable.
• Docs live under docs/ (full hierarchy in docs/AGENTS.md); per-directory AGENTS.md files document every leaf.

System Architecture Overview

A short summary lives here; full architecture diagrams (system overview, module-dependency graph, per-stage data flow, configuration-system flow) are maintained in AGENTS.md and docs/core/architecture.md. In short:

• Entry points: ./run.sh (interactive or --pipeline) and uv run python scripts/runner/execute_pipeline.py --project <name> [--core-only]; numbered orchestrators under scripts/ include 00_*.py through 07_*.py (setup → copy, LLM, executive report — see scripts/AGENTS.md).
• Orchestration: the pipeline runs Setup → Tests → Analysis → Render → Validate → Copy, with optional LLM Review and LLM Translations stages.
• Core systems: importable infrastructure/ packages (Layer 1; live list in docs/_generated/COUNTS.md) plus per-project projects/{name}/src/ algorithms (Layer 2); see docs/_generated/COUNTS.md for the live module list.
• Data flow: project source + manuscript markdown + config.yaml flow through the pipeline into output/<name>/{pdf,figures,data,reports}/.
• Quality assurance: infra ≥60 % and project ≥90 % coverage gates, no-mocks policy, deterministic seeds, real PDF/markdown validation.
• Configuration: projects/{name}/manuscript/config.yaml plus environment overrides feed PDF metadata, LaTeX preamble, figure labels, and validation rules.

Directory Overview with Documentation Links:

Directory	Purpose	Documentation
infrastructure/	Generic build/validation tools (Layer 1)	infrastructure/AGENTS.md
scripts/	Entry point orchestrators	scripts/AGENTS.md
tests/	Infrastructure test suite	tests/AGENTS.md
projects/{name}/src/	Project-specific scientific code (Layer 2)	Per-project AGENTS.md
projects/{name}/tests/	Project test suite	Per-project AGENTS.md
docs/	Documentation hub	docs/documentation-index.md
projects/{name}/manuscript/	Research manuscript sections	Per-project AGENTS.md
output/	Generated public evidence plus local build output	Final evidence is tracked selectively; runtime residue is ignored

📚 Explore Documentation: See docs/documentation-index.md for documentation structure

🔑 Key Architectural Principles

The repository follows a thin orchestrator pattern: business logic lives only in infrastructure/ and projects/{name}/src/; scripts coordinate, never implement. Tests prohibit mock frameworks and prefer real execution; remaining pytest.monkeypatch dependency replacements are explicitly inventoried rather than treated as proof of a mock-free suite. Coverage gates remain strict. Full narrative + benefits: docs/architecture/thin-orchestrator-summary.md, docs/core/architecture.md.

✨ Key Features

• Test-driven development with ≥60 % infra and ≥90 % project coverage gates (docs/core/workflow.md).
• Automated script execution via thin-orchestrator scripts under projects/{name}/scripts/ (scripts/AGENTS.md).
• Markdown-to-PDF pipeline with cross-referenced manuscripts and figure integration (docs/usage/markdown-template-guide.md, docs/modules/pdf-validation.md).
• Validated build system with 16 declared stages, a default 10-stage core+LLM path, an 8-stage --core-only path, and CI gates (docs/RUN_GUIDE.md).
• Generic + reusable — drop the same infrastructure/ into any project that follows the layout (docs/usage/template-description.md).

🔒 Security & Monitoring

LLM input sanitization (infrastructure.llm.core.sanitization), security validators (infrastructure.core.security), runtime health checks (infrastructure.core.runtime.health_check), rate limiting, and HTTP security headers. Full surface and worked usage examples: docs/development/security.md.

🛠️ Installation & Setup

Prerequisites: pandoc and a TeX distribution (texlive-xetex on Debian/Ubuntu, MacTeX on macOS). Python deps install with uv sync (project interpreter is .venv/bin/python; the template targets Python 3.10+ (requires-python in pyproject.toml) and CI tests infrastructure on 3.10–3.13, with .python-version pinning 3.12 as the local default). Add per-project deps with uv run python scripts/maintenance/manage_workspace.py add <package> --project <name>. To generate a manuscript, follow the Quickstart at the top.

Layer 1 is also a standard Python distribution. Build it with uv build, or install the wheel attached to a GitHub release. Installation provides the research-template command (research-template --help). Optional pip extras mirror the major capability groups, for example research-project-template[rendering,publishing]; repository contributors should continue to use uv sync --group ... for development groups.

🐳 Docker Support

docker-compose up (or docker build -t research-template . && docker run -it research-template) builds a reproducible image with pandoc, TeX, Ollama LLM server support, persistent model/output volumes, and hot-reload. See Dockerfile and docker-compose.yml.

🔧 Customization

Project Metadata Configuration

Two configuration paths exist: edit projects/{name}/manuscript/config.yaml (recommended) or export AUTHOR_NAME / AUTHOR_ORCID / AUTHOR_EMAIL / PROJECT_TITLE / DOI environment variables (env vars override the YAML file). The YAML schema (paper title, authors with ORCID, publication DOI, keywords, optional LLM translations block) and a worked example are documented once in CLAUDE.md and AGENTS.md; both files also list every available field. See projects/{name}/manuscript/config.yaml.example for the full template. Applied configuration drives PDF metadata, LaTeX document properties (see docs/reference/copypasta.md for preamble examples), generated file headers, and cross-reference systems.

Adding Project-Specific Scripts

Place Python scripts under projects/{name}/scripts/. They must follow the thin orchestrator pattern: import computation from projects/{name}/src/ or infrastructure/, handle only I/O / visualization / orchestration, print output paths to stdout for manifest collection, and never implement algorithms inline. Worked examples and the full pattern walkthrough live in scripts/AGENTS.md and docs/architecture/thin-orchestrator-summary.md.

Manuscript Structure

Per-project manuscript files live in projects/{name}/manuscript/: config.yaml, preamble.md, zero-padded numbered chapter files (00_abstract.md onward), optional S01_*.md supplements, and 99_references.md. Exact chapter slugs vary per project — the canonical exemplar is projects/templates/template_code_project/manuscript/; the numbering system and slug rules are authoritative in docs/usage/manuscript-numbering-system.md.

📊 Testing

TDD with strict coverage gates: infrastructure ≥ 60 %, projects ≥ 90 %. No unit-level mock frameworks — tests use real data, real files; HTTP boundaries use pytest-httpserver (in-process test server). The project pipeline runs a focused pipeline-smoke infrastructure contract plus the selected project's full coverage suite, so ordinary renders do not rerun the entire repository test matrix. Run the full infrastructure gate explicitly with uv run python scripts/pipeline/stage_01_test.py --infra-only --infra-scope full; run a project suite with uv run python scripts/pipeline/stage_01_test.py --project-only --project <name>. Per-suite commands and coverage report flags are documented in tests/AGENTS.md and docs/development/testing/testing-guide.md; live coverage / test counts live in docs/_generated/COUNTS.md.

Output

Working outputs: projects/{name}/output/. Public exemplars retain only deterministic deliverables and evidence under that tree; runtime state (.checkpoints/, .pipeline/, logs/, telemetry, snapshots) and renderer intermediates are local-only and regenerated by the pipeline. Multi-project mode adds output/executive_summary/ as a disposable copied-output surface.

🔍 How It Works

Two entry points — ./run.sh (interactive or --pipeline) and uv run python scripts/runner/execute_pipeline.py --project <name> [--core-only].

Pipeline (canonical phrasing — keep in sync with CLAUDE.md and AGENTS.md): The default pipeline.yaml declares 16 named stages: 8 core stages, 2 optional LLM stages, 2 opt-in ebook/metadata stages, 2 opt-in bundle/archival stages, and 2 opt-in science/provenance stages (Connector Search, Provenance Record). Default full runs include the 10 core+LLM stages (Clean Output Directories plus nine numbered stages). --core-only runs 8 stages by excluding LLM-tagged and opt-in stages. Ebook, metadata, bundle, archival, science, and provenance stages are declared for contracts but invoked separately when needed (directly via their scripts/pipeline/stage_*.py entry points).

Stage	Script	Tags	Failure mode
0 Clean Output Directories	built-in _run_clean_outputs	core, clean	soft fail
1 Environment Setup	scripts/pipeline/stage_00_setup.py	core	hard fail
2 Infrastructure Tests	scripts/pipeline/stage_01_test.py --infra-only --verbose --infra-scope pipeline-smoke	core, tests	configurable tolerance
3 Project Tests	scripts/pipeline/stage_01_test.py --project-only --verbose	core, tests	configurable tolerance
4 Project Analysis	scripts/pipeline/stage_02_analysis.py	core	hard fail
5 Connector Search	scripts/pipeline/stage_08_connector_search.py	science	skipped if not configured
6 Provenance Record	scripts/pipeline/stage_09_provenance_record.py --stage Connector Search	provenance	skipped if not configured
7 PDF Rendering	scripts/pipeline/stage_03_render.py	core	hard fail
8 Output Validation	scripts/pipeline/stage_04_validate.py	core	PDF/bookends and artifact/provenance failures block; optional-format structure remains a warning + report
9 LLM Scientific Review	scripts/pipeline/stage_06_llm_review.py --reviews-only	llm	skipped if Ollama absent
10 LLM Translations	scripts/pipeline/stage_06_llm_review.py --translations-only	llm	skipped if Ollama absent
11 Copy Outputs	scripts/pipeline/stage_05_copy.py	core	soft fail
12 Ebook Generation	scripts/pipeline/stage_11_ebook.py	core, ebook	soft fail
13 Metadata Package	scripts/pipeline/stage_12_metadata.py	core, metadata	soft fail
14 Executable Bundle	scripts/runner/bundle_executable.py	bundle	soft fail
15 Archival Publication	scripts/runner/archive_publication.py	archival	soft fail

Full per-stage flowchart, failure/skip transitions, and the script-to-stage mapping for --core-only live in AGENTS.md and docs/RUN_GUIDE.md. Workflow narrative: docs/core/workflow.md. Architecture narrative: docs/core/architecture.md.

📚 Documentation Index

The full per-file documentation index lives in docs/documentation-index.md (authoritative; counts drift, so it is not duplicated here). Top-level entry points:

• System reference: AGENTS.md, CLAUDE.md, docs/AGENTS.md
• Walkthroughs: docs/guides/getting-started.md, docs/core/how-to-use.md (12 skill levels), docs/reference/quick-start-cheatsheet.md
• Architecture: docs/core/architecture.md, docs/architecture/thin-orchestrator-summary.md, docs/architecture/two-layer-architecture.md
• Pipeline & build: docs/RUN_GUIDE.md, docs/CLOUD_DEPLOY.md
• Modules: docs/modules/modules-guide.md, infrastructure/AGENTS.md
• Quality / testing: tests/AGENTS.md, docs/development/testing/testing-guide.md
• Agent code navigation: docs/guides/codegraph-local.md, docs/guides/leann-local.md (optional local indexes; never committed artifacts)
• Best practices: docs/best-practices/best-practices.md
• Live facts (auto-derived): docs/_generated/COUNTS.md, docs/_generated/active_projects.md
• Per-directory AGENTS.md files in infrastructure/, scripts/, tests/, docs/, and every projects/{name}/{src,scripts,tests,manuscript}/.

🤝 Contributing

contribution guide | Code of conduct | Roadmap

We welcome contributions! To contribute:

1. Ensure all tests pass with coverage requirements met - Testing Guide
2. Follow the established project structure - Architecture
3. Add tests for new functionality - Workflow
4. Update documentation as needed - Documentation Guide
5. Maintain thin orchestrator pattern - scripts use src/ methods - Pattern Guide

Recent Improvements:

• Build system optimizations - Performance Optimization
• Test suite enhancements
• Simplified directory structure with markdown/ elimination

📄 License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

📚 Citation

The machine-readable CITATION.cff is the single source of truth (GitHub's "Cite this repository" widget reads it). If you use this template in your research, please cite:

Cite the current release. Earlier versions retain their own Zenodo DOIs; the version-independent concept DOI always resolves to the latest.

BibTeX:

@software{friedman_template_2026,
  author    = {Daniel Ari Friedman},
  title     = {A template/ approach to Reproducible Generative Research:
               Architecture and Ergonomics from Configuration through Publication},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19139090},
  url       = {https://doi.org/10.5281/zenodo.19139090}
}

Plain text: Daniel Ari Friedman. (2026). A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication. Zenodo. https://doi.org/10.5281/zenodo.19139090

🆘 Troubleshooting

Common issue catalog — failing tests, missing pandoc/xelatex, PDF quality, LLM unavailability — lives in docs/operational/troubleshooting/README.md and docs/reference/faq.md. Pipeline entry points and flags: docs/RUN_GUIDE.md. PDF validator: docs/modules/pdf-validation.md.

🔄 Migration from Other Projects

To adapt this template: copy infrastructure/ and scripts/, mirror the projects/{name}/{src,tests,scripts,manuscript}/ layout, adopt config.yaml (see AGENTS.md), and validate by running the pipeline. Worked examples: docs/usage/examples.md, docs/best-practices/migration-guide.md.

🏗️ Architecture Benefits

Thin orchestrator pattern delivers single-source-of-truth business logic, high testability (≥90 % project coverage), reusability across projects, and CI-gated quality. Full benefits + rationale: docs/core/architecture.md.

---

Quick Navigation by Task

Task	Start here
Assistants / Cursor	.cursorrules, CLAUDE.md, AGENTS.md
Write documents	docs/guides/getting-started.md, docs/usage/markdown-template-guide.md
Add figures	docs/guides/figures-and-analysis.md, docs/usage/visualization-guide.md
Fix issues	docs/operational/troubleshooting/README.md, docs/reference/faq.md
Understand architecture	docs/core/architecture.md, docs/architecture/two-layer-architecture.md
Configure system	docs/operational/config/configuration.md, AGENTS.md
Run pipeline	docs/RUN_GUIDE.md
Contribute code	docs/development/contributing.md, docs/rules/AGENTS.md
Find all docs	docs/documentation-index.md
Check the backlog	TO-DO.md
See what changed	CHANGELOG.md

---

🎉 Get Started Now

Ready to begin? Choose your path:

1. New User? → Start with Quick Start or docs/guides/getting-started.md
2. Developer? → Read docs/core/architecture.md and docs/core/workflow.md
3. Need Help? → Check docs/reference/faq.md or docs/operational/troubleshooting/README.md
4. Explore All Docs? → Browse docs/documentation-index.md

📚 Documentation Hub: All documentation is organized in the docs/ directory with guides for every aspect of the template.