@docxology/template-10

🤖 AGENTS.md - System Documentation

instructionscodexclaude

Install

agr install @docxology/template-10 --target claude

Writes 1 file into .claude/skills/, pinned to git-beb3f748.

.claude/skills/template-10/AGENTS.md

Document

🤖 AGENTS.md - System Documentation

🎯 System Overview

This document provides documentation for the Research Project Template system, ensuring understanding of all functionality, configuration options, and operational procedures.

📄 Publication

Title: A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication DOI: 10.5281/zenodo.19139090 · Record: zenodo.org/records/19139090

template/ applies Infrastructure as Code to the research lifecycle: version-controlled manuscripts, tests, provenance, and a declared pipeline DAG. Layer 1 (infrastructure/) is separated from Layer 2 (self-contained projects under projects/). Current measured counts and stage facts live in docs/_generated/COUNTS.md; re-derive them instead of copying literals into prose. Each directory carries README.md + AGENTS.md; infrastructure packages usually add SKILL.md for agent routing. Full paper, metrics, and claims: Zenodo record above and root README.md.

Documentation map

Entry	Role
`README.md`	Onboarding, documentation hub links, exemplar table
`.cursorrules`	Cursor agents: layer rules, CI scope, editing discipline
`CLAUDE.md`	Command cheat sheet, patterns; keep in sync with this file for pipeline wording
This file (`AGENTS.md`)	Full reference: stages, validation, modules, troubleshooting
`docs/documentation-index.md`	Flat index of long-lived docs
`docs/_generated/active_projects.md`	Authoritative public CI/documentation project names — never hard-code rotating private paths in docs
`docs/_generated/COUNTS.md`	Measured coverage and counts; refresh after changing gates or discovery
`.github/README.md`	GitHub: CI overview, templates, Dependabot
`.github/AGENTS.md`	Actions job names, coverage gates, local reproduction commands
`TO-DO.md`	Current backlog, scoped by acceptance line
`CHANGELOG.md`	Release history

For assistants and automation

Read order: README.md → CLAUDE.md → this file for anything not covered there.

Ground truth: Public CI/documentation project names come from docs/_generated/active_projects.md (infrastructure.project.public_scope). Runtime discover_projects() remains broader for local private symlinked workspaces. Measured numbers for documentation claims belong in docs/_generated/COUNTS.md; avoid inventing statistics.

Definition of done (code): Ruff clean on the full public lint surface from uv run python -m infrastructure.project.public_scope lint-paths, and mypy clean on the import-safe source paths from ... source-paths; tests exercise real behaviour, introduce no prohibited mock framework, and do not increase semantic dependency-replacement debt; coverage still meets 60% (infra) / 90% (project src/) unless CI documents a rotating-project exception (.github/AGENTS.md).

Hooks: .pre-commit-config.yaml — commit stage runs Ruff and mypy; pre-push adds no-mocks verification, a short pytest smoke module, Bandit (-c bandit.yaml, exclusions in YAML), and infrastructure.skills check + check-all-exports. Install: pre-commit install and pre-commit install --hook-type pre-push after uv sync.

Architecture: Business logic only in infrastructure/ or projects/{name}/src/. Scripts orchestrate; violating this breaks the test and documentation contract (Thin orchestrator pattern).

Optional CodeGraph: CodeGraph may be used as a local agent navigation index for source-code questions and impact analysis. It is not a dependency, pipeline stage, CI requirement, or manuscript evidence source. .codegraph/ is ignored and rejected by the generated-artifact guard if force-added. Use docs/guides/codegraph-local.md and uv run python scripts/maintenance/codegraph_local.py commands .; initialize private associated projects from their canonical private checkout, not through public template/projects/<name> symlinks.

Optional LEANN: LEANN may be used as a local semantic-retrieval companion for agent navigation. It is not a dependency, pipeline stage, CI requirement, MCP default, or manuscript evidence source. .leann/ is ignored and rejected by the generated-artifact guard if force-added. Use docs/guides/leann-local.md; build indexes from tracked public files or from the canonical private checkout, not through public template/projects/<name> symlinks.

Local agent memory (gitignored): When present, read .cursor/hooks/state/continual-learning-memory.json for durable user preferences and workspace facts. Never commit this file or write Learned sections into root AGENTS.md. Schema: .cursor/hooks/state/continual-learning-memory.example.json. Load/save helpers: infrastructure.core.agent_memory.

Public output hygiene: Canonical public exemplars may track deterministic publication evidence and release contracts under projects/templates/<name>/output/: final PDFs, figures, analysis data, hydrated manuscripts, and stable validation/publication registries. Runtime checkpoints, .pipeline/ state, logs, telemetry, stage snapshots, pipeline reports, and LaTeX/slide build intermediates are local-only and ignored. Recreate them through the canonical pipeline when diagnosing or rendering; do not publish them as exemplar evidence.

Template authoring & operational notes

Generic, Layer-1 facts for working in this repository.

Manuscript variables are injected, not hand-authored. Per-project metrics, counts, and variables come from output/data/manuscript_variables.json at render time. For template_code_project, the default pipeline calls generate_variables(..., require_analysis_outputs=True) via projects/{name}/scripts/z_generate_manuscript_variables.py and fails when output/data/optimization_results.csv is absent; pass --allow-draft only for intentional early drafts. PDF Publishing Information reads publication.doi, optional publication.repository_url, and publication.repository_label from projects/{name}/manuscript/config.yaml via infrastructure/rendering/_pdf_latex_helpers.py.
Validation & rendering pitfalls. Content-validation diagnostics use stable dotted IDs from infrastructure/validation/content/diagnostic_codes.py (MarkdownCode, BibtexCode); every new DiagnosticEvent must pass code=…, and renaming an existing code is a breaking change for downstream jq/rg filters. Fast manuscript pre-flight: uv run python -m infrastructure.validation.cli prerender projects/<project>/manuscript --repo-root .. Multi-pass PDF rendering continues when pass 1 wrote output despite recoverable Missing $ errors so later passes resolve forward references. Mermaid: unquoted // line comments; stadium nodes [/label/] close with /]; combined-PDF Mermaid via Chrome headless or mmdc, else verbatim figure fallback. FIGURE_WIDTH_* values must be bare fractions (e.g. 0.9); the alt-text comment belongs before \begin{figure}; prefer inline $...$ over $...$ in Markdown list items.
Entry points & gates. run.sh and secure_run.sh source only scripts/shell/shell_bootstrap.sh; menu and argparse live in infrastructure.orchestration. scripts/shell/bash_utils.sh serves backup/health scripts and tests, not pipeline entrypoints. Exemplar doc/code drift: scripts/audit/check_template_drift.py → infrastructure.project.drift.run_drift_checks() on PUBLIC_PROJECT_NAMES (--project, --strict). Layer 1 module size: scripts/gates/module_line_count_check.py and uv run python -m infrastructure.core.health (module-line-count). The health registry also runs the executable methods contract and scripts/gates/public_capabilities.py across the canonical public roster. Opt-in gates under scripts/gates/ report status: "skipped" under skipped_tools when tools are missing. bandit.yaml exclude_dirs skips rotating/private trees so CI stays strict on infrastructure/, scripts/, and public exemplars.

Confidentiality invariant (this is a PUBLIC repo)

.gitignore ignores projects/* and negates only the public canonical exemplar trees under projects/templates/ (plus the repo-level projects/*.md docs). The public exemplar roster is derived from infrastructure.project.public_scope.PUBLIC_PROJECT_NAMES and documented in docs/_generated/active_projects.md; do not hand-maintain a second allowlist in prose. Those public canonical exemplars are the only project trees ever git-tracked/pushed. Confidential/private work lives in a separate, external private repository whose location is configured with TEMPLATE_PRIVATE_PROJECTS_ROOT or .private_projects_root; the simplified sidecar normally has working/ and archive/, with optional ongoing/ (long-lived projects with no publication target) plus legacy active/, published/, and other/ folders still supported by the linker. run.sh/infrastructure.orchestration sync existing lifecycle folders into matching typed subfolders under projects/: working/* → projects/working/*, ongoing/* → projects/ongoing/*, archive/* → projects/archive/*, and optional active/* → projects/active/* (rendered). Only projects/templates/* and optional projects/active/* are discovered/rendered by default.

Every path under projects/ other than templates/ — especially the local-only working/, archive/, optional active/, published/, other/ mirrors — is local-only and must never be committed. This is enforced, not conventional: scripts/audit/check_tracked_all.py fails the CI lint job and the pre-push pre-push-quick hook if any non-template project path is tracked (a git add -f cannot slip past it). Consult docs/_generated/active_projects.md before hard-coding any project path in docs.

The same invariant covers three sibling top-level resource-pool directories — fonds/, rules/, tools/ (each analogous to projects/: only their templates/ subfolder is git-tracked, working//archive/ are LOCAL-ONLY, and each gets the same run.sh/infrastructure.orchestration sidecar auto-sync, independently overridable via TEMPLATE_FONDS_ROOT/TEMPLATE_SKIP_FOND_LINK_SYNC, TEMPLATE_RULES_ROOT/TEMPLATE_SKIP_RULE_LINK_SYNC, TEMPLATE_TOOLS_ROOT/TEMPLATE_SKIP_TOOL_LINK_SYNC). scripts/audit/check_tracked_all.py runs all four confidentiality checks (offending_tracked_projects/fonds/rules/tools in infrastructure/project/git_guards.py) in one pass; the narrower scripts/audit/check_tracked_projects.py still runs standalone but is no longer wired into CI or pre-commit.

Operational gotchas: running all projects/*/tests/ in one pytest process fails when projects each ship tests/conftest under the tests.conftest package name — run one project test directory per pytest invocation (with --cov-append to merge coverage), or follow .github/workflows/ci.yml. resolve_project_root accepts a qualified <subfolder>/<name> path (e.g. templates/template_code_project, active/demo) and resolves it directly under projects/. For a bare name it prefers projects/active/<name>/ (the hot seat) when that tree has project markers, then projects/working/<name>/, then a flat standalone projects/<name>/, falling back to projects/active/<name>/ for error messages.

Agent skills hub: docs/prompts/SKILL.md (template-workflows); regenerate via uv run python -m infrastructure.skills write and uv run python -m infrastructure.skills write-index. Repository-scoped context-engineering skills live under .agents/skills/ with a pinned source lock and cross-runtime sync/check command. Each canonical exemplar under projects/templates/ also ships its own .agents/skills/<name>/SKILL.md; both public lanes are included in .cursor/skill_manifest.json, the generated skills index, MCP list_skills, and Hermes/agentskills.io runtimes.

📋 Table of Contents

Assistants: For assistants and automation

Core Architecture
Directory-Level Documentation
Configuration System
Rendering Pipeline
Validation Systems
Testing Framework
Output Formats
Advanced Modules
Troubleshooting
Maintenance

🏗️ Core Architecture

Two-Layer Architecture

Layer 1: Infrastructure (Generic - Reusable)

infrastructure/ - Generic build/validation tools (reusable across projects)
scripts/ - Entry point orchestrators (core pipeline or full pipeline via ./run.sh)
tests/ - Infrastructure and integration tests

Layer 2: Projects (Project-Specific - Customizable)

projects/{name}/src/ - Research algorithms and analysis (domain-specific per project)
projects/{name}/tests/ - Project test suite
projects/{name}/scripts/ - Project analysis scripts (thin orchestrators)
projects/{name}/manuscript/ - Research manuscript
projects/{name}/output/ - Working outputs during pipeline execution
output/{name}/... - Final deliverables after pipeline completion

Thin Orchestrator Pattern

CRITICAL: All business logic resides in projects/{name}/src/ modules. Scripts are thin orchestrators that:

Root Entry Points (Generic):

Coordinate build pipeline stages
Discover and invoke projects/{name}/scripts/ for specified project
Handle I/O, orchestration only
Work with ANY project structure (single or multi-project)

Project Scripts (Project-Specific):

Import from projects/{name}/src/ for computation
Import from infrastructure/ for utilities
Orchestrate domain-specific workflows
Handle I/O and visualization

Violation of this pattern breaks the architecture.

Multi-Project Support

The template now supports multiple independent projects within a single repository:

Project Discovery:

Projects are discovered automatically from projects/ directory
Each project must have src/ and tests/ directories
Projects are validated for structural completeness

Project Isolation:

Each project has its own source code, tests, manuscript, and scripts
Working outputs are stored in projects/{name}/output/
Final deliverables are organized in output/{name}/...

Orchestration Options:

Run individual projects: --project {name}
Run all projects sequentially: --all-projects
Interactive project selection menu
Backward compatibility with single-project workflows

Active projects (under projects/): the set rotates as workspaces are promoted, archived, or moved. Authoritative names at any moment are only in docs/_generated/active_projects.md (regenerate after layout changes). The only path guaranteed to remain the control-positive exemplar for docs and commands is projects/templates/template_code_project/ (optimization research exemplar).

Private projects normally live in a separate external private repository (location configurable via TEMPLATE_PRIVATE_PROJECTS_ROOT or .private_projects_root) and are symlinked by lifecycle into matching typed subfolders under projects/. The simplified private sidecar uses working/ and archive/ by default; optional ongoing/ (long-lived projects with no publication target) plus legacy active/, published/, and other/ folders are still recognized when present. working/* mirrors into projects/working/*, ongoing/* into projects/ongoing/*, and archive/* into projects/archive/* for explicit inspection/rendering; optional active/* mirrors into projects/active/* for discovery/rendering. Use uv run python -m infrastructure.orchestration link-projects --dry-run to inspect the planned links, TEMPLATE_PRIVATE_PROJECTS_ROOT or .private_projects_root to override the sibling repo, and TEMPLATE_SKIP_LINK_SYNC=1 to disable auto-sync for a command.

Note: Exemplars such as blake_bimetalism, traditional_newspaper, area_handbook, density_bioscales may live under projects/archive/. In-progress trees live under projects/working/ until retired or explicitly rendered. Active names are listed in docs/_generated/active_projects.md.

📂 Project Organization: Rendered vs Non-Rendered Subfolders

Rendered Projects (`projects/templates/` + `projects/active/`)

Projects under projects/templates/ (tracked exemplars) and projects/active/ (hot-seat set) are actively discovered and executed by infrastructure:

Discovered by infrastructure.project.discovery.discover_projects() with qualified names templates/<name> and active/<name>
Listed in run.sh interactive menu
Executed by the canonical pipeline stages under scripts/pipeline/ (for example, stage_01_test.py and stage_02_analysis.py)
Outputs generated in projects/<subfolder>/{name}/output/ and copied to output/<subfolder>/{name}/

Non-Rendered Projects (`working/`, `ongoing/`, `archive/`, optional legacy mirrors)

Projects under projects/working/, projects/ongoing/, and projects/archive/ are preserved for explicit targeted work but not executed by default. Optional legacy projects/published/ and projects/other/ mirrors are treated the same way when present:

NOT discovered by default infrastructure discovery functions
NOT listed in the normal run.sh menu
NOT executed by all-project pipeline scripts
Preserved for in-progress work, long-lived no-publication-target work (ongoing/), historical reference, and explicit qualified commands such as --project working/<name> or --project ongoing/<name>

Project Lifecycle

Retiring in the sidecar: Move working/{name}/ → archive/{name}/ Resuming in the sidecar: Move archive/{name}/ → working/{name}/ Explicit render: From the template checkout, run uv run python scripts/pipeline/stage_03_render.py --project working/{name} after link-projects.

Projects are automatically discovered when a deliberately restored sidecar active/{name} entry is synced into projects/active/{name}. Normal sidecar work can also be rendered explicitly with a qualified name such as working/{name} without moving it into the default render set.

In-Progress Projects (`projects/working/`)

An intermediate staging area for projects that are under active development but not yet ready to run through the full pipeline. Projects here:

NOT discovered by infrastructure discovery functions
NOT listed in run.sh menu
NOT executed by any pipeline scripts
Useful for drafting new project scaffolding before explicitly rendering with a qualified name or deliberately restoring into optional projects/active/

Current in-progress projects: the roster rotates every checkout, so it is deliberately not hard-coded here — run ls projects/working/ for the live set (not executed by default by ./run.sh; render explicitly with a qualified project name such as working/<name>). Rendered projects are listed only in docs/_generated/active_projects.md.

To render by default: restore/sync the project through the sidecar's optional active/{name} folder so it appears as projects/active/{name}. Keep ordinary backburner work in sidecar working/ or archive/.

📚 Repository Structure

The template separates generic infrastructure from project-specific code:

flowchart TB
    ROOT[template<br/>Generic template repository]

    ROOT --> INFRA[infrastructure<br/>Layer 1 · generic build · validation tools]
    ROOT --> SCRIPTS[scripts<br/>Pipeline stage orchestrators 00–07]
    ROOT --> TESTS[tests<br/>Infrastructure test suite]
    ROOT --> DOCS[docs<br/>Documentation hub · 300+ files]
    ROOT --> PROJECTS[projects<br/>Typed subfolders · templates+active rendered]
    ROOT --> OUT[output<br/>Final deliverables · organized by project]

    INFRA --> I_DOCS[AGENTS.md · README.md · SKILL.md]
    INFRA --> I_CONFIG[config<br/>Repo-wide configuration]
    INFRA --> I_DOCKER[docker<br/>Container specs]
    INFRA --> I_SUB[Layer 1 packages listed in<br/>`infrastructure/AGENTS.md` —<br/>counts from COUNTS.md]

    PROJECTS --> P_README[README.md · multi-project guide]
    PROJECTS --> P_STUB[_test_project<br/>Stub · output/ only · not discovered]
    PROJECTS --> P_CODE[template_code_project<br/>Guaranteed control-positive exemplar]
    PROJECTS --> P_OTHER[&lt;name&gt;<br/>Additional discovered projects]

    P_CODE --> P_C_SRC[src · tests · scripts · manuscript · output<br/>+ pyproject.toml]

    OUT --> O_CODE[template_code_project<br/>Project outputs]
    OUT --> O_DOTS[&lt;other projects&gt;]

    classDef root fill:#0f172a,stroke:#0f172a,color:#fff
    classDef l1 fill:#1e3a8a,stroke:#0f172a,color:#fff
    classDef l2 fill:#0f766e,stroke:#0f172a,color:#fff
    classDef gen fill:#7c2d12,stroke:#0f172a,color:#fff
    class ROOT root
    class INFRA,SCRIPTS,TESTS,DOCS,I_CONFIG,I_DOCKER,I_SUB,I_DOCS l1
    class PROJECTS,P_CODE,P_OTHER,P_STUB,P_README,P_C_SRC l2
    class WIP,ARCH,OUT,O_CODE,O_DOTS gen

📚 Directory-Level Documentation

Each directory contains documentation for easy navigation:

Generic Infrastructure (Reusable)

Directory	AGENTS.md	README.md	Purpose
`infrastructure/`	AGENTS.md	README.md	Generic build/validation tools (Layer 1)
`scripts/`	AGENTS.md	README.md	Generic entry point orchestrators
`tests/`	AGENTS.md	README.md	Infrastructure test suite

Project-Specific (Customizable)

Directory	AGENTS.md	README.md	Purpose
`projects/templates/template_code_project/`	AGENTS.md	README.md	Code-centric exemplar (canonical, always present)
`projects/templates/template_prose_project/`	AGENTS.md	README.md	Prose-centric exemplar (canonical, always present)
`projects/templates/template_active_inference/`	AGENTS.md	README.md	Active Inference multi-track exemplar (canonical, always present)
`projects/templates/template_advanced_literature_review/`	AGENTS.md	README.md	Advanced multi-phase literature-review exemplar with phase provenance and offline replay (canonical, always present)
`projects/templates/template_autopoiesis/`	AGENTS.md	README.md	Combinatoric-grammar project-generation exemplar (canonical, always present)
`projects/templates/template_autoresearch_project/`	AGENTS.md	README.md	AutoResearch exemplar (canonical, always present)
`projects/templates/template_autoscientists/`	AGENTS.md	README.md	AutoScientists coordination-mechanism testbed exemplar (canonical, always present)
`projects/templates/template_data_descriptor/`	AGENTS.md	README.md	Dataset descriptor/data-paper exemplar with schema, provenance, and quality gates (canonical, always present)
`projects/templates/template_eda_notebook/`	AGENTS.md	README.md	EDA notebook exemplar with notebook-to-src binding and deterministic analysis outputs (canonical, always present)
`projects/templates/template_formal/`	AGENTS.md	README.md	Strongly typed multiagent colony exemplar with session-typed protocols and Lean/TLA+ formal side-specs (canonical, always present)
`projects/templates/template_gold_refinement/`	AGENTS.md	README.md	Gold-refining metallurgical analogy for manuscript composition (ore → nine-nines, mega-madlib token injection) (canonical, always present)
`projects/templates/template_literature_meta_analysis/`	AGENTS.md	README.md	Literature meta-analysis exemplar — multi-engine retrieval, de-dup, full-text, embeddings, bibliometrics; default term `modafinil` (canonical, always present)
`projects/templates/template_madlib/`	AGENTS.md	README.md	Conditional token-injection manuscript exemplar with QA probes and authoring contract (canonical, always present)
`projects/templates/template_methods_paper/`	AGENTS.md	README.md	Methods-paper exemplar — controlled-method specification DSL, staged validation, deterministic compilation, informed by BPL (canonical, always present)
`projects/templates/template_newspaper/`	AGENTS.md	README.md	Newspaper layout-engine exemplar (canonical, always present)
`projects/templates/template_pitch_deck/`	AGENTS.md	README.md	Pitch deck / slide deck scaffold exemplar (canonical, always present)
`projects/templates/template_pools_rules_tools/`	AGENTS.md	README.md	Fonds/rules/tools resource-pool integration exemplar (canonical, always present)
`projects/templates/template_redacted_report/`	AGENTS.md	README.md	Formal redaction and release-review exemplar with authority, ledger, and mosaic-risk gates (canonical, always present)
`projects/templates/template_registered_report/`	AGENTS.md	README.md	Registered-report/preregistration exemplar with locked hypotheses, outcomes, analysis plan, and deviation ledger (canonical, always present)
`projects/templates/template_sia/`	AGENTS.md	README.md	SIA harness exemplar (canonical, always present)
`projects/templates/template_storybook/`	AGENTS.md	README.md	Full-page illustrated storybook exemplar (canonical, always present)
`projects/templates/template_template/`	AGENTS.md	README.md	Meta-template exemplar (canonical, always present)
`projects/templates/template_textbook/`	AGENTS.md	README.md	Modular fillable-textbook scaffold exemplar (canonical, always present)
`projects/templates/template_search_project/`	AGENTS.md	README.md	Literature-search exemplar (canonical, always present)
Rotating projects (e.g. `actinf_policy_entanglement_lean`, private symlinked workspaces)	see project tree when checked out under a typed subfolder	see project tree when checked out under a typed subfolder	See `docs/_generated/active_projects.md` for current rendered roster; ordinary sidecar work rotates between `projects/working/` and `projects/archive/`

In-progress projects live under projects/working/ and are not executed by default pipeline discovery. Render one explicitly with a qualified project name such as working/<name>, or deliberately restore it through optional sidecar active/ only when it should enter default discovery. The roster rotates every checkout, so it is deliberately not hard-coded here (cf. the rotation rule above — hard-coding rotating project paths is the recurring staleness defect this guidance prevents): run ls projects/working/ for the live set, and see docs/_generated/active_projects.md for the rendered roster.

Ongoing projects live under projects/ongoing/ when present — long-lived work with no publication target, not discovered or executed by default. Render one explicitly with a qualified project name such as ongoing/<name>. The roster rotates every checkout and is deliberately not hard-coded here: run ls projects/ongoing/ for the live set.

Archived projects live under projects/archive/ when present, but the roster is checkout-specific and is not discovered or executed by default. Resume by moving them back to sidecar working/, render explicitly with archive/<name> when appropriate, or deliberately restore through optional sidecar active/ for default discovery. Use ls projects/archive/ for local inspection. The authoritative list of rendered projects is in docs/_generated/active_projects.md. Regenerate it after layout changes: uv run python scripts/docgen/active_projects.py.

Documentation Directories

Directory	AGENTS.md	README.md	Purpose
`docs/`	AGENTS.md	README.md	Project documentation hub

Documentation Navigation

For detailed information:

Read directory-specific AGENTS.md files for details
Each AGENTS.md covers architecture, usage, and best practices

For quick reference:

Check directory-specific README.md files for fast answers
Each README.md provides quick start and essential commands

Root documentation:

This file (root AGENTS.md) - System overview
README.md - Project quick start and introduction

Directory Structure

flowchart TB
    ROOT[template<br/>Generic Template]

    ROOT --> INFRA[infrastructure<br/>Layer 1 · generic build/validation tools]
    ROOT --> DOCS[docs<br/>Documentation hub]
    ROOT --> CUR[.cursor<br/>Editor configuration]
    ROOT --> SCR[scripts<br/>Pipeline stage entry points]
    ROOT --> TS[tests<br/>Infrastructure tests]
    ROOT --> PR[projects<br/>Typed subfolders · templates+active rendered]
    ROOT --> OUT[output<br/>Final deliverables · disposable]
    ROOT --> PYPROJ[pyproject.toml<br/>Root configuration]

    INFRA --> I_DOCS[AGENTS.md · README.md · SKILL.md]
    INFRA --> I_CFG[config<br/>.env.template · secure_config.yaml]
    INFRA --> I_DOCKER[docker<br/>Dockerfile · docker-compose.yml]
    INFRA --> I_MODULES[build_verifier.py · figure_manager.py · ...]

    DOCS --> D_FILES[AGENTS.md · README.md ·<br/>CLOUD_DEPLOY.md · PAI.md · RUN_GUIDE.md]

    CUR --> C_FILES[.cursorrules · .cursorignore · README.md]

    SCR --> S_DOCS[AGENTS.md · README.md]
    SCR --> S_STAGES[scripts/pipeline/stage_00_setup.py<br/>stage_01_test.py<br/>stage_02_analysis.py<br/>stage_03_render.py<br/>stage_04_validate.py<br/>stage_05_copy.py]

    TS --> T_FILES[AGENTS.md · README.md · test_*.py]

    PR --> PR_CODE[template_code_project<br/>Optimization exemplar · active]
    PR --> PR_OTHER[&lt;name&gt;<br/>additional discovered projects]
    PR_CODE --> PRC_LAYOUT[src · tests · scripts ·<br/>manuscript · output<br/>+ pyproject.toml]

    classDef root fill:#0f172a,stroke:#0f172a,color:#fff
    classDef l1 fill:#1e3a8a,stroke:#0f172a,color:#fff
    classDef l2 fill:#0f766e,stroke:#0f172a,color:#fff
    classDef gen fill:#7c2d12,stroke:#0f172a,color:#fff
    class ROOT,PYPROJ root
    class INFRA,DOCS,CUR,SCR,TS,I_DOCS,I_CFG,I_DOCKER,I_MODULES,D_FILES,C_FILES,S_DOCS,S_STAGES,T_FILES l1
    class PR,PR_CODE,PR_OTHER,PRC_LAYOUT l2
    class WIP,ARC,OUT gen

Documentation in each directory:

AGENTS.md - Detailed directory-specific documentation
README.md - Quick reference and navigation

Note on src/ directory:

Root src/ no longer exists (was empty shells)
All code is in infrastructure/ (generic) or projects/{name}/src/ (project-specific)
This separation enables reusability across projects

⚙️ Configuration System

Configuration File (Recommended)

The system supports configuration through a YAML file, providing a centralized, version-controllable way to manage all paper metadata.

Location: projects/{name}/manuscript/config.yaml Template: projects/{name}/manuscript/config.yaml.example

Example configuration:

paper:
  title: "Novel Optimization Framework"
  subtitle: ""  # Optional
  version: "1.0"

authors:
  - name: "Dr. Jane Smith"
    orcid: "0000-0000-0000-1234"
    email: "jane.smith@university.edu"
    affiliation: "University of Example"
    corresponding: true

publication:
  doi: "10.5281/zenodo.12345678"  # Optional
  journal: ""  # Optional
  volume: ""  # Optional
  pages: ""  # Optional

keywords:
  - "optimization"
  - "machine learning"

metadata:
  license: "Apache-2.0"
  language: "en"

# LLM Review Settings (optional)
llm:
  reviews:
    enabled: true
    types:
      - executive_summary  # Default: single review
      # Uncomment to enable additional reviews:
      # - quality_review
      # - methodology_review
      # - improvement_suggestions
  translations:
    enabled: true  # Set to false to disable translation generation
    languages:
      - zh  # Default: single translation (Chinese Simplified)
      # Uncomment to enable additional languages:
      # - hi  # Hindi
      # - ru  # Russian

Benefits:

✅ Version controllable (can be committed to git)
✅ Single file for all metadata
✅ Supports multiple authors with affiliations
✅ Structured format (YAML)
✅ Easy to edit and maintain

Environment Variables (Alternative Method)

Environment variables are supported as an alternative configuration method and take precedence over config file values:

Variable	Default	Description
`AUTHOR_NAME`	`"Project Author"`	Primary author name
`AUTHOR_ORCID`	`"0000-0000-0000-0000"`	Author ORCID identifier
`AUTHOR_EMAIL`	`"author@example.com"`	Author contact email
`DOI`	`""`	Digital Object Identifier (optional)
`PROJECT_TITLE`	`"Project Title"`	Project/research title
`LOG_LEVEL`	`1`	Logging verbosity (0=DEBUG, 1=INFO, 2=WARN, 3=ERROR)

Priority order:

Environment variables (highest priority - override config file)
Config file (projects/{name}/manuscript/config.yaml)
Default values (lowest priority)

Configuration Examples

Using Configuration File (Recommended)

# Edit projects/{name}/manuscript/config.yaml with your information
vim projects/{name}/manuscript/config.yaml

# Build with config file values
uv run python scripts/pipeline/stage_03_render.py --project {name}

Using Environment Variables

export AUTHOR_NAME="Dr. Jane Smith"
export PROJECT_TITLE="Novel Optimization Framework"
export AUTHOR_EMAIL="jane.smith@university.edu"
export AUTHOR_ORCID="0000-0000-0000-1234"
export DOI="10.5281/zenodo.12345678"  # Optional

uv run python scripts/pipeline/stage_03_render.py

Verbose Logging

export LOG_LEVEL=0  # Show all debug messages
uv run python scripts/pipeline/stage_03_render.py

Runtime Configuration

Configuration is read at runtime by scripts/pipeline/stage_03_render.py and applied to:

PDF metadata (title, author, date)
LaTeX document properties
Generated file headers
Cross-reference systems
Title page generation

🚀 Rendering Pipeline

Pipeline Execution

The template provides three entry points for pipeline execution:

Main Entry Point (Recommended)

# Routes to manuscript operations
./run.sh

Manuscript Operations

# Interactive menu with manuscript operations
./run.sh

# Non-interactive: default full pipeline — 10 core+LLM stages; pipeline.yaml declares four additional opt-in ebook/metadata/bundle/archival stages. --core-only drops LLM and opt-in stages and leaves 8.
./run.sh --pipeline

LLM review stages use the local Ollama workflow documented in infrastructure/llm/README.md. Canonical smoke commands:

ollama serve
ollama pull gemma3:4b
uv run pytest tests/infra_tests/llm/ -m requires_ollama -v

Secure Pipeline (`secure_run.sh`)

Two steps: (1) Pipeline run — the same DAG as the normal pipeline through Python PipelineRunner / PipelineExecutor — not by shelling out to ./run.sh; (2) Steganography pass — run SteganographyProcessor on the resulting PDFs.

Pipeline run is skipped when --steganography-only. When the pipeline run executes, --project <name> is required (single project per invocation). For steganography-only with no --project, PDFs for all discovered projects are processed.

For argv shaping into the secure subcommand from the same thin shell as ./run.sh, use ./run.sh --secure-run (see run.sh). ./secure_run.sh always execs python -m infrastructure.orchestration secure and does not replicate the full interactive main menu by itself.

Steganography pass: post-processes PDFs (companion *_steganography.pdf, .hashes.json manifest). Original PDFs stay untouched.

# Interactive path that forwards to the secure subcommand (same orchestration CLI as ./run.sh)
./run.sh --secure-run

# Full pipeline + steganography for one project
./secure_run.sh --project template_code_project

# Core DAG only (no LLM stages) + steganography
./secure_run.sh --project template_code_project --core-only

# Re-process existing PDFs only (omit --project to cover every discovered project)
./secure_run.sh --steganography-only --project template_code_project

# Multi-project: run pipelines separately, then harden all PDFs without re-running pipelines
./secure_run.sh --steganography-only

Output files:

flowchart LR
    PDF[projects/&lt;name&gt;/output/pdf]
    PDF --> A[&lt;name&gt;_combined.pdf<br/>Standard output · untouched]
    PDF --> B[&lt;name&gt;_combined_steganography.pdf<br/>Steganographically hardened copy]
    PDF --> C[&lt;name&gt;_combined.hashes.json<br/>SHA-256/SHA-512 integrity manifest]

    classDef d fill:#0f172a,stroke:#0f172a,color:#fff
    classDef f fill:#0f766e,stroke:#0f172a,color:#fff
    class PDF d
    class A,B,C f

Steganographic techniques: diagonal watermark overlays, QR + barcode strips, PDF metadata/XMP injection, SHA-256/SHA-512 hash manifests, invisible text layers, optional AES-256 password encryption.

Configuration (infrastructure/config/secure_config.yaml):

Controls all steganography settings. Any steganography: block in a project's manuscript/config.yaml overrides these repo-level defaults. Key fields:

steganography:
  overlays_enabled: true       # Diagonal watermark
  barcodes_enabled: true       # QR + Code128 strip
  metadata_enabled: true       # PDF metadata + XMP
  hashing_enabled: true        # SHA-256/512 manifest
  encryption_enabled: false    # AES-256 password (set pdf_password to enable)
  overlay_mode: "text"         # "text" | "qr" | "none"
  overlay_text: "CONFIDENTIAL"
  overlay_opacity: 0.08        # 0.02 subtle → 0.30 strong
  output_suffix: "_steganography"

Entry Point Comparison

./run.sh: Main entry point — interactive menu or pipeline run. Bash progress: [0/9] clean, then [1/9]–[9/9] for nine tracked steps (labels from STAGE_NAMES, kept in sync with pipeline.yaml).
./run.sh --pipeline: Non-interactive full DAG; optional LLM stages may skip if Ollama is unavailable.
./run.sh --secure-run: Forwards to the secure orchestration subcommand (same Python CLI as bare ./run.sh; use when you want argv shaping from the main shell).
./secure_run.sh: Ensures steganography extras (uv sync --group steganography), then python -m infrastructure.orchestration secure. --project is required when running the pipeline phase (omit only for --steganography-only across all projects). See Secure Pipeline above.
uv run python scripts/runner/execute_pipeline.py --project {name} --core-only: Core DAG only — 8 stages in default infrastructure/core/pipeline/pipeline.yaml (LLM-tagged stages excluded); no LLM dependencies.

Pipeline Stages

Full Pipeline Stages — the default pipeline.yaml declares 16 named stages: 8 core stages, 2 optional LLM stages, 2 opt-in ebook/metadata stages, 2 opt-in bundle/archival stages, and 2 opt-in science/provenance stages (Connector Search, Provenance Record). Default full runs include the 10 core+LLM stages (Clean Output Directories plus nine numbered stages). run.sh displays that default path as [0/9] for clean and [1/9]–[9/9] for the nine numbered stages. --core-only runs 8 stages by excluding LLM-tagged and opt-in stages.

[0/9] Clean Output Directories - Clean working and final output directories (pre-step)

Environment Setup - Verify system requirements and dependencies
Infrastructure Tests - Run the focused pipeline-smoke infrastructure contract (may be skipped; full coverage gate is explicit)
Project Tests - Run project test suite (90% coverage minimum)
Project Analysis - Execute projects/{name}/scripts/ analysis workflows
PDF Rendering - Generate manuscript PDFs and figures
Output Validation - Validate all generated outputs
LLM Scientific Review - AI-powered manuscript analysis (optional, requires Ollama)
LLM Translations - Multi-language technical abstract generation (optional, requires Ollama)
Copy Outputs - Copy final deliverables to root output/ directory

Opt-in long-horizon stages (added 2026-05-20; NOT in default core or --core-only runs — enable via --tags ebook, --tags metadata, --tags bundle, or --tags archival):

Ebook Generation (scripts/pipeline/stage_11_ebook.py, tag ebook) — Generate EPUB, MOBI, and DOCX ebooks from the combined markdown manuscript. Gracefully skips (exit 2) when the combined markdown is absent. Invoke: uv run python scripts/pipeline/stage_11_ebook.py --project <name>.
Metadata Package (scripts/pipeline/stage_12_metadata.py, tag metadata) — Generate ONIX 3.0 XML, metadata.json, and OPF skeleton from manuscript/config.yaml. Gracefully skips (exit 2) when config.yaml is absent. Invoke: uv run python scripts/pipeline/stage_12_metadata.py --project <name>.
Executable Bundle (scripts/runner/bundle_executable.py, tag bundle) — Produce a container + lockfile + agent-runnable manifest.json for the project, parallel to PDF as the durable artifact. Design: docs/maintenance/stage-10-executable-bundle.md.
Archival Publication (scripts/runner/archive_publication.py, tag archival) — Mirror the executable bundle to multiple independent archival targets (Zenodo, Software Heritage, IPFS via Pinata/Web3.Storage). Defaults to dry-run; pass --commit to actually deposit. Design: docs/maintenance/archival-targets.md.

Infrastructure Tests Behavior:

Single project pipeline mode: Stage 2 runs scripts/pipeline/stage_01_test.py --infra-only --infra-scope pipeline-smoke, a focused real suite for DAG execution, advisory controls, evidence/profile/benchmark extension points, doc invariants, and tracked-artifact guards. This keeps project rebuilds fast without hiding the full repo gate.
Full infrastructure gate: Run uv run python scripts/pipeline/stage_01_test.py --infra-only --infra-scope full (or the direct pytest command in the verification guide) for the coverage-bearing repository suite.
Multi-project mode (--all-projects): Infrastructure tests run once for all projects at the start, then are skipped for individual project executions to avoid redundant testing. This is shown in logs as "Running infrastructure tests once for all projects..." followed by "Skipping stage: Infrastructure Tests" for each project.

Multi-Project Executive Reporting (--all-projects mode only):

Executive Reporting - Cross-project metrics, summaries, and visual dashboards (generated after all projects, not as a numbered stage)

Stage numbering (canonical phrasing — keep in sync with CLAUDE.md and README.md):

The default pipeline.yaml declares 16 named stages: 8 core stages, 2 optional LLM stages, 2 opt-in ebook/metadata stages, 2 opt-in bundle/archival stages, and 2 opt-in science/provenance stages (Connector Search, Provenance Record). Default full runs include the 10 core+LLM stages (Clean Output Directories plus nine numbered stages). --core-only runs 8 stages by excluding LLM-tagged and opt-in stages. Ebook, metadata, bundle, archival, science, and provenance stages are declared for contracts but invoked separately when needed (directly via their scripts/pipeline/stage_*.py entry points).

Manual Execution Options

Individual Stage Execution:

# Environment setup
uv run python scripts/pipeline/stage_00_setup.py --project {name}

# Test execution (combined infra + project)
uv run python scripts/pipeline/stage_01_test.py --project {name}

# Project analysis scripts
uv run python scripts/pipeline/stage_02_analysis.py --project {name}

# PDF rendering
uv run python scripts/pipeline/stage_03_render.py --project {name}

# Output validation
uv run python scripts/pipeline/stage_04_validate.py --project {name}

# Copy outputs
uv run python scripts/pipeline/stage_05_copy.py --project {name}

# LLM manuscript review (optional, requires Ollama)
uv run python scripts/pipeline/stage_06_llm_review.py --project {name}

# Generate executive report (multi-project only)
uv run python scripts/pipeline/stage_07_executive_report.py --project {name}

Validation Tools:

# Validate markdown files
uv run python -m infrastructure.validation.cli markdown projects/{name}/manuscript/

# Validate PDF outputs
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf

Validation Systems

PDF Validation

# Validate generated PDF for issues (per-project)
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf

# With verbose output
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf --verbose

# Specific PDF file
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf

Validation Checks:

Unresolved references (??)
Missing citations ([?])
LaTeX warnings and errors
Document structure integrity
Word count and content preview

Markdown Validation

# Validate all markdown files
uv run python -m infrastructure.validation.cli markdown projects/{name}/manuscript/

# Strict mode (fail on any issues)
uv run python -m infrastructure.validation.cli markdown projects/{name}/manuscript/ --strict

Validation Checks:

Image reference resolution
Cross-reference integrity
Equation label validation
Link formatting
Mathematical notation

Test Coverage

See docs/_generated/COUNTS.md for current status from live test runs.

# Run via orchestrator
uv run python scripts/pipeline/stage_01_test.py --project {name}

# Manual with reports
uv run pytest tests/infra_tests/ --cov=infrastructure --cov-report=html
uv run pytest projects/{name}/tests/ --cov=projects/{name}/src --cov-report=html

Requirements:

projects/{name}/src/ : 90% minimum (per-project standalone gate; exemplars meet it)
combined-union public-project gate (scripts/pipeline/stage_01_test.py --project-only --all-projects --public-projects, DEFAULT_FAIL_UNDER) : 75% — deliberately lower than the per-project floor because per-project suites only cover their own src/ while the union denominator spans the public exemplar source set. Local --all-projects without --public-projects still runs every discovered project, including rotating private symlinks. Per-project floors remain authoritative.
infrastructure/ : 60% minimum

Tests should use real data and computation. The lexical no-mocks gate and the semantic inventory are both enforced: dependency replacements must remain at zero, while environment/path isolation is classified separately and permitted.

Testing Framework

Mock-framework prohibition and semantic-stand-in inventory

CRITICAL REQUIREMENT: Do not introduce MagicMock, mocker.patch, unittest.mock, or another mocking framework. Prefer real data, local services, subprocesses, and dependency injection. The lexical CI gate proves only that prohibited framework imports/calls are absent; --inventory separately records environment isolation and existing monkeypatch.setattr/setitem dependency replacement.

uv run python scripts/audit/verify_no_mocks.py
uv run python scripts/audit/verify_no_mocks.py --inventory --max-dependency-replacements 0

This policy ensures:

Tests validate actual behavior, not mocked behavior
Integration points are truly tested
Code is tested in realistic conditions
No false confidence from mocked tests

No-Mocks Implementation Patterns

HTTP API Testing: Use pytest-httpserver for local test servers

# BEFORE (mocked)
with patch('requests.post') as mock_post:
    mock_post.return_value = MagicMock(status_code=200, json=lambda: {"result": "ok"})

# AFTER (HTTP)
def test_api_call(ollama_test_server):
    # ollama_test_server fixture provides HTTP server
    config = OllamaClientConfig(base_url=ollama_test_server.url_for("/"))
    client = LLMClient(config)
    response = client.query("test")  # HTTP request
    assert "response" in response.lower()

CLI Testing: Execute subprocess commands instead of mocking sys.argv

# BEFORE (mocked)
with patch('sys.argv', ['cli.py', 'validate', 'file.pdf']):
    cli.main()

# AFTER (subprocess)
result = subprocess.run(
    ['python', '-m', 'infrastructure.validation.cli', 'validate', 'file.pdf'],
    capture_output=True, text=True
)
assert result.returncode == 0

PDF Generation: Create PDFs with reportlab instead of mocking PDF libraries

# BEFORE (mocked)
with patch.dict('sys.modules', {'pdfplumber': mock_pdfplumber}):
    result = extract_text(pdf_file)

# AFTER (PDF)
from reportlab.pdfgen import canvas
c = canvas.Canvas(str(pdf_file))
c.drawString(100, 750, "Test content")
c.save()

result = extract_text(pdf_file)  # PDF processing
assert "Test content" in result

File System Operations: Use temp files and directories

# BEFORE (mocked)
with patch('builtins.open') as mock_open:
    mock_open.return_value.__enter__.return_value.read.return_value = "content"

# AFTER (files)
def test_file_operation(tmp_path):
    test_file = tmp_path / "test.txt"
    test_file.write_text("content")
    result = read_file(test_file)  # File operation
    assert result == "content"

External Tool Testing: Keep default tests deterministic; mark opt-in tool/credential tests or fail with setup guidance

# BEFORE (mocked subprocess)
with patch('subprocess.run') as mock_run:
    mock_run.return_value = MagicMock(returncode=0)

# AFTER (tool execution in an opt-in marker group)
@pytest.mark.requires_latex
def test_pandoc_conversion(tmp_path):
    pandoc = shutil.which('pandoc')
    assert pandoc, "Install pandoc before running requires_latex tests"
    md_file = tmp_path / "test.md"
    md_file.write_text("# Test")
    pdf_file = tmp_path / "test.pdf"

    result = subprocess.run([pandoc, str(md_file), '-o', str(pdf_file)])
    assert result.returncode == 0
    assert pdf_file.exists()

Test Structure

Tests follow the thin orchestrator pattern principles:

Import methods from projects/{name}/src/ or infrastructure/ modules
Use data and computation
Validate actual behavior; avoid adding semantic dependency replacements
Ensure reproducible, deterministic results

Test Categories

Unit Tests (test_*.py) - Individual function validation
Integration Tests - Script and pipeline integration
Validation Tests - PDF and markdown quality checks

Running Tests

# All tests via orchestrator (recommended)
uv run python scripts/pipeline/stage_01_test.py

# Specific test file
uv run pytest projects/{name}/tests/test_example.py -v

# Infrastructure tests with coverage
uv run pytest tests/infra_tests/ --cov=infrastructure --cov-report=html

# Project tests with coverage
uv run pytest projects/{name}/tests/ --cov=projects/{name}/src --cov-report=html

📤 Output Formats

Generated Files Structure

flowchart TB
    OUT[output]
    OUT --> P[project<br/>Project-specific outputs]
    P --> PDF[pdf<br/>PDF documents]
    P --> TEX[tex<br/>LaTeX source files]
    P --> FIG[figures<br/>Generated figures]
    P --> DATA[data<br/>Generated datasets]
    P --> HTML[project_combined.html<br/>HTML version for IDE]

    PDF --> PDF_SECT[01_abstract.pdf · 02_introduction.pdf · ...]
    PDF --> PDF_COMB["{name}_combined.pdf<br/>Full manuscript"]

    TEX --> TEX_FILES[01_abstract.tex · project_combined.tex · ...]

    FIG --> FIG_FILES[example_figure.png · convergence_plot.png · ...]

    DATA --> DATA_FILES[example_data.csv · convergence_data.npz · ...]

    classDef d fill:#0f172a,stroke:#0f172a,color:#fff
    classDef f fill:#0f766e,stroke:#0f172a,color:#fff
    class OUT,P,PDF,TEX,FIG,DATA d
    class PDF_SECT,PDF_COMB,TEX_FILES,FIG_FILES,DATA_FILES,HTML f

PDF Versions

Standard PDF ({project}_combined.pdf, e.g. template_code_project_combined.pdf)
- Professional printing format
- Optimized for LaTeX rendering
- Cross-references and citations
IDE-Friendly PDF (project_combined_ide_friendly.pdf)
- for text editor viewing
- Better font rendering in IDEs
- Simplified layout for screen reading
HTML Version (project_combined.html)
- Web browser compatible
- IDE integration
- Interactive features (when available)

🧪 Advanced Modules

The template includes reusable infrastructure modules for scientific development; see docs/modules/modules-guide.md and docs/_generated/COUNTS.md for the live module list and counts. Selected module examples:

🔒 Core Utilities (`infrastructure/core/`)

Enterprise-grade security and system monitoring.

Key Features:

Input Sanitization: LLM prompt validation and threat detection
Security Monitoring: Security event tracking and alerting
Rate Limiting: Configurable request rate limiting with monitoring
Health Checks: System health monitoring with component-level status
Security Headers: HTTP security header implementation

Usage:

from infrastructure.llm.core.sanitization import sanitize_llm_input
from infrastructure.core.security import get_security_validator
from infrastructure.core import SystemHealthChecker

# Validate LLM input with security checks
sanitized = sanitize_llm_input(user_prompt)

# Perform system health check
checker = SystemHealthChecker()
if checker.is_healthy():
    status = checker.get_health_status()

🔍 Validation System (`infrastructure/validation/`)

Comprehensive file integrity, manuscript structures, cross-references, and output validation.

Key Features:

File Integrity: Hash-based verification of output files
Cross-Reference Validation: LaTeX reference integrity checking
Data Consistency: Format and structure validation
Academic Standards: Compliance with writing standards
Build Artifact Verification: Output validation

Usage:

from infrastructure.validation.integrity import verify_output_integrity, generate_integrity_report

report = verify_output_integrity(output_dir)
print(generate_integrity_report(report))

🔬 Scientific Development (`infrastructure/scientific/`)

Scientific computing best practices and tools.

Modular Structure:

stability.py - Numerical stability checking
benchmarking.py - Performance benchmarking
documentation.py - API documentation generation
validation.py - Best practices validation
templates.py - Research workflow templates

Key Features:

Numerical Stability: Algorithm stability testing
Performance Benchmarking: Execution time and memory analysis
Scientific Documentation: API documentation generation
Best Practices Validation: Code quality assessment
Research Workflow Templates: Reproducible experiment templates

Usage:

from infrastructure.scientific import check_numerical_stability, benchmark_function

stability = check_numerical_stability(your_function, test_inputs)
benchmark = benchmark_function(your_function, test_inputs)

🤖 LLM Integration (`infrastructure/llm/`)

Local LLM assistance for research workflows.

Key Features:

Ollama Integration: Local model support (privacy-first)
Template System: Pre-built prompts for common research tasks
Context Management: Multi-turn conversation handling
Streaming Support: Response generation
Model Fallback: Automatic fallback to alternative models
Token Counting: Track usage and costs

Research Templates:

Abstract summarization
Code documentation
Data interpretation
Section drafting assistance
Citation formatting
Technical abstract translation (Chinese, Hindi, Russian)

Usage:

from infrastructure.llm import LLMClient

client = LLMClient()
summary = client.apply_template("summarize_abstract", text=abstract)
response = client.query("What are the key findings?")

🎨 Rendering System (`infrastructure/rendering/`)

Multi-format output generation from single source.

Key Features:

PDF Rendering: Professional LaTeX-based PDFs
Presentation Slides: Beamer (PDF) and reveal.js (HTML) slides
Web Output: Interactive HTML with MathJax
Scientific Posters: Large-format poster generation
Format-Agnostic: Single source, multiple outputs
Quality Validation: Automated output checking

Usage:

from infrastructure.rendering import RenderManager

manager = RenderManager()
pdf = manager.render_pdf("manuscript.tex")
slides = manager.render_slides("presentation.md", format="revealjs")
html = manager.render_web("manuscript.md")
all_outputs = manager.render_all("manuscript.md")

🚀 Publishing Module (`infrastructure/publishing/`)

Automated publishing to academic platforms.

Module Structure:

core.py - Publication metadata extraction, DOI validation, citation generation
api.py - Platform API clients (Zenodo, arXiv, GitHub)
citations.py - Citation helpers (BibTeX CLI target plus APA/MLA library helpers)
metadata.py - Publication metadata management
platforms.py - Platform-specific integration logic

Key Features:

Zenodo Integration: Upload with DOI minting
arXiv Preparation: Submission package creation
GitHub Releases: Automated release management
Metrics Tracking: Download and citation tracking
Distribution Packages: Publication bundles

Usage:

from infrastructure.publishing import (
    extract_publication_metadata,
    publish_to_zenodo,
    create_github_release,
    prepare_arxiv_submission
)

# Extract metadata
metadata = extract_publication_metadata([Path("manuscript.md")])

# Publish to Zenodo
result = publish_to_zenodo(metadata, files, token)
print(result.doi)

# Create GitHub release
release = create_github_release(metadata, files, token)

# Prepare arXiv submission
package = prepare_arxiv_submission(metadata, files)

Module Integration

All advanced modules follow the thin orchestrator pattern:

Business logic in infrastructure/ modules with test coverage
Orchestration in separate utility scripts
Integration with existing build pipeline
Testing ensuring reliability
Documentation for each module's functionality

Additional opt-in modules:

Scientific Connector Registry (infrastructure/search/connectors/) — uniform Connector protocol over 8 science databases; python -m infrastructure.search.connectors list-dbs / search — see infrastructure/search/connectors/AGENTS.md
Provenance DAG (infrastructure/provenance/) — content-addressed artifact tracking and lineage — see infrastructure/provenance/AGENTS.md
Research prompts (infrastructure/research/) — reusable LLM prompt templates for research workflows — see infrastructure/research/AGENTS.md

Testing Coverage: measured per module in docs/development/coverage-gaps.md; suite locations under tests/infra_tests/.

✅ Security — tests/infra_tests/core/
✅ Health Check — tests/infra_tests/core/
✅ Input Sanitization — tests/infra_tests/llm/
✅ Integrity — tests/infra_tests/validation/
✅ Publishing — tests/infra_tests/publishing/
✅ Scientific Dev — tests/infra_tests/scientific/
✅ Build Verifier — tests/infra_tests/validation/
✅ LLM Integration — tests/infra_tests/llm/ (measured coverage → docs/development/coverage-gaps.md)
✅ Rendering System — tests/infra_tests/rendering/ (measured coverage → docs/development/coverage-gaps.md)
✅ Reporting — tests/infra_tests/reporting/ (measured coverage → docs/development/coverage-gaps.md)

Accessing Outputs

# Open combined PDF
open output/{name}/pdf/{name}_combined.pdf

# Open HTML version in browser
open output/{name}/{name}_combined.html

# List all generated files
ls -la output/{name}/

# Check PDF validation
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf

🔧 Troubleshooting

Quick Reference

General Troubleshooting: docs/operational/troubleshooting/
Common Errors: docs/operational/troubleshooting/common-errors.md
LLM Review Issues: docs/operational/troubleshooting/llm-review.md
Checkpoint/Resume: docs/operational/config/checkpoint-resume.md
Performance Issues: docs/operational/config/performance-optimization.md
Headless / Cloud Deploy: docs/CLOUD_DEPLOY.md ☁️

Common Issues

Tests Failing

# Ensure coverage requirements met for both suites
uv run python scripts/pipeline/stage_01_test.py

# Or run individually with coverage reports
uv run pytest tests/infra_tests/ --cov=infrastructure --cov-fail-under=60
uv run pytest projects/{name}/tests/ --cov=projects/{name}/src --cov-fail-under=90

Project Fails In `--all-projects` But Passes Standalone

A multi-project run (./run.sh --all-projects) executes projects sequentially over a long wall time; a project can fail there yet pass when re-run on its own. Triage:

# Re-run only the failed project (the multi-project summary prints this command)
./run.sh --project {name} --pipeline --core-only --skip-infra

# Or resume the whole run from the last good checkpoint
./run.sh --all-projects --pipeline --resume

FileNotFoundError: .../projects/{name}/.venv/bin/python at Project Tests — the project's .venv/ directory survived with a dangling bin/python symlink (its base interpreter was moved/removed, e.g. a relocated miniforge/uv install). The test runner now detects this via resolve_test_python() (infrastructure/core/runtime/_python_env.py): it only uses the project interpreter when that file actually exists and otherwise falls back to the workspace interpreter, so a stale .venv/ no longer crashes the stage. To remove the stale env entirely: rm -rf projects/{name}/.venv (it is regenerated on demand; the analysis stage uses uv run which self-heals it).
Transient Project Tests / PDF Rendering failures under load — flaky failures during a 38-min multi-project sweep that do not reproduce standalone. Confirm health directly: a project's own gate (projects/{name}/output/reports/test_results.json → total_failed: 0) and render artifacts (projects/{name}/output/pdf/*_combined.pdf or output/{name}/pdf/*_combined.pdf present, with no ^! lines in adjacent .log files). If clean, re-run that single project.

`validate_docs_and_figures.py` exit 1 at Project Analysis (rotating project)

Some rotating/private projects ship a read-only self-validator. Its run-report, session, and output-link checks often assert a completed canonical run (run_all.py / later pipeline stages). Run as a plain analysis script it should treat not-yet-generated artifacts as non-blocking warnings; pass --strict to require the full canonical run as a hard gate. Keep project-specific artifact paths in the project docs, not in this root manual.

Scripts Failing

# Run scripts individually to debug
uv run python scripts/pipeline/stage_02_analysis.py --project {name}

# Check import errors
uv run python -c "import importlib; importlib.import_module('projects.{name}.src')"

PDF Generation Issues

# Check LaTeX installation
which xelatex

# Validate LaTeX packages (pre-flight check)
uv run python -m infrastructure.rendering.latex_package_validator

# Validate markdown first
uv run python -m infrastructure.validation.cli markdown projects/{name}/manuscript/

# Check compilation logs
ls projects/{name}/output/pdf/*_compile.log output/{name}/pdf/*_compile.log

Missing LaTeX Package Errors:

If you see "File *.sty not found" during PDF rendering:

Identify the missing package from the error message

Install via tlmgr (BasicTeX package manager):

sudo tlmgr update --self
sudo tlmgr install multirow cleveref doi newunicodechar

Verify installation:

/usr/local/texlive/2025basic/bin/universal-darwin/kpsewhich multirow.sty

Run pre-flight validation:

uv run python -m infrastructure.rendering.latex_package_validator

Common missing packages in BasicTeX:

multirow, cleveref, doi, newunicodechar - Require installation
bm, subcaption - Already included (part of tools and caption)

Alternative: Install full MacTeX (~4 GB) instead of BasicTeX (~100 MB):

brew install --cask mactex

Missing Dependencies

# Install system dependencies
# Ubuntu/Debian:
sudo apt-get install -y pandoc texlive-xetex texlive-fonts-recommended fonts-dejavu

# macOS (BasicTeX - minimal):
brew install pandoc
brew install --cask basictex
sudo tlmgr update --self
sudo tlmgr install multirow cleveref doi newunicodechar

# macOS (MacTeX):
brew install pandoc
brew install --cask mactex

Debug Mode

# Enable verbose logging
export LOG_LEVEL=0
uv run python scripts/pipeline/stage_03_render.py --project {name}

# Run with debug output
uv run python -m infrastructure.validation.cli pdf output/{name}/pdf/{name}_combined.pdf --verbose

Log Files

Key log files for debugging:

projects/{name}/output/pdf/*_compile.log - working LaTeX compilation logs
projects/{name}/output/pdf/_combined_manuscript.md - combined markdown source when retained by the renderer
output/{name}/pdf/ - final copied PDF deliverables
Test output from pytest runs

🛠️ Maintenance

System Updates

Update Dependencies

# Update Python packages
uv sync

# Update system packages
sudo apt-get update && sudo apt-get upgrade

Version Control

# Check current status
git status

# Stage changes
git add .

# Commit with descriptive message
git commit -m "feat: add validation feature"

Backup Strategy

# Clean outputs before backup
uv run python -c "from pathlib import Path; from infrastructure.core.files import clean_output_directories; clean_output_directories(Path('.'), '{name}')"

# Backup source files only
tar -czf project_backup.tar.gz projects/{name}/src/ projects/{name}/tests/ projects/{name}/scripts/ projects/{name}/manuscript/ docs/

Adding Features

Business Logic → Add to projects/{name}/src/
Tests → Add to projects/{name}/tests/
Scripts → Add to projects/{name}/scripts/ (use projects/{name}/src/ methods)
Documentation → Update relevant .md files
Validation → Ensure coverage requirements met

Performance Optimization

Parallel Testing: Use pytest-xdist for faster test runs
Caching: Enable pytest caching for repeated runs
Incremental Builds: Only rebuild changed components
Performance Monitoring: Automatic bottleneck detection in pipeline summary
Resource Tracking: Memory and CPU usage reporting (when enabled)

See docs/operational/config/performance-optimization.md for optimization guide.

Checkpoint and Resume

The pipeline includes automatic checkpointing for resume capability:

# Resume from last checkpoint
uv run python scripts/runner/execute_pipeline.py --project {name} --core-only --resume
./run.sh --pipeline --resume

# Start fresh (clears checkpoint on success)
uv run python scripts/runner/execute_pipeline.py --project {name} --core-only
./run.sh --pipeline

Features:

Automatic checkpoint after each successful stage
Checkpoint validation before resume
Graceful handling of corrupted checkpoints
Preserves pipeline start time and stage durations

See docs/operational/config/checkpoint-resume.md for documentation.

📚 References

Internal Documentation

README.md - Project overview and quick start
docs/CLOUD_DEPLOY.md - Headless / cloud server deployment ☁️
docs/RUN_GUIDE.md - Full pipeline orchestration reference
docs/core/how-to-use.md - Usage guide
docs/core/architecture.md - System design details
docs/core/workflow.md - Development workflow
projects/README.md - Multi-project management guide
docs/documentation-index.md - Full documentation hub

External Resources

Pandoc Manual - Document conversion
LaTeX Wikibook - LaTeX documentation
Python Testing - Testing framework

🎯 Best Practices

Development Workflow

Write tests first (TDD)
Ensure coverage requirements met
Follow thin orchestrator pattern
Validate all outputs
Update documentation
Commit with clear messages

Project Structure

Working outputs: projects/{name}/output/ (generated during pipeline)
Final deliverables: output/{name}/ (copied by stage 5)
Source code: projects/{name}/src/
Tests: projects/{name}/tests/
Scripts: projects/{name}/scripts/
Manuscript: projects/{name}/manuscript/

Code Quality

Type Hints: All public APIs must have type annotations
Documentation: Clear docstrings for all functions
Error Handling: Graceful failure with informative messages
Consistency: Follow established patterns and conventions

System Reliability

Deterministic Outputs: All generation must be reproducible
Validation: Check all aspects of output quality
Error Recovery: Handle failures gracefully with clear messages
Performance Monitoring: Track execution time and resource usage

✅ System Status: OPERATIONAL

All systems confirmed functional with exemplar projects:

✅ Multi-project pipeline: default 10-stage DAG in pipeline.yaml (including LLM stages) + optional executive reporting for --all-projects
✅ Test coverage excellence: All active projects meet coverage requirements
✅ Publication-quality outputs: Professional PDFs, cross-referenced manuscripts, automated figures
✅ Mathematical rigor: Advanced equations, theorem proofs, convergence analysis
✅ Testing: Edge cases, performance benchmarks, type safety validation
✅ Documentation: AGENTS.md/README.md across all directories
✅ Data testing: Zero mocks, integration testing
✅ Infrastructure robustness: Fixed critical bugs, improved error handling

Environment Management:

✅ Matplotlib auto-configuration (headless operation via MPLBACKEND=Agg)
✅ Optional dependency handling (python-dotenv graceful fallback)
✅ Test failure tolerance (MAX_TEST_FAILURES environment variable)
✅ LaTeX path management (BasicTeX/MacTeX support)
✅ Docker containerization (infrastructure/docker/Dockerfile + docker-compose.yml)

Modules (v2.1):

✅ Security System (tests) - Input sanitization and monitoring
✅ Health Check System (tests) - System health monitoring
✅ Input Sanitization (tests) - LLM prompt validation
✅ LLM Integration — tests in tests/infra_tests/llm/ (measured coverage → docs/development/coverage-gaps.md)
✅ Rendering System — tests in tests/infra_tests/rendering/ (measured coverage → docs/development/coverage-gaps.md)
✅ Publishing API (integrated) - Zenodo, arXiv, GitHub automation
✅ Multi-project architecture (projects/{name}/ structure)

Audit Status:

✅ High code coverage across infrastructure and project gates (live % → docs/_generated/COUNTS.md, docs/development/coverage-gaps.md)
✅ Zero mock methods - all tests use data and HTTP calls
✅ All .cursorrules standards implemented
✅ compliance with thin orchestrator pattern
✅ Production-ready build pipeline (core pipeline + full pipeline with optional LLM stages)
✅ Reproducible outputs (deterministic with fixed seeds)
✅ Graceful degradation for optional features
✅ Multi-project support (projects/{name}/ structure)
✅ manuscript reference validation (all citations, figures, equations, sections resolved)
✅ HTTP testing with pytest-httpserver (no mocks for API calls)

Stage	Script	Tags	Failure mode
0 Clean Output Directories	built-in `_run_clean_outputs`	`core`, `clean`	soft fail
1 Environment Setup	`scripts/pipeline/stage_00_setup.py`	`core`	hard fail
2 Infrastructure Tests	`scripts/pipeline/stage_01_test.py --infra-only --verbose --infra-scope pipeline-smoke`	`core`, `tests`	configurable tolerance
3 Project Tests	`scripts/pipeline/stage_01_test.py --project-only --verbose`	`core`, `tests`	configurable tolerance
4 Project Analysis	`scripts/pipeline/stage_02_analysis.py`	`core`	hard fail
5 Connector Search	`scripts/pipeline/stage_08_connector_search.py`	`science`	skipped if not configured
6 Provenance Record	`scripts/pipeline/stage_09_provenance_record.py --stage Connector Search`	`provenance`	skipped if not configured
7 PDF Rendering	`scripts/pipeline/stage_03_render.py`	`core`	hard fail
8 Output Validation	`scripts/pipeline/stage_04_validate.py`	`core`	PDF/bookends and artifact/provenance failures block; optional-format structure remains a warning + report
9 LLM Scientific Review	`scripts/pipeline/stage_06_llm_review.py --reviews-only`	`llm`	skipped if Ollama absent
10 LLM Translations	`scripts/pipeline/stage_06_llm_review.py --translations-only`	`llm`	skipped if Ollama absent
11 Copy Outputs	`scripts/pipeline/stage_05_copy.py`	`core`	soft fail
12 Ebook Generation	`scripts/pipeline/stage_11_ebook.py`	`core`, `ebook`	soft fail
13 Metadata Package	`scripts/pipeline/stage_12_metadata.py`	`core`, `metadata`	soft fail
14 Executable Bundle	`scripts/runner/bundle_executable.py`	`bundle`	soft fail
15 Archival Publication	`scripts/runner/archive_publication.py`	`archival`	soft fail

Directory & sub-document map

Navigation links to in-tree documents (keeps them discoverable):

Repository README

Describes docxology/template as a whole, which may contain artifacts other than this one. Where this artifact had no useful description of its own, its summary was taken from here.

🚀 Research Project Template

📄 Published: A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication — DOI: 10.5281/zenodo.19139090

Template Repository - Click "Use this template" to create a research project with this structure

Quickstart

Just cloned the repo? Do this:

git clone <this-repo> && cd template
uv sync (installs the root environment, including deterministic dependencies for all public template exemplars)
./run.sh (interactive menu) or ./run.sh --pipeline --project templates/template_code_project --core-only (non-interactive, no LLM)
PDFs land in output/templates/<project>/pdf/. Logs in output/templates/<project>/logs/.
Run ./run.sh --help for all flags. The always-present roster is generated from PUBLIC_PROJECT_NAMES in docs/_generated/active_projects.md.

Repurposing for your research? See docs/repurposing-architectures.md — maps every reusable architecture (DAG pipeline, two-layer separation, evidence registry, multi-format rendering, MCP server, publishing stack) to its module and adoption path.

For deeper guidance see docs/guides/getting-started.md and docs/RUN_GUIDE.md.

Thin-orchestrator gates: uv run python scripts/audit/check_template_drift.py --strict, uv run python scripts/gates/module_line_count_check.py, uv run python -m infrastructure.core.health — details in docs/architecture/thin-orchestrator-summary.md.

Assistants and editors: .cursorrules summarizes architecture and tooling for Cursor; CLAUDE.md is the command cheat sheet; AGENTS.md is the full system manual (pipeline, validation, configuration). For routable agent workflows, start at docs/prompts/SKILL.md and the generated skill index docs/_generated/skills_index.md.

Contributors and CI: GitHub Actions, Dependabot, and PR/issue templates live under .github/README.md (agent entry point, doc map, CI inventory) and .github/AGENTS.md (job names, thresholds, troubleshooting).

Local hooks: After uv sync, run pre-commit install and pre-commit install --hook-type pre-push to mirror Ruff, mypy, Bandit, and smoke tests locally (see .pre-commit-config.yaml).

Strict Mermaid/PDF checks: Run npm ci at the repository root. The Python documentation and rendering gates automatically resolve the pinned node_modules/.bin/mmdc; add that directory to PATH only when invoking mmdc directly. The existing Chrome resolver supplies the browser executable.

A system for research and development projects. This template provides a test-driven structure with automated PDF generation, professional documentation, and validated build pipelines.

🧭 Positioning (honest framing)

This is primarily Daniel Ari Friedman's research operating system, made public and Apache 2.0-licensed so other researchers can fork it if helpful. It is not a one-size-fits-all template — it is opinionated, Python+pytest+LaTeX-flavored, and tuned to the kind of work Daniel does (Active Inference, computational biology, cognitive security). Honest framing ages better than wishful adoption metrics.

If your workflow looks similar (TDD-on-research-code, Markdown→PDF, multi-project monorepo, optional local-LLM draft assistance, deterministic + watermarked outputs, Zenodo DOI publishing), the template will probably save you time. If it doesn't look similar, a lighter alternative (Quarto, MyST, Cookiecutter-data-science) may serve you better. See MAINTAINERS.md for ownership and STATUS.md for per-subsystem freshness so you can judge what's actively maintained vs dormant.

Long-horizon viability guides — toolchain migration, regression testing, archival redundancy, local CI, and the design for a future executable-bundle stage — live in docs/maintenance/.

🎯 What This Template Provides

This is a GitHub Template Repository that gives you:

✅ Multi-project support - Run multiple projects in one repository
✅ Project structure with clear separation of concerns
✅ Test-driven development setup with coverage requirements
✅ Automated PDF generation from markdown sources
✅ Thin orchestrator pattern for maintainable code
✅ Methods orchestration linking pipeline contracts, methods prose, artifacts, and evidence
✅ Executable methods contracts with DAG, script, artifact, and verification validation
✅ Bounded parallel quick testing with isolated project processes and serial oracles
✅ Public exemplar capability inventory covering structure and declared skip reasons
✅ Ready-to-use utilities for any research project
✅ Professional documentation structure (full inventory: docs/documentation-index.md)
✅ Advanced quality analysis and document metrics
✅ Reproducibility tools for scientific workflows
✅ Integrity verification and validation
✅ Publishing tools for academic dissemination
✅ Scientific development best practices
✅ Reporting with error aggregation and performance metrics
✅ Local Ollama workflow documented in infrastructure/llm/README.md and docs/operational/troubleshooting/llm-review.md

🗺️ Choose Your Path

Pick the entry point that matches your goal:

Cold start, human or agent (install prerequisites, run the canonical exemplar once): START_HERE.md.
New users (write docs, generate PDFs): start with Quickstart, then docs/guides/getting-started.md and docs/reference/quick-start-cheatsheet.md.
Developers (figures, data analysis, automation): docs/core/architecture.md, docs/architecture/thin-orchestrator-summary.md, docs/guides/methods-orchestration.md, docs/core/workflow.md, docs/guides/figures-and-analysis.md.
Contributors (improve the template): docs/development/contributing.md, docs/development/code-of-conduct.md, docs/development/roadmap.md, docs/development/security.md.
Advanced (system internals, modules, deep technical dive): AGENTS.md, docs/RUN_GUIDE.md, docs/modules/modules-guide.md, docs/documentation-index.md.

🧭 Documentation Hub

📚 Documentation Index | 📖 Documentation Guide | 🔍 Quick Reference

The template ships with a large documentation corpus under docs/. The full hierarchical map (with mermaid diagram) lives in docs/AGENTS.md; the authoritative per-file index lives in docs/documentation-index.md (rely on that index, not a hard-coded file count, which drifts). Top-level layout:

docs/core/ — essential reading: how-to-use, architecture, workflow
docs/guides/ — progressive walkthroughs by skill level (1–12)
docs/operational/ — build, configuration, troubleshooting, performance
docs/reference/ — FAQ, cheatsheet, common workflows, API reference
docs/architecture/ — two-layer architecture, thin orchestrator, decision tree
docs/usage/ — examples, showcase, markdown writing guide
docs/modules/ — module-by-module guides
docs/development/ — contributing, testing, roadmap

🤖 Agentic operation and SKILLS

Agents should load the smallest applicable workflow before editing. The routing surface is first-class and available via two skill discovery channels:

Hermes / agentskills.io project-local skills

Every canonical exemplar under projects/templates/ now ships its own .agents/skills/<name>/SKILL.md with YAML frontmatter discoverable by Hermes and agentskills.io runtimes. Each skill captures when to use, quick reference (pytest / analysis / render commands), pitfalls, and cross-refs for that template. Load the skill by its name (e.g. template-code-project, template-active-inference) when working inside that exemplar.

Skill	Template	When to load
`template-active-inference`	`template_active_inference`	Use this template when several independent research tracks must compose into one manuscript whose claims stay consistent where the tracks overlap — here: a closed-form analytical oracle, a pymdp simulation harness, a Lean formalization boundary, and shared GNN/ontology notation.
`template-advanced-literature-review`	`template_advanced_literature_review`	Use this template when one review question needs distinct retrieval phases, phase-specific filters, and explicit cross-phase provenance.
`template-autoresearch-project`	`template_autoresearch_project`	Use this template when you need a bounded, offline AutoResearch loop: deterministic ML candidate evaluation over a fixed local dataset, with evidence-linked claims, machine-readable ledgers, artifact-integrity manifests, and deferred human-review gates.
`template-autopoiesis`	`template_autopoiesis`	Use this template when you need to generate runnable project trees deterministically — not a manuscript, but a whole child project (its own `src/`, `tests/`, `scripts/`, and `manuscript/`) selected by a seed from a combinatoric grammar, with recompute-based provenance verification and a falsifiable honesty manifest against green-by-construction test theater.
`template-code-project`	`template_code_project`	Use this template for code-driven computational research: algorithms in `src/`, numerical experiments with deterministic seeds, automated publication-quality figures, and a manuscript that reports the computed results.
`template-data-descriptor`	`template_data_descriptor`	Use this template when the contribution is a published dataset or data paper: a schema contract, file inventory, data dictionary, provenance chain, license boundary, quality checks, and machine-readable descriptor must all stay consistent before publication.
`template-eda-notebook`	`template_eda_notebook`	Use this template for exploratory data analysis on tabular data: load a dataset, surface missingness, compute descriptive statistics and per-group means, rank features by correlation, and produce a few diagnostic figures.
`template-formal`	`template_formal`	Use this template when the research subject is the type architecture itself: illegal-state-unrepresentable design, session-typed protocols, affine/linear resource-handle discipline, or a decentralized (no-shared-global-state) multiagent simulation that needs its own local storage and local networking per agent.
`template-literature-meta-analysis`	`template_literature_meta_analysis`	Use it whenever the research object is a body of literature about a topic and you want every reported number to trace to committed, regenerable artifacts.
`template-madlib`	`template_madlib`	Use this template when you need configuration-driven manuscript generation with auditable token provenance, conditional section structure, explicit/default field visibility, failure-boundary reporting, and a reviewer-visible authoring contract: lexicon categories, section titles, narrative moves, method steps, design principles, pipeline phases, evaluation criteria, QA probes, failure modes, authoring obligations, visualization controls, audit rules, and slots are declared in YAML; `src/` deterministically expands those declarations; and the manuscript receives large-grain `{{TOKEN}}` bodies only after the source code has generated the supporting artifacts.
`template-newspaper`	`template_newspaper`	Use this template when you need data-driven, large-format print layout: multi-page broadsheets/tabloids with precise column geometry, typography-first constraints, and strict content/engine separation (YAML editions in `content/`, pure-Python ReportLab engine in `src/`).
`template-pools-rules-tools`	`template_pools_rules_tools`	Use this project when you need to: - Demonstrate how a research project integrates multiple resource directories (fonds, tools, rules) in a single pipeline - Validate that your fonds, tools, and rule infrastructure modules are correctly wired and discoverable - Onboard new teams to the three-resource architecture with a concrete, runnable example - Test cross-cutting concern integration where fonds supply data, rules govern validation, and tools execute transforms - Extend the architecture by adding new resource types; copy this project as a starting point for integration testing This template is ideal for platform teams, CI engineers, and infrastructure maintainers who need a self-validating meta-project that exercises all three resource layers without coupling to a specific domain science.
`template-prose-project`	`template_prose_project`	Use this template for manuscript-focused editorial pipelines: readability gates (Flesch-Kincaid bands), structural checking, citation/BibTeX validation, and prose-quality review workflows where the document itself is the artifact under test.
`template-autoscientists`	`template_autoscientists`	Use this template when you need to isolate and measure agent-coordination mechanisms: coordinated teams vs single-thread baselines under matched budgets, deterministic ablation studies, and honest per-mechanism effect reporting (including nulls).
`template-gold-refinement`	`template_gold_refinement`	Use this template for analogical manuscript composition research: projects that map a scientific domain onto a refinement pipeline and generate the manuscript through deterministic mega-madlib token injection.
`template-methods-paper`	`template_methods_paper`	Use this template when the paper you are writing describes a methodology rather than reports results: a procedure, protocol, or specification language, where the contribution is the controlled vocabulary and its guarantees (dimensional safety, staged validation, deterministic compilation) rather than a numeric outcome.
`template-redacted-report`	`template_redacted_report`	Use this template when a report needs formal redaction before release: classification ceilings, source-control markings, redaction decisions, authority review, reviewer approvals, source-safe hash ledgers, residual-risk checks, mosaic-risk checks, and a public audit packet must be validated before any sanitized narrative is published.
`template-registered-report`	`template_registered_report`	Use this template when the core method is a registered report or preregistered replication: hypotheses, outcomes, exclusion rules, analysis plan, power or sensitivity rationale, deviation ledger, and confirmatory-versus-exploratory claim boundaries must be locked before results are interpreted.
`template-search-project`	`template_search_project`	Use this template for literature-review and evidence-synthesis pipelines: multi-backend search (arXiv, Crossref, local corpora, optional Paperclip), deduplication, BibTeX generation, and LLM-assisted per-paper and corpus-level synthesis.
`template-sia`	`template_sia`	Use this template when you need a self-improvement-agent evaluation harness: Meta → Target → Feedback generation loops, public/private task splits to detect overfitting, deterministic fixture replay for testability, and fail-closed loop validation.
`template-storybook`	`template_storybook`	Use this template when you need full-page illustrated PDF storytelling: picture-book pages, symbolic scenes, character-generation methods, page-level orchestrators, and deterministic raster art assembled into a print-ready PDF.
`template-template`	`template_template`	Use this template when your research subject is the repository itself — programmatic introspection of architecture, pipeline DAGs, module inventories, and security layers, rendered as a manuscript whose every metric is computed live (autopoietic: the paper regenerates itself from the code it describes).
`template-textbook`	`template_textbook`	Use this template for book-length manuscripts: parts → chapters → labs → question banks declared in a single `config.yaml`, with auto-numbering, deterministic figure/diagram generation, and structural-contract tests that keep hundreds of pages from drifting.
`template-pitch-deck`	`template_pitch_deck`	Use this template when you need a pitch, grant report, or recurring stakeholder update treated as a build artifact — bound to live repository facts, validated for unresolved tokens and pitch-deck cliché, and regenerable byte-for-byte from source rather than hand-maintained in a proprietary slide tool.

Generated infrastructure skills

In addition to the project-local skills, the repo generates infrastructure-level SKILL.md files from live discovery:

Shared context-engineering skills: .agents/skills/ carries a pinned, provenance-checked Agent Skills collection discoverable by Codex/OpenAI and Hermes-compatible runtimes. Use uv run python -m infrastructure.skills runtime-status to audit Codex, Claude Code, and Hermes parity, or ... runtime-install to create reversible user-level links from the pinned shared store.
Workflow router: docs/prompts/SKILL.md (template-workflows) routes broad requests such as full audits, pipeline debugging, code changes, tests, validation, manuscript work, and release checks to exactly one child workflow.
Agentic-use hardening: docs/prompts/agentic-use/SKILL.md covers skill inventory, routing checks, .cursor/skill_manifest.json, and generated skill-index maintenance.
Infrastructure module skills: infrastructure/SKILL.md is the Layer-1 hub; pair the relevant infrastructure/<module>/SKILL.md with that module's AGENTS.md before editing code.
Script and resource-pool skills: scripts/*/SKILL.md files and public fonds/templates/, rules/templates/, and tools/templates/ skills are included when present, without scanning private lifecycle roots.
Human skill index: docs/_generated/skills_index.md lists all discovered skills. Regenerate after skill changes with uv run python -m infrastructure.skills write-index; refresh the editor manifest with uv run python -m infrastructure.skills write; verify both with uv run python -m infrastructure.skills check and uv run python -m infrastructure.skills check-contracts. The same inventory is returned by the opt-in stdio MCP server's list_skills tool via uv run python -m infrastructure.mcp_server.

🔀 Multi-Project Support

The repo can host multiple research projects in parallel. Each project owns its own src/, tests/, manuscript/, scripts/, and output/ directory under projects/<name>/. Layer-1 infrastructure is shared.

Permanent canonical exemplars — always present and tracked in git:

Exemplar	When to use
`template_active_inference`	Use this template when several independent research tracks must compose into one manuscript whose claims stay consistent where the tracks overlap — here: a closed-form analytical oracle, a pymdp simulation harness, a Lean formalization boundary, and shared GNN/ontology notation.
`template_advanced_literature_review`	Use this template when one review question needs distinct retrieval phases, phase-specific filters, and explicit cross-phase provenance.
`template_autoresearch_project`	Use this template when you need a bounded, offline AutoResearch loop: deterministic ML candidate evaluation over a fixed local dataset, with evidence-linked claims, machine-readable ledgers, artifact-integrity manifests, and deferred human-review gates.
`template_autopoiesis`	Use this template when you need to generate runnable project trees deterministically — not a manuscript, but a whole child project (its own `src/`, `tests/`, `scripts/`, and `manuscript/`) selected by a seed from a combinatoric grammar, with recompute-based provenance verification and a falsifiable honesty manifest against green-by-construction test theater.
`template_code_project`	Use this template for code-driven computational research: algorithms in `src/`, numerical experiments with deterministic seeds, automated publication-quality figures, and a manuscript that reports the computed results.
`template_data_descriptor`	Use this template when the contribution is a published dataset or data paper: a schema contract, file inventory, data dictionary, provenance chain, license boundary, quality checks, and machine-readable descriptor must all stay consistent before publication.
`template_eda_notebook`	Use this template for exploratory data analysis on tabular data: load a dataset, surface missingness, compute descriptive statistics and per-group means, rank features by correlation, and produce a few diagnostic figures.
`template_formal`	Use this template when the research subject is the type architecture itself: illegal-state-unrepresentable design, session-typed protocols, affine/linear resource-handle discipline, or a decentralized (no-shared-global-state) multiagent simulation that needs its own local storage and local networking per agent.
`template_literature_meta_analysis`	Use it whenever the research object is a body of literature about a topic and you want every reported number to trace to committed, regenerable artifacts.
`template_madlib`	Use this template when you need configuration-driven manuscript generation with auditable token provenance, conditional section structure, explicit/default field visibility, failure-boundary reporting, and a reviewer-visible authoring contract: lexicon categories, section titles, narrative moves, method steps, design principles, pipeline phases, evaluation criteria, QA probes, failure modes, authoring obligations, visualization controls, audit rules, and slots are declared in YAML; `src/` deterministically expands those declarations; and the manuscript receives large-grain `{{TOKEN}}` bodies only after the source code has generated the supporting artifacts.
`template_newspaper`	Use this template when you need data-driven, large-format print layout: multi-page broadsheets/tabloids with precise column geometry, typography-first constraints, and strict content/engine separation (YAML editions in `content/`, pure-Python ReportLab engine in `src/`).
`template_pools_rules_tools`	Use this project when you need to: - Demonstrate how a research project integrates multiple resource directories (fonds, tools, rules) in a single pipeline - Validate that your fonds, tools, and rule infrastructure modules are correctly wired and discoverable - Onboard new teams to the three-resource architecture with a concrete, runnable example - Test cross-cutting concern integration where fonds supply data, rules govern validation, and tools execute transforms - Extend the architecture by adding new resource types; copy this project as a starting point for integration testing This template is ideal for platform teams, CI engineers, and infrastructure maintainers who need a self-validating meta-project that exercises all three resource layers without coupling to a specific domain science.
`template_prose_project`	Use this template for manuscript-focused editorial pipelines: readability gates (Flesch-Kincaid bands), structural checking, citation/BibTeX validation, and prose-quality review workflows where the document itself is the artifact under test.
`template_autoscientists`	Use this template when you need to isolate and measure agent-coordination mechanisms: coordinated teams vs single-thread baselines under matched budgets, deterministic ablation studies, and honest per-mechanism effect reporting (including nulls).
`template_gold_refinement`	Use this template for analogical manuscript composition research: projects that map a scientific domain onto a refinement pipeline and generate the manuscript through deterministic mega-madlib token injection.
`template_methods_paper`	Use this template when the paper you are writing describes a methodology rather than reports results: a procedure, protocol, or specification language, where the contribution is the controlled vocabulary and its guarantees (dimensional safety, staged validation, deterministic compilation) rather than a numeric outcome.
`template_redacted_report`	Use this template when a report needs formal redaction before release: classification ceilings, source-control markings, redaction decisions, authority review, reviewer approvals, source-safe hash ledgers, residual-risk checks, mosaic-risk checks, and a public audit packet must be validated before any sanitized narrative is published.
`template_registered_report`	Use this template when the core method is a registered report or preregistered replication: hypotheses, outcomes, exclusion rules, analysis plan, power or sensitivity rationale, deviation ledger, and confirmatory-versus-exploratory claim boundaries must be locked before results are interpreted.
`template_search_project`	Use this template for literature-review and evidence-synthesis pipelines: multi-backend search (arXiv, Crossref, local corpora, optional Paperclip), deduplication, BibTeX generation, and LLM-assisted per-paper and corpus-level synthesis.
`template_sia`	Use this template when you need a self-improvement-agent evaluation harness: Meta → Target → Feedback generation loops, public/private task splits to detect overfitting, deterministic fixture replay for testability, and fail-closed loop validation.
`template_storybook`	Use this template when you need full-page illustrated PDF storytelling: picture-book pages, symbolic scenes, character-generation methods, page-level orchestrators, and deterministic raster art assembled into a print-ready PDF.
`template_template`	Use this template when your research subject is the repository itself — programmatic introspection of architecture, pipeline DAGs, module inventories, and security layers, rendered as a manuscript whose every metric is computed live (autopoietic: the paper regenerates itself from the code it describes).
`template_textbook`	Use this template for book-length manuscripts: parts → chapters → labs → question banks declared in a single `config.yaml`, with auto-numbering, deterministic figure/diagram generation, and structural-contract tests that keep hundreds of pages from drifting.
`template_pitch_deck`	Use this template when you need a pitch, grant report, or recurring stakeholder update treated as a build artifact — bound to live repository facts, validated for unresolved tokens and pitch-deck cliché, and regenerable byte-for-byte from source rather than hand-maintained in a proprietary slide tool.

Test and coverage figures are representative; confirm against docs/_generated/COUNTS.md after substantive changes.

Choosing an exemplar: every exemplar README opens with a ## When to use this template section, and the generated differentiation map in docs/_generated/exemplar_roster.md collects them into one "copy THIS when…" table (regenerate with uv run python scripts/docgen/exemplar_roster.py; sync is test-enforced).

The permanent exemplars share the same core layout and verification checklist. The code/prose exemplars also carry the 12-file project docs/ hub (agent_instructions.md, style_guide.md, syntax_guide.md, testing_philosophy.md, rendering_pipeline.md, faq.md, quickstart.md, output_conventions.md, troubleshooting.md, architecture.md, AGENTS.md, README.md). New projects copy whichever exemplar is closest in shape and adjust from there. See projects/AGENTS.md for the full comparison.

Publication metadata for every public exemplar is generated from project config and sidecars into docs/_generated/publication_records.md, and the GitHub-facing table in .github/README.md is auto-injected from that same source. To publish a project modularly, start with the Publication runbook: it covers the standalone public GitHub mirror, first real Zenodo DOI, new-version releases, optional mirrors, status blocks, and archival handoff.

Public Exemplar Outputs And Mirrors

Every canonical exemplar under projects/templates/ is tracked in this monorepo, including its project-local output/ tree. The copied release artifacts under output/templates/<name>/ are tracked as well, so a clone of docxology/template contains both the source and the latest rendered public artifacts. Public output files above 50 MB remain excluded by the generated-artifact guard; private or rotating project outputs remain blocked.

Each exemplar also has a standalone docxology/template_* GitHub repository linked to its Zenodo concept and latest version DOI. The current matrix is docs/_generated/publication_records.md. The standalone repository must exist before scripts/publish/publish_project_release.py can create a release there; the release script publishes the GitHub release asset and Zenodo deposit, but it does not create the repository itself. To regenerate any exemplar from the monorepo:

git clone https://github.com/docxology/template
cd template
uv sync
./run.sh --project templates/template_code_project --pipeline --core-only
uv run python scripts/pipeline/stage_04_validate.py --project templates/template_code_project
uv run python scripts/pipeline/stage_05_copy.py --project templates/template_code_project

Replace template_code_project with any public exemplar name from the table above. The standalone repositories are publication mirrors; use this monorepo when you need the shared infrastructure, full render pipeline, or cross-template validation.

The canonical exemplars also ship project-local composability overlays: domain_profile.yaml declares review gates, source policy, artifact expectations, and benchmark rubric preferences; experiment_plan.yaml declares design-validation conditions, primary metric direction, expected figures/tables, baselines, and ablations. These files are declarative inputs for validation and benchmark tooling; they do not generate experiments or run autonomous agents.

🔒 Confidentiality. This is a public template repo. Only the canonical exemplars above (under projects/templates/) are git-tracked/pushed — .gitignore ignores projects/* and negates only projects/templates/. Any other project you add under projects/ (research, client, or confidential work) stays local-only and is never committed; scripts/audit/check_tracked_all.py blocks any accidental commit in the pre-push hook and CI.

Private lifecycle projects. In Daniel's working checkout, confidential projects live outside this public repo at $TEMPLATE_PRIVATE_PROJECTS_ROOT. The simplified sidecar uses working/ and archive/; optional ongoing/ (long-lived projects with no publication target) plus legacy active/, published/, and other/ folders are still supported when present. run.sh and python -m infrastructure.orchestration auto-sync existing folders into matching typed subfolders under projects/: working/* into projects/working/*, ongoing/* into projects/ongoing/*, archive/* into projects/archive/*, and optional active/* into projects/active/*. templates/ and optional active/ links behave like native rendered entries; working/, ongoing/, and archive/ links are visible for explicit targeted work but are not default-rendered. Inspect without changing the tree: uv run python -m infrastructure.orchestration link-projects --dry-run. Override the sibling path with TEMPLATE_PRIVATE_PROJECTS_ROOT or .private_projects_root; disable auto-sync with TEMPLATE_SKIP_LINK_SYNC=1. The symlinked project keeps working outputs at projects/<subfolder>/<name>/output/ (the private target), while final deliverables still copy to output/<subfolder>/<name>/ in this template checkout.

Other entries rotate between projects/working/ and projects/archive/ as work progresses. Never hard-code their paths in long-lived docs — consult docs/_generated/active_projects.md (authoritative public scope, regenerated from infrastructure.project.public_scope) and docs/_generated/COUNTS.md instead.

Common commands:

./run.sh                                     # Interactive project selection
./run.sh --project templates/template_code_project --pipeline
./run.sh --all-projects --pipeline           # All discovered projects sequentially
./secure_run.sh --steganography-only --project templates/template_code_project  # Re-watermark PDFs
mkdir -p projects/my_research/{src,tests,manuscript,scripts}  # Scaffold new project

Lifecycle: rendered = projects/templates/ plus optional projects/active/ (discovered, executed). The simplified private sidecar normally uses working/ and archive/ (plus optional ongoing/ for long-lived work with no publication target); render sidecar projects explicitly with a qualified name such as working/{name} or ongoing/{name}. See projects/PROJECTS_PARADIGM.md for lifecycle, slug rules, and discovery semantics.

🚀 Quick Start {#quick-start}

See the Quickstart at the top of this file for the canonical clone-to-PDF flow. For headless cloud deployment use docs/CLOUD_DEPLOY.md (uv is installed automatically when you run ./run.sh --pipeline). Beginner walkthrough: docs/guides/getting-started.md. One-page command reference: docs/reference/quick-start-cheatsheet.md. Twelve-level usage guide: docs/core/how-to-use.md.

System Status

Current state is captured in docs/_generated/COUNTS.md (updated from discovery, test runs, and CI configuration).

Key elements:

Active projects listed via discover_projects()
Coverage enforced at 60% (infrastructure) and 90% (projects)
Tests run with real data and computations
Commands standardized to uv run
Outputs organized per project under output/{name}/

See docs/_generated/COUNTS.md and docs/development/testing/testing-guide.md for details.

🎓 Skill-Based Learning Paths

Twelve progressive levels — Document Creation (1–3), Figures & Automation (4–6), Test-Driven Development (7–9), System Architecture (10–12) — are documented end-to-end in docs/core/how-to-use.md. Per-band walkthroughs: docs/guides/getting-started.md, docs/guides/figures-and-analysis.md, docs/guides/testing-and-reproducibility.md, docs/guides/extending-and-automation.md.

🏗️ Project Structure

Two-layer architecture:

Layer 1 — infrastructure/ (generic, reusable): build, validation, rendering, LLM, publishing, etc. Plus scripts/ (entry-point orchestrators) and tests/ (infrastructure tests, ≥60 % coverage).
Layer 2 — projects/<name>/ (project-specific, customizable): src/ (algorithms, ≥90 % coverage), tests/, scripts/ (thin orchestrators), manuscript/ (markdown sections + config.yaml).
Output is generated under each exemplar's project-local projects/templates/<name>/output/ tree. Final deterministic evidence (publication PDFs, figures, analysis data, hydrated manuscripts, and release/validation registries) may be tracked for public exemplars; checkpoints, logs, telemetry, pipeline snapshots, and LaTeX/slide build intermediates are disposable, ignored, and regeneratable.
Docs live under docs/ (full hierarchy in docs/AGENTS.md); per-directory AGENTS.md files document every leaf.

System Architecture Overview

A short summary lives here; full architecture diagrams (system overview, module-dependency graph, per-stage data flow, configuration-system flow) are maintained in AGENTS.md and docs/core/architecture.md. In short:

Entry points: ./run.sh (interactive or --pipeline) and uv run python scripts/runner/execute_pipeline.py --project <name> [--core-only]; numbered orchestrators under scripts/ include 00_*.py through 07_*.py (setup → copy, LLM, executive report — see scripts/AGENTS.md).
Orchestration: the pipeline runs Setup → Tests → Analysis → Render → Validate → Copy, with optional LLM Review and LLM Translations stages.
Core systems: importable infrastructure/ packages (Layer 1; live list in docs/_generated/COUNTS.md) plus per-project projects/{name}/src/ algorithms (Layer 2); see docs/_generated/COUNTS.md for the live module list.
Data flow: project source + manuscript markdown + config.yaml flow through the pipeline into output/<name>/{pdf,figures,data,reports}/.
Quality assurance: infra ≥60 % and project ≥90 % coverage gates, no-mocks policy, deterministic seeds, real PDF/markdown validation.
Configuration: projects/{name}/manuscript/config.yaml plus environment overrides feed PDF metadata, LaTeX preamble, figure labels, and validation rules.

Directory Overview with Documentation Links:

Directory	Purpose	Documentation
`infrastructure/`	Generic build/validation tools (Layer 1)	infrastructure/AGENTS.md
`scripts/`	Entry point orchestrators	scripts/AGENTS.md
`tests/`	Infrastructure test suite	tests/AGENTS.md
`projects/{name}/src/`	Project-specific scientific code (Layer 2)	Per-project `AGENTS.md`
`projects/{name}/tests/`	Project test suite	Per-project `AGENTS.md`
`docs/`	Documentation hub	docs/documentation-index.md
`projects/{name}/manuscript/`	Research manuscript sections	Per-project `AGENTS.md`
`output/`	Generated public evidence plus local build output	Final evidence is tracked selectively; runtime residue is ignored

📚 Explore Documentation: See docs/documentation-index.md for documentation structure

🔑 Key Architectural Principles

The repository follows a thin orchestrator pattern: business logic lives only in infrastructure/ and projects/{name}/src/; scripts coordinate, never implement. Tests prohibit mock frameworks and prefer real execution; remaining pytest.monkeypatch dependency replacements are explicitly inventoried rather than treated as proof of a mock-free suite. Coverage gates remain strict. Full narrative + benefits: docs/architecture/thin-orchestrator-summary.md, docs/core/architecture.md.

✨ Key Features

Test-driven development with ≥60 % infra and ≥90 % project coverage gates (docs/core/workflow.md).
Automated script execution via thin-orchestrator scripts under projects/{name}/scripts/ (scripts/AGENTS.md).
Markdown-to-PDF pipeline with cross-referenced manuscripts and figure integration (docs/usage/markdown-template-guide.md, docs/modules/pdf-validation.md).
Validated build system with 16 declared stages, a default 10-stage core+LLM path, an 8-stage --core-only path, and CI gates (docs/RUN_GUIDE.md).
Generic + reusable — drop the same infrastructure/ into any project that follows the layout (docs/usage/template-description.md).

🔒 Security & Monitoring

LLM input sanitization (infrastructure.llm.core.sanitization), security validators (infrastructure.core.security), runtime health checks (infrastructure.core.runtime.health_check), rate limiting, and HTTP security headers. Full surface and worked usage examples: docs/development/security.md.

🛠️ Installation & Setup

Prerequisites: pandoc and a TeX distribution (texlive-xetex on Debian/Ubuntu, MacTeX on macOS). Python deps install with uv sync (project interpreter is .venv/bin/python; the template targets Python 3.10+ (requires-python in pyproject.toml) and CI tests infrastructure on 3.10–3.13, with .python-version pinning 3.12 as the local default). Add per-project deps with uv run python scripts/maintenance/manage_workspace.py add <package> --project <name>. To generate a manuscript, follow the Quickstart at the top.

Layer 1 is also a standard Python distribution. Build it with uv build, or install the wheel attached to a GitHub release. Installation provides the research-template command (research-template --help). Optional pip extras mirror the major capability groups, for example research-project-template[rendering,publishing]; repository contributors should continue to use uv sync --group ... for development groups.

🐳 Docker Support

docker-compose up (or docker build -t research-template . && docker run -it research-template) builds a reproducible image with pandoc, TeX, Ollama LLM server support, persistent model/output volumes, and hot-reload. See Dockerfile and docker-compose.yml.

🔧 Customization

Project Metadata Configuration

Two configuration paths exist: edit projects/{name}/manuscript/config.yaml (recommended) or export AUTHOR_NAME / AUTHOR_ORCID / AUTHOR_EMAIL / PROJECT_TITLE / DOI environment variables (env vars override the YAML file). The YAML schema (paper title, authors with ORCID, publication DOI, keywords, optional LLM translations block) and a worked example are documented once in CLAUDE.md and AGENTS.md; both files also list every available field. See projects/{name}/manuscript/config.yaml.example for the full template. Applied configuration drives PDF metadata, LaTeX document properties (see docs/reference/copypasta.md for preamble examples), generated file headers, and cross-reference systems.

Adding Project-Specific Scripts

Place Python scripts under projects/{name}/scripts/. They must follow the thin orchestrator pattern: import computation from projects/{name}/src/ or infrastructure/, handle only I/O / visualization / orchestration, print output paths to stdout for manifest collection, and never implement algorithms inline. Worked examples and the full pattern walkthrough live in scripts/AGENTS.md and docs/architecture/thin-orchestrator-summary.md.

Manuscript Structure

Per-project manuscript files live in projects/{name}/manuscript/: config.yaml, preamble.md, zero-padded numbered chapter files (00_abstract.md onward), optional S01_*.md supplements, and 99_references.md. Exact chapter slugs vary per project — the canonical exemplar is projects/templates/template_code_project/manuscript/; the numbering system and slug rules are authoritative in docs/usage/manuscript-numbering-system.md.

📊 Testing

TDD with strict coverage gates: infrastructure ≥ 60 %, projects ≥ 90 %. No unit-level mock frameworks — tests use real data, real files; HTTP boundaries use pytest-httpserver (in-process test server). The project pipeline runs a focused pipeline-smoke infrastructure contract plus the selected project's full coverage suite, so ordinary renders do not rerun the entire repository test matrix. Run the full infrastructure gate explicitly with uv run python scripts/pipeline/stage_01_test.py --infra-only --infra-scope full; run a project suite with uv run python scripts/pipeline/stage_01_test.py --project-only --project <name>. Per-suite commands and coverage report flags are documented in tests/AGENTS.md and docs/development/testing/testing-guide.md; live coverage / test counts live in docs/_generated/COUNTS.md.

Output

Working outputs: projects/{name}/output/. Public exemplars retain only deterministic deliverables and evidence under that tree; runtime state (.checkpoints/, .pipeline/, logs/, telemetry, snapshots) and renderer intermediates are local-only and regenerated by the pipeline. Multi-project mode adds output/executive_summary/ as a disposable copied-output surface.

🔍 How It Works

Two entry points — ./run.sh (interactive or --pipeline) and uv run python scripts/runner/execute_pipeline.py --project <name> [--core-only].

Pipeline (canonical phrasing — keep in sync with CLAUDE.md and AGENTS.md): The default pipeline.yaml declares 16 named stages: 8 core stages, 2 optional LLM stages, 2 opt-in ebook/metadata stages, 2 opt-in bundle/archival stages, and 2 opt-in science/provenance stages (Connector Search, Provenance Record). Default full runs include the 10 core+LLM stages (Clean Output Directories plus nine numbered stages). --core-only runs 8 stages by excluding LLM-tagged and opt-in stages. Ebook, metadata, bundle, archival, science, and provenance stages are declared for contracts but invoked separately when needed (directly via their scripts/pipeline/stage_*.py entry points).

Stage	Script	Tags	Failure mode
0 Clean Output Directories	built-in `_run_clean_outputs`	`core`, `clean`	soft fail
1 Environment Setup	`scripts/pipeline/stage_00_setup.py`	`core`	hard fail
2 Infrastructure Tests	`scripts/pipeline/stage_01_test.py --infra-only --verbose --infra-scope pipeline-smoke`	`core`, `tests`	configurable tolerance
3 Project Tests	`scripts/pipeline/stage_01_test.py --project-only --verbose`	`core`, `tests`	configurable tolerance
4 Project Analysis	`scripts/pipeline/stage_02_analysis.py`	`core`	hard fail
5 Connector Search	`scripts/pipeline/stage_08_connector_search.py`	`science`	skipped if not configured
6 Provenance Record	`scripts/pipeline/stage_09_provenance_record.py --stage Connector Search`	`provenance`	skipped if not configured
7 PDF Rendering	`scripts/pipeline/stage_03_render.py`	`core`	hard fail
8 Output Validation	`scripts/pipeline/stage_04_validate.py`	`core`	PDF/bookends and artifact/provenance failures block; optional-format structure remains a warning + report
9 LLM Scientific Review	`scripts/pipeline/stage_06_llm_review.py --reviews-only`	`llm`	skipped if Ollama absent
10 LLM Translations	`scripts/pipeline/stage_06_llm_review.py --translations-only`	`llm`	skipped if Ollama absent
11 Copy Outputs	`scripts/pipeline/stage_05_copy.py`	`core`	soft fail
12 Ebook Generation	`scripts/pipeline/stage_11_ebook.py`	`core`, `ebook`	soft fail
13 Metadata Package	`scripts/pipeline/stage_12_metadata.py`	`core`, `metadata`	soft fail
14 Executable Bundle	`scripts/runner/bundle_executable.py`	`bundle`	soft fail
15 Archival Publication	`scripts/runner/archive_publication.py`	`archival`	soft fail

Full per-stage flowchart, failure/skip transitions, and the script-to-stage mapping for --core-only live in AGENTS.md and docs/RUN_GUIDE.md. Workflow narrative: docs/core/workflow.md. Architecture narrative: docs/core/architecture.md.

📚 Documentation Index

The full per-file documentation index lives in docs/documentation-index.md (authoritative; counts drift, so it is not duplicated here). Top-level entry points:

System reference: AGENTS.md, CLAUDE.md, docs/AGENTS.md
Walkthroughs: docs/guides/getting-started.md, docs/core/how-to-use.md (12 skill levels), docs/reference/quick-start-cheatsheet.md
Architecture: docs/core/architecture.md, docs/architecture/thin-orchestrator-summary.md, docs/architecture/two-layer-architecture.md
Pipeline & build: docs/RUN_GUIDE.md, docs/CLOUD_DEPLOY.md
Modules: docs/modules/modules-guide.md, infrastructure/AGENTS.md
Quality / testing: tests/AGENTS.md, docs/development/testing/testing-guide.md
Agent code navigation: docs/guides/codegraph-local.md, docs/guides/leann-local.md (optional local indexes; never committed artifacts)
Best practices: docs/best-practices/best-practices.md
Live facts (auto-derived): docs/_generated/COUNTS.md, docs/_generated/active_projects.md
Per-directory AGENTS.md files in infrastructure/, scripts/, tests/, docs/, and every projects/{name}/{src,scripts,tests,manuscript}/.

🤝 Contributing

contribution guide | Code of conduct | Roadmap

We welcome contributions! To contribute:

Ensure all tests pass with coverage requirements met - Testing Guide
Follow the established project structure - Architecture
Add tests for new functionality - Workflow
Update documentation as needed - Documentation Guide
Maintain thin orchestrator pattern - scripts use src/ methods - Pattern Guide

Recent Improvements:

Build system optimizations - Performance Optimization
Test suite enhancements
Simplified directory structure with markdown/ elimination

📄 License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

📚 Citation

The machine-readable CITATION.cff is the single source of truth (GitHub's "Cite this repository" widget reads it). If you use this template in your research, please cite:

Cite the current release. Earlier versions retain their own Zenodo DOIs; the version-independent concept DOI always resolves to the latest.

BibTeX:

@software{friedman_template_2026,
  author    = {Daniel Ari Friedman},
  title     = {A template/ approach to Reproducible Generative Research:
               Architecture and Ergonomics from Configuration through Publication},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19139090},
  url       = {https://doi.org/10.5281/zenodo.19139090}
}

Plain text: Daniel Ari Friedman. (2026). A template/ approach to Reproducible Generative Research: Architecture and Ergonomics from Configuration through Publication. Zenodo. https://doi.org/10.5281/zenodo.19139090

🆘 Troubleshooting

Common issue catalog — failing tests, missing pandoc/xelatex, PDF quality, LLM unavailability — lives in docs/operational/troubleshooting/README.md and docs/reference/faq.md. Pipeline entry points and flags: docs/RUN_GUIDE.md. PDF validator: docs/modules/pdf-validation.md.

🔄 Migration from Other Projects

To adapt this template: copy infrastructure/ and scripts/, mirror the projects/{name}/{src,tests,scripts,manuscript}/ layout, adopt config.yaml (see AGENTS.md), and validate by running the pipeline. Worked examples: docs/usage/examples.md, docs/best-practices/migration-guide.md.

🏗️ Architecture Benefits

Thin orchestrator pattern delivers single-source-of-truth business logic, high testability (≥90 % project coverage), reusability across projects, and CI-gated quality. Full benefits + rationale: docs/core/architecture.md.

Quick Navigation by Task

Task	Start here
Assistants / Cursor	`.cursorrules`, `CLAUDE.md`, `AGENTS.md`
Write documents	`docs/guides/getting-started.md`, `docs/usage/markdown-template-guide.md`
Add figures	`docs/guides/figures-and-analysis.md`, `docs/usage/visualization-guide.md`
Fix issues	`docs/operational/troubleshooting/README.md`, `docs/reference/faq.md`
Understand architecture	`docs/core/architecture.md`, `docs/architecture/two-layer-architecture.md`
Configure system	`docs/operational/config/configuration.md`, `AGENTS.md`
Run pipeline	`docs/RUN_GUIDE.md`
Contribute code	`docs/development/contributing.md`, `docs/rules/AGENTS.md`
Find all docs	`docs/documentation-index.md`
Check the backlog	`TO-DO.md`
See what changed	`CHANGELOG.md`

🎉 Get Started Now

Ready to begin? Choose your path:

New User? → Start with Quick Start or docs/guides/getting-started.md
Developer? → Read docs/core/architecture.md and docs/core/workflow.md
Need Help? → Check docs/reference/faq.md or docs/operational/troubleshooting/README.md
Explore All Docs? → Browse docs/documentation-index.md

📚 Documentation Hub: All documentation is organized in the docs/ directory with guides for every aspect of the template.

Trustgrade A

passBody integrity
Whether the stored document is plausibly the kind of file the artifact declares, rather than something fetched by mistake.
passType matchnot applicable to this artifact type
Whether the artifact is really the kind of thing its metadata claims it is.
passFreshness
How long since the source repository was last pushed to.
passPrompt injection
Scans the artifact's own text for instructions aimed at your agent rather than at you.
passLicense
Whether the source repository declares an SPDX license permissive enough to redistribute.

How the grade is calculated

Each check contributes 0 points when it passes, 1 when it warns, and 2 when it fails. The total maps to a letter:

Aevery check passed
Bone warning
Ctwo warnings
Dprompt injection or body integrity failed, or three warnings
Fone of those failed, and something else is wrong

These are automated hygiene checks, not a security audit, and not a dependency or vulnerability scan. A grade of A means nothing was flagged — not that the artifact is safe.

Versions

git-beb3f748bb312026-08-04

@docxology/template-10

Install

Document

🤖 AGENTS.md - System Documentation

🎯 System Overview

📄 Publication

Documentation map

For assistants and automation

Template authoring & operational notes

Confidentiality invariant (this is a PUBLIC repo)

📋 Table of Contents

🏗️ Core Architecture

Two-Layer Architecture

Layer 1: Infrastructure (Generic - Reusable)

Layer 2: Projects (Project-Specific - Customizable)

Thin Orchestrator Pattern

Multi-Project Support

📂 Project Organization: Rendered vs Non-Rendered Subfolders

Rendered Projects (projects/templates/ + projects/active/)

Non-Rendered Projects (working/, ongoing/, archive/, optional legacy mirrors)

Project Lifecycle

In-Progress Projects (projects/working/)

📚 Repository Structure

📚 Directory-Level Documentation

Generic Infrastructure (Reusable)

Project-Specific (Customizable)

Documentation Directories

Documentation Navigation

Directory Structure

⚙️ Configuration System

Configuration File (Recommended)

Environment Variables (Alternative Method)

Configuration Examples

Using Configuration File (Recommended)

Using Environment Variables

Verbose Logging

Runtime Configuration

🚀 Rendering Pipeline

Pipeline Execution

Main Entry Point (Recommended)

Manuscript Operations

Secure Pipeline (secure_run.sh)

Entry Point Comparison

Pipeline Stages

Manual Execution Options

Validation Systems

PDF Validation

Markdown Validation

Test Coverage

Testing Framework

Mock-framework prohibition and semantic-stand-in inventory

No-Mocks Implementation Patterns

Test Structure

Test Categories

Running Tests

📤 Output Formats

Generated Files Structure

PDF Versions

🧪 Advanced Modules

🔒 Core Utilities (infrastructure/core/)

🔍 Validation System (infrastructure/validation/)

🔬 Scientific Development (infrastructure/scientific/)

🤖 LLM Integration (infrastructure/llm/)

🎨 Rendering System (infrastructure/rendering/)

🚀 Publishing Module (infrastructure/publishing/)

Module Integration

Accessing Outputs

🔧 Troubleshooting

Quick Reference

Common Issues

Tests Failing

Project Fails In --all-projects But Passes Standalone

validate_docs_and_figures.py exit 1 at Project Analysis (rotating project)

Scripts Failing

PDF Generation Issues

Missing Dependencies

Debug Mode

Log Files

🛠️ Maintenance

System Updates

Rendered Projects (`projects/templates/` + `projects/active/`)

Non-Rendered Projects (`working/`, `ongoing/`, `archive/`, optional legacy mirrors)

In-Progress Projects (`projects/working/`)

Secure Pipeline (`secure_run.sh`)

🔒 Core Utilities (`infrastructure/core/`)

🔍 Validation System (`infrastructure/validation/`)

🔬 Scientific Development (`infrastructure/scientific/`)

🤖 LLM Integration (`infrastructure/llm/`)

🎨 Rendering System (`infrastructure/rendering/`)

🚀 Publishing Module (`infrastructure/publishing/`)

Project Fails In `--all-projects` But Passes Standalone

`validate_docs_and_figures.py` exit 1 at Project Analysis (rotating project)