Test Badge Toolchain Documentation

run_pytest_repo.sh and publish_repo_badges.sh

PMIverse / PMISuite Testing & Badge Publishing Pipeline

1. Overview

This document describes the end–to–end test and badge pipeline built around two scripts:

run_pytest_repo.sh – repository‑wide Odoo addon test runner that: - runs pytest-odoo per addon, - produces logs, JUnit XML, and coverage reports, - generates SVG badges per addon and an overall summary, and - maintains a latest symlink for consumers.
publish_repo_badges.sh – badge publisher that: - copies the latest SVG badges from the test logs into a repository, - writes them under .badges/tests/, - optionally commits and pushes changes, enabling self‑hosted README badges.

Together they provide a fully self‑contained testing status surface for Odoo repositories, without external badge services.

Canonical PMIVerse policy requires focused addon selection for normal work. Use --addons for the addons being changed; a repository-wide run requires explicit approval. The runner creates and removes disposable PostgreSQL databases.

2. `run_pytest_repo.sh` – Repo‑Wide Test Runner

2.1 Purpose

run_pytest_repo.sh runs pytest for:

all addons with a tests/ directory under a repository, or
a provided list of addon directories.

It is pytest‑odoo based and:

isolates addons into separate disposable databases,
can run addons sequentially or in parallel,
produces structured logs and reports in a timestamped directory,
generates summary metadata (CSV + JSON), and
creates SVG badges for each addon and for the overall test status.

2.2 Usage

run_pytest_repo.sh --repo <repo_root> [options] [-- PYTEST_ARGS]
run_pytest_repo.sh --addons <addon_dir1> [<addon_dir2> ...] [options] [-- PYTEST_ARGS]

Options

Option	Description
`--repo <path>`	Repository root containing multiple addon directories.
`--addons <paths...>`	Explicit list of addon directories (each must contain `__manifest__.py`).
`--pattern <glob>`	Include pattern for addon directory names (default: `*`).
`--exclude <glob>`	Exclude pattern for addon directory names (optional).
`--db-prefix <prefix>`	Prefix for disposable databases (default: `test_`).
`--parallel <N>`	Maximum concurrent addon jobs (default: automatic, at least 4 when feasible).
`--stop-on-first-fail`	Stop processing after the first failing addon.
`--only-changed [<ref>]`	Run only addons changed vs git ref (default ref: `origin/18.0`).
`--no-junit`	Disable JUnit XML report generation.
`--no-coverage`	Disable coverage measurement, even if available.
`--keep-db`	Keep test databases instead of dropping them before and after the run.
`-h`, `--help`	Show inline help and exit.

Any arguments after -- are passed directly to pytest (PYTEST_ARGS).

--keep-db is forwarded to each addon runner before that separator. It preserves the database on both successful and failed test runs and skips the batch runner's pre-run and post-run cleanup. Without it, disposable database cleanup remains enabled.

Parallel execution uses a rolling worker pool: whenever an addon finishes, the next queued addon starts in the available slot without waiting for other running addons. At most N addons run concurrently, and all started jobs finish before the summary is generated. Parallel execution requires Bash 4.3 or newer.

2.3 Input Discovery

There are two modes of selecting addons:

Explicit list – --addons <addon_dir1> <addon_dir2> ...
- Each directory must contain __manifest__.py.
- Must have a tests/ subdirectory to be considered.
Repository discovery – --repo <repo_root>
- The script scans for all __manifest__.py files under repo_root.
- It filters directories with a tests/ subdirectory.
- It applies include (--pattern) and exclude (--exclude) glob filters.
- Optionally restricts to changed addons only with --only-changed [<ref>] using git diff.

If no addons with tests/ are found, the script exits with code 2.

2.4 Output Structure

A new log root is created per run:

/srv/pmiverse/logs/tests/<timestamp>/
    summary.txt
    summary.json
    <addonA>.log
    <addonA>.exit
    <addonA>/
        junit.xml
        coverage.xml
        htmlcov/...
        coverage.pct
    <addonB>.log
    <addonB>.exit
    <addonB>/
        ...
    badges/
        summary.svg
        <addon>_status.svg
        <addon>_coverage.svg

Additionally, a symlink is maintained:

/srv/pmiverse/logs/tests/latest → /srv/pmiverse/logs/tests/<timestamp>/

The latest symlink provides a stable location used by publish_repo_badges.sh.

2.5 Per‑Addon Execution

Each addon may declare test-only Odoo apps in tests/odoo_dependencies.txt, one addon name per line. The delegated runner validates this file and installs those apps together with the target addon before pytest. Blank lines and full-line # comments are allowed. A nonempty declaration requires a disposable test_* database. No separate database preparation or --keep-db is needed to exercise these integrations on a fresh run. Production manifest dependencies remain unchanged; required apps must be available on the configured Odoo addons path, and installation failures fail the run.

For each addon:

A dedicated database name is derived:

<db-prefix><addon_name>

The per‑addon runner is invoked:

/srv/pmiverse/_pmiconfig/scripts/test/run_pytest_addon.sh \
    -a <addon_dir>/tests -d <db_name> -- <per-addon pytest args>

Output is tee’d into <addon>.log.
The exit code is recorded in <addon>.exit.

Per‑addon pytest arguments may include:

JUnit XML output (if not disabled)
Coverage plugin configuration (if enabled and available)
Any additional PYTEST_ARGS from the command line

2.6 Coverage Handling

If coverage is enabled (default) the script:

Tries to detect pytest-cov, using: - the instance venv’s Python, or
- the instance venv’s pytest, or
- the global pytest.
If pytest-cov is unavailable, coverage is switched off and a message is printed.

When coverage is active, per‑addon arguments include:

-p pytest_cov
--cov=odoo.addons.<addon_name>
--cov-report=term-missing:skip-covered
--cov-report=xml:<addon_dir>/coverage.xml
--cov-report=html:<addon_dir>/htmlcov

A coverage.pct file is derived from the XML line-rate attribute and stored as a plain percentage (e.g. 92.5).

2.7 Summary Files

`summary.txt`

A CSV summary:

Addon,Status,ExitCode,Log
addon_a,PASSED,0,/srv/pmiverse/logs/tests/<ts>/addon_a.log
addon_b,FAILED,1,/srv/pmiverse/logs/tests/<ts>/addon_b.log
...

`summary.json`

A minimal JSON array with per‑addon entries:

[
  {
    "addon": "addon_a",
    "status": "PASSED",
    "exit": 0,
    "log": "/srv/.../addon_a.log",
    "junit": "/srv/.../addon_a/junit.xml",
    "coverage_xml": "/srv/.../addon_a/coverage.xml",
    "coverage_pct": "92.5",
    "htmlcov": "/srv/.../addon_a/htmlcov"
  }
]

This is suitable for API consumption, dashboards, or more advanced reporting.

2.8 Badge Generation

The script generates SVG badges under:

/srv/pmiverse/logs/tests/<timestamp>/badges/

Per‑Addon Badges

For each addon:

<addon>_status.svg – status badge (PASSED/FAILED).
<addon>_coverage.svg – coverage badge, if coverage was computed.

Colors:

Status:
Green #4c1 for PASSED
Red #e05d44 for FAILED
Coverage:
Green #4c1 for ≥ 90%
Yellow #dfb317 for ≥ 75% and < 90%
Red #e05d44 for < 75%
Grey #9f9f9f if coverage is not available.

Overall Summary Badge

summary.svg
Displays tests label and value <passed>/<total> passed.
Green if all pass, red otherwise.

The badge layout is compatible with common “shields‑style” badges (rectangular, left label, right value).

2.9 Exit Codes

0 – All addons passed.
2 – No addons with tests/ found.
3 – One or more addons failed.
Other non‑zero – Argument errors, environment issues, or early aborts.

2.10 Example Invocations

Run all addons in a repo, single process:

./run_pytest_repo.sh --repo /srv/pmiverse/_pmisuite18/repos/pmiverse/testing

Run only changed addons vs origin/18.0:

./run_pytest_repo.sh --repo /path/repo --only-changed

Run with 4 parallel jobs and extra pytest args:

./run_pytest_repo.sh --repo /path/repo --parallel 4 -- --maxfail=1 -q

Disable coverage and JUnit:

./run_pytest_repo.sh --repo /path/repo --no-junit --no-coverage

3. `publish_repo_badges.sh` – Repository‑Local Badge Publisher

3.1 Purpose

publish_repo_badges.sh copies the latest SVG badges generated by run_pytest_repo.sh into a target repository:

<repo>/.badges/tests/

Optionally, it commits and pushes those updates, enabling READMEs to render badges from the repository itself, without external services.

3.2 Usage

publish_repo_badges.sh --repo <repo_root> \
    [--source <logs_dir>] \
    [--commit] \
    [--branch <name>]

Options

Option	Description
`--repo <path>`	Target repository root (required).
`--source <path>`	Source badges directory (default: `/srv/pmiverse/logs/tests/latest/badges`).
`--commit`	Commit and push badge updates.
`--branch <name>`	Branch to commit into (default: current branch).
`-h`, `--help`	Show inline help.

3.3 Source and Destination

Default source directory:

/srv/pmiverse/logs/tests/latest/badges/

Destination directory inside the repository:

<repo_root>/.badges/tests/

All *.svg files from source are copied into the destination.

3.4 README Integration

Once badges have been published into the repo, the README can embed them as:

## Test Status

![Overall Tests](./.badges/tests/summary.svg)
![Addon A](./.badges/tests/addon_a_status.svg)
![Addon A Coverage](./.badges/tests/addon_a_coverage.svg)

These badges are versioned with the repository and remain valid even without external CI badge URLs.

3.5 Optional Commit & Push

External write: --commit creates a commit and pushes it. It requires explicit approval and must not be used as part of read-only validation.

When --commit is passed:

The script optionally checks out the given branch (--branch).
Adds .badges/tests/*.svg to the Git index.
If there are changes, creates a commit with message:

chore(badges): update test badges

Pushes to the current branch:

git push origin <current_branch>

If there are no changes to SVG files, no commit is made.

3.6 Failure Modes

Condition	Behaviour
`--repo` missing	Error + usage output, non‑zero exit.
Source directory missing	Error with explicit path, non‑zero exit.
No SVG files present	Copy attempts but safely continues; nothing is committed.
Git checkout fails	Non‑zero exit (when using `--branch`).
No diff to commit	No commit created; informational message.

4. Typical CI Pipeline

A typical PMIverse CI job might chain both scripts:

# 1. Run repository tests and generate badges
./run_pytest_repo.sh --repo "$CI_PROJECT_DIR" --parallel 4 -- --maxfail=1 -q

# 2. Publish badges into the repo and push
./publish_repo_badges.sh --repo "$CI_PROJECT_DIR" --commit --branch "$CI_COMMIT_REF_NAME"

This produces:

Structured logs and reports in /srv/pmiverse/logs/tests/<timestamp>/,
Up‑to‑date SVG badges under <repo>/.badges/tests/, and
README badges that reflect the latest CI run.

5. Maintenance Notes

Keep run_pytest_addon.sh stable and version‑controlled together with this documentation.
If coverage rules change (e.g. per‑package cov configuration), update the coverage arguments in run_pytest_repo.sh accordingly.
When adding new badge types (e.g. lint status, migration tests), extend the generate_badge_svg calls and consider separate subdirectories.
For multi‑repo setups, standardize on /srv/pmiverse/logs/tests/ as the single aggregation point.

6. License

See the project’s LICENSE and top‑level README for full legal details.

Test Badge Toolchain Documentation

1. Overview

2. run_pytest_repo.sh – Repo‑Wide Test Runner

2.1 Purpose

2.2 Usage

Options

2.3 Input Discovery

2.4 Output Structure

2.5 Per‑Addon Execution

2.6 Coverage Handling

2.7 Summary Files

summary.txt

summary.json

2.8 Badge Generation

Per‑Addon Badges

Overall Summary Badge

2.9 Exit Codes

2.10 Example Invocations

3. publish_repo_badges.sh – Repository‑Local Badge Publisher

3.1 Purpose

3.2 Usage

Options

3.3 Source and Destination

3.4 README Integration

3.5 Optional Commit & Push

3.6 Failure Modes

4. Typical CI Pipeline

5. Maintenance Notes

6. License

2. `run_pytest_repo.sh` – Repo‑Wide Test Runner

`summary.txt`

`summary.json`

3. `publish_repo_badges.sh` – Repository‑Local Badge Publisher