Count test totals correctly for dashboards (#280)

* Account test totals correctly for dashboards

* Add blurb to the dev guide on skipping tests

* Remove extra newline

* Default to 0 if "skipped" isn't found

Co-authored-by: Mathew Odden <1471252+mrodden@users.noreply.github.com>

---------

Co-authored-by: Mathew Odden <1471252+mrodden@users.noreply.github.com>
This commit is contained in:
charleshofer 2025-03-14 16:57:08 -05:00 committed by GitHub
parent 9cc545254c
commit 022da913e6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 20 additions and 1 deletions

View File

@ -60,6 +60,7 @@ jobs:
path: ./dist/*.whl
- name: Run tests
env:
ROCM_TEST_INCLUDE_SKIPS: "1"
GPU_COUNT: "8"
GFX: "gfx90a"
run: |

View File

@ -30,7 +30,7 @@ This guide lays out how to do some dev operations, what branches live in this re
If upstream reviewers request some changes to the new PR before merging, you can add
or modify commits on the new `-upstream` feature branch.
b. If this is an urgent change that we want in `rocm-main` right now but also want upstream,
add the `open-upstream` label, merge your PR, and then follow the link that
add the `open-upstream` label, merge your PR, and then follow the link that
c. If this is a change that we only want to keep in `rocm/jax` and not push into upstream,
squash and merge your PR.
@ -63,3 +63,7 @@ development tasks. These all live in `.github/workflows`.
| ROCm Open Upstream PR | `rocm-open-upstream-pr.yml` | Add the `open-upstream` label to a PR | Copies changes from a PR aimed at `rocm-main` into a new PR aimed at upstream's `main` |
| ROCm Nightly Upstream Sync | `rocm-nightly-upstream-sync.yml` | Runs nightly, can be triggered manually via Actions | Opens a PR that merges changes from upstream `main` into our `rocm-main` branch |
# Test Guidlines
Use `pytest.mark.xfail` or `pytest.mark.xfail(run=False)` to skip failing tests that we will fix later. Use these for tests that we do eventually want to pass, but that we will skip for now to keep CI green. Continue to use `unittest.skip` or `self.skip` for tests that aren't applicable to AMD hardware.

14
tests/conftest.py Normal file
View File

@ -0,0 +1,14 @@
import os
import pytest
INCLUDE_SKIPS = os.getenv("ROCM_TEST_INCLUDE_SKIPS", default=False)
@pytest.hookimpl(optionalhook=True)
def pytest_json_modifyreport(json_report):
"""Get rid of skipped tests in reporting. We only care about xfails."""
if (not INCLUDE_SKIPS
and "summary" in json_report
and "total" in json_report["summary"]):
json_report["summary"]["unskipped_total"] = json_report["summary"]["total"] - json_report["summary"].get("skipped", 0)
del json_report["summary"]["total"]