diff --git a/.github/workflows/rocm-ci.yml b/.github/workflows/rocm-ci.yml
index 3eda5116d..1f58ffe8e 100644
--- a/.github/workflows/rocm-ci.yml
+++ b/.github/workflows/rocm-ci.yml
@@ -60,6 +60,7 @@ jobs:
           path: ./dist/*.whl
       - name: Run tests
         env:
+          ROCM_TEST_INCLUDE_SKIPS: "1"
           GPU_COUNT: "8"
           GFX: "gfx90a"
         run: |
diff --git a/rocm-downstream-dev-guide.md b/rocm-downstream-dev-guide.md
index c7e5578cf..ff86eeb52 100644
--- a/rocm-downstream-dev-guide.md
+++ b/rocm-downstream-dev-guide.md
@@ -30,7 +30,7 @@ This guide lays out how to do some dev operations, what branches live in this re
      If upstream reviewers request some changes to the new PR before merging, you can add
      or modify commits on the new `-upstream` feature branch.
   b. If this is an urgent change that we want in `rocm-main` right now but also want upstream,
-     add the `open-upstream` label, merge your PR, and then follow the link that 
+     add the `open-upstream` label, merge your PR, and then follow the link that
   c. If this is a change that we only want to keep in `rocm/jax` and not push into upstream,
      squash and merge your PR.
 
@@ -63,3 +63,7 @@ development tasks. These all live in `.github/workflows`.
 | ROCm Open Upstream PR      | `rocm-open-upstream-pr.yml`      | Add the `open-upstream` label to a PR                | Copies changes from a PR aimed at `rocm-main` into a new PR aimed at upstream's `main` |
 | ROCm Nightly Upstream Sync | `rocm-nightly-upstream-sync.yml` | Runs nightly, can be triggered manually via Actions  | Opens a PR that merges changes from upstream `main` into our `rocm-main` branch        |
 
+# Test Guidlines
+
+Use `pytest.mark.xfail` or `pytest.mark.xfail(run=False)` to skip failing tests that we will fix later. Use these for tests that we do eventually want to pass, but that we will skip for now to keep CI green. Continue to use `unittest.skip` or `self.skip` for tests that aren't applicable to AMD hardware.
+
diff --git a/tests/conftest.py b/tests/conftest.py
new file mode 100644
index 000000000..78e56a831
--- /dev/null
+++ b/tests/conftest.py
@@ -0,0 +1,14 @@
+import os
+import pytest
+
+
+INCLUDE_SKIPS = os.getenv("ROCM_TEST_INCLUDE_SKIPS", default=False)
+
+@pytest.hookimpl(optionalhook=True)
+def pytest_json_modifyreport(json_report):
+    """Get rid of skipped tests in reporting. We only care about xfails."""
+    if (not INCLUDE_SKIPS
+            and "summary" in json_report
+            and "total" in json_report["summary"]):
+        json_report["summary"]["unskipped_total"] = json_report["summary"]["total"] - json_report["summary"].get("skipped", 0)
+        del json_report["summary"]["total"]