Skip to main content
A run’s selection decides which tests it executes. The usual way to select tests is by tag. You can save a selection on an environment so every run of that environment uses it, and change any part of it when you trigger a run.

Select tests by tag

Tag your tests in Playwright:
Then run only the tagged tests:
Use exclude_tags to skip tests, for example --exclude-tags @destructive on an environment that points at production. Empirical adds a missing @ to each tag, so smoke and @smoke select the same tests. A tag matches as a substring of the test’s tags: @smoke also selects a test tagged @smoke-nightly. tags does not filter by test title. A title such as checkout with a saved card becomes @checkout with a saved card, matches no test, and the run ends with empty_selection. To run specific tests, pass their ids in test_case_ids (--tests in the CLI).

Selection fields

The trigger request and the environment use the same field names with the same meaning. Most selections need only tags and exclude_tags. Use the project fields when your Playwright config splits tests into projects you want to run separately, such as one project per browser.

How Empirical combines them

A test runs when it passes every filter:
  1. If tags is set, it carries one of the tags.
  2. If test_case_ids is set, its id is in the list.
  3. Its Playwright project matches playwright_projects_match and no playwright_projects_ignore pattern.
  4. It carries none of the exclude_tags.
exclude_tags applies last, so it only removes tests. Playwright setup projects that the selected tests depend on run too. When nothing matches, the run ends with the empty_selection error code.

Environment defaults and request overrides

For each field, Empirical takes the first value it finds:
  1. The trigger request.
  2. For a rerun, the run it reruns.
  3. The run’s environment.
  4. Empty.
A field you send replaces the environment’s value for that run. Empirical doesn’t merge the two lists. Send an empty list to clear a field: "tags": [] runs the environment’s selection without its tags. The environment’s tags and exclude_tags apply to a run that sends test_case_ids. To run a test the environment’s tags don’t select, send "tags": [] with the ids. Empirical stores the resolved selection on the run. Changing the environment later doesn’t change runs that already exist.

Save a selection on an environment

Add the fields to the environment’s entry in .empiricalrun/ENVIRONMENTS.yaml:
A field you leave out of the entry resets to its default when Empirical syncs the file: no tags, and every project.

Select tests for one run

With the CLI:
With the API:
See the CLI reference for every flag and the API reference for every field.

Rerun failed tests

A rerun keeps its source run’s selection, including Re-run all tests. To rerun only the tests that failed, open Re-run on a completed run and click Re-run failed tests, or send "only_failed": true to the rerun API. When the source run has no failed tests, an API rerun with only_failed runs the source run’s whole selection.