Skip to content

Tests: Remove implicit dependencies on some leaky state - #13091

Open
mdawaffe wants to merge 5 commits into
WordPress:trunkfrom
mdawaffe:fix/test-isolation-issues
Open

Tests: Remove implicit dependencies on some leaky state#13091
mdawaffe wants to merge 5 commits into
WordPress:trunkfrom
mdawaffe:fix/test-isolation-issues

Conversation

@mdawaffe

Copy link
Copy Markdown
Contributor

There are a bunch of PHPUnit tests that implicitly depend on state leaked from other tests. This leaky state isn't observed in "normal" test runs, but can be easily seen by running:

npm run test:php -- --order-by=random

Leaky state can cause several problems:

  • A test may fail when run by itself (if it depends on state leaked by a previously run test)
  • A test class may fail when run by itself
  • A test may pass only because of some leaked state, not because the test is testing what it's supposed to test.
  • etc.

Trac ticket: https://core.trac.wordpress.org/ticket/65893

Use of AI Tools

AI assistance: Yes
Tool: Claude Code
Model: Opus 5
Used for: Analysis for failing tests, initial implementation; final implementation reviewed/edited by me.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

`self::$theme_root` should be reverted in `WP_Font_Face_UnitTestCase`

`self::$requires_switch_theme_fixtures` needs to be set before calling
`parent::set_up_before_class()`.
…itemaps`

`@runInSeparateProcess` reruns `tests/phpunit/includes/bootstrap.php`,
which deletes all posts, which changes the conditions of the test.

In particular, `test_disable_sitemap_should_return_404()` is supposed to
test what happens if sitemaps are disabled, not what happens when there
are no posts in the sitemap.
…self.

These tests call `->render_control_template_scripts()`, which depends on a
normally lazy-loaded file: `wp-includes/media-template.php`.

Previous tests in these classes load that file as a side-effect of calling
`$widget->enqueue_admin_scripts()`. When testing these tests individually,
that file is never loaded. Load it explicitly.
Test methods that register new block types should unregister them as well.

Otherwise, the state leaks into later tests.
@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props mdawaffe.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants