cf. #5701 (comment)

It appears that none of our tests exercise plugins.load_plugins. We should try to make the tests use that instead of a mock. If that's not possible, we should probably try to run some end-to-end (CLI + real music files) tests in CI.

EDIT: Or maybe we do use load_plugins, but there appears to be some difference, see the discussion in the linked comment thread.