Conversation
…sue #24)\n\nAdds a test to reproduce the 'inotify instance limit reached' error on Linux by rapidly creating many concurrent cache waits using the pickle backend. The test is skipped on non-Linux systems and is designed to be informative, not flaky.\n\nAlso updates CI to lower the inotify instance limit on Linux/local jobs, increasing the likelihood of hitting the error in CI.\n\nReferences: https://github.com/python-cachier/cachier/issues/24\n\n- Follows project and Python best practices for test isolation, OS-specific logic, and CI configuration.\n- Test is marked with @pytest.mark.pickle and uses pytest.skip for clarity.\n- CI change is Linux/local only, with clear comments.\n\nThis helps document and track the resource exhaustion issue, and provides a reproducible test for future backend or watchdog improvements.
for more information, see https://pre-commit.ci
- Increase thread count to 4x the system limit (up to 4096) - Make the slow function slower (0.5s instead of 0.1s) - Increase wait_for_calc_timeout to 0.1s - Force test to fail instead of skip when limit not hit - Add more debugging output to understand what's happening This should help reproduce the inotify instance limit issue in CI.
for more information, see https://pre-commit.ci
- Test now FAILS when inotify instance limit is reached (bug exists) - Test now PASSES when no inotify errors occur (bug is fixed) - This makes the test useful for verifying fixes to the inotify issue - Updated error messages to reflect the new logic
- Test is now marked as expected to fail with @pytest.mark.xfail - Will show as XFAIL when bug exists (expected) - Will show as XPASS when bug is fixed (pleasant surprise) - Provides clear reason for expected failure
for more information, see https://pre-commit.ci
…h, ternary, string wrapping)
for more information, see https://pre-commit.ci
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds a Linux-only pytest to reproduce the “inotify instance limit reached” error for the pickle backend and updates CI to lower the inotify instance limit on Linux/local jobs.
- Introduce
test_inotify_instance_limit_reachedmarked as Linux-only and xfail intests/test_pickle_core.py - Add a GitHub Actions step to reduce
fs.inotify.max_user_instancesto 128 for Linux/local CI runs
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/test_pickle_core.py | New test_inotify_instance_limit_reached function to exhaust inotify instances via concurrent cache calls |
| .github/workflows/ci-test.yml | CI step to lower inotify instance limit on Linux/local runners to force the test to hit the limit |
Comments suppressed due to low confidence (1)
tests/test_pickle_core.py:631
- The test uses
threading.Threadbutthreadingis never imported in this scope, which will raise aNameError. Please addimport threadingbefore using it.
import queue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a Linux-only test to reproduce the "inotify instance limit reached" error (see Issue #24) by rapidly creating many concurrent cache waits using the pickle backend.
@pytest.mark.pickleand is skipped on non-Linux systems.Additionally, the CI workflow is updated to lower the inotify instance limit on Linux/local jobs, increasing the likelihood of hitting the error in CI.
Rationale
Enables tracking of how we handle issue #24