Skip to content

[ty] Infer constant membership in inline list and set literals - #28676

Open
carljm wants to merge 1 commit into
mainfrom
cjm/inline-literal-membership
Open

carljm wants to merge 1 commit into
mainfrom
cjm/inline-literal-membership

Conversation

@carljm

@carljm carljm commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Inline list and set membership tests now preserve the values of their individual elements, allowing ty to determine when a condition is always true or false. With the target platform set to Linux, both sys.platform in ["linux", "windows"] and sys.platform in {"linux", "windows"} select only the matching branch, so an assignment there is inferred as Literal[1].

Reuse the precise element extraction used for iteration and membership narrowing without changing the inferred container type. Lists use the existing tuple membership equality evaluator; sets are folded only when their elements and the searched value have known builtin literal hashing and equality. Fixed-length unpacking also respects tuple subclasses that override __iter__.

Context is that in python/typing#2173 people are wanting to specify that sys.platform in ["linux", "windows"] should work. This makes it work in ty so that we can be fine with any outcome of that specification change.

Test plan

Mdtests cover platform-dependent branch reachability; positive and negative membership in inline lists and sets; inferred literal values and unions; fixed-length unpacking and tuple iterator overrides; stored mutable containers; custom equality; comparison chains; and preservation of enum-set narrowing.

Ecosystem results

The ecosystem comparison adds four redundant-condition-strict warnings:

  • paasta: a parameter annotated as a tuple or None is checked against strings, so the exclusion is always true.
  • SciPy: the loop's literal method names exclude both values in the membership test, making the condition always false.
  • steam.py: the known slot names exclude _state, so the membership filter is redundant.
  • dd-trace-py: envier dynamically replaces configuration markers with strings through setattr, which ty does not model. The resulting false positive exposes an existing limitation already present for equality and tuple membership. We accept that limitation for this change.

All four changes were reproduced and minimized against the exact compared revisions. Flaky diagnostic differences are excluded from this summary.

@carljm carljm added the ty Multi-file analysis & type inference label Sep 17, 2026
@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 97.87%. The percentage of expected errors that received a diagnostic held steady at 97.78%. The number of fully passing files held steady at 130/145.

@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
sphinx 166.69MB 166.72MB +0.02% (31.31kB)
prefect 461.17MB 461.19MB +0.00% (18.77kB)
trio 92.25MB 92.25MB +0.00% (1.34kB)
flake8 38.25MB 38.25MB +0.00% (1008.00B)

Significant changes

Click to expand detailed breakdown

sphinx

Name Old New Diff Outcome
infer_expression_types_impl 11.52MB 11.52MB +0.08% (9.30kB)
infer_definition_types 11.57MB 11.58MB +0.05% (5.63kB)
all_narrowing_constraints_for_expression 1.65MB 1.66MB +0.15% (2.54kB)
BoundMethodType 1.10MB 1.10MB +0.21% (2.34kB)
MemberLookupKey 3.01MB 3.01MB +0.08% (2.34kB)
Type<'db>::class_member_with_policy_inner_ 3.28MB 3.29MB +0.07% (2.33kB)
member_lookup_with_policy_inner 3.65MB 3.65MB +0.05% (1.97kB)
infer_statement_types_impl 518.14kB 519.26kB +0.22% (1.12kB)
analyze_condition 422.41kB 423.43kB +0.24% (1.02kB)
infer_unpack_types 265.71kB 266.67kB +0.36% (984.00B)
analyze_non_terminal_call 509.12kB 509.52kB +0.08% (416.00B)
loop_header_reachability 279.69kB 280.00kB +0.11% (312.00B)
evaluate_projected_narrowing_checkpoint 252.80kB 253.11kB +0.12% (312.00B)
implicit_attribute_names 127.01kB 127.16kB +0.12% (160.00B)
try_metaclass_inner 186.10kB 186.26kB +0.08% (160.00B)
... 5 more

prefect

Name Old New Diff Outcome
infer_unpack_types 453.32kB 459.48kB +1.36% (6.16kB)
Type<'db>::class_member_with_policy_inner_ 9.94MB 9.94MB +0.04% (3.80kB)
BoundMethodType 2.70MB 2.71MB +0.14% (3.76kB)
MemberLookupKey 7.09MB 7.10MB +0.05% (3.76kB)
member_lookup_with_policy_inner 9.14MB 9.15MB +0.04% (3.63kB)
infer_expression_types_impl 23.72MB 23.72MB +0.01% (2.36kB)
FunctionType 6.74MB 6.73MB -0.02% (1.68kB) ⬇️
GenericContext 2.17MB 2.17MB -0.05% (1.11kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 3.97MB 3.97MB -0.02% (880.00B) ⬇️
TypeVarSetInner 1.07MB 1.07MB -0.07% (740.00B) ⬇️
Type<'db>::apply_specialization_inner_ 2.84MB 2.84MB -0.02% (672.00B) ⬇️
TypeVarInference 1.44MB 1.44MB -0.04% (600.00B) ⬇️
StaticClassLiteral<'db>::implicit_attribute_inner_ 1.15MB 1.15MB +0.05% (576.00B)
Specialization 5.08MB 5.08MB -0.01% (560.00B) ⬇️
member_lookup_with_policy_and_receiver_inner 2.05MB 2.05MB +0.02% (504.00B)
... 23 more

trio

Name Old New Diff Outcome
infer_unpack_types 93.00kB 93.75kB +0.81% (768.00B)
FunctionType 1.38MB 1.38MB -0.05% (688.00B) ⬇️
BoundMethodType 373.04kB 373.65kB +0.16% (624.00B)
MemberLookupKey 1.15MB 1.15MB +0.05% (624.00B)
Type<'db>::class_member_with_policy_inner_ 1.29MB 1.29MB +0.04% (576.00B)
member_lookup_with_policy_inner 1.24MB 1.24MB +0.04% (520.00B)
GenericContext 570.29kB 569.85kB -0.08% (456.00B) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 945.31kB 945.00kB -0.03% (320.00B) ⬇️
Type<'db>::cached_materialization_::interned_arguments 266.88kB 266.56kB -0.12% (320.00B) ⬇️
TypeVarSetInner 262.04kB 261.75kB -0.11% (296.00B) ⬇️
infer_expression_types_impl 5.95MB 5.95MB +0.00% (264.00B)
infer_deferred_types 1.20MB 1.20MB +0.02% (260.00B)
StaticClassLiteral<'db>::try_mro_unspecialized_ 105.03kB 105.27kB +0.23% (248.00B)
TypeVarInference 372.31kB 372.07kB -0.06% (240.00B) ⬇️
Specialization 1.13MB 1.13MB -0.02% (224.00B) ⬇️
... 24 more

flake8

Name Old New Diff Outcome
infer_unpack_types 27.30kB 27.53kB +0.86% (240.00B)
infer_definition_types 1.17MB 1.17MB +0.02% (232.00B)
try_metaclass_inner 31.22kB 31.37kB +0.48% (152.00B)
code_generator_of_static_class 18.98kB 19.07kB +0.49% (96.00B)
inheritance_cycle_inner 9.57kB 9.66kB +0.98% (96.00B)
StaticClassLiteral<'db>::slot_definition_ 1.30kB 1.39kB +6.59% (88.00B)
implicit_attribute_names 15.26kB 15.32kB +0.36% (56.00B)
use_def_map 30.57kB 30.62kB +0.15% (48.00B)

@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
redundant-condition-strict 4 0 0
Total 4 0 0

Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.

Raw diff:

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/internal/settings/errortracking.py:43:48 warning[redundant-condition-strict] Condition `config._report_handled_errors in ["all", "user", "third_party"]` is always false

paasta (https://github.com/yelp/paasta)
+ paasta_tools/paastaapi/api_client.py:211:12 warning[redundant-condition-strict] Condition `response_type not in ["file", "bytes"]` is always true

scipy (https://github.com/scipy/scipy)
+ scipy/optimize/tests/test__root.py:25:16 warning[redundant-condition-strict] Condition `method in ('linearmixing', 'excitingmixing')` is always false

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/chat.py:172:65 warning[redundant-condition-strict] Condition `slot not in {"_state"}` is always true

Full report with detailed diff (timing results)

@carljm

carljm commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author
Full ecosystem report for ab05864

PR #28676 ecosystem summary

The PR adds four redundant-condition-strict warnings. Three identify redundant membership conditions under the projects' existing annotations. The fourth, in dd-trace-py, is a runtime false positive caused by an existing limitation in modeling dynamically replaced configuration attributes. The same limitation already produces equality warnings on the merge base; the PR exposes it in inline list membership as well.

Affected projects

1. dd-trace-py: dynamically replaced configuration attributes

Report entry: ddtrace/internal/settings/errortracking.py:43

The PR adds warning[redundant-condition-strict] Condition config._report_handled_errors in ["all", "user", "third_party"] is always false. At runtime, envier replaces the configuration marker with a parsed string through setattr, so this condition can be true. Both revisions infer the attribute as EnvVariable[str], following the factory's return annotation. Envier supplies a mypy plugin to model this transformation; ordinary annotations do not express it.

The merge base already reports the subsequent equalities with "user", "third_party", and "all" as always false. Equivalent tuple membership also warns on the merge base. The new warning therefore extends existing comparison reasoning to the inline list; it does not demonstrate a new error in how list membership compares the inferred types. We accept this existing dynamic-inference limitation, consistent with the current treatment of equality and tuple membership.

Related ty issue: ty#2016 discusses limitations of dynamic setattr effects; it is not an envier-specific issue.

class EnvVariable:
    pass

class ErrorTrackingConfig:
    _report_handled_errors = EnvVariable()

config = ErrorTrackingConfig()
setattr(config, "_report_handled_errors", "all")
# Merge base: no diagnostic
# PR: warning[redundant-condition-strict] Condition `config._report_handled_errors in ["all"]` is always false
if config._report_handled_errors in ["all"]:
    pass

The minimized condition evaluates to True at runtime.

2. paasta: a tuple parameter checked against strings

Report entry: paasta_tools/paastaapi/api_client.py:211

The parameter is annotated as Optional[Tuple[Any]], but the condition tests response_type not in ["file", "bytes"]. The merge base reports no diagnostic at this condition; the PR correctly reports that it is always true under the existing non-strict equality policy. An ordinary tuple or None cannot match either string. This exposes an ineffective check under the annotation. Strict equality semantics continue to leave the result uncertain because a tuple subclass can override equality.

def check(response_type: tuple[object]):
    # Merge base: no diagnostic
    # PR: warning[redundant-condition-strict] Condition `response_type not in ["file"]` is always true
    if response_type not in ["file"]:
        pass

3. SciPy: a literal union cannot match either excluded method

Report entry: scipy/optimize/tests/test__root.py:25

The loop iterates over seven method names, excluding both linearmixing and excitingmixing. The PR correctly identifies method in ('linearmixing', 'excitingmixing') as always false. This also improves tuple membership: the shared equality evaluator now recognizes that every member of a union of builtin literals has reflexive equality.

for method in ["hybr", "lm"]:
    # Merge base: no diagnostic
    # PR: warning[redundant-condition-strict] Condition `method in ("linearmixing",)` is always false
    if method in ("linearmixing",):
        pass

4. steam.py: excluding an absent slot name

Report entry: steam/chat.py:172

Both revisions infer _BaseMember.__slots__ as tuple[Literal["_user"], Literal["_cs_channel"]]. The generator excludes "_state", which is absent from those slots. The PR correctly reports slot not in {"_state"} as always true. The source reduces to a constant set membership condition without imports or class definitions.

# Merge base: no diagnostic
# PR: warning[redundant-condition-strict] Condition `"_user" not in {"_state"}` is always true
if "_user" not in {"_state"}:
    pass

Reproduction

  • Detailed report: ecosystem-analyzer report
  • Actions run: run 35281371799, attempt 1
  • Ruff comparison: eb3751b19c to 91cfc6f349, the merge of PR head ab0586463e
  • ecosystem-analyzer: a31ef389f3
  • mypy-primer: 3058720299
  • Dependency cutoff: 2026-09-17T22:18:41Z
  • Project Python: dd-trace-py, paasta, SciPy, and steam.py use Python 3.11.
  • Project analysis mode: all four projects use non-strict settings.
  • Comparison method: run both exact-revision profiling binaries with the PR ecosystem configuration installed as user-level configuration and each pinned project environment. Use ty check for SciPy; ty check paasta_tools for paasta; ty check steam for steam.py; and ty check --exclude tests/appsec/iast/fixtures/aspects/str/non_utf8_content.py --exclude tests/appsec/iast/fixtures/ast/str/non_utf8_content.py for dd-trace-py. Each command includes --python <project-venv> --output-format concise, with TY_CONFIG_FILE unset and no strict-analysis overrides. The standalone minimized examples need only ty check <file> --python-version 3.11 with the same user-level configuration.

@carljm
carljm marked this pull request as ready for review September 17, 2026 22:43
@carljm
carljm requested a review from a team as a code owner September 17, 2026 22:43
@astral-sh-bot
astral-sh-bot Bot requested a review from dcreager September 17, 2026 22:43
@carljm
carljm requested review from AlexWaygood and removed request for dcreager September 17, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant