Skip to content

Conversation

@moreal
Copy link
Contributor

@moreal moreal commented Jan 22, 2026

This pull request updates types module to v3.14.2. While doing it, it fixes also async-related feature. This pull request's base is generated by #6827.

Summary by CodeRabbit

  • New Features

    • Generators that act as iterable coroutines are now recognized as awaitable, improving async behavior.
    • Yield-from and await interactions now handle coroutine-iterable sources more consistently.
  • Bug Fixes

    • Reduces spurious TypeError cases when awaiting or yielding from wrapped coroutine-like generators.

✏️ Tip: You can customize this high-level summary in your review settings.

Co-Authored-By: CPython Developers <>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 22, 2026

📝 Walkthrough

Walkthrough

Adds a new ITERABLE_COROUTINE code flag and updates VM await/yield from resolution so generators marked with this flag are treated as awaitable iterables where applicable, with checks applied in compiler bytecode flags, async generator await logic, and frame iteration/await handling.

Changes

Cohort / File(s) Summary
Bytecode flag extension
crates/compiler-core/src/bytecode.rs
Add ITERABLE_COROUTINE = 0x0100 to CodeFlags bitflags.
Awaitable resolution (async generator)
crates/vm/src/builtins/asyncgenerator.rs
PyAnextAwaitable.get_awaitable_iter now recognizes PyGenerator instances with CO_ITERABLE_COROUTINE and returns the generator as the awaitable iterator; otherwise falls back to __await__ or TypeError.
Frame execution (await / yield-from)
crates/vm/src/frame.rs
GetAwaitable and GetYieldFromIter updated to treat generators bearing COROUTINE or ITERABLE_COROUTINE consistently for await and yield from use; non-iterable coroutines still error.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Caller
    participant Frame
    participant PyAnextAwaitable
    participant PyGenerator

    Caller->>Frame: attempt "await" / "yield from" on object
    Frame->>PyAnextAwaitable: resolve awaitable (GetAwaitable / GetYieldFromIter)
    alt obj is PyGenerator with ITERABLE_COROUTINE
        PyAnextAwaitable->>PyGenerator: return generator as awaitable iterator (no __await__ call)
        PyGenerator-->>Frame: iterator returned
    else other object
        PyAnextAwaitable->>PyGenerator: (not applicable) or call __await__ on object
        PyAnextAwaitable-->>Frame: awaitable or raise TypeError
    end
    Frame->>Caller: proceed with await/yield-from using returned iterator or propagate error
    rect rgba(0,128,0,0.5)
    Note over Frame,PyAnextAwaitable: Flag check for ITERABLE_COROUTINE is central to branching
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

  • Remove _dis module #6690: Modifies CodeFlags and coroutine-related flag semantics — closely related to adding/reshaping coroutine flags used here.
  • PyAnextAwaitable  #6427: Changes PyAnextAwaitable.get_awaitable_iter logic — directly overlaps awaitable resolution adjustments in this PR.
  • sys.set_asyncgen_hook #6439: Updates awaitable resolution and async-generator handling in VM/frame paths — touches the same call sites and control flow.

Poem

🐰 A tiny flag hops into place,
Marking generators with gentle grace.
Now await and yield-from find their way,
As coroutines dance and softly play.
Hooray! 🥕✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Title check ⚠️ Warning The title 'Update types from v3.14.2' refers only to a types module version update, but the changeset primarily adds ITERABLE_COROUTINE flag support across multiple files and enhances async/coroutine handling logic—the main technical change is not captured in the title. Revise the title to reflect the primary change, such as 'Add ITERABLE_COROUTINE flag support for async generators' or 'Support ITERABLE_COROUTINE flag in coroutine handling', which better describes the substantial code additions.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

Code has been automatically formatted

The code in this PR has been formatted using:

  • cargo fmt --all
    Please pull the latest changes before pushing again:
git pull origin update-3.14.2-types

@github-actions
Copy link
Contributor

github-actions bot commented Jan 22, 2026

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[+] lib: cpython/Lib/_opcode_metadata.py
dependencies:

  • _opcode_metadata
    dependent tests: (13 tests)
    • opcode: test__opcode test_code test_dis test_peepholer
      • dis: test_ast test_compile test_compiler_assemble test_dtrace test_fstring test_inspect test_opcache test_patma test_positional_only_arg

[+] lib: cpython/Lib/pydoc_data
dependencies:

  • pydoc_data (native: system.)
    dependent tests: (no tests depend on pydoc_data)

[+] lib: cpython/Lib/types.py
[+] test: cpython/Lib/test/test_types.py
dependencies:

  • types
    dependent tests: (39 tests)
  • types: test___all__ test_ast test_asyncgen test_builtin test_call test_collections test_compile test_compiler_assemble test_decorators test_descr test_dis test_doctest test_dtrace test_dynamicclassattribute test_email test_exception_group test_fstring test_funcattrs test_generators test_genericalias test_importlib test_inspect test_listcomps test_marshal test_opcache test_os test_pprint test_pyclbr test_raise test_subclassinit test_subprocess test_tempfile test_trace test_traceback test_types test_typing test_unittest test_xml_etree test_xml_etree_c

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@moreal moreal force-pushed the update-3.14.2-types branch from 1e5e723 to b25479a Compare January 22, 2026 06:22
moreal and others added 4 commits January 22, 2026 17:52
- Add ITERABLE_COROUTINE (0x0100) to CodeFlags in bytecode.rs
- Update frame.rs to check for both COROUTINE and ITERABLE_COROUTINE flags
  when validating 'yield from' on coroutine objects
- Remove False and workaround in types.coroutine() now that the flag is supported
- Remove @unittest.expectedFailure from test_async_def and test_genfunc

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Generators decorated with @types.coroutine have the CO_ITERABLE_COROUTINE
flag set, making them awaitable. The GetAwaitable instruction now properly
handles this case instead of only checking for __await__ method.

This fixes the "object generator can't be used in 'await' expression" error
that occurred with asyncio.sleep(0) which uses @types.coroutine decorated
__sleep0() generator internally.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Generators decorated with @types.coroutine have the CO_ITERABLE_COROUTINE
flag and can be used in await expressions. The PyAnextAwaitable's
get_awaitable_iter method was missing this check, causing builtin anext()
to fail with "object generator can't be used in 'await' expression" when
used with such generators.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
For generators without CO_ITERABLE_COROUTINE flag, try to get __await__
method instead of returning an error immediately. This matches CPython's
_PyCoro_GetAwaitableIter behavior which allows generator subclasses that
define __await__ to be used in await expressions.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@moreal moreal force-pushed the update-3.14.2-types branch from df2a9ce to b76dde1 Compare January 22, 2026 08:52
@moreal moreal marked this pull request as ready for review January 22, 2026 10:25
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/vm/src/frame.rs (1)

1008-1032: Allow generator subclasses with __await__ to be awaited.

The current code uses downcast_ref::<PyGenerator>() which matches any generator, including subclasses, then immediately returns a TypeError if the ITERABLE_COROUTINE flag is absent. This blocks generator subclasses that implement __await__, preventing the fallback __await__ lookup.

Only generators that fail to have the ITERABLE_COROUTINE flag AND lack an __await__ method should error. The check should be restructured to only apply the generator-specific logic when the flag is actually set, allowing non-flagged generators to fall through to the __await__ fallback.

🔧 Suggested fix (preserve __await__ fallback for non-flagged generators)
-                } else if let Some(generator) = awaited_obj.downcast_ref::<PyGenerator>() {
-                    // Generator with CO_ITERABLE_COROUTINE flag can be awaited
-                    // (e.g., generators decorated with `@types.coroutine`)
-                    if generator
-                        .as_coro()
-                        .frame()
-                        .code
-                        .flags
-                        .contains(bytecode::CodeFlags::ITERABLE_COROUTINE)
-                    {
-                        awaited_obj
-                    } else {
-                        return Err(vm.new_type_error(format!(
-                            "object {} can't be used in 'await' expression",
-                            awaited_obj.class().name(),
-                        )));
-                    }
-                } else {
+                } else if awaited_obj
+                    .downcast_ref::<PyGenerator>()
+                    .is_some_and(|generator| {
+                        generator
+                            .as_coro()
+                            .frame()
+                            .code
+                            .flags
+                            .contains(bytecode::CodeFlags::ITERABLE_COROUTINE)
+                    })
+                {
+                    awaited_obj
+                } else {
                     let await_method = vm.get_method_or_type_error(
                         awaited_obj.clone(),
                         identifier!(vm, __await__),
                         || {
                             format!(
                                 "object {} can't be used in 'await' expression",
                                 awaited_obj.class().name(),
                             )
                         },
                     )?;
                     let result = await_method.call((), vm)?;
                     // Check that __await__ returned an iterator
                     if !PyIter::check(&result) {
                         return Err(vm.new_type_error(format!(
                             "__await__() returned non-iterator of type '{}'",
                             result.class().name()
                         )));
                     }
                     result
                 };

@moreal moreal marked this pull request as draft January 22, 2026 10:34
@moreal moreal marked this pull request as ready for review January 23, 2026 13:06
@youknowone youknowone merged commit 5c7f6a2 into RustPython:main Jan 23, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants