Skip to content

Add shorthand aliases for json and pickle serializers#2401

Open
selwin wants to merge 10 commits into
masterfrom
serializer-shortcuts
Open

Add shorthand aliases for json and pickle serializers#2401
selwin wants to merge 10 commits into
masterfrom
serializer-shortcuts

Conversation

@selwin
Copy link
Copy Markdown
Collaborator

@selwin selwin commented May 10, 2026

Summary by CodeRabbit

Release Notes

  • New Features

    • Serializer configuration now supports shorthand strings ('json', 'pickle') for queues, workers, and CLI commands, simplifying setup from full class paths.
  • Documentation

    • Updated guides to demonstrate simplified serializer configuration using shorthand syntax.
  • Tests

    • Added test coverage for serializer alias resolution.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 10, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds string aliases for serializer configuration in RQ. Users can now pass serializer='json' or serializer='pickle' to Queue and Worker constructors, and use --serializer json on the CLI instead of requiring full import paths. The change introduces a PickleSerializer alias and SERIALIZER_ALIASES mapping in the core resolver, updates comprehensive documentation and tests to demonstrate shorthand usage, and includes end-to-end CLI test coverage.

Changes

Serializer Shorthand Aliases

Layer / File(s) Summary
Core serializer resolver
rq/serializers.py
Introduce PickleSerializer alias for DefaultSerializer, add SERIALIZER_ALIASES mapping for 'json' and 'pickle', and update resolve_serializer() to return PickleSerializer on falsy input and resolve string inputs via aliases before falling back to import_attribute.
Unit tests for aliases
tests/test_serializers.py, tests/test_queue.py, tests/test_worker.py
Update serializer unit tests to assert PickleSerializer as default and test alias resolution; add Queue and Worker unit tests asserting serializer aliases resolve to correct implementations.
CLI integration tests
tests/test_cli.py
Add end-to-end CLI tests verifying rq worker and rq worker-pool accept --serializer json option for jobs enqueued with JSONSerializer.
User documentation
docs/docs/jobs.md, docs/docs/workers.md
Update jobs and workers documentation to show Queue and Worker configuration using serializer='json'/'pickle' shorthands and --serializer json CLI option; document PickleSerializer as default for worker-pool.
Minor docstring update
rq/queue.py
Update QueueArgs docstring formatting.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

A rabbit hops through queues so bright,
With JSON shorthands, oh what delight!
No imports long, just 'json' will do,
The serializers dance, both old and new! 🐰✨
Aliases simple, the defaults ring true!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add shorthand aliases for json and pickle serializers' directly and clearly summarizes the main change: introducing shorthand string aliases ('json', 'pickle') for serializer configuration throughout the codebase.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch serializer-shortcuts

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/docs/jobs.md`:
- Line 271: Remove the leading shell prompt from the command example: replace
the snippet "$ rq worker --serializer json" with a plain command line "rq worker
--serializer json" so it no longer triggers markdownlint MD014; update the code
block containing that exact string accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e88d7c1e-2b37-4f1a-a1d8-ce17bed2348e

📥 Commits

Reviewing files that changed from the base of the PR and between 6b3955c and 94a0b40.

📒 Files selected for processing (9)
  • docs/docs/jobs.md
  • docs/docs/workers.md
  • rq/cli/helpers.py
  • rq/cli/workers.py
  • rq/serializers.py
  • tests/test_cli.py
  • tests/test_queue.py
  • tests/test_serializers.py
  • tests/test_worker.py

Comment thread docs/docs/jobs.md

```console
$ rq worker --serializer rq.serializers.JSONSerializer
$ rq worker --serializer json
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix markdownlint MD014 in the console command example.

Line 271 includes a $ prompt, which triggers the configured lint rule for command snippets without output.

📎 Suggested doc-only fix
- $ rq worker --serializer json
+ rq worker --serializer json
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
$ rq worker --serializer json
rq worker --serializer json
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 271-271: Dollar signs used before commands without showing output

(MD014, commands-show-output)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/docs/jobs.md` at line 271, Remove the leading shell prompt from the
command example: replace the snippet "$ rq worker --serializer json" with a
plain command line "rq worker --serializer json" so it no longer triggers
markdownlint MD014; update the code block containing that exact string
accordingly.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/docs/workers.md`:
- Around line 251-253: Remove the leading shell prompt character from the
example command in the Markdown fenced code block so it shows the raw command;
specifically update the console block that contains "rq worker --serializer
json" (currently prefixed with "$") to remove the "$" prefix and leave just the
command "rq worker --serializer json".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b5f56905-6944-49d2-a48a-57505816f91d

📥 Commits

Reviewing files that changed from the base of the PR and between 94a0b40 and 0796a12.

📒 Files selected for processing (7)
  • docs/docs/jobs.md
  • docs/docs/workers.md
  • rq/cli/helpers.py
  • rq/defaults.py
  • rq/serializers.py
  • rq/worker_pool.py
  • tests/test_serializers.py
✅ Files skipped from review due to trivial changes (1)
  • rq/cli/helpers.py

Comment thread docs/docs/workers.md
Comment on lines 251 to 253
```console
$ rq worker --serializer rq.serializers.JSONSerializer
$ rq worker --serializer json
```
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove the shell prompt in this command example (markdownlint MD014).

Line 252 includes a leading $ for a command without output; that triggers MD014.

📎 Suggested fix
 ```console
-$ rq worker --serializer json
+rq worker --serializer json
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion

🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 252-252: Dollar signs used before commands without showing output

(MD014, commands-show-output)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/docs/workers.md` around lines 251 - 253, Remove the leading shell prompt
character from the example command in the Markdown fenced code block so it shows
the raw command; specifically update the console block that contains "rq worker
--serializer json" (currently prefixed with "$") to remove the "$" prefix and
leave just the command "rq worker --serializer json".

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
rq/queue.py (1)

920-983: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Queue.parse_args() return shape change is a public API compatibility risk.

On Line [963], switching to a bare tuple removes named-field access for callers of this public method. Any integration doing parsed.job_id/parsed.timeout will break at runtime.

Please keep a named return type (QueueArgs/NamedTuple) or make this method private before changing its contract.

Proposed compatibility-safe adjustment
-        return (
+        return QueueArgs(
             f,
             timeout,
             description,
             result_ttl,
             ttl,
             failure_ttl,
             depends_on,
             job_id,
             at_front,
             meta,
             retry,
             repeat,
             on_success,
             on_failure,
             on_stopped,
             pipeline,
             unique,
             args,
             kwargs,
         )
# Outside this segment (module scope), reintroduce the named return type.
# from typing import NamedTuple
#
# class QueueArgs(NamedTuple):
#     f: FunctionReferenceType
#     timeout: int | None
#     description: str | None
#     result_ttl: int | None
#     ttl: int | None
#     failure_ttl: int | None
#     depends_on: JobDependencyType | None
#     job_id: str | None
#     at_front: bool
#     meta: dict | None
#     retry: Retry | None
#     repeat: Repeat | None
#     on_success: Callback | Callable[..., Any] | None
#     on_failure: Callback | Callable[..., Any] | None
#     on_stopped: Callback | Callable[..., Any] | None
#     pipeline: Pipeline | None
#     unique: bool
#     args: tuple | list | None
#     kwargs: dict | None
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rq/queue.py` around lines 920 - 983, The change to Queue.parse_args removed
the named return type, breaking public API consumers that expect attribute
access (e.g., parsed.job_id); restore a named return by reintroducing a
QueueArgs NamedTuple (or dataclass) at module scope with fields matching the
returned values (f, timeout, description, result_ttl, ttl, failure_ttl,
depends_on, job_id, at_front, meta, retry, repeat, on_success, on_failure,
on_stopped, pipeline, unique, args, kwargs) and change Queue.parse_args to
return QueueArgs(...) instead of a bare tuple; alternatively, if you intend to
break the public contract, mark parse_args private, but prefer the QueueArgs
approach to preserve compatibility.
🧹 Nitpick comments (1)
rq/worker_registration.py (1)

73-78: ⚡ Quick win

Prefer explicit None narrowing over # type: ignore in get_keys().

On Line 77, # type: ignore suppresses a useful type-safety check. Switching to explicit is not None branching keeps behavior the same and avoids masking future None regressions.

♻️ Proposed refactor
-    if queue:
+    if queue is not None:
         redis = queue.connection
         redis_key = WORKERS_BY_QUEUE_KEY % queue.name
     else:
-        redis = connection  # type: ignore
+        assert connection is not None
+        redis = connection
         redis_key = REDIS_WORKER_KEYS
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rq/worker_registration.py` around lines 73 - 78, Replace the "# type: ignore"
by explicitly narrowing None for the fallback connection in get_keys(): instead
of "redis = connection  # type: ignore", branch on "if connection is not None"
and assign redis = connection there, otherwise handle the missing connection
case (raise a clear exception or return an empty result) so nullability is
checked explicitly; keep the existing queue branch that sets redis_key =
WORKERS_BY_QUEUE_KEY % queue.name and the fallback redis_key =
REDIS_WORKER_KEYS, and reference the variables queue, connection,
WORKERS_BY_QUEUE_KEY and REDIS_WORKER_KEYS to locate the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rq/worker/base.py`:
- Around line 1192-1193: The code checks results[7] after building a pipeline
which is brittle; instead capture the pipeline index for the job heartbeat
before adding it and use that dynamic index to inspect the job heartbeat result
and decide whether to call self.connection.delete(job.key). Concretely: record
the pipeline length (e.g., prior_len = len(pipeline.commands) or the return
position) immediately before calling job.heartbeat()/pipeline.hset for the job,
then after executing the pipeline read results[prior_len] (or the stored index)
rather than results[7]; update the references around heartbeat(),
execution.heartbeat(), job.heartbeat(), results and the
self.connection.delete(job.key) call and add a short comment explaining the
indexing approach.

In `@rq/worker/worker_classes.py`:
- Around line 191-198: The spawned "-c" Python source currently embeds values
via raw string interpolation (self.key, job.id, queue.name) which can break on
quotes/newlines and risk injection; update the code that builds the "-c" string
to escape these values using a Python literal-safe representation (e.g. use
repr() or json.dumps on self.key, job.id, queue.name) before inserting them, and
replace the direct interpolations used with Worker.find_by_key, Job.fetch, and
Queue(...) so the child process receives safely quoted literals.

---

Outside diff comments:
In `@rq/queue.py`:
- Around line 920-983: The change to Queue.parse_args removed the named return
type, breaking public API consumers that expect attribute access (e.g.,
parsed.job_id); restore a named return by reintroducing a QueueArgs NamedTuple
(or dataclass) at module scope with fields matching the returned values (f,
timeout, description, result_ttl, ttl, failure_ttl, depends_on, job_id,
at_front, meta, retry, repeat, on_success, on_failure, on_stopped, pipeline,
unique, args, kwargs) and change Queue.parse_args to return QueueArgs(...)
instead of a bare tuple; alternatively, if you intend to break the public
contract, mark parse_args private, but prefer the QueueArgs approach to preserve
compatibility.

---

Nitpick comments:
In `@rq/worker_registration.py`:
- Around line 73-78: Replace the "# type: ignore" by explicitly narrowing None
for the fallback connection in get_keys(): instead of "redis = connection  #
type: ignore", branch on "if connection is not None" and assign redis =
connection there, otherwise handle the missing connection case (raise a clear
exception or return an empty result) so nullability is checked explicitly; keep
the existing queue branch that sets redis_key = WORKERS_BY_QUEUE_KEY %
queue.name and the fallback redis_key = REDIS_WORKER_KEYS, and reference the
variables queue, connection, WORKERS_BY_QUEUE_KEY and REDIS_WORKER_KEYS to
locate the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c50dd2af-dbd8-4042-877c-306c3b83a8f6

📥 Commits

Reviewing files that changed from the base of the PR and between 0796a12 and e826cc0.

📒 Files selected for processing (7)
  • rq/cron.py
  • rq/queue.py
  • rq/scripts.py
  • rq/serializers.py
  • rq/worker/base.py
  • rq/worker/worker_classes.py
  • rq/worker_registration.py
✅ Files skipped from review due to trivial changes (1)
  • rq/scripts.py

Comment thread rq/worker/base.py Outdated
Comment thread rq/worker/worker_classes.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rq/serializers.py`:
- Around line 61-64: Replace the runtime assert with an explicit type check:
instead of "assert not isinstance(serializer, str)" perform an if
isinstance(serializer, str): raise TypeError(...) to validate the public
argument named serializer before the existing isinstance(serializer, Serializer)
check; keep the subsequent NotImplementedError for non-Serializer objects but
use the explicit check to ensure user-provided string serializers produce a
clear, stable error rather than an AssertionError that can be stripped by -O.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 522a308f-34d5-4971-999e-287d887ee381

📥 Commits

Reviewing files that changed from the base of the PR and between e826cc0 and 6cad293.

📒 Files selected for processing (1)
  • rq/serializers.py

Comment thread rq/serializers.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant