Bug description
parse_json_markdown fails whenever the model writes anything containing a } or ]
after the JSON — a citation such as [1], or a see [docs] line after the closing
code fence.
This is a regression from #41959 (commit 67f7a55, 2026-09-08). That change switched
extraction from fence-anchored to bracket-anchored, slicing from the first {/[ to
the last }/] in the whole string. Any bracket in trailing prose is therefore
taken to be the end of the JSON, and the resulting slice does not parse.
Before #41959 all three examples below parsed correctly.
The function is used to read LLM output in
core/rag/retrieval/dataset_retrieval.py (multi-dataset routing) and
core/llm_generator/output_parser/rule_config_generator.py, so an otherwise valid
model response is rejected.
Steps to reproduce
from libs.json_in_md_parser import parse_json_markdown
parse_json_markdown('```json\n{"a": 1}\n```\nSee [docs] for more.')
parse_json_markdown('{"action": "Final Answer", "action_input": "done"}\n\nReferences: [1]')
parse_json_markdown('Result: {"a": 1}. Note [1].')Expected behavior
All three return the JSON object: {'a': 1}, {'action': 'Final Answer', 'action_input': 'done'}, {'a': 1}.
Actual behavior
All three raise json.decoder.JSONDecodeError:
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 9)
json.decoder.JSONDecodeError: Extra data: line 3 column 1 (char 52)
json.decoder.JSONDecodeError: Extra data: line 1 column 9 (char 8)
Comparing the current implementation against the one immediately before #41959:
| input |
before #41959 |
current main |
```json\n{"a": 1}\n```\nSee [docs] for more. |
{'a': 1} |
JSONDecodeError |
{"action": ...}\n\nReferences: [1] |
parsed |
JSONDecodeError |
Result: {"a": 1}. Note [1]. |
{'a': 1} |
JSONDecodeError |
```json\n{"a": 1}\n``` |
{'a': 1} |
{'a': 1} |
{"a": 1} |
{'a': 1} |
{'a': 1} |
Trailing prose is clearly meant to be tolerated already: the existing test
test_parse_and_check_json_markdown_handles_think_fenced_and_raw_variants feeds
```json\n[...]\n```, error: Expecting value: line 1 column 1 (char 0) and passes —
only because that trailing text happens to contain no bracket.
Proposed fix
Decode the first complete JSON value at the opening bracket with
json.JSONDecoder().raw_decode(json_string, start_index) instead of slicing to the last
bracket. This keeps everything #41959 set out to do — anchoring on brackets, immunity to
backticks inside JSON strings and in leading prose — and makes trailing content
irrelevant regardless of what it contains.
One deliberate behavior change comes with it. test_parse_and_check_json_markdown_multiple_blocks_fails
currently pins the greedy behavior, and its own comment describes it as a limitation
rather than a requirement:
The current implementation is greedy and will match from the first opening fence to
the last closing fence, causing JSON decode failure.
Ignoring everything after the first complete JSON value necessarily means a second
fenced block is ignored too, so that case returns the first object instead of raising.
That seems strictly more useful for reading model output, but it is a semantic change,
so please say if you would rather keep it raising and I will adjust.
Environment
main at commit ad9a5be. Python 3.12.
I have the fix and regression tests ready and will open a PR referencing this issue.
Bug description
parse_json_markdownfails whenever the model writes anything containing a}or]after the JSON — a citation such as
[1], or asee [docs]line after the closingcode fence.
This is a regression from #41959 (commit 67f7a55, 2026-09-08). That change switched
extraction from fence-anchored to bracket-anchored, slicing from the first
{/[tothe last
}/]in the whole string. Any bracket in trailing prose is thereforetaken to be the end of the JSON, and the resulting slice does not parse.
Before #41959 all three examples below parsed correctly.
The function is used to read LLM output in
core/rag/retrieval/dataset_retrieval.py(multi-dataset routing) andcore/llm_generator/output_parser/rule_config_generator.py, so an otherwise validmodel response is rejected.
Steps to reproduce
Expected behavior
All three return the JSON object:
{'a': 1},{'action': 'Final Answer', 'action_input': 'done'},{'a': 1}.Actual behavior
All three raise
json.decoder.JSONDecodeError:Comparing the current implementation against the one immediately before #41959:
main```json\n{"a": 1}\n```\nSee [docs] for more.{'a': 1}JSONDecodeError{"action": ...}\n\nReferences: [1]JSONDecodeErrorResult: {"a": 1}. Note [1].{'a': 1}JSONDecodeError```json\n{"a": 1}\n```{'a': 1}{'a': 1}{"a": 1}{'a': 1}{'a': 1}Trailing prose is clearly meant to be tolerated already: the existing test
test_parse_and_check_json_markdown_handles_think_fenced_and_raw_variantsfeeds```json\n[...]\n```, error: Expecting value: line 1 column 1 (char 0)and passes —only because that trailing text happens to contain no bracket.
Proposed fix
Decode the first complete JSON value at the opening bracket with
json.JSONDecoder().raw_decode(json_string, start_index)instead of slicing to the lastbracket. This keeps everything #41959 set out to do — anchoring on brackets, immunity to
backticks inside JSON strings and in leading prose — and makes trailing content
irrelevant regardless of what it contains.
One deliberate behavior change comes with it.
test_parse_and_check_json_markdown_multiple_blocks_failscurrently pins the greedy behavior, and its own comment describes it as a limitation
rather than a requirement:
Ignoring everything after the first complete JSON value necessarily means a second
fenced block is ignored too, so that case returns the first object instead of raising.
That seems strictly more useful for reading model output, but it is a semantic change,
so please say if you would rather keep it raising and I will adjust.
Environment
mainat commit ad9a5be. Python 3.12.I have the fix and regression tests ready and will open a PR referencing this issue.