Skip to content

Add JSON::ParserError#json_path - #1062

Open
ojuschugh1 wants to merge 3 commits into
ruby:masterfrom
ojuschugh1:parser-error-json-path
Open

Add JSON::ParserError#json_path#1062
ojuschugh1 wants to merge 3 commits into
ruby:masterfrom
ojuschugh1:parser-error-json-path

Conversation

@ojuschugh1

@ojuschugh1 ojuschugh1 commented Aug 17, 2026

Copy link
Copy Markdown

Implements the ParserError#json_path idea from #954: a JSONPath-style string locating a parse error in the document.

begin
  JSON.parse('{"user": {"roles": [1, {"admin": xyz}]}}')
rescue JSON::ParserError => e
  e.message   # => "unexpected character: 'xyz}]}}' at line 1 column 34"
  e.json_path # => "$.user.roles[1].admin"
end

JSON.parse('{"x": {"a": 1, "a": 2}}', allow_duplicate_key: false)
# => ParserError, json_path == "$.x.a"

The path is reconstructed lazily at raise time: the frame stack still holds every enclosing container at that point and their keys are still on the rvalue stack, so nothing is added to the happy path. benchmark/parser.rb before/after shows no difference outside noise.

A few choices worth reviewing:

  • For duplicate key errors the path points at the duplicated key itself ($.x.a) rather than the containing object, which seemed closer to what the reporter needs. Line/column keep pointing at the object's opening brace, so the two diverge slightly there. Happy to make it point at the object instead if you'd rather keep them consistent.
  • Keys that aren't plain identifiers use bracket notation with escaping: $["hello world"], $["a\"b"]. symbolize_names and on_load-transformed keys are handled.
  • It also works for ResumableParser, where line/column can't be accurate: an error raised mid-feed still reports the exact path.
  • cParser_parse released the spilled stacks before raising the end-of-input error. That was fine while nothing read parser state during the raise, but the path reconstruction does (the 100k-deep minefield fixture segfaulted), so the raise now happens before the release. The existing comment already documents that skipping the release on the exception path doesn't leak.
  • Not implemented for the Java parser, json_path returns nil there like line/column do. Can look at that as a follow-up if there's interest.

Closes #954

@byroot

byroot commented Aug 17, 2026

Copy link
Copy Markdown
Member

Code looks great, I'll merge it soon.

Happy to make it point at the object instead if you'd rather keep them consistent.

No I think this was the good call. I might even change it for the line/column too, as I think it can simplify the parser a little bit.

on_load-transformed keys are handled.

Hum, I need to check this a bit deeper, because in theory on_load can return about anything, so in some cases you just can't handle it. I'd be fine with just not generating the path in such case, but I'll dig a bit deeper.

It also works for ResumableParser, where line/column can't be accurate:

Yep, I like that.

Can look at that as a follow-up if there's interest.

Yes, adding position and json_path to the Java parser would be welcome in a followup.

@byroot
byroot force-pushed the parser-error-json-path branch from b04624e to fbd308c Compare August 17, 2026 07:13
@byroot
byroot force-pushed the parser-error-json-path branch from fbd308c to c9bd21b Compare August 17, 2026 07:16
@byroot

byroot commented Aug 17, 2026

Copy link
Copy Markdown
Member

, because in theory on_load can return about anything, so in some cases you just can't handle it.

I forced push your branch with an extra on_load test that show how it can be broken:

Failure: test_parse_error_json_path_on_load(JSONParserTest): [JSON::ParserError] exception expected, not #<NoMethodError: undefined method 'inspect' for an instance of BasicObject>.
test/json/json_parser_test.rb:949:in 'JSONParserTest#assert_parse_error_at'
test/json/json_parser_test.rb:864:in 'JSONParserTest#test_parse_error_json_path_on_load'
     861:   end
     862: 
     863:   def test_parse_error_json_path_on_load
  => 864:     assert_parse_error_at "$.a.b.c" do
     865:       JSON.load('{"a": {"b": {"c":', -> (obj) {
     866:         if String === obj
     867:           BasicObject.new

I think we should just not build the path it if a Hash key isn't a String/Symbol or an array index isn't an Integer.
Alternatively we could just not build it at all when on_load is passed, but that might be a bit too extreme.

@ojuschugh1

Copy link
Copy Markdown
Author

Went with your first suggestion: the path stops at the first key that isn't a String or Symbol, so the valid prefix is kept ($.a if only "b" was replaced by the proc, $ if all keys were). That also removes the inspect fallback, so building the path can't call into user code at all anymore. Extended your test with the partial case.

Comment thread ext/json/ext/parser/parser.c Outdated
Comment on lines +670 to +674
if (plain) {
rb_str_cat_cstr(path, ".");
rb_str_cat(path, RSTRING_PTR(key), len);
return true;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had the realization that it would probably be simpler to just push the raw objects in an array, and build the json_path string in Ruby. That would also help share the code with the eventual JRuby version.

Comment thread ext/json/ext/parser/parser.c
This isn't a performance sensitive path, and that logic can be shared
with the eventual JRuby implementation.
@byroot
byroot force-pushed the parser-error-json-path branch from d77424e to 8550be8 Compare August 18, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Subtle breaking change with custom object_class and duplicate keys

2 participants