dox4/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2026-08-17 01:13:32 +08:00
Code Issues Packages Projects Releases Wiki Activity
90,216 Commits 34 Branches 1,193 Tags
Commit Graph

189 Commits

Author SHA1 Message Date
Étienne Barrié
b4bfbcaddc
Optimize Symbol generation in strict mode 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2025-02-06 16:02:03 +09:00
Étienne Barrié
f865148e19
Fix JSON::Coder to call as_json proc for NaN and Infinity 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2025-02-06 16:02:03 +09:00
Jean Boussier
98e1c2845a [ruby/json] Refactor convert_UTF8_to_JSON to split searching and escaping code 
The goal is to be able to dispatch to more optimized search implementations
without having to duplicate the escaping code.

Somehow, this is a few % faster already:

```
== Encoding activitypub.json (52595 bytes)
ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after     2.257k i/100ms
Calculating -------------------------------------
               after     22.930k (± 1.3%) i/s   (43.61 μs/i) -    115.107k in   5.020814s

Comparison:
              before:    21604.0 i/s
               after:    22930.1 i/s - 1.06x  faster

== Encoding citm_catalog.json (500298 bytes)
ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after   137.000 i/100ms
Calculating -------------------------------------
               after      1.397k (± 1.1%) i/s  (715.57 μs/i) -      6.987k in   5.000408s

Comparison:
              before:     1344.4 i/s
               after:     1397.5 i/s - 1.04x  faster

== Encoding twitter.json (466906 bytes)
ruby 3.4.1 (2024-12-25 revision https://github.com/ruby/json/commit/48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after   249.000 i/100ms
Calculating -------------------------------------
               after      2.464k (± 1.8%) i/s  (405.81 μs/i) -     12.450k in   5.054131s

Comparison:
              before:     2326.5 i/s
               after:     2464.2 i/s - 1.06x  faster
```

https://github.com/ruby/json/commit/8fb5ae807f
 2025-02-03 10:05:25 +09:00
Jean Boussier
50e34fd768
Allow JSON::Fragment to be used even in strict mode 2025-01-28 15:42:20 +09:00
Étienne Barrié
89e316ad06
Introduce JSON::Coder 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2025-01-28 15:41:47 +09:00
tompng
86b262179d [ruby/json] Reject invalid number: - -.1 -e0 
https://github.com/ruby/json/commit/b9bfeecfa9
 2025-01-20 14:20:55 +01:00
tompng
525d7a68e4 [ruby/json] Raise parse error on invalid comments 
https://github.com/ruby/json/commit/2f57f40467
 2025-01-20 14:20:55 +01:00
tompng
c026e44bb5 [ruby/json] Fix parsing incomplete unicode escape "\uaaa" 
https://github.com/ruby/json/commit/86c0d4eb7e
 2025-01-20 14:20:55 +01:00
Étienne Barrié
e8676cada8 [ruby/json] Introduce JSON::Fragment 
https://github.com/ruby/json/commit/9e3500f345

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2025-01-20 14:20:55 +01:00
Jean Boussier
33708f2dc4 [ruby/json] Fix a regression in the parser with leading / 
Ref: https://github.com/ruby/ruby/pull/12598

This could lead to an infinite loop.

https://github.com/ruby/json/commit/f8cfa2696a
 2025-01-20 10:31:56 +01:00
Jean Boussier
ef585744c0 Finalize Kevin's handrolled parser. 
And get rid of the Ragel parser.

This is 7% faster on activitypub, 15% after on twitter and 11% faster
on citm_catalog.

There might be some more optimization opportunities, I did a quick
optimization pass to fix a regression in string parsing, but other
than that I haven't dug much in performance.
 2025-01-20 16:09:00 +09:00
Jean Boussier
91918bb01f [ruby/json] Refactor JSONFixturesTest 
https://github.com/ruby/json/commit/c69963fbe5
 2025-01-16 09:01:11 +09:00
Jean Boussier
599fbeaffa [ruby/json] Refactor JSON::Ext::Parser to split configuration and parsing state 
Ref: https://github.com/ruby/json/pull/718

The existing `Parser` interface is pretty bad, as it forces to
instantiate a new instance for each document.

Instead it's preferable to only take the config and do all the
initialization needed, and then keep the parsing state on the
stack on in ephemeral memory.

This refactor makes the `JSON::Coder` pull request much easier to
implement in a performant way.

https://github.com/ruby/json/commit/c8d5236a92

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
 2025-01-14 09:08:02 +01:00
Hiroshi SHIBATA
9f5ca6332b Refactor to omit JSON::GenericObject tests 2025-01-08 17:12:19 +09:00
Jean Boussier
1510d72bec [ruby/json] Fix generate(script_safe: true) to not confuse unrelated characters 
Fix: https://github.com/ruby/json/issues/715

The first byte check was missing.

https://github.com/ruby/json/commit/93a7f8717d
 2024-12-05 09:16:22 +01:00
Jean Boussier
693a793521 JSON::GeneratorError expose invalid object 
Fix: https://github.com/ruby/json/issues/710

Makes it easier to debug why a given tree of objects can't
be dumped as JSON.

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
 2024-11-26 15:11:05 +09:00
Jean Boussier
ee0de3fd4e [ruby/json] JSON.dump: write directly into the provided IO 
Ref: https://github.com/ruby/json/issues/524

Rather than to buffer everything in memory.

Unfortunately Ruby doesn't provide an API to write into
and IO without first allocating a string, which is a bit
wasteful.

https://github.com/ruby/json/commit/f017af6c0a
 2024-11-26 15:11:05 +09:00
Peter Zhu
64827b15e8 [ruby/json] Fix $VERBOSE restore 
Manually sync this commit that was missed.

https://github.com/ruby/json/commit/6ae9155
 2024-11-11 15:13:24 -05:00
Jean Boussier
1b6a26e23a [ruby/json] JSON.load_file: explictly load the file as UTF-8 
Fix: https://github.com/ruby/json/issues/697

This way even if `Encoding.default_external` is set to a weird value
the document will be parsed just fine.

https://github.com/ruby/json/commit/3a8505a8fa
 2024-11-11 09:40:11 +09:00
Nobuyoshi Nakada
8254f6492c [ruby/json] Categorize deprecated warning 
https://github.com/ruby/json/commit/1acce7aceb
 2024-11-06 23:31:30 +01:00
Jean Boussier
4120f2babd [ruby/json] Appease ruby/ruby CI 
https://github.com/ruby/json/commit/bc2c970ce4
 2024-11-05 18:00:36 +01:00
Jean Boussier
633f65e471 [ruby/json] Strip whitespaces 
https://github.com/ruby/json/commit/e85107197b
 2024-11-05 18:00:36 +01:00
Jean Boussier
ca8f21ace8 [ruby/json] Resync 2024-11-05 18:00:36 +01:00
Benoit Daloze
e8522f06b5 [ruby/json] Re-enable passing test 
https://github.com/ruby/json/commit/82d21f01c5
 2024-11-05 18:00:36 +01:00
Jean Boussier
f664e7eaab [ruby/json] Add tests for the behavior of JSON.generate with base types subclasses 
Ref: https://github.com/ruby/json/pull/674
Ref: https://github.com/ruby/json/pull/668

The behavior on such case it quite unclear, the goal here is to
figure out whatever was the behavior on Cext version of `json 2.7.0`
and get all implementations to converge.

We can then decide to make them all behave differently if we so wish.

https://github.com/ruby/json/commit/614921dcef
 2024-11-05 18:00:36 +01:00
Jean Boussier
ed22e68379 [ruby/json] JSON::Ext::Parser mark the name cache entries when not on the heap 
This is somewhat dead code as unless you are using `JSON::Parser.new`
direcltly we never allocate `JSON::Ext::Parser` anymore.

But still, we should mark all its reference in case some code out there
uses that.

Followup: #675

https://github.com/ruby/json/commit/8bf74a977b
 2024-11-05 18:00:36 +01:00
Jean Boussier
ef5565f5d1 JSON.generate: call to_json on String subclasses 
Fix: https://github.com/ruby/json/issues/667

This is yet another behavior on which the various implementations
differed, but the C implementation used to call `to_json` on String
subclasses used as keys.

This was optimized out in e125072130229e54a651f7b11d7d5a782ae7fb65
but there is an Active Support test case for it, so it's best to
make all 3 implementation respect this behavior.
 2024-11-01 13:04:24 +09:00
Jean Boussier
3782600f0f [ruby/json] Emit warnings when dumping binary strings 
Because of it's Ruby 1.8 heritage, the C extension doesn't care
much about strings encoding. We should get stricter over time.

https://github.com/ruby/json/commit/42402fc13f
 2024-11-01 13:04:24 +09:00
Jean Boussier
f2b8829df0 Deprecate unsafe default options of JSON.load 
[Feature #19528]

Ref: https://bugs.ruby-lang.org/issues/19528

`load` is understood as the default method for serializer kind of libraries, and
the default options of `JSON.load` has caused many security vulnerabilities over the
years.

The plan is to do like YAML/Psych, deprecate these default options and direct
users toward using `JSON.unsafe_load` so at least it's obvious it should be
used against untrusted data.
 2024-11-01 13:04:24 +09:00
Jean Boussier
cc2e67a138 Elide Generator::State allocation until a to_json method has to be called 
Fix: https://github.com/ruby/json/issues/655

For very small documents, the biggest performance gap with alternatives is
that the API impose that we allocate the `State` object. In a real world app
this doesn't make much of a difference, but when running in a micro-benchmark
this doubles the allocations, causing twice the amount of GC runs, making us
look bad.

However, unless we have to call a `to_json` method, the `State` object isn't
visible, so with some refactoring, we can elude that allocation entirely.

Instead we allocate the State internal struct on the stack, and if we need
to call a `to_json` method, we allocate the `State` and spill the struct on
the heap.

As a result, `JSON.generate` is now as fast as re-using a `State` instance,
as long as only primitives are generated.

Before:
```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   598.654k i/100ms
                json   400.542k i/100ms
                  oj   533.353k i/100ms
Calculating -------------------------------------
        json (reuse)      6.371M (± 8.6%) i/s  (156.96 ns/i) -     31.729M in   5.059195s
                json      4.120M (± 6.6%) i/s  (242.72 ns/i) -     20.828M in   5.090549s
                  oj      5.622M (± 6.4%) i/s  (177.86 ns/i) -     28.268M in   5.061473s

Comparison:
        json (reuse):  6371126.6 i/s
                  oj:  5622452.0 i/s - same-ish: difference falls within error
                json:  4119991.1 i/s - 1.55x  slower

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   248.125k i/100ms
                json   215.255k i/100ms
                  oj   217.531k i/100ms
Calculating -------------------------------------
        json (reuse)      2.628M (± 6.1%) i/s  (380.55 ns/i) -     13.151M in   5.030281s
                json      2.185M (± 6.7%) i/s  (457.74 ns/i) -     10.978M in   5.057655s
                  oj      2.217M (± 6.7%) i/s  (451.10 ns/i) -     11.094M in   5.044844s

Comparison:
        json (reuse):  2627799.4 i/s
                  oj:  2216824.8 i/s - 1.19x  slower
                json:  2184669.5 i/s - 1.20x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   641.334k i/100ms
                json   322.745k i/100ms
                  oj   642.450k i/100ms
Calculating -------------------------------------
        json (reuse)      7.133M (± 6.5%) i/s  (140.19 ns/i) -     35.915M in   5.068201s
                json      4.615M (± 7.0%) i/s  (216.70 ns/i) -     22.915M in   5.003718s
                  oj      6.912M (± 6.4%) i/s  (144.68 ns/i) -     34.692M in   5.047690s

Comparison:
        json (reuse):  7133123.3 i/s
                  oj:  6911977.1 i/s - same-ish: difference falls within error
                json:  4614696.6 i/s - 1.55x  slower
```

After:

```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   572.751k i/100ms
                json   457.741k i/100ms
                  oj   512.247k i/100ms
Calculating -------------------------------------
        json (reuse)      6.324M (± 6.9%) i/s  (158.12 ns/i) -     31.501M in   5.023093s
                json      6.263M (± 6.9%) i/s  (159.66 ns/i) -     31.126M in   5.017086s
                  oj      5.569M (± 6.6%) i/s  (179.56 ns/i) -     27.661M in   5.003739s

Comparison:
        json (reuse):  6324183.5 i/s
                json:  6263204.9 i/s - same-ish: difference falls within error
                  oj:  5569049.2 i/s - same-ish: difference falls within error

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   258.505k i/100ms
                json   242.335k i/100ms
                  oj   220.678k i/100ms
Calculating -------------------------------------
        json (reuse)      2.589M (± 9.6%) i/s  (386.17 ns/i) -     12.925M in   5.071853s
                json      2.594M (± 6.6%) i/s  (385.46 ns/i) -     13.086M in   5.083035s
                  oj      2.250M (± 2.3%) i/s  (444.43 ns/i) -     11.255M in   5.004707s

Comparison:
        json (reuse):  2589499.6 i/s
                json:  2594321.0 i/s - same-ish: difference falls within error
                  oj:  2250064.0 i/s - 1.15x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   656.373k i/100ms
                json   644.135k i/100ms
                  oj   650.283k i/100ms
Calculating -------------------------------------
        json (reuse)      7.202M (± 7.1%) i/s  (138.84 ns/i) -     36.101M in   5.051438s
                json      7.278M (± 1.7%) i/s  (137.40 ns/i) -     36.716M in   5.046300s
                  oj      7.036M (± 1.7%) i/s  (142.12 ns/i) -     35.766M in   5.084729s

Comparison:
        json (reuse):  7202447.9 i/s
                json:  7277883.0 i/s - same-ish: difference falls within error
                  oj:  7036115.2 i/s - same-ish: difference falls within error

```
 2024-11-01 13:04:24 +09:00
Benoit Daloze
88b411464d [ruby/json] Skip test failing on JRuby 
https://github.com/ruby/json/commit/0f0b16b3f5
 2024-11-01 13:04:24 +09:00
Benoit Daloze
eb19156a28 [ruby/json] Add test for parsing broken strings 
https://github.com/ruby/json/commit/850bd077c4
 2024-11-01 13:04:24 +09:00
Jean Boussier
ebfa178b72 [ruby/json] Setup ruby_memcheck 
Hoping it might find the leak reported in https://github.com/ruby/json/issues/460

https://github.com/ruby/json/commit/08635312e5
 2024-11-01 13:04:24 +09:00
Jean Boussier
b094ee3f23
Handle all formatting configs potentially being nil. 
Fix: https://github.com/ruby/json/issues/653

I don't think this was really fully supported in the past, but
it kinda worked with some of the implementations.
 2024-10-29 13:25:01 +09:00
Jean Boussier
a5bd0c638a [ruby/json] Workaround rubygems $LOAD_PATH bug 
Ref: https://github.com/ruby/json/issues/647
Ref: https://github.com/rubygems/rubygems/pull/6490

Older rubygems are executing `extconf.rb` with a broken `$LOAD_PATH`
causing the `json` gem native extension to be loaded with the stdlib
version of the `.rb` files.

This fails with

```
json/common.rb:82:in `initialize': wrong number of arguments (given 1, expected 0) (ArgumentError)
```

Since this is just for `extconf.rb` we can probably just accept that
extra argument and ignore it.

The bug was fixed in rubygems 3.4.9 / 2023-03-20

https://github.com/ruby/json/commit/1f5e849fe0
 2024-10-26 18:44:15 +09:00
Jean Boussier
3daf16e51f [ruby/json] Cleanup test_helper.rb 
https://github.com/ruby/json/commit/49de571dd8
 2024-10-26 18:44:15 +09:00
Jean Boussier
7314275548 json_pure: fix ractor compatibility 
This actually never worked, because the test was always testing
the ext version from the stdlib, never the pure version nor the
current ext version.
 2024-10-26 18:44:15 +09:00
Jean Boussier
b1d417dc7b [ruby/json] Cleaner .encode / .force_encoding 
https://github.com/ruby/json/commit/cecf04fdfc
 2024-10-26 18:44:15 +09:00
Jean Boussier
1045b9f820 [ruby/json] Modernize heredocs 
https://github.com/ruby/json/commit/fb25e94aea
 2024-10-26 18:44:15 +09:00
Jean Boussier
bfdf02ea72 pretty_generate: don't apply object_nl / array_nl for empty containers 
Fix: https://github.com/ruby/json/issues/437

Before:

```json
{
  "foo": {
  },
  "bar": [
  ]
}
```

After:

```json
{
  "foo": {},
  "bar": []
}
```
 2024-10-26 18:44:15 +09:00
Jean Boussier
fc9f0cb8c5 [ruby/json] JSON.dump / String#to_json: raise on invalid encoding 
This regressed since 2.7.2.

https://github.com/ruby/json/commit/35407d6635
 2024-10-26 18:44:15 +09:00
Benoit Daloze
2c6e3bc71e Raise the correct exception in fast_serialize_string 
* Related to https://github.com/ruby/json/issues/344
 2024-10-26 18:44:15 +09:00
Jean Boussier
70f554efb4 [ruby/json] raise_parse_error: avoid UB 
Fix: https://github.com/ruby/json/pull/625

Declaring the buffer in a sub block cause bugs on some compilers.

https://github.com/ruby/json/commit/90967c9eb0
 2024-10-26 18:44:15 +09:00
Étienne Barrié
44aef5e852 [ruby/json] Drop compatibility for missing Array#permutation (Ruby <= 1.8.6) 
https://github.com/ruby/json/commit/b02091ed44

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Étienne Barrié
82f7550f65 Use frozen string literals 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Étienne Barrié
11348c583f Use Encoding constants, String#b 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Jean Boussier
18cc663aef [ruby/json] Add test coverage for JSON.load with a Proc 
Fix: https://github.com/ruby/json/issues/438

https://github.com/ruby/json/commit/9dd89eaac8
 2024-10-26 18:44:15 +09:00
Jean Boussier
9045258c88 [ruby/json] Limit the size of ParserError exception messages 
Fix: https://github.com/ruby/json/issues/534

Only include up to 32 bytes of unparseable the source.

https://github.com/ruby/json/commit/f44995cfb6
 2024-10-26 18:44:15 +09:00
Stephen Humphries
326a21d441
Relax Pure::Parser's comment regex... 
...to allow any character sequence, including "/*", before then end
sequence of a multi-line ANSI C-style comment
.
 2024-10-18 11:31:42 +09:00
Jean Boussier
8feed977a0 [ruby/json] Assume Encoding is defined 
https://github.com/ruby/json/commit/8713aa4812
 2024-10-18 11:30:55 +09:00