Allow extension projects to update marked dependency sections after
`create_makefile` has resolved their source and object lists.
Share the dependency scanner with `tool/mkdepend.rb` so bundled and
upstream extensions can generate the same dependency files.
Replace `tool/update-deps` with a Ruby source scanner so dependency
updates no longer require GNU make, a configured build, or compiler
preprocessor output.
Keep compact source mappings and scanner declarations in dependency
files, and expand complete rules into the build directory. Avoid the
temporary fixture headers required by the compiler-driven prototype.
Use the generated rules from GNU make, BSD make, and NMake builds.
Support out-of-tree and read-only source trees, and retain expanded
dependencies shipped in release archives when baseruby is unavailable.
And move the encoding convertion logic in another function with NOINLINE.
The overwelming majority of strings are correctly encoded, so we
want to inline the very cheap check, however we don't want to
inline the much larger piece of code required to re-encode the string.
https://github.com/ruby/json/commit/cfbe356b4f
Co-Authored-By: Jean Boussier <jean.boussier@gmail.com>
Fix: https://github.com/ruby/json/issues/929
When calling `cState_partial_generate` from `mHash_to_json` or other
`to_json` funcs, `VState` becomes unreachable very quickly, hence the
compiler may optimize it out of the stack, and make it invisible to
the GC stack scanning. This is particularly liekly given how aggressively
we inline.
Repro:
```ruby
require 'json'
test_data = {
"flag" => true,
"data" => 10000.times.map { [1.0] },
:flag => false,
}
10.times do
test_data.to_json
end
```
But in practice the cause was just that the issued warning calls
Hash#inspect on a big hash, which triggers GC.
So it can be triggered even more reliably with:
```ruby
require 'json'
module JSON
module Common
def self.on_mixed_keys_hash(...)
GC.start
end
end
end
test_data = {
"flag" => true,
"data" => 10000.times.map { [1.0] },
:flag => false,
}
test_data.to_json
```
https://github.com/ruby/json/commit/79b6e168ba
When serializing an Array, and one of the elements of the Array requires
calling `to_json`, if the depth is changed, it will be used for the next
entries, which wasn't the case before
https://github.com/ruby/json/commit/5abd43490714, and is not the case with
TruffleRuby and JRuby.
Additionally, with TruffleRuby and JRuby the state's depth after the
`to_json` call is used to close the Array, which isn't the case with
CRuby.
https://github.com/ruby/json/commit/386b36fde5
For `JSON.generate` and `JSON::State#generate_new`, don't copy
generate_json_data::depth to JSON_Generator_State::depth.
In `JSON.generate`, the JSON_Generator_State is on the stack and
discarded anyway. In `JSON::State#generate_new`, we copy the struct to
avoid mutating the original one.
https://github.com/ruby/json/commit/873b29ea34
Commit https://github.com/ruby/json/commit/44df509dc2de fixed it for StandardError, but other exceptions and
jumps are also possible. Use rb_ensure() to release FBuffer instead of
rb_rescue().
A reproducer:
o = Object.new
def o.to_json(a) = throw :a
a = ["make heap allocation"*100, o]
10.times do
100_000.times do
catch(:a) { JSON(a) }
end
puts `ps -o rss= -p #{$$}`
end
https://github.com/ruby/json/commit/9b7b648ecd
Fix: 90616277e3 (r168784389)
Because the `depth` counter is inside `JSON::State` it can't be used
concurrently, and in case of a circular reference the counter may be
left at the max value.
The depth counter should be moved outside `JSON_Generator_State` and
into `struct generate_json_data`, but it's a larger refactor.
In the meantime, `JSON::Coder` calls `State#generate_new` so I changed
that method so that it first copy the state on the stack.
https://github.com/ruby/json/commit/aefa671eca
Because both strings and symbols keys are serialized the same,
it always has been possible to generate documents with duplicated
keys:
```ruby
>> puts JSON.generate({ foo: 1, "foo" => 2 })
{"foo":1,"foo":2}
```
This is pretty much always a mistake and can cause various
issues because it's not guaranteed how various JSON parsers
will handle this.
Until now I didn't think it was possible to catch such case without
tanking performance, hence why I only made the parser more strict.
But I finally found a way to check for duplicated keys cheaply enough.
