dox4/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2026-08-08 15:08:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
100,908 Commits 31 Branches 1,193 Tags
Commit Graph

329 Commits

Author SHA1 Message Date
Nobuyoshi Nakada
e0fc139e1b
[Misc #22206] Derive generated dependencies from Makefile targets 2026-07-23 00:54:12 +09:00
Nobuyoshi Nakada
bfb05e41be
[Misc #22206] Unify dependency output mappings 
Use depends declarations for emitted dependency names instead of
overloading generated declarations or rewriting thread headers in
code.  This keeps scanning, generation, and dependency expansion
separate and makes platform-selected thread dependencies explicit.
 2026-07-23 00:54:12 +09:00
Nobuyoshi Nakada
b92606aebf
[Misc #22206] Declare conditional dependency scan rules 
Move fixed preprocessor macro handling into dependency declarations so
the scan inputs and emitted dependencies are visible in depend files.
Add source-specific define and undef declarations for `parse.y` and
`ripper.y`.
 2026-07-23 00:54:12 +09:00
Nobuyoshi Nakada
7d4461e162
[Misc #22206] Generate dependencies without compiler builds 
Replace `tool/update-deps` with a Ruby source scanner so dependency
updates no longer require GNU make, a configured build, or compiler
preprocessor output.

Keep compact source mappings and scanner declarations in dependency
files, and expand complete rules into the build directory.  Avoid the
temporary fixture headers required by the compiler-driven prototype.

Use the generated rules from GNU make, BSD make, and NMake builds.
Support out-of-tree and read-only source trees, and retain expanded
dependencies shipped in release archives when baseruby is unavailable.
 2026-07-23 00:04:47 +09:00
Nobuyoshi Nakada
fda83cd2d6
Promote Ripper DSL array elements to VALUE 
Ensure arguments passed through `rb_ary_new_from_args` have type
`VALUE`, including when the statement-expression optimization is
unavailable.  This avoids undefined behavior in variadic calls.
 2026-07-21 15:57:41 +09:00
Jean Boussier
0fd945f3de Stop including onigmo.h in rregexp.h 
Followup: https://github.com/ruby/ruby/pull/17671

[Bug #22184]

`onigmo.h` contains a lot of declarations that can easily
conflict with third party code, hence it's better to keep
it opaque. That however requires a few hacks.
 2026-07-07 09:21:13 +02:00
Jean Boussier
b58a6024a0 Make T_REGEXP embedded 
When embeddeding `struct re_pattern_buffer`, `RRegexp` end up
488B large, which result in a rather minimal 24B waste in 512B slots.

This however requires a few changes:
  - Stop relying on `RREGEXP_PTR == NULL` to check for initialization.
  - `reg_copy` requires a new `onig_reg_copy_body` API.
 2026-07-06 11:56:40 +02:00
John Hawthorn
3d861274e6 Introduce RMATCH_{BEG,END,NREGS} helpers 
These are internal-only helpers which can be used instead of the
RMATCH_REGS struct directly. RMATCH_REGS is just a pointer offset from
the RMatch VALUE itself, so this should not significantly affect
codegen.

The motivation for this is that it's both simpler, and should move us
towards being able to replace the storage for RMATCH, and to be able to
store the positions embedded instead of in separate malloc memory.
 2026-05-08 13:41:12 -07:00
Ernie Miller
9aca729140 Improve comparison error message. 
In certain cases, things like Array#sort can result in a confusing error
message. For instance where a and b are characters in a string,
`"string"`:

```ruby
array.sort { |a, b| string.index(a) <=> string.index(b) }
```

If one of the index calls returns nil, we will get "comparison of String
with String failed", which is somewhat unhelpful, since it's easy to be
confused, given that what is really being compared is a Fixnum or
NilClass (the cause of the error). Yes, as far as Array#sort is
concerned, the two characters are the things being sorted, but it's
useful to call attention to the return value of the comparison in this
case.

This patch adds a "reason" argument to rb_cmperr, which will provide an
error message of "comparison of String with String failed: comparator
returned nil" in the case above, or, in the case of:

```ruby
1.upto('10').to_a
```

it will provide the message: "comparison of Fixnum with String failed:
coercion was not possible"
 2026-03-06 14:02:00 +09:00
Jean Boussier
6cd4549060 Optimize File.join common use case 
`File.join` is a hotspot for common libraries such as Zeitwerk
and Bootsnap. It has a fairly flexible signature, but 99% of
the time it's called with just two (or a small number of) UTF-8 strings.

If we optimistically optimize for that use case we can cut down a large
number of type and encoding checks, significantly speeding up the method.

The one remaining expensive check we could try to optimize is `str_null_check`.
Given it's common to use the same base string for joining, we could memoize it.
Also we could precompute it for literal strings.

```
compare-ruby: ruby 4.1.0dev (2026-01-17T14:40:03Z master 00a3b71eaf) +PRISM [arm64-darwin25]
built-ruby: ruby 4.1.0dev (2026-01-18T12:10:38Z spedup-file-join 069bab58d4) +PRISM [arm64-darwin25]
warming up....

|              |compare-ruby|built-ruby|
|:-------------|-----------:|---------:|
|two_strings   |      2.475M|    9.444M|
|              |           -|     3.82x|
|many_strings  |    551.975k|    2.346M|
|              |           -|     4.25x|
|array         |    514.946k|  522.034k|
|              |           -|     1.01x|
|mixed         |    621.236k|  633.189k|
|              |           -|     1.02x|
```
 2026-01-18 16:31:31 +01:00
Peter Zhu
7379b9ed78 Optimize rb_mark_generic_ivar for T_DATA and T_STRUCT 
T_DATA and T_STRUCT could have ivars but might not use the generic_fields_tbl.
This commit skips lookup in the generic_fields_tbl for those cases.
 2026-01-09 17:31:10 +01:00
Earlopain
b3216bc1e1 Fix Ripper::Lexer::State#[] for to_s 
The instance variable is called `to_s`, not `event`.
 2026-01-09 09:33:16 +09:00
Satoshi Tagomori
aaa1234702 update referenced filenames from namespace to box 2025-11-07 13:14:54 +09:00
Jeremy Evans
0b23a8db60 Update dependencies for addition of set.h to public headers 2025-07-11 15:24:23 +09:00
Satoshi Tagomori
382645d440 namespace on read 2025-05-11 23:32:50 +09:00
Jean Boussier
3ec7bfff2e Use a set_table for rb_vm_struct.unused_block_warning_table 
Now that we have a hash-set implementation we can use that
instead of a hash-table with a static value.
 2025-04-27 11:59:28 +02:00
ydah
eafba0d5d3 Use LRAMA instead of BISON 2025-01-14 17:20:02 +09:00
yui-knk
84f0d2dd41 Stop passing -t and -v on ripper build 
Both of them are debug option.
Let's use `YFLAGS` for parse.y build if needed.
 2025-01-13 09:23:31 +09:00
ydah
de8deb5f9c Remove unused token definitions for tRPAREN in Ripper and parser files 2025-01-07 07:17:55 +09:00
Nobuyoshi Nakada
871124c3ea
[DOC] Stop document Ripper::Lexer 
`:nodoc:` seems not working for inner classes.
 2024-12-25 13:07:22 +09:00
Peter Zhu
407f8b8716 Fix memory leak in Ripper for indented heredocs 
The allocated parser string is never freed, which causes a memory leak.

The following code leaks memory:

    Ripper.sexp_raw(DATA.read)

    __END__
    <<~EOF
      a
        #{1}
      a
    EOF
 2024-09-25 08:56:14 -04:00
S-H-GAMELINKS
95d26ee41e Reuse dedent_string function in rb_ruby_ripper_dedent_string function 
This change is reduce Ruby C API dependency for Universal Parser.
Reuse dedent_string functions in rb_ruby_ripper_dedent_string functions and remove dependencies on rb_str_modify and rb_str_set_len from the parser.
 2024-09-22 12:22:20 +09:00
Nobuyoshi Nakada
97449338d6 [Bug #20649] Allow nil as 2nd argument of assign_error 
Fallback to the last token element in that case, for the backward
compatibilities.
 2024-07-24 22:18:36 +09:00
Nobuyoshi Nakada
18fcec23bf
ripper: Introduce RIPPER_ID macro instead of ripper_id_ macros 2024-06-08 13:20:46 +09:00
Nobuyoshi Nakada
4c7ec5e014
ripper: Extend to represent array access and splat 2024-05-21 13:52:30 +09:00
Nobuyoshi Nakada
66446b9435
ripper: Allow parenthesized comma in options 2024-05-21 13:52:29 +09:00
Nobuyoshi Nakada
47efdae602
ripper: Preserve indentation 2024-05-21 13:52:29 +09:00
Nobuyoshi Nakada
147134b474
ripper: Remove rb_ripper_none 
Now it is used only for wheter `opt_paren_args` is `none`.  Introduce
a new special node to distinguish an empty parentheses from it .
 2024-05-21 13:52:29 +09:00
Nobuyoshi Nakada
2e765c20db
ripper: Short hand for rb_ary_new_from_args 2024-05-21 13:52:29 +09:00
Nobuyoshi Nakada
2d92a4afba
ripper: Make $:n to refer each grammar values 
Ripper DSL uses these values for callbacks, but does not need indexes.
 2024-05-21 13:52:29 +09:00
yui-knk
cf74ff714a Change return value of gets function to be rb_parser_string_t * instead of VALUE 
This change reduces parser's dependency on ruby object.
 2024-05-04 11:59:10 +09:00
Nobuyoshi Nakada
a6308ca958 ripper: Move DSL line pattern 2024-04-29 08:38:23 +09:00
ydah
f9cf923af2 Use user defined parameterizing rules 2024-04-29 08:38:23 +09:00
卜部昌平
c844968b72 ruby tool/update-deps --fix 2024-04-27 21:55:28 +09:00
yui-knk
33929ef995 Move encoding object conversion outside of parser 
Reduce the parser's dependence on `VALUE` and `rb_enc_from_encoding`.
 2024-04-23 13:11:46 +09:00
Nobuyoshi Nakada
afa0d58580
Adjust indent [ci skip] 2024-04-23 09:21:38 +09:00
yui-knk
2992e1074a Refactor parser compile functions 
Refactor parser compile functions to reduce the dependence
on ruby functions.
This commit includes these changes

1. Refactor `gets`, `input` and `gets_` of `parser_params`

Parser needs two different data structure to get next line, function (`gets`) and input data (`input`).
However `gets_` is used for both function (`call`) and input data (`ptr`).
`call` is used for managing general callback function when `rb_ruby_parser_compile_generic` is used.
`ptr` is used for managing the current pointer on String when `parser_compile_string` is used.
This commit changes parser to used only `gets` and `input` then removes `gets_`.

2. Move parser_compile functions and `gets` functions from parse.y to ruby_parser.c

This change reduces the dependence on ruby functions from parser.

3. Change ruby_parser and ripper to take care of `VALUE input` GC mark

Move the responsibility of calling `rb_gc_mark` for `VALUE input` from parser to ruby_parser and ripper.
`input` is arbitrary data pointer from the viewpoint of parser.

4. Introduce rb_parser_compile_array function

Caller of `rb_parser_compile_generic` needs to take care about GC because ruby_parser doesn’t know
about the detail of `lex_gets` and `input`.
Introduce `rb_parser_compile_array` to reduce the complexity of ast.c.
 2024-04-23 07:20:22 +09:00
yui-knk
38b8bdb8ea Remove undefined function's prototype declaration 
89cfc152071 removed the definition of these functions.
 2024-04-14 10:51:16 +09:00
yui-knk
e816ab0b0c Remove rb_imemo_tmpbuf_t from parser 
No parser semantic value types are `VALUE` then no need to
use imemo for managing semantic value stack anymore.
 2024-04-02 19:37:27 +09:00
yui-knk
799e854897 [Feature #20331] Simplify parser warnings for hash keys duplication and when clause duplication 
This commit simplifies warnings for hash keys duplication and when clause duplication,
based on the discussion of https://bugs.ruby-lang.org/issues/20331.
Warnings are reported only when strings are same to ohters.
 2024-04-02 08:26:58 +09:00
S-H-GAMELINKS
060a71d4e7 Fix Ripper memory allocation size when enabled Universal Parser 
The size of `struct parser_params` is 8 bytes difference in `ripper_s_allocate` and `rb_ruby_parser_allocate` when the universal parser is
enabled.
This causes a situation where `*r->p` is not fully initialized in `ripper_s_allocate` as shown below.

```console
(gdb) p *r->p
$2 = {heap = 0x0, lval = 0x0, yylloc = 0x0, lex = {strterm = 0x0, gets = 0x0, input = 0, string_buffer = {head = 0x0, last = 0x0}, lastlin
e = 0x0,
    nextline = 0x0, pbeg = 0x0, pcur = 0x0, pend = 0x0, ptok = 0x0, gets_ = {ptr = 0, call = 0x0}, state = EXPR_NONE, paren_nest = 0, lpar
_seen = 0,
    debug = 0, has_shebang = 0, token_seen = 0, token_info_enabled = 0, error_p = 0, cr_seen = 0, value = 0, result = 0, parsing_thread = 0, s_value = 0,
    s_lvalue = 0, s_value_stack = 2097}
````

This seems to cause `double free or corruption (!prev)` and SEGV.
So, fixing this by introduce `rb_ripper_parser_params_allocate` and `rb_ruby_parser_config` functions for Ripper, and `struct parser_params` same size is returned.
 2024-03-21 18:10:02 +09:00
Nobuyoshi Nakada
e670892497
Remove no longer needed matching 2024-03-17 18:47:18 +09:00
Nobuyoshi Nakada
9e470ebdcd
Revert "Remove flip-flop usages from build scripts" 
This reverts commit 301fa452f7a9cdea922103e9c50d85a2d5652d0d.
 2024-03-17 18:28:28 +09:00
Jean Boussier
09d8c99cdc Ensure test suite is compatible with --frozen-string-literal 
As preparation for https://bugs.ruby-lang.org/issues/20205
making sure the test suite is compatible with frozen string
literals is making things easier.
 2024-03-14 17:56:15 +01:00
yui-knk
7cb8fd7800 Move ripper_validate_object to ripper_init.c.tmpl 2024-02-20 19:19:31 +09:00
yui-knk
89cfc15207 [Feature #20257] Rearchitect Ripper 
Introduce another semantic value stack for Ripper so that
Ripper can manage both Node and Ruby Object separately.
This rearchitectutre of Ripper solves these issues.
Therefore adding test cases for them.

* [Bug 10436] https://bugs.ruby-lang.org/issues/10436
* [Bug 18988] https://bugs.ruby-lang.org/issues/18988
* [Bug 20055] https://bugs.ruby-lang.org/issues/20055

Checked the differences of `Ripper.sexp` for files under `/test/ruby`
are only on test_pattern_matching.rb.
The differences comes from the differences between
`new_hash_pattern_tail` functions between parser and Ripper.
Ripper `new_hash_pattern_tail` didn’t call `assignable` then
`kw_rest_arg` wasn’t marked as local variable.
This is also fixed by this commit.

```
--- a/./tmp/before/test_pattern_matching.rb
+++ b/./tmp/after/test_pattern_matching.rb
@@ -3607,7 +3607,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [984, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [985, 10]]],
+                    [:var_ref, [:@ident, “a”, [985, 10]]],
                     :==,
                     [:hash, nil]]],
                   nil]]],
@@ -3662,7 +3662,7 @@
                  [:in,
                   [:hshptn, nil, [], [:var_field, [:@ident, “a”, [993, 13]]]],
                   [[:binary,
-                    [:vcall, [:@ident, “a”, [994, 10]]],
+                    [:var_ref, [:@ident, “a”, [994, 10]]],
                     :==,
                     [:hash,
                      [:assoclist_from_args,
@@ -3813,7 +3813,7 @@
                    [:command,
                     [:@ident, “raise”, [1022, 10]],
                     [:args_add_block,
-                     [[:vcall, [:@ident, “b”, [1022, 16]]]],
+                     [[:var_ref, [:@ident, “b”, [1022, 16]]]],
                      false]]],
                   [:else, [[:var_ref, [:@kw, “true”, [1024, 10]]]]]]]],
                nil,
@@ -3876,7 +3876,7 @@
                      [:@int, “0”, [1033, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1033, 20]]],
+                     [:var_ref, [:@ident, “b”, [1033, 20]]],
                      :==,
                      [:hash, nil]]]],
                   nil]]],
@@ -3946,7 +3946,7 @@
                      [:@int, “0”, [1042, 15]]],
                     :“&&“,
                     [:binary,
-                     [:vcall, [:@ident, “b”, [1042, 20]]],
+                     [:var_ref, [:@ident, “b”, [1042, 20]]],
                      :==,
                      [:hash,
                       [:assoclist_from_args,
@@ -5206,7 +5206,7 @@
                      [[:assoc_new,
                        [:@label, “c:“, [1352, 22]],
                        [:@int, “0”, [1352, 25]]]]]],
-                   [:vcall, [:@ident, “r”, [1352, 29]]]],
+                   [:var_ref, [:@ident, “r”, [1352, 29]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5299,7 +5299,7 @@
                       [:assoc_new,
                        [:@label, “c:“, [1367, 34]],
                        [:@int, “0”, [1367, 37]]]]]],
-                   [:vcall, [:@ident, “r”, [1367, 41]]]],
+                   [:var_ref, [:@ident, “r”, [1367, 41]]]],
                   false]]],
                [:binary,
                 [:call,
@@ -5931,7 +5931,7 @@
              [:in,
               [:hshptn, nil, [], [:var_field, [:@ident, “r”, [1533, 11]]]],
               [[:binary,
-                [:vcall, [:@ident, “r”, [1534, 8]]],
+                [:var_ref, [:@ident, “r”, [1534, 8]]],
                 :==,
                 [:hash,
                  [:assoclist_from_args,
```
 2024-02-20 17:33:58 +09:00
yui-knk
928f388415 [DOC] Fix Ripper DSL input example 
'!' suffix is needed for event dispatch.
 2024-01-30 22:49:22 +09:00
yui-knk
ee7f63ebba Make lastline and nextline to be rb_parser_string 
This commit changes `struct parser_params` lastline and nextline
from `VALUE` (String object) to `rb_parser_string_t *` so that
dependency on Ruby Object is reduced.
`parser_string_buffer_t string_buffer` is added to `struct parser_params`
to manage `rb_parser_string_t` pointers of each line. All allocated line
strings are freed in `rb_ruby_parser_free`.
 2024-01-23 08:58:16 +09:00
KJ Tsanaktsidis
61da90c1b8 Mark asan fake stacks during machine stack marking 
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.

[Bug #20001]
 2024-01-19 09:55:12 +11:00
yui-knk
52d9e55903 Statically allocate parser config 2024-01-12 21:17:41 +09:00