This reverts commit 994257ab06072df38de024e70a60aa9a87e36089.
I saw some failures in CI that are probably related to the change.
Example:
```
1) Failure:
TestMonitor#test_timedwait [/Users/runner/work/ruby/ruby/src/test/monitor/test_monitor.rb:282]:
```
This starvation problem has not been an issue in real apps afaik, so for now it's best to revert
it and think of a better solution.
Continually locking a mutex m can lead to starvation if all other threads are on the waitq of m.
See https://bugs.ruby-lang.org/issues/21840 for more details.
Solution:
When a thread `T1` wakes up `T2` during mutex unlock but `T1` or any other thread successfully acquires it
before `T2`, then we record the `running_time` of the thread during mutex acquisition. Then during unlock, if
that thread's running_time is less than the saved running time, we set it back to the saved time.
Fixes [Bug #21840]
Thread::Queue spends a significant amount of time in array functions,
checking for invariants we know aren't a problem, and whether the backing
array need to reordered.
By using a ring buffer we can remove a lot of overhead (~23% faster).
```
$ hyperfine './miniruby --yjit /tmp/q.rb' './miniruby-qrb --yjit /tmp/q.rb'
Benchmark 1: ./miniruby --yjit /tmp/q.rb
Time (mean ± σ): 1.050 s ± 0.191 s [User: 0.988 s, System: 0.004 s]
Range (min … max): 0.984 s … 1.595 s 10 runs
Benchmark 2: ./miniruby-qrb --yjit /tmp/q.rb
Time (mean ± σ): 844.2 ms ± 3.1 ms [User: 840.4 ms, System: 2.8 ms]
Range (min … max): 838.6 ms … 848.9 ms 10 runs
Summary
./miniruby-qrb --yjit /tmp/q.rb ran
1.24 ± 0.23 times faster than ./miniruby --yjit /tmp/q.rb
```
```
q = Queue.new([1, 2, 3, 4, 5, 6, 7, 8])
i = 2_000_000
while i > 0
i -= 1
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
end
```
Mutexes spend a significant amount of time in `rb_fiber_serial`
because it can't be inlined (except with LTO).
The fiber struct is opaque the so function can't be defined as inlineable.
Ideally the while fiber struct would not be opaque to the rest of
Ruby core, but it's tricky to do.
Instead we can store the fiber serial in the execution context
itself, and make its access cheaper:
```
$ hyperfine './miniruby-baseline --yjit /tmp/mut.rb' './miniruby-inline-serial --yjit /tmp/mut.rb'
Benchmark 1: ./miniruby-baseline --yjit /tmp/mut.rb
Time (mean ± σ): 4.011 s ± 0.084 s [User: 3.977 s, System: 0.011 s]
Range (min … max): 3.950 s … 4.245 s 10 runs
Benchmark 2: ./miniruby-inline-serial --yjit /tmp/mut.rb
Time (mean ± σ): 3.495 s ± 0.150 s [User: 3.448 s, System: 0.009 s]
Range (min … max): 3.340 s … 3.869 s 10 runs
Summary
./miniruby-inline-serial --yjit /tmp/mut.rb ran
1.15 ± 0.05 times faster than ./miniruby-baseline --yjit /tmp/mut.rb
```
```ruby
i = 10_000_000
mut = Mutex.new
while i > 0
i -= 1
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
end
```
Previously this held a pointer to the Fiber itself, which requires
marking it (which was only implemented recently, prior to that it was
buggy). Using a monotonically increasing integer instead allows us to
avoid having a free function and keeps everything simpler.
My main motivations in making this change are that the root fiber lazily
allocates self, which makes the writebarrier implementation challenging
to do correctly, and wanting to avoid sending Mutexes to the remembered
set when locked by a short-lived Fiber.
Mutexes were being improperly unlocked on thread cleanup. This bug was
introduced in 050a8954395.
We must keep a reference from the mutex to the thread, because if the fiber
is collected before the mutex is, then we cannot unlink it from the thread in
`mutex_free`. If it's not unlinked from the the thread when it's freed, it
causes bugs in `rb_thread_unlock_all_locking_mutexes`.
We now mark the fiber when a mutex is locked, and the thread is marked
as well. However, a fiber can still be freed in the same GC cycle as the
mutex, so the reference to the thread is still needed.
The reason we need to mark the fiber is that `mutex_owned_p()` has an ABA
issue where if the fiber is collected while it's locked, a new fiber could be
allocated at the same memory address and we could get false positives.
Fixes [Bug #21342]
Co-authored-by: John Hawthorn <john@hawthorn.email>
Documented the necessity of calling `wait` in a loop. We modified the
example to demonstrate the idiomatic use, and added a third thread `a2`
to demonstrate another reason that necessitates the loop.
Mentioned spurious wake-up in the doc.
assert does not print the bug report, only the file and line number of
the assertion that failed. RUBY_ASSERT prints the full bug report, which
makes it much easier to debug.
This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
* Revert "Extract `do_mutex_lock_check_interrupts` to try and fix `ppc64le`. (#8393)"
This reverts commit 5184b40dd4dc446660cd35c3e53896324e95b317.
* .travis.yml: Try default gcc 9.4.0 instead of gcc-10 in ppc64le and s390x.
Use gcc 9.4.0 instead of gcc-10 to avoid the current failures by a possible GCC
10 compiler bug in the Travis ppc64le and s390x cases. And it also aligns with
RubyCI Ubuntu ppc64le and s390x where the default gcc is used.
---------
Co-authored-by: Jun Aruga <jaruga@ruby-lang.org>
We found some tests were hanging in `do_mutex_lock`, specifically the
fiber scheduler autoload test. After much investigation, it may be a code
generation bug. Because we didn't change the code, but only extracted it
into a separate function, and it appears to fix the problem.
It's possible (but very rare) to have a race condition between setting
`mutex->fiber = NULL` and `thread_mutex_remove(th, mutex)` which results
in the following bug:
```
[BUG] invalid keeping_mutexes: Attempt to unlock a mutex which is not locked
```
Fixes <https://bugs.ruby-lang.org/issues/19480>.
Fixes the following compilation warnings:
thread_sync.c:908:48: warning: taking address of packed member of `struct rb_queue` may result in an unaligned pointer value [-Waddress-of-packed-member]
thread_sync.c:1181:48: warning: taking address of packed member of `struct rb_queue` may result in an unaligned pointer value [-Waddress-of-packed-member]
[Bug #19105]
If no fiber scheduler is registered and the fiber that
owns the lock and the one that try to acquire it
both belong to the same thread, we're in a deadlock case.
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
[Feature #18982]
Instead of introducing an `exception: false` argument to have `non_block`
return nil rather than raise, we can clearly document that a timeout of 0
immediately returns.
The code is refactored a bit to avoid doing a time calculation in
such case.
When I removed the SizeQueue#push timeout from my PR, I forgot to
update the `queue_sleep` parameters to be a `queue_sleep_arg`.
Somehow this worked on most archs, but on Solaris/Sparc it would
legitimately crash when trying to access the `timeout` and `end`
members of the struct.
rb_ary_tmp_new suggests that the array is temporary in some way, but
that's not true, it just creates an array that's hidden and not on the
transient heap. This commit renames it to rb_ary_hidden_new.
