Encode the safety argument of the epilogue reorder as VM_ASSERTs: a
non-last dying thread still owns its scheduler slot when it leaves the
living set (its VM-lock work stays a counted running thread), and the
last thread's reverse order is only safe because it has no successor
(running == NULL, so the removal cannot join a barrier). Also check
winding_cnt against underflow at the reclaim decrement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fork taken while the parent holds the VM barrier (e.g. rb_gc_before_fork)
leaves the child with barrier_waiting set and a barrier owner that no longer
exists, so the next barrier in the child could never complete. Reset the
barrier fields in thread_sched_atfork like the rest of the scheduler state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The is_running check added by 3fa1c8708 guarded against a terminating
thread joining the barrier after leaving the running set. Since the
previous commit such a thread leaves the living set before handing over
its scheduler slot, so every joiner is a running-set member again.
Restore the unconditional count and assert the invariant instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`rb_ractor_sched_barrier_join` unconditionally did `barrier_waiting_cnt++`.
A thread can reach it (via `vm_lock_enter`) while its Ractor's
`sched.running` is set but the thread is not yet in the VM running set
(`sched.is_running == false`) -- e.g. resuming from a blocking IO wait.
Counting such a thread pushes `barrier_waiting_cnt` past `running_cnt - 1`
and trips `VM_ASSERT(running_cnt - 1 >= barrier_waiting_cnt)` in
`ractor_sched_barrier_completed_p`; in release builds the unsigned
`(running_cnt - barrier_waiting_cnt) == 1` completion test can underflow.
Increment `barrier_waiting_cnt` only when `cr->threads.sched.is_running`
(O(1), toggled together with running_threads add/del). A not-running
thread is parked at the barrier regardless, so it is safe to wait without
counting.
NOTE (open): this guards the count at the join site. A stricter fix would
prevent a not-running thread from reaching barrier_join at all; that needs
care so it still waits for the barrier rather than proceeding during
compaction. Left for review.
Reproduction is currently coupled with a separate pre-existing compaction
use-after-poison on a blocked-IO thread's STR_TMPLOCK buffer (only the
IO-read blocking path triggers this barrier window), so a standalone
passing test awaits that fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The nt scheduling loop reclaimed a terminated coroutine thread's context using
the return value of coroutine_transfer() (through thread_sched_switch0),
assuming it was the resuming context (the dead coroutine). That holds for the
amd64 asm backend, but the ucontext backend returns the transfer *target*
instead, so the loop never recognized the terminal transfer: the reclaim always
saw a live thread, and the loop always unlocked the sched lock -- which the
dying thread had already released in coroutine_thread_terminated -- and
double-unlocked it. Benign on glibc, but FreeBSD's pthread_mutex_unlock returns
EPERM (crash), and it corrupts the scheduler (hang) elsewhere.
Have the terminating coroutine record its own context in nt->dead_co (per
native thread) right before its final transfer; the loop reads it after switch0
returns, independent of coroutine_transfer()'s backend-dependent return value.
It is set and read on the same native thread with nothing running in between,
so it cannot be overwritten before it is consumed.
coroutine_transfer0() and thread_sched_switch0() no longer need to return the
resuming context, so revert them to void (the ASan-only local remains).
Reproduced on Linux with --with-coroutine=ucontext (100% hang before, clean
after); amd64 and ucontext both pass bootstraptest thread/ractor/fiber.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
thread_sched_atfork empties the global ready queue with
ccan_list_head_init but leaves vm->ractor.sched.grq_cnt at whatever the
parent had, so a child forked while any Ractor was enqueued keeps a stale
non-zero count for an empty list. VM_CHECK_MODE builds then fail
"grq_size(vm, cr) == vm->ractor.sched.grq_cnt" in ractor_sched_deq
(reproducible on master with RUBY_MN_THREADS=1: fork in a loop that also
creates Ractors); on regular builds the counter only feeds assertions and
debug logs, so the corruption stays silent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Redesign the teardown of M:N (coroutine) threads around the natural call
structure nt_start -> co_start -> thread_start_func_2: when the thread body
returns, its epilogue (coroutine_thread_terminated) finishes all rb_thread_t /
rb_ractor_t business while th is still valid, and co_start then makes exactly
one final transfer back to the native thread's own context (nt_start's loop),
where the nt reclaims the dead coroutine's context. The resume itself proves
the final transfer's register save completed, so the reclaim needs no list,
no handshake and no extra locking.
This deletes the zombie-threads machinery (VM-global list, sched.finished
publish, the GC mark/reap pass) and closes its UAF window: publishing
`finished` before the terminal transfer let the GC free the Thread wrapper --
whose coroutine context the transfer still saves into -- mid-teardown.
* struct rb_thread_context bundles a coroutine thread's execution-owned
resources: the coroutine_context (kept first, so th->sched.context points
into the block) plus its machine stack, the stashed final-transfer target
(nt), and a dead flag. co_start's epilogue sets the flag and transfers via
the stashed nt, touching tctx alone -- th may already be collected by then.
The nt loop passes every coroutine it resumes from to
thread_sched_reclaim_from(), which frees dead ones (stack back to the
pool). A live coroutine cannot be resumed elsewhere, die and be freed
while the nt loop still inspects it: a live yield arrives with the sched
lock handed off, and no other nt can schedule that thread again until the
nt loop's own unlock.
* coroutine_thread_terminated (run from thread_start_func_2, where the
upstream non-coroutine teardown also runs, while th is still valid and
listed) does: wake the next thread as upstream's thread_sched_to_dead
does; stash th->nt into tctx and detach th->sched.context; leave the
living set; clear the current ec; and, for a shared-nt non-last thread,
enqueue the Ractor's next runnable thread (not earlier: waking it from
to_dead would let another thread of the same Ractor execute while this
teardown still runs). Everything that reads th happens before the
living-set removal, after which the GC may collect th (and, for a
Ractor's main thread, the Ractor).
* interrupt_lock is now destroyed in thread_free instead of during teardown
(thread_cleanup_func): while th is in its Ractor's living set, anyone --
terminate_all on the Ractor's terminating main thread, Thread#kill /
#raise -- may lock it, and destroying it mid-teardown leaves a window
where a concurrent interrupter locks a destroyed mutex (EINVAL; the
upstream non-coroutine path has the same, smaller, window). At thread_free
time the wrapper is unreachable AND off the living set (listed threads
are marked), so no interrupter can exist. The atfork cleanup reinitializes
the copied lock, which may have been held at the instant of fork.
* The Thread wrapper's dfree only reclaims the context of a thread that
never started; a terminated one cleared th->sched.context in its epilogue.
VM shutdown needs no extra synchronization: past the living-set removal the
epilogue touches only its own context block, the TLS and nt->nt_context,
none of which VM destruct frees (SNT native threads and the machine-stack
pool are intentionally not torn down at exit, as before).
Verified: bootstraptest 2050 PASS; test_thread / test_ractor / test_fiber /
test_gc / test_gc_compact / thread_queue 282 tests 0F0E; Ractor x Thread x
IO teardown churn 150/150 + 60/60 (exercises DNT promotion, terminate_all
racing dying threads, and the double-enqueue fixed in the previous commit);
fork with live Ractors + child GC 20/20; ASAN clean including unreferenced
Thread/Ractor collection (the old zombie window) and mixed stress;
RGENGC_CHECK_MODE=2 btest 2050 + churn; TSan warning profile identical to
master (the new functions appear in no race report); YJIT
move-under-GC.stress; kill/join races; Thread object retention across
Ractor death; exit-while-winding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A runnable thread whose Ractor was enqueued to the global ready queue can
be served without a dequeue: a thread waking up in wait_running_turn hands
its native thread over by a direct transfer, and the hot-thread path
steals `running` back outright. Both leave the Ractor's grq entry behind,
orphaned. The entry itself is harmless (the dequeuing nt sees a served
Ractor and skips), but the accounting breaks: the next legitimate enqueue
double-lists the one grq_node embedded in the sched, corrupting the queue
(observed on master as pthread_mutex_lock EINVAL / crashes; about 1/20
runs of a Ractor x Thread x IO churn under a saturated shared-nt pool).
Restore the invariant "enqueued <=> runnable and unserved" by cancelling
the entry at both direct-service points (ractor_sched_cancel_enq). The
grq_node is kept self-linked whenever it is not enqueued, so membership is
read off the node itself: under the per-Ractor sched lock a self-linked
read is decisive (enqueuers hold that lock too), and the common direct
switch -- whose transition never enqueued -- pays no lock at all; a linked
read can race only with a dequeue, so recheck under the grq lock before
ccan_list_del_init.
ractor_sched_enq now unconditionally rb_bugs on a still-linked node: the
double-list is queue corruption either way, and a VM_CHECK_MODE-only
assert cannot catch this race in practice (CHECK builds perturb the
timing; the previous VM_CHECK-only scan never fired while plain builds
reproduced the corruption).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ThreadSanitizer cannot follow Ruby's userspace coroutine stack switches, so
its per-thread shadow stack leaks across a switch and eventually faults
inside libtsan. Create a TSan fiber per coroutine and switch to it at each
transfer, mirroring the existing AddressSanitizer annotations. The main
context borrows the OS thread's implicit fiber and must not destroy it.
Only amd64 is annotated here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hot thread cannot steal back control when sched->running is an MN
thread, and goes to sleep with sched->runnable_hot_th still pointing to
itself, firing VM_ASSERT(sched->runnable_hot_th != th) at the next
wakeup. Drop the hint before sleeping since the thread is no longer
spinning. On a VM_CHECK_MODE build with RUBY_MN_THREADS=1:
Thread.new { loop {} }
100.times { File.read(IO::NULL) }
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the "TODO: CPU num?". Uses `sysconf(_SC_NPROCESSORS_ONLN)` with
a fallback to 8, guarded by `#if defined(HAVE_SYSCONF) && defined(_SC_NPROCESSORS_ONLN)`.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a race between ubf_ractor_wait and `rb_ractor_sched_wakeup` (Port#send,Ractor#send).
It could occur with a sequence like this:
1) r1 t1: r2#send (rb_ractor_sched_wakeup(r2, t2))
2) ubf called for r2, but the caller gets descheduled right before acquiring RACTOR_LOCK(r2)
in ubf_ractor_wait
3) r2 t2 wakes up from #send call, takes RACTOR_LOCK, clears ubf, waiter is
no longer on its stack.
4) ubf resumes, acquires RACTOR_LOCK(r2), tries to touch waiter
(use_after_return)
This was caught by ASAN:
https://ci.rvm.jp/results/trunk_asan@ruby-sp1/6247788
Patch 08372635f7 fixed a race condition on ubfs, but it's only valid if right after
a call to `ubf_clear`, we assume the ubf function cannot be in the middle of running.
This patch removes an optimization in `ubf_clear` that violates that assumption. In short,
`ubf_clear` needs to take `th->interrupt_lock` unconditionally both to avoid deadlocks and to be
able to reason about when ubfs can be run.
This should fix CI errors like https://ci.rvm.jp/results/trunk-jemalloc@ruby-sp2-noble-docker/6242153.
The error was in test_timeout.rb, which had a deadlock during VM shutdown.
```ruby
r = Ractor.new do
begin
Timeout.timeout(0.1) { sleep }
rescue Timeout::Error
:ok
end
end.value
assert_equal :ok, r
```
The deadlock happened during `rb_ractor_terminate_interrupt_main_thread` with 2 ractors:
1) r1 t1: UBF called with t2->interrupt_lock (ubf = ubf_waiting)
2) r2 t2: ubf cleared from previous thread_sched_wait_events_call (but no lock taken, because of optimization)
3) r2 t2: thread_sched_wait_events: acquire thread_sched_lock(t2) (caller calling native_sleep() in loop)
4) r2 t2: ubf_set: try to acquire t2->interrupt_lock [block]
5) r1 t1: try to acquire thread_sched_lock(t2) [block, deadlock]
t2 needs to block on t2->interrupt_lock in step 2 until the ubf has completed. Only then can it register a new
ubf in the next `native_sleep` iteration.
Registering a ubf was considered problematic in some cases because it could
result in lock ordering inversions with the ubf function itself. I believe
this is the reason that in patch be1bbd5b7d, the ubf was registered outside of
the `thread_sched_lock`.
For example, `thread_sched_to_waiting_until_wakeup()` (native_sleep)
should register the ubf with the thread_sched_lock (TSL) taken. The ordering
is (TSL -> UBF lock). During the ubf call itself, since the UBF lock is
always acquired during the call, the ordering is (UBF lock -> TSL). This
inversion was avoided by registering the ubf outside the TSL.
This inversion is benign in this case, though. Since there can be no ubf on the
thread before the call to `ubf_set`, and this ubf is the only place where the
(UBF lock -> TSL lock) ordering is done, it is okay to register it with the TSL.
In fact registering the ubf outside the TSL can result in problamatic race conditions,
so we now register the ubf with the TSL lock.
This fixes a race condition whereby a newly registered ubf is called before the thing it's
protecting (the sleep) is actually run. The sleeping thread can never be interrupted if this
happens.
Fixes [Bug #21926]
When removing a thread from `running_threads`, if we're on a shared
native thread and we're running a dedicated task, we need to wake
the timer thread so it can create a new SNT if necessary. We only
do this if it's waiting forever without the 10ms quantum timeout
for now, because max 10ms of wait is considered "good enough".
In the future, perhaps we can force the timer thread to wake if this
becomes an issue (`timer_thread_wakeup_force`).
Fixes [Bug #21504]
timer_thread_check_exceed() was returning true when the remaining time
was less than 1ms, treating it as "too short time". This caused
sub-millisecond sleeps (like sleep(0.0001)) to return immediately
instead of actually sleeping.
The fix removes this optimization that was incorrectly short-circuiting
short sleep durations. Now the timeout is only considered exceeded when
the actual deadline has passed.
Note: There's still a separate performance issue where MN_THREADS mode
is slower for sub-millisecond sleeps due to the timer thread using
millisecond-resolution polling. This will require a separate fix to
use sub-millisecond timeouts in kqueue/epoll.
[Bug #21836]
When sleeping with `sleep`, currently the main thread can get woken up from sigchld
from any thread (subprocess exited). The timer thread wakes up the main thread when this
happens, as it checks for signals. The main thread then executes the ruby sigchld handler
if one is registered and is supposed to go back to sleep immediately. This is not ideal but
it's the way it's worked for a while. In commit 8d8159e7d8 I added writes to `th->status`
before and after `wait_running_turn` in `thread_sched_to_waiting_until_wakeup`, which is
called from `sleep`. This is usually the right way to set the thread's status, but `sleep`
is an exception because the writes to `th->status` are done in `sleep_forever`. There's a
loop that checks `th->status` in `sleep_forever`. When the main thread got woken up from
sigchld it saw the changed `th->status` and continued to run the main thread instead of
going back to sleep.
The following script shows the error. It was returning instead of sleeping forever.
```ruby
t = Thread.new do
sleep 0.3
`echo hello` # Spawns subprocess
puts "Subprocess exited"
end
puts "Main thread sleeping..."
result = sleep # Should block forever
puts "sleep returned: #{result.inspect}"
```
Fixes [Bug #21812]
Fix race between timer thread dequeuing waiting thread and thread
skipping sleeping due to being dequeued. We now use `th->event_serial` which
is protected by `thread_sched_lock`. When a thread is put on timer thread's waiting
list, the event serial is saved on the item. The timer thread checks
that the saved serial is the same as current thread's serial before
calling `thread_sched_to_ready`.
The following script (taken from a test in `test_thread.rb` used to crash on
scheduler debug assertions. It would likely crash in non-debug mode as well.
```ruby
def assert_nil(val)
if val != nil
raise "Expected #{val} to be nil"
end
end
def assert_equal(expected, actual)
if expected != actual
raise "Expected #{expected} to be #{actual}"
end
end
def test_join2
ok = false
t1 = Thread.new { ok = true; sleep }
Thread.pass until ok
Thread.pass until t1.stop?
t2 = Thread.new do
Thread.pass while ok
t1.join(0.01)
end
t3 = Thread.new do
ok = false
t1.join
end
assert_nil(t2.value)
t1.wakeup
assert_equal(t1, t3.value)
ensure
t1&.kill&.join
t2&.kill&.join
t3&.kill&.join
end
rs = 30.times.map do
Ractor.new do
test_join2
end
end
rs.each(&:join)
```
This race condition was found when calling `Thread#join` with a timeout
inside a ractor. The race is between the polling thread waking up the
thread and the `ubf` getting called (`ubf_event_waiting`). The error was
that the ubf or polling thread would set the thread as ready, but then
the other function would do the same.
Fixes [Bug #21614]
We allocate the stack of the main thread using malloc, but we never set
malloc_stack to true and context_stack. If we fork, the main thread may
no longer be the main thread anymore so it reports memory being leaked
in RUBY_FREE_AT_EXIT.
This commit allows the main thread to free its own VM stack at shutdown.
In rb_ractor_sched_wait() (ex: Ractor.receive), we acquire
RACTOR_LOCK(cr) and then thread_sched_lock(cur_th). However, on wakeup
if we're a dnt, in thread_sched_wait_running_turn() we acquire
thread_sched_lock(cur_th) after condvar wakeup and then RACTOR_LOCK(cr).
This lock inversion can cause a deadlock with rb_ractor_wakeup_all()
(ex: port.send(obj)), where we acquire RACTOR_LOCK(other_r) and then
thread_sched_lock(other_th).
So, the error happens:
nt 1: Ractor.receive
rb_ractor_sched_wait() after condvar wakeup in thread_sched_wait_running_turn():
- thread_sched_lock(cur_th) (condvar) # acquires lock
- rb_ractor_lock_self(cr) # deadlock here: tries to acquire, HANGS
nt 2: port.send
ractor_wakeup_all()
- RACTOR_LOCK(port_r) # acquires lock
- thread_sched_lock # tries to acquire, HANGS
To fix it, we now unlock the thread_sched_lock before acquiring the
ractor_lock in rb_ractor_sched_wait().
Script that reproduces issue:
```ruby
require "async"
class RactorWrapper
def initialize
@ractor = Ractor.new do
Ractor.recv # Ractor doesn't start until explicitly told to
# Do some calculations
fib = ->(x) { x < 2 ? 1 : fib.call(x - 1) + fib.call(x - 2) }
fib.call(20)
end
end
def take_async
@ractor.send(nil)
Thread.new { @ractor.value }.value
end
end
Async do |task|
10_000.times do |i|
task.async do
RactorWrapper.new.take_async
puts i
end
end
end
exit 0
```
Fixes [Bug #21398]
Co-authored-by: John Hawthorn <john.hawthorn@shopify.com>
We never freed any resources of rb_native_thread at fork because it would
cause it to hang. This is because it called rb_native_cond_destroy for
condition variables. We can't call rb_native_cond_destroy here because
according to the specs of pthread_cond_destroy:
Attempting to destroy a condition variable upon which other threads
are currently blocked results in undefined behavior.
Specifically, glibc's pthread_cond_destroy waits on all the other listeners.
Since after forking all the threads are dead, the condition variable's
listeners will never wake up, so it will hang forever.
This commit changes it to only free the memory and none of the condition
variables.
* Added `Ractor::Port`
* `Ractor::Port#receive` (support multi-threads)
* `Rcator::Port#close`
* `Ractor::Port#closed?`
* Added some methods
* `Ractor#join`
* `Ractor#value`
* `Ractor#monitor`
* `Ractor#unmonitor`
* Removed some methods
* `Ractor#take`
* `Ractor.yield`
* Change the spec
* `Racotr.select`
You can wait for multiple sequences of messages with `Ractor::Port`.
```ruby
ports = 3.times.map{ Ractor::Port.new }
ports.map.with_index do |port, ri|
Ractor.new port,ri do |port, ri|
3.times{|i| port << "r#{ri}-#{i}"}
end
end
p ports.each{|port| pp 3.times.map{port.receive}}
```
In this example, we use 3 ports, and 3 Ractors send messages to them respectively.
We can receive a series of messages from each port.
You can use `Ractor#value` to get the last value of a Ractor's block:
```ruby
result = Ractor.new do
heavy_task()
end.value
```
You can wait for the termination of a Ractor with `Ractor#join` like this:
```ruby
Ractor.new do
some_task()
end.join
```
`#value` and `#join` are similar to `Thread#value` and `Thread#join`.
To implement `#join`, `Ractor#monitor` (and `Ractor#unmonitor`) is introduced.
This commit changes `Ractor.select()` method.
It now only accepts ports or Ractors, and returns when a port receives a message or a Ractor terminates.
We removes `Ractor.yield` and `Ractor#take` because:
* `Ractor::Port` supports most of similar use cases in a simpler manner.
* Removing them significantly simplifies the code.
We also change the internal thread scheduler code (thread_pthread.c):
* During barrier synchronization, we keep the `ractor_sched` lock to avoid deadlocks.
This lock is released by `rb_ractor_sched_barrier_end()`
which is called at the end of operations that require the barrier.
* fix potential deadlock issues by checking interrupts just before setting UBF.
https://bugs.ruby-lang.org/issues/21262
Although it almost certainly works in this case, volatile is best not
used for multi-threaded code. Using atomics instead avoids warnings from
TSan.
This also simplifies some logic, as system_working was previously only
ever assigned to 1, so --system_working <= 0 should always return true
(unless it underflowed).
Rework ractors so that any ractor action (Ractor.receive, Ractor#send, Ractor.yield, Ractor#take,
Ractor.select) will operate on the thread that called the action. It will put that thread to sleep if
it's a blocking function and it needs to put it to sleep, and the awakening action (Ractor.yield,
Ractor#send) will wake up the blocked thread.
Before this change every blocking ractor action was associated with the ractor struct and its fields.
If a ractor called Ractor.receive, its wait status was wait_receiving, and when another ractor calls
r.send on it, it will look for that status in the ractor struct fields and wake it up. The problem was that
what if 2 threads call blocking ractor actions in the same ractor. Imagine if 1 thread has called Ractor.receive
and another r.take. Then, when a different ractor calls r.send on it, it doesn't know which ruby thread is associated
to which ractor action, so what ruby thread should it schedule? This change moves some fields onto the ruby thread
itself so that ruby threads are the ones that have ractor blocking statuses, and threads are then specifically scheduled
when unblocked.
Fixes [#17624]
Fixes [#21037]
When a dedicated native thread fails to create (OS can't create more
threads), don't add the ruby-level thread to the thread scheduler. If
it's added, then next time the thread is scheduled to run it will sleep
forever, waiting for a native thread that doesn't exist to pick it up.
This commit adds an environment variable `RUBY_THREAD_TIMESLICE` for
specifying the default thread quantum in milliseconds. You can adjust
this variable to tune throughput, which is especially useful on
multithreaded systems that are mixing CPU bound work and IO bound work.
The default quantum remains 100ms.
[Feature #20861]
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Many libraries should be loaded on the main ractor because of
setting constants with unshareable objects and so on.
This patch allows to call `requore` on non-main Ractors by
asking the main ractor to call `require` on it. The calling ractor
waits for the result of `require` from the main ractor.
If the `require` call failed with some reasons, an exception
objects will be deliverred from the main ractor to the calling ractor
if it is copy-able.
Same on `require_relative` and `require` by `autoload`.
Now `Ractor.new{pp obj}` works well (the first call of `pp` requires
`pp` library implicitly).
[Feature #20627]
This code was working around a bug in the Linux kernel. It was
previously possible for the kernel to place heap pages in a region where
the stack was allowed to grow into, and then therefore run out of usable
stack memory before RLIMIT_STACK was reached.
This bug was fixed in Linux commit
c204d21f22
for kernel 4.13 in 2017. Therefore, in 2024, we should be safe to delete
this workaround.
[Bug #20804]
