fix: deadlock using IRB integration on Rails applications

[jvlara]

Related issues

Maybe #1166
#1145
I think there are others, but I'm not entirely sure

Description

When using debugger in a Rails console with IRB integration (RUBY_DEBUG_IRB_CONSOLE=true), hitting a breakpoint causes a fatal deadlock: "No live threads left. Deadlock?"

The root cause is a race condition in thread_stopper: leave_subsession pops @subsession_stack and sets @tc = nil before thread_stopper.disable takes full effect. In that window, the thread_stopper TracePoint fires on the main thread (already back in IRB's eval loop), and since @tc is nil, the guard tc == @tc no longer protects it. The main thread is paused via on_pause, but the session server is already blocked on @q_evt.pop -- neither can wake the other.

The fix adds next unless in_subsession? as the first guard in thread_stopper. Once leave_subsession has popped the stack, in_subsession? returns false and the TracePoint becomes a no-op even if it hasn't been disabled yet.

How to reproduce

Tested with Rails 8.1.3 and Ruby 3.3.9, but the issue is not Rails-version-specific -- it occurs whenever background threads are running (e.g. ActiveRecord Pool Reaper), which triggers the race condition in thread_stopper.

rails new my_app \
  --minimal \
  --api \
  --skip-git \
  --skip-docker \
  --skip-keeps \
  --skip-action-mailer \
  --skip-action-mailbox \
  --skip-action-text \
  --skip-active-job \
  --skip-active-storage \
  --skip-action-cable \
  --skip-asset-pipeline \
  --skip-javascript \
  --skip-hotwire \
  --skip-jbuilder \
  --skip-test \
  --skip-system-test \
  --skip-bootsnap \
  --skip-dev-gems \
  --skip-thruster \
  --skip-rubocop \
  --skip-brakeman \
  --skip-bundler-audit \
  --skip-ci \
  --skip-kamal \
  --skip-solid \
  --skip-bundle

Then modify the Gemfile:

source "https://rubygems.org"

gem "rails", "~> 8.1.3"
gem "sqlite3", ">= 2.1"
gem "puma", ">= 5.0"
gem "tzinfo-data", platforms: %i[ windows jruby ]

gem "debug"
gem "irb"

Create a script.rb file:

x = 1
debugger

Then run:

RUBY_DEBUG_IRB_CONSOLE=true bin/rails console

load "script.rb"   # hits breakpoint
continue           # => "No live threads left. Deadlock?"

With this fix applied, the second continue returns to the prompt normally.

[ko1]

Thank you！