[pull] master from ruby:master - #1388
Merged
Merged
Conversation
heap_pages_free_unused_pages called heap_page_free once per empty page, and each call took page_pool.lock twice: once in global_page_index_remove and once in page_pool_release. A full drain of N pages cost 2N lock cycles This commit links pages into a local chain using free_next and then releases the whole chain under a single page_pool.lock.
This commit initializes the page_pool.lock with PTHREAD_MUTEX_ERRORCHECK in RGENGC_CHECK_MODE builds and asserts that the the mutex lock is EDEADLK.
heap_pages_free_batch locked page_pool.lock once per page on the non-mmap path, through global_page_index_remove.
The redirect handling only compared hosts, so an https source redirecting to http on the same host had the request's user and password copied onto a plaintext connection. Gem::RemoteFetcher already refuses such a redirect. ruby/rubygems@56e1b63333 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CRT _tzname is encoded in the active code page, while zone_str() assumed the UTF-8 converted name from GetDynamicTimeZoneInformation. Tag strings from the _tzname fallback with the encoding of GetACP(), and stop asserting the locale encoding on Windows, where the assertion restated the bug. Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
`update-src` truncated revision.h to force the regeneration, but the blank file also defeated the guard in file2lastrev.rb that keeps the existing content when no VCS is available, so `make up` on a released tarball dropped `RUBY_REVISION` and `RUBY_FULL_REVISION`. Removing the timestamp alone is enough to force the regeneration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bumps the github-actions group with 1 update in the / directory: [taiki-e/install-action](https://github.com/taiki-e/install-action). Updates `taiki-e/install-action` from 2.87.5 to 2.87.6 - [Release notes](https://github.com/taiki-e/install-action/releases) - [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md) - [Commits](taiki-e/install-action@5bf6ce0...7b8d471) --- updated-dependencies: - dependency-name: taiki-e/install-action dependency-version: 2.87.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
TYPE_UCLASS only compared the built-in type before RBASIC_SET_CLASS, so a struct could be relabelled as a struct class with a different number of members, and its accessors then read and wrote past the allocated slots. Apply the member count check that TYPE_STRUCT already has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Net::HTTPHeader#chunked? matched the token anywhere in Transfer-Encoding, so a response carrying `chunked, gzip` was chunk-framed and the bytes after the terminating chunk were left unread on the connection. See RFC 9112 Section 6.3: https://www.rfc-editor.org/rfc/rfc9112.html#section-6.3 If a Transfer-Encoding header field is present in a response and the chunked transfer coding is not the final encoding, the message body length is determined by reading the connection until it is closed by the server. ruby/net-http@e70762aaf9 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Once chunked is not the final transfer coding, read_body_0 fell through to Content-Length and framed the body by it, which leaves the client and the server disagreeing about where the response ends. See RFC 9112 Section 6.3: https://www.rfc-editor.org/rfc/rfc9112.html#section-6.3 If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length. ruby/net-http@9fb5b93b52 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rb_ractor_terminate_all() interrupts the other Ractors and waits on a condvar of its own for them to finish, dropping the VM lock for the wait and taking it back after. That was rb_vm_cond_timedwait(), whose only caller it was; rb_vm_cond_wait() had none at all. Move the wait to rb_ractor_sched_wait_terminate() in thread_sched.c and give the native thread back for its duration, as a blocking region does. Waiting natively holds whatever native thread the caller runs on, and an M:N caller runs on one from the shared pool -- so it would wait, holding the pool, for Ractors that need the pool to finish. It does not happen today, the caller being the main Ractor's main thread with a native thread of its own, but a scheduler-blind wait in the scheduler's way is worth keeping out of vm_sync.c. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`RUBY_MN_THREADS=1` only affects threads created after it is read: the main thread keeps a native thread of its own and is woken through its condvar, so handing control to the main thread costs an order of magnitude more than handing it between any two other threads (6.4us against 0.56us for a Queue round trip here). With `RUBY_MN_THREADS=2` the running main thread is turned into an M:N thread in place (thread_sched_main_to_shared), rather than spawning a thread for the program body and leaving the main native thread idle: * the process stack becomes the main thread's coroutine context (coroutine_initialize_main), the way nt_start's own stack is a shared thread's context; * the main native thread joins the shared pool with a stack of its own for its scheduling loop (nt_loop_co), which starts on the first transfer into it -- the main thread's first park. From then on any shared native thread may resume the main thread, and this one serves any Ractor. nt_start's shared loop is split out as nt_shared_loop so that both entries share it. The main native thread never retires, its loop being on a stack only it could free: ractor_sched_deq takes a can_retire flag and native_thread_dedicated_dec always lets it rejoin. The wait in rb_ractor_terminate_all had to give its native thread back first, or exit hung with the Ractors it waits for having nothing to run on; that is the commit before this one. The VM barrier waits natively in the same way and has the same exposure, and is not touched here, being no more reachable under `=2` than before. Several places assumed the main thread owns the process's initial native thread: * native_thread_init_stack set the process stack's range for any thread starting on that native thread, over the pool stack an M:N thread already owns; the case is now keyed on th->sched.context. * thread_sched_switch passed to_dead for a thread whose status is THREAD_KILLED, which the main thread already is while rb_ractor_terminate_all parks it. A thread ends only through co_start's epilogue transfer. * thread_sched_atfork left the forker's nt->running_th and nt->retiring as the parent had them; retire eligibility is decided against the process's main native thread, refreshed there. * rb_thread_free_native_thread (RUBY_FREE_AT_EXIT) destroyed the native thread hosting the main thread even when shared, along with the altstack registered on the thread running it. RUBY_FREE_AT_EXIT now leaves more behind: the main native thread and its context and stack, since a thread is parked in its loop until the process ends, and the main thread's own coroutine context, since the free-at-exit path does not reach rb_threadptr_sched_free. Leak-checker baselines move by that much. The OS thread name is no longer set from the Ruby thread for M:N threads. One shared native thread runs many Ruby threads over its life, so the name described whichever one happened to start on it; under `=2` that renamed the process itself, the thread group leader's comm being what ps and pkill show. Thread#name= was already skipped for M:N threads for the same reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Ractor's threads have been on the M:N scheduler since it was added, with no way to turn that off: RUBY_MN_THREADS only ever decided whether the main Ractor joined them. There is no way, then, to tell an M:N bug from a Ractor bug, and a C extension that keeps state per native thread cannot be used from a Ractor at all. RUBY_MN_THREADS=-1 gives every Ractor's threads a native thread of their own, as the main Ractor's have by default. With it the setting reads as one ladder: -1 nothing, 0 (the default) a Ractor's threads, 1 the main Ractor's threads too, 2 the main thread as well. The shared pool is left unmade when nothing is M:N, rather than minting native threads that no thread can park on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The same character count clamp reaches the backward scan in str_rindex() and the pos == 0 shortcut in rb_str_rindex(), where a pattern with wider characters than the receiver is still compared past its last byte. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
String#chomp steps back by the encoding's minimum character width before testing for a newline, which reads before the first byte of the receiver when it holds less than one whole character. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The BOM written ahead of the first character was not counted, so that
character can need 6 bytes for UTF-16 and 8 for UTF-32 against a declared
max_output of 4. Since the engine lets a transcoder write straight into
the caller's window once max_output bytes are free, a window of exactly 4
overran it:
ec = Encoding::Converter.new("UTF-8", "UTF-16")
ec.primitive_convert("\u{1F600}", "X" * 4092, 4092, 4)
# -e:1: [BUG] probable buffer overflow: 4098 for 4096
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A katakana held back for a possible sound mark is flushed by the next
character, costing a designation and two data bytes before that
character's own designation and data, so one call can write 9 bytes
against a declared max_output of 5:
ec = Encoding::Converter.new("CP51932", "CP50220")
ec.primitive_convert("\x8e\xb6\x8e\xe0".b, "X" * 4088, 4088, 8)
# -e:1: [BUG] probable buffer overflow: 4097 for 4096
9 is past the 8 byte inline write buffer in transcode.c, so this
transcoder now allocates one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e extraction root `extract_tar_gz` only ran relative link targets through `File.expand_path`, so an absolute target kept its `..` components. A target such as `<destination_dir>/../../etc` therefore passed the prefix check and resolved outside the extraction root. ruby/rubygems@a34a62c3a1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Avoid quadratic parse time on long and/or chains logop() builds a left-associative and/or chain into a right-leaning tree, and found the insertion point by walking the whole right spine from the top on every operator. For a chain such as `a && a && ... && a` that walk is O(n) per operator, so parsing the chain is O(n^2): 32k operators took seconds. Cache the tail of the chain most recently built by logop(), so an operator that extends the same chain reaches the insertion point in constant time, making the parse linear. The cached tail is validated before use (it must still be a node of the chain's type whose nd_2nd is not another such node), so a stale entry from an earlier parse falls back to the scan. The resulting tree is unchanged: AST and compiled bytecode are byte-identical to before across left-associative, parenthesized, and mixed and/or chains.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )