Repository navigation
[pull] master from tensorflow:master - #8846
Merged
Merged
Conversation
…erand`/`PositionOfResult`. These accessors map an HLO operand number, or a result shape index, to a position in the kernel argument list. The value going in and the value coming out live in different index spaces, and calling both of them an "index" invited the reader to mix them up. `KernelArguments` already calls the outgoing value a position internally (`ArgumentPositions`), so this brings the public names in line with that vocabulary. Pure rename, no behavior change. PiperOrigin-RevId: 985207402
…s larger than 4 GiB Imported from GitHub PR openxla/xla#47282 ## 📝 Summary of Changes `ZlibOutputBuffer::Append` passes data that does not fit in the staging buffer directly to zlib by assigning the `size_t` byte count to `z_stream::avail_in`, which is zlib's 32-bit `uInt`. A single `Append` of 4 GiB or more is truncated to `size % 2^32` bytes: the write reports OK, `Close()` produces a well-formed gzip stream, and everything past the truncated count is silently lost. This change chunks the direct-deflate path so zlib is fed at most `uInt` max bytes per round, and adds a round-trip regression test that appends 4 GiB + 1 MiB in one call. The test lives in its own `size = "large"` target (`zlib_buffers_large_append_test`) because it needs ~4 GiB of memory for the input string; the existing `zlib_buffers_test` stays `small`. This completes the fix started in 783463ddeb, which addressed the size comparisons but not the `avail_in` assignment (see the reporter's follow-up in the issue). ## 🎯 Justification Fixes #28159: profiling a large workload makes `jax.profiler.stop_trace()` write the whole Chrome-trace JSON through one `Append` call in `WriteGzippedDataToFile` (`xla/tsl/profiler/rpc/client/save_profile.cc`); above 4 GiB the resulting `<host>.trace.json.gz` decompresses to a truncated JSON document. Fixing `Append` itself (rather than chunking in `save_profile.cc`) also covers the other users of this class (`record_writer`, `xla/service/dump.cc`). Reproduced at HEAD: appending 2^32 + 2^20 bytes produced a file that decompresses to exactly 2^20 bytes (`size % 2^32`). ## 🚀 Kind of Contribution 🐛 Bug Fix ## 🧪 Unit Tests: - New target `//xla/tsl/lib/io:zlib_buffers_large_append_test` (`size = "large"`, ~4 GiB peak RSS, 8 s at `Z_BEST_SPEED`): single 4 GiB + 1 MiB `Append`, round-tripped through `ZlibInputStream` with full content verification and an end-of-stream check. Fails before this change (first read hits `OUT_OF_RANGE: EOF reached` - the file decompresses to exactly `size % 2^32` = 1 MiB), passes after. - All 13 test targets in `//xla/tsl/lib/io/...` pass with the change. - For appends below 4 GiB the loop runs exactly one iteration with the same zlib call sequence as before, so existing behavior is unchanged (covered by the existing round-trip tests, which exercise the direct-deflate branch with data larger than the staging buffer). ## 🧪 Execution Tests: Not applicable (host-side I/O library; no device execution involved). Copybara import of the project: -- 391f419e530e1ed5184d114f29eadc2354abd631 by Stanislav Bardyuk <sbardyuk@google.com>: Fix silent truncation of ZlibOutputBuffer appends larger than 4 GiB ZlibOutputBuffer::Append assigns the size_t byte count of data that does not fit in the staging buffer directly to zlib's 32-bit avail_in, so a single Append of 4 GiB or more is truncated to size % 2^32 bytes while reporting success and producing a well-formed gzip stream. This is how jax.profiler.stop_trace() corrupts <host>.trace.json.gz for large traces: the whole trace JSON goes through one Append in WriteGzippedDataToFile. Feed the direct-deflate path at most UINT_MAX bytes per round instead, and add a regression test that round-trips a single 4 GiB + 1 MiB append (own size = "large" target; the test needs ~4 GiB for the input string). Completes 783463ddeb, which fixed the size comparisons but not the avail_in assignment. Fixes #28159. Merging this change closes #47282 PiperOrigin-RevId: 985215524
Imported from GitHub PR openxla/xla#47219 📝 Summary of Changes Upgrade curl from the previous pin to **8.21.0** and adapt XLA/TSL libcurl integration. - Bump curl archive pin in `workspace2.bzl` (`strip_prefix`, URL, SHA256). - Refresh `third_party/curl.BUILD` for the 8.21.0 source layout (explicit `lib/`/`src/` lists; vquic `cf-*` renames and new modules). - Update `CurlHttpRequest` for newer libcurl APIs (`CURLOPT_UPLOAD` instead of deprecated `CURLOPT_PUT`; option/signature type updates). - Adjust `curl_http_request_test` fakes for the updated options/signatures. - `USE_SECTRANSP` was Secure Transport, which curl 8.15 removed. `USE_APPLE_SECTRUST` is curl 8.17’s replacement for using Apple’s OS trust store for certificate verification. 🎯 Justification Keeps XLA/TSL’s vendored curl up to date for security and compatibility. Curl releases after the previous pin fix multiple CVEs (credential leaks, connection reuse, TLS/HTTP2 issues, etc.). TensorFlow imports XLA via Copybara, so these changes must land in openxla/xla first; companion TF PR: #115842 🚀 Kind of Contribution ♻️ Cleanup (dependency/security upgrade) 📊 Benchmark (for Performance Improvements) N/A — dependency upgrade, not a performance change. 🧪 Unit Tests: No new XLA compiler unit tests. Existing `curl_http_request_test` updated for the new libcurl option types/signatures. 🧪 Execution Tests: N/A — no HLO/GPU execution changes. Validated locally by building `@curl` / curl HTTP request targets from the companion TensorFlow tree. Copybara import of the project: -- f2754cd3bdd4b1ed36c9dbb3ed555c2e1bf8eaec by spesherov <9691985+spesherov@users.noreply.github.com>: Upgrade curl to 8.21.0 and update libcurl integration. Bump the curl archive pin and refresh curl.BUILD for the 8.21.0 source layout. Update CurlHttpRequest for CURLOPT_UPLOAD and newer libcurl option/signature types. Merging this change closes #47219 PiperOrigin-RevId: 985224070
This patch disabled BLAKE3 x86_64 SIMD assembly in LLVM's Bazel overlay to work around a 2022 Windows build failure where MSVC's ml64.exe failed on Unix .S assembly files. Upstream LLVM now selects the Windows GNU assembly files for clang-cl, and XLA Windows builds use clang-cl instead of MSVC. Removing this patch also restores SIMD-accelerated BLAKE3 hashing on x86_64 platforms. PiperOrigin-RevId: 985256166
…la_gpu_autotune_level is 0. - This particular condition of changing the instruction filter depending on the autotune level can lead to inconsistency and confusion. PiperOrigin-RevId: 985277541
PiperOrigin-RevId: 985290320
… cpu_only_tests lit_test_suite PiperOrigin-RevId: 985329328
Riegeli's Writer abstraction doesn't imply any compression, so this simply allows for flexibility in the storage format, such as to Cord rather than string. In order to do so, we need to create a quick shim between Writer and llvm's raw_ostream, which handles the buffering and flushing already. PiperOrigin-RevId: 985337262
…unkEmitter, allowing AOT/offline compilation of HLO modules with custom calls whose FFI handlers are not registered in the compilation binary (matching the behavior of xla_gpu_mock_custom_calls on GPU). - Add xla_cpu_mock_custom_calls field to DebugOptions in xla.proto and debug_options_flags.cc. - In CPU ThunkEmitter::EmitCustomCallThunk, ignore NOT_FOUND errors for custom call handler lookups when xla_cpu_mock_custom_calls is enabled. - Add mock_custom_call_test unit test for CPU. Testing: Added mock_custom_call_test unit test for CPU and tested compilation of HLO modules containing custom calls. PiperOrigin-RevId: 985344122
…ith absl::bit_cast Imported from GitHub PR openxla/xla#49146 ## [XLA:CPU] Replace strict-aliasing-violating type punning with absl::bit_cast ### Problem In `onednn_ops_rewriter.cc` and `onednn_op_thunk_test.cc`, converting the IEEE-754 bit representation of `float` epsilon to an `int32_t` config field used pointer-dereference type-punning (`*reinterpret_cast<int32_t*>(&eps)`). Dereferencing an aliased pointer of an incompatible type violates the C++ strict aliasing rule ([basic.lval]), which is undefined behavior and risks miscompilations under aggressive optimization levels. ### Solution Replace the pointer type-puns with `absl::bit_cast<int32_t>(...)`. This preserves the exact IEEE-754 bit pattern without numeric conversion while adhering strictly to standard aliasing rules. ### Changes - `xla/service/cpu/onednn_ops_rewriter.cc`: Include `absl/base/casts.h` and use `absl::bit_cast<int32_t>(eps)`. - `xla/service/cpu/BUILD`: Add `@com_google_absl//absl/base:casts` to `onednn_ops_rewriter` deps. - `xla/backends/cpu/runtime/onednn/onednn_op_thunk_test.cc`: Include `absl/base/casts.h` and use `absl::bit_cast<int32_t>(epsilon)`. - `xla/backends/cpu/runtime/onednn/BUILD`: Add `@com_google_absl//absl/base:casts` to `onednn_op_thunk_test` deps. ### Testing - `git diff --check`: Passed with no whitespace or syntax formatting issues. - Verified exact 4-file diff matches project include and dependency conventions. ### Risk Low risk. Behavior-preserving change that preserves identical bit-level values while removing undefined behavior. Copybara import of the project: -- 166e8f6bd7c144aa220220b81aa96ffcbd943da8 by saketh reddy pingili <pingilisakethreddy@gmail.com>: [XLA:CPU] Replace strict-aliasing-violating type punning with absl::bit_cast -- a502f377ac91ce49e01175e0b87489e38ffb3d14 by saketh reddy pingili <pingilisakethreddy@gmail.com>: Fix absl dependency target to @com_google_absl//absl/base Merging this change closes #49146 PiperOrigin-RevId: 985346019
PiperOrigin-RevId: 985354575
…ascal` PiperOrigin-RevId: 985357669
Adds command_buffer_profile_annotation to the VLOG(2) message in the command buffer conversion pass to make it easier to identify which command buffer is being created. Example: Creating command buffer thunk command_buffer_1 with the following thunks: loop_xor_fusion, sort.1, loop_and_convert_fusion PiperOrigin-RevId: 985360175
Split CopyInsertion::AddSpecialCaseCopies into ComputeSpecialCaseCopies and InsertSpecialCaseCopies. A backend can now compute its own special case copies from the same alias analysis instead of building a second one. The inserted copies are unchanged. PiperOrigin-RevId: 985376755
PiperOrigin-RevId: 985380666
… BF16 does not exist. Pass GpuComputeCapability into DotAlgorithmRewriter so it can check target architecture capabilities. Pre-Ampere GPUs lack native BF16 Tensor Cores, where BF16 matmuls are upcast to F32 for performance. PiperOrigin-RevId: 985383850
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )