Sync with Microsoft ONNX Runtime - 15082026 - #1259
Open
ai-fw-intg wants to merge 29 commits into
Open
Conversation
This pull request strengthens validation for image tensor dimensions and improves robustness in both the core library and test code. The main focus is to ensure that image tensor height and width are always positive and do not exceed the maximum allowed integer size, preventing potential overflows or invalid memory access. It also updates related tests and utility functions to cover these cases. **Validation Improvements:** * Added explicit checks in `CreateImageFeatureDescriptor` to ensure image tensor height and width are positive and no greater than `INT32_MAX`, throwing `E_INVALIDARG` if not (`winml/lib/Api.Ort/OnnxruntimeDescriptorConverter.cpp`). * In `ConvertSoftwareBitmapToGPUTensor`, changed buffer size calculations to use `UINT64`, added validation for positive tensor dimensions, and ensured the upload size does not exceed `SIZE_T` limits. Also, verified output resource bounds and used safe arithmetic functions to prevent overflows (`winml/lib/Api.Image/VideoFrameToTensorConverter.cpp`). [[1]](diffhunk://#diff-0433ecea41bd9c8ac8df1cc801435820b985a89b7175992149d9846d1640103cL603-R622) [[2]](diffhunk://#diff-0433ecea41bd9c8ac8df1cc801435820b985a89b7175992149d9846d1640103cL636-R653) [[3]](diffhunk://#diff-0433ecea41bd9c8ac8df1cc801435820b985a89b7175992149d9846d1640103cL627-R644) **Test Enhancements:** * Added a new test, `RejectOversizedImageDimensions`, to verify that models with image dimensions exceeding `INT32_MAX` are correctly rejected (`winml/test/api/LearningModelAPITest.cpp`, `LearningModelAPITest.h`). [[1]](diffhunk://#diff-333193022bf7c1f14c1cd007c564615f78d18d7e5131dcaaf381f6a7abadd3faR277-R286) [[2]](diffhunk://#diff-333193022bf7c1f14c1cd007c564615f78d18d7e5131dcaaf381f6a7abadd3faR345) [[3]](diffhunk://#diff-ac423ebe6007e7e5068341e624f86aad1e761b541fe6ff074a48f84e05fcfe9fR22) [[4]](diffhunk://#diff-ac423ebe6007e7e5068341e624f86aad1e761b541fe6ff074a48f84e05fcfe9fR48) * Updated the `CreateModel` utility to support an `image_input` flag, allowing creation of image-typed model inputs for testing validation logic (`winml/test/common/protobufHelpers.cpp`, `protobufHelpers.h`). [[1]](diffhunk://#diff-a950cbbe1cffe6d8654c7b52f4e2a26ba7b47f44e7e51483909d585c68d3f073L212-R212) [[2]](diffhunk://#diff-a950cbbe1cffe6d8654c7b52f4e2a26ba7b47f44e7e51483909d585c68d3f073R295-R297) [[3]](diffhunk://#diff-60f379241dc17a8d28e628bd8b85bff62c0accbfbcf01aaa282bd922a785a37eL24-R26) **General Code Quality:** * Included `<limits>` where needed to support the new validation checks and ensure portability and correctness. [[1]](diffhunk://#diff-0433ecea41bd9c8ac8df1cc801435820b985a89b7175992149d9846d1640103cR10) [[2]](diffhunk://#diff-ea694b0a467611dec54406349f03f521ba2f946422b7bf9c7e2de3b71444c749R7) [[3]](diffhunk://#diff-333193022bf7c1f14c1cd007c564615f78d18d7e5131dcaaf381f6a7abadd3faR8-R9) These changes help prevent invalid image tensor shapes from being processed, improving the reliability and safety of the codebase.
This pull request adds input validation to the `RoiAlign` operator to ensure that all Region of Interest (ROI) coordinate values are finite, improving robustness and error reporting. It also introduces a corresponding unit test to verify this behavior. **Input validation improvements:** * Added a check in `roialign.cc` to ensure all values in the `rois` input tensor are finite (i.e., not NaN or infinity), returning an error if any non-finite value is found. **Testing enhancements:** * Added a new test case `RoiCoordinatesMustBeFinite` in `roialign_test.cc` to verify that the operator fails with an appropriate error message when the `rois` input contains a NaN value. * Included `<limits>` header in `roialign_test.cc` to support the use of `std::numeric_limits` for generating NaN values in tests.
This pull request improves the robustness of the Slice optimization logic in the transpose optimizer by adding an additional shape validation check and a corresponding unit test. The main changes ensure that Slice operations with a `starts` input longer than the input tensor's rank are not incorrectly optimized. **Validation improvements:** * Updated `HandleSlice` in `onnx_transpose_optimization.cc` to add a check that the `starts` tensor's length does not exceed the input tensor's rank, preventing invalid optimizations. **Testing:** * Added a new test `TestSliceDefaultAxesStartsLengthExceedsRankNoOpt` in `transpose_optimizer_test.cc` to verify that Slice nodes with `starts` longer than the input rank are not optimized, ensuring correct behavior.
This pull request improves the robustness and correctness of categorical node folding in the tree ensemble implementation by refactoring the subtree comparison logic to prevent infinite loops caused by cycles, and by adding a test to ensure cycles are properly detected and rejected. **Categorical Node Folding and Cycle Detection Improvements:** * Refactored the `CheckIfSubtreesAreEqual` method in `tree_ensemble_common.h` to use an iterative approach with explicit cycle detection, replacing the previous recursive implementation. This prevents stack overflows and infinite loops when cycles are present in the tree structure. (`onnxruntime/core/providers/cpu/ml/tree_ensemble_common.h`, [[1]](diffhunk://#diff-098f04f4ce424d09171371b1dc5893d29e38c42a0f880639775de9e0fec67241L115-R116) [[2]](diffhunk://#diff-098f04f4ce424d09171371b1dc5893d29e38c42a0f880639775de9e0fec67241L411-R462) * Modified the categorical node folding logic in `AddNodes` to maintain a set of visited categorical nodes and enforce that no node is revisited, raising an error if a cycle is detected. (`onnxruntime/core/providers/cpu/ml/tree_ensemble_common.h`, [onnxruntime/core/providers/cpu/ml/tree_ensemble_common.hR537-R544](diffhunk://#diff-098f04f4ce424d09171371b1dc5893d29e38c42a0f880639775de9e0fec67241R537-R544)) * Added `#include <set>` to support the new cycle detection logic. (`onnxruntime/core/providers/cpu/ml/tree_ensemble_common.h`, [onnxruntime/core/providers/cpu/ml/tree_ensemble_common.hR8](diffhunk://#diff-098f04f4ce424d09171371b1dc5893d29e38c42a0f880639775de9e0fec67241R8)) **Testing:** * Added a new unit test `TreeRegressorRejectsCategoricalCycle` to verify that the implementation correctly detects and rejects cycles in categorical nodes, ensuring the model fails gracefully with an appropriate error message. (`onnxruntime/test/providers/cpu/ml/treeregressor_test.cc`, [onnxruntime/test/providers/cpu/ml/treeregressor_test.ccR768-R789](diffhunk://#diff-08b3495816c68f145657ecff63d7b5f3d56813586ec62f7324c22977e70e336bR768-R789))
This pull request adds robust support for string tensors in the ONNX Runtime Rust bindings. It introduces new error handling for string tensor operations, implements correct extraction and decoding of string data, and refactors the output tensor data handling to accommodate both borrowed and owned data. These changes ensure that string tensors are handled safely and correctly, with clear error reporting for invalid or unsupported operations. **String tensor support and extraction:** * Added new error variants to `OrtError` for string tensor-specific failures, such as data length retrieval, content extraction, invalid offsets, and UTF-8 decoding errors. * Implemented `extract_string_tensor` to correctly extract and decode string tensor data using ONNX Runtime's dedicated APIs. This includes validation of offsets and UTF-8 content, with comprehensive error handling. * Updated the `OrtOutput` enum conversion to use the new string tensor extraction logic, ensuring string tensors are properly decoded and wrapped. **Output tensor data handling:** * Refactored `WithOutputTensor` to use a new `OutputTensorData` enum, allowing it to own decoded string data or borrow numeric data, enabling safe handling of both types. * Modified the `view` method of `WithOutputTensor` to support both borrowed and owned data pointers, ensuring compatibility with the new string tensor logic. * Prevented string tensors from being accessed through the generic `WithOutputTensor<T>` implementation, returning a descriptive error instead. **Testing and validation:** * Added unit tests for string tensor content decoding, including cases for valid strings, empty strings, invalid offsets, and invalid UTF-8 data.
This pull request improves the memory management and safety of async inference execution in the ONNX Runtime C# API, particularly around the `RunAsync` method. The main changes ensure that arrays passed to native code are properly pinned and unpinned, reducing the risk of memory errors, and that resources are disposed of correctly in both success and failure scenarios. The test code is also updated to more rigorously exercise object lifetimes. ### Memory management and resource safety improvements * The `CallbackHost` class now implements `IDisposable` and manages the pinning and unpinning of input and output arrays using `GCHandle`, ensuring that native code receives stable pointers and that handles are freed even on exceptions. [[1]](diffhunk://#diff-622c8600020a433468d4a0e82a320fdc478e0c685c2d92c86c04dac351af4d04L1146-R1150) [[2]](diffhunk://#diff-622c8600020a433468d4a0e82a320fdc478e0c685c2d92c86c04dac351af4d04R1162-R1176) [[3]](diffhunk://#diff-622c8600020a433468d4a0e82a320fdc478e0c685c2d92c86c04dac351af4d04R1188-R1268) * The `OrtCallback` and `RunAsyncInternal` methods are updated to explicitly dispose of the `CallbackHost` and free handles in all code paths, further strengthening resource safety. [[1]](diffhunk://#diff-622c8600020a433468d4a0e82a320fdc478e0c685c2d92c86c04dac351af4d04R1135) [[2]](diffhunk://#diff-622c8600020a433468d4a0e82a320fdc478e0c685c2d92c86c04dac351af4d04R1188-R1268) ### Native interop changes * The native P/Invoke signature for `OrtRunAsync` is updated to pass pointers (`IntPtr`) instead of managed arrays, matching the new pinned memory approach. * Calls to the native API (`OrtRunAsync`) are updated to use the new pointer properties from `CallbackHost` instead of directly passing arrays. ### Test improvements * The async inference test now explicitly creates and nulls out a `RunOptions` instance and forces garbage collection, testing that managed resources are not prematurely collected while native work is outstanding. These changes collectively make async inference safer and more robust, especially in scenarios with concurrent or long-running operations.
## Description Add Branch Target Identification support to MLAS AArch64 assembly when the compiler enables __ARM_FEATURE_BTI_DEFAULT. - emit BTI C landing pads at exported assembly function entries - emit the ELF GNU_PROPERTY_AARCH64_FEATURE_1_BTI note - cover both conventional MLAS assembly macros and generated portable SVE assembly - preserve output for builds without BTI enabled This allows Android consumers to link MLAS objects with -z force-bti without disabling BTI hardening. ## Validation - ONNX Runtime lintrunner passes for all changed files - all 29 MLAS AArch64 assembly files cross-assemble with -mbranch-protection=standard - all 29 resulting objects advertise AArch64 feature: BTI - conventional and generated SVE entries begin with BTI C only when BTI is enabled - all 29 objects link successfully with LLD --fatal-warnings -z force-bti
This pull request introduces a new method to the `PrepackedWeightsForGraph` class to improve how prepacked weights are managed, particularly when saving models. The main change is the addition of a `DiscardAndReplaceWithReferenceIfSaving` method, which either replaces a prepacked weight with a reference or removes it, depending on whether the save mode is enabled. Corresponding updates are made to usage sites and comprehensive unit tests are added to verify the new behavior. **Enhancements to prepacked weights management:** * Added the `DiscardAndReplaceWithReferenceIfSaving` method to the `PrepackedWeightsForGraph` class. This method replaces an existing entry with a reference to another weight when in save mode, or removes it from the container otherwise. [[1]](diffhunk://#diff-beaf819b642665d791baf27f6931969a7ed44776f27939c16660038b2b76e760R111-R121) [[2]](diffhunk://#diff-8f7cf3c97f7f9c93fd52a849ade7a4da4375b05f0debe357a8e52dd2090c8e2cR124-R129) * Updated the call site in `SessionState::PrepackConstantInitializedTensors` to use the new method, ensuring correct handling of prepacked weights during model save operations. **Testing improvements:** * Added new unit tests in `tensorutils_test.cc` to verify the behavior of `DiscardAndReplaceWithReferenceIfSaving` in both save mode and non-save mode, ensuring correct reference replacement and removal of weights as appropriate.
This pull request introduces a small refactor to the Whisper encoder subgraph validation logic and adds a new unit test to improve code clarity and test coverage. The main changes are the extraction of input name validation into a dedicated function and the addition of a test to check for invalid input names. **Refactoring and validation improvements:** * Extracted the input name validation logic from `WhisperEncoderSubgraph::Validate` into a new helper function `ValidateWhisperEncoderInputNames` in `subgraph_whisper_encoder.cc` and declared it in the header file `subgraph_whisper_encoder.h` for improved code reuse and readability. [[1]](diffhunk://#diff-1c7fc3b879a5909572c7a602415544b6b8a37c15efdec3b1b417bdb3dc47a491R17-R24) [[2]](diffhunk://#diff-ca786f63a9d798e56cbc0905436e0c7c675e30d2a88144f77721ecbdff1d329cR13-R14) * Updated `WhisperEncoderSubgraph::Validate` to use the new `ValidateWhisperEncoderInputNames` function, simplifying the validation code and fixing an error in the previous input name check for the decoder input. **Testing improvements:** * Added a new unit test `WhisperEncoderSubgraphTest.ReportsInvalidSecondInputName` in `beam_search_test.cc` to verify that the validation function correctly reports an error when the decoder input name is incorrect. This increases test coverage for the new validation logic. [[1]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R6) [[2]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R16) [[3]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R27-R36)
This pull request improves the robustness and error handling of the TreeEnsemble implementation in ONNX Runtime. It introduces stricter validation for tree structure attributes, adds cycle detection, and expands the test suite to cover more invalid input scenarios. **Validation and error handling improvements:** * Added checks to ensure `tree_roots` only contains valid node indices, preventing out-of-range root references. * Added comprehensive validation for node and leaf indices within the tree traversal, including checks for out-of-range indices and missing required node data. * Introduced cycle and shared node detection by tracking visited nodes during traversal, enforcing that each internal node is only visited once. [[1]](diffhunk://#diff-2b4c3aa0fba01debf9814c19f34ee80759e62f8ea9ce4a28fa05bdd27ebd4e7eR309) [[2]](diffhunk://#diff-2b4c3aa0fba01debf9814c19f34ee80759e62f8ea9ce4a28fa05bdd27ebd4e7eR323-R351) **Test coverage enhancements:** * Added the `RunInvalidTreeStructureTest` helper and new test cases to verify the operator correctly rejects out-of-range roots, out-of-range child nodes, and cycles in the tree structure. [[1]](diffhunk://#diff-9bce50df70fddc092ce6fc5351812c621343f682a15ca178c34e6dd9415e9a39R139-R167) [[2]](diffhunk://#diff-9bce50df70fddc092ce6fc5351812c621343f682a15ca178c34e6dd9415e9a39R389-R400)
This pull request introduces improvements to the handling and testing of string data in the `ScatterND` operator. The main changes include a fix to disable parallelization for string data types and the addition of new test cases to verify correct behavior with duplicate indices and string concatenation. Improvements to string data handling: * Disabled parallel execution (`tp = nullptr`) for `std::string` data types in the `ScatterNDDispatchTarget` struct to prevent potential issues with concurrent string operations. Expanded test coverage: * Added `ScatterND_string_duplicate_indices` test to verify that when duplicate indices are present, the last update value is used for string data. * Added `ScatterND_string_add_duplicate_indices` test to verify that when the reduction attribute is set to `"add"`, updates to the same string index are concatenated in order.
This pull request improves how tensor element types are handled in the ONNX Runtime Rust bindings. It introduces more robust error handling for unsupported and undefined tensor element types, replacing previous unchecked conversions with a safe and explicit conversion mechanism. The most important changes are: **Error Handling Improvements:** * Added a new `OrtError::UnsupportedTensorElementType(i64)` variant to represent unsupported tensor types, and updated the `UndefinedTensorElementType` error message for clarity. **Type Conversion and Safety:** * Implemented `TryFrom<sys::ONNXTensorElementDataType>` for `TensorElementDataType`, returning descriptive errors for undefined or unsupported types instead of using unsafe `transmute`. * Updated code in `dangerous` module to use the new `try_from` conversion for tensor element types, eliminating unsafe casting and ensuring errors are handled explicitly. **Testing:** * Added unit tests to verify that supported types convert successfully, undefined types return the correct error, and unsupported types are properly rejected. **Code Cleanliness:** * Added necessary imports for `TryFrom` in `session.rs` to support the new conversion logic.
This pull request strengthens input validation for attention-related operators and adds corresponding unit tests to ensure that large attribute values do not cause integer overflows. The changes affect the `LinearAttention`, `LongformerAttentionBase`, and GPT subgraph implementations, improving reliability and error reporting. **Input validation improvements:** * [`onnxruntime/contrib_ops/cpu/bert/linear_attention.cc`](diffhunk://#diff-5c4a68eb89a2f94cd428e245a6d7f15a6713138b94ccf04d8d8de61edfce4834L44-R55): Added checks to ensure `q_num_heads` and `kv_num_heads` attributes are within the range `[1, INT_MAX]`, preventing integer overflows during initialization. * [`onnxruntime/contrib_ops/cpu/bert/longformer_attention_base.h`](diffhunk://#diff-1da5f76768826ff4730d0c18c326a4581e3257a889df80237ec0d36b8eccc486L28-R39): Added validation to ensure `num_heads` and `window` attributes are within `[1, INT_MAX]`. * [`onnxruntime/contrib_ops/cpu/transformers/subgraph_gpt.cc`](diffhunk://#diff-1c3c36abe873809b84c5b9b0c4620cdd9acbac51c55cee11b2925440fd62f8b3R176-R182): Added checks to ensure certain tensor dimensions do not exceed `INT_MAX`, preventing unsafe casts. **Testing enhancements:** * [`onnxruntime/test/contrib_ops/linear_attention_op_test.cc`](diffhunk://#diff-3dd3dc17608e4a4603256f802204c16de8f89b61878359375421b659545295a5R1385-R1436): Added new tests to verify that the `LinearAttention` operator correctly rejects out-of-range `q_num_heads` and `kv_num_heads` values, ensuring the new validation logic is exercised. **Code maintenance:** * Included `<limits>` header where necessary to support the new validation checks. [[1]](diffhunk://#diff-5c4a68eb89a2f94cd428e245a6d7f15a6713138b94ccf04d8d8de61edfce4834R12) [[2]](diffhunk://#diff-1da5f76768826ff4730d0c18c326a4581e3257a889df80237ec0d36b8eccc486R7) [[3]](diffhunk://#diff-1c3c36abe873809b84c5b9b0c4620cdd9acbac51c55cee11b2925440fd62f8b3R10) --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
### Description <!-- Describe your changes. --> Ensure CUDA `Abs` returns `+0.0` for both `+0.0` and `-0.0`, with a regression test checking the output sign bits. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> The existing implementation negates `+0.0`, producing `-0.0` instead. This fixes the signed-zero behavior. Fixes microsoft#31162
Previous, the packaging pipeline was split into two (cuda 12.8 and cuda 13.x). This PR updates the test pipelines and split in the same way.
) The output rank validation added in microsoft#31708 wrongly falls back ops whose ONNX output rank differs from the WebNN operand rank: - InstanceNormalization, Conv (conv1d) and MatMul (1-D input) reshape the input to a WebNN-valid rank and reshape the output back. Skip the output rank check exactly when that workaround fires, mirroring the existing input-side skips. Also make the InstanceNormalization input-side check symmetric (only validate the 4D direct path). - Gather with 0-D indices yields a valid 0-D output, but the WebNN gather output rankRange is reported with min=1; permit 0-D output.
This change aligns CUDA architecture selections across the CUDA plugin EP and the related Python, C API, TensorRT, and Node.js packaging pipelines. Linux Python packaging now passes architecture lists separately for x64 and aarch64, while the other Linux packaging scripts use the same per-CUDA-version x64 lists. ## CUDA Architecture Matrix | OS | CUDA | CUDA architectures | | --- | --- | --- | | Windows x64 | 12.8 | 61;75;86;89;120a | | Windows x64 | 13.0 | 75;80;86;89;120a | | Linux x64 | 12.8 | 60;70;75;80;86;89;90a;120a | | Linux x64 | 13.0 | 75;80;86;89;90a;120a | | Linux aarch64 | 13.0 | 89;90a;120a;121a | The `a` suffix denotes the architecture-specific CMake form such as `90-real` or `120-real`; all entries in the pipelines are emitted with the `-real` suffix. CUDA 12.8 has no Linux aarch64 packaging configuration in these pipelines. ## Key Changes - Updated `plugin-cuda-pipeline.yml` as the reference architecture matrix. - Split Python packaging parameters into Windows x64, Linux x64, and Linux aarch64 CUDA architecture settings. - Forwarded the Linux architecture setting through `py-linux-gpu-stage.yml`, `run_python_dockerbuild.sh`, and `build_linux_python_package.sh`. - Aligned C API, TensorRT C API, and Node.js Linux packaging scripts with the plugin Linux x64 settings. - Removed `120-virtual` from the affected packaging configurations so they match the plugin EP configuration. ## Validation - `bash -n` passes for all modified Linux packaging scripts. - Modified pipeline YAML files parse successfully with PyYAML. - `git diff --check` passes.
This pull request improves the robustness of the FastGelu fusion optimization by ensuring malformed nodes are properly skipped and adds a test to verify this behavior. The main changes include stricter input validation in the fusion logic and a new unit test. **Fusion logic improvements:** * Added explicit checks for the number of inputs (`InputDefs().size()`) in `Mul` and `Pow` nodes within the `FastGeluFusion` optimizer to ensure only well-formed nodes are considered for fusion. [[1]](diffhunk://#diff-8f18e5c2ad33a6cc11a340f2c0ff3ce5ad63beed1dcc31ea49f1ff409ef030c9R39) [[2]](diffhunk://#diff-8f18e5c2ad33a6cc11a340f2c0ff3ce5ad63beed1dcc31ea49f1ff409ef030c9L89-R91) [[3]](diffhunk://#diff-8f18e5c2ad33a6cc11a340f2c0ff3ce5ad63beed1dcc31ea49f1ff409ef030c9R119) **Testing enhancements:** * Introduced a new test, `FastGeluFusionSkipsMalformedScaleMul`, that modifies a model to create a malformed `Mul` node and verifies that the fusion optimizer correctly skips it (i.e., does not produce a `FastGelu` node).
This pull request strengthens the validation logic for Group Query Attention (GQA) fusion in ONNX Runtime by adding stricter shape checks for projection tensors and guarding against integer overflows in hidden size calculations. It also introduces a new unit test to ensure that the fusion is skipped when projection tensor shapes do not match the expected configuration. ### Improved validation and error handling: * Added the `ProjectionTensorShapesMatch` function to verify that the Q, K, and V projection tensor shapes match the expected dimensions based on quantization and head attributes. The fusion is skipped if the shapes do not match. [[1]](diffhunk://#diff-0e6d8470d28199117b315d0bbb3324d532555e0861f8fe088565cc1944450208R257-R280) [[2]](diffhunk://#diff-0e6d8470d28199117b315d0bbb3324d532555e0861f8fe088565cc1944450208L484-R532) * Added checks to prevent integer overflows when calculating `q_hidden_size`, `kv_hidden_size`, and `output_hidden_size`, ensuring all head attributes are positive and calculations are safe. * Included `<limits>` for use of `std::numeric_limits` in overflow checks. ### Testing enhancements: * Added a new test, `GroupQueryAttentionFusionSkipsMismatchedProjectionSizesTest`, to verify that fusion is correctly skipped when projection tensor sizes do not match the head attributes. * Updated the GQA fusion test builder to allow specifying the number of GQA heads, supporting more flexible test scenarios. [[1]](diffhunk://#diff-7a86cd9c3c03984228cbb6918b82e9dda0f1de51b505dbc25d323cc413d89028L965-R966) [[2]](diffhunk://#diff-7a86cd9c3c03984228cbb6918b82e9dda0f1de51b505dbc25d323cc413d89028L1050-R1051)
This pull request introduces an element count validation for CUDA `QuantizeLinear` and `DequantizeLinear` operators to ensure they do not process more than `INT32_MAX` elements, preventing potential overflows and undefined behavior. It also adds corresponding unit tests to verify this constraint. **Element Count Validation:** * Added a new function `ValidateQDQElementCount` in `quantize_linear.h` to check that the number of elements does not exceed `INT32_MAX`, returning an error if the limit is exceeded. * Integrated `ValidateQDQElementCount` into both `QuantizeLinear<T, U>::ComputeInternal` and `DequantizeLinear<T, U>::ComputeInternal` to enforce the element count constraint during operator execution. [[1]](diffhunk://#diff-2fc9bd2dcb3b392eb85409ae26189d0a290ba6d8261cbeda4a92f83434b113c2L193-R197) [[2]](diffhunk://#diff-2fc9bd2dcb3b392eb85409ae26189d0a290ba6d8261cbeda4a92f83434b113c2R411) **Testing:** * Added a unit test `CudaElementCountRange` in `quantize_linear_test.cc` to confirm that `ValidateQDQElementCount` accepts the maximum allowed value and rejects values above the limit.
This pull request strengthens shape inference and input validation for the `MatMulFpQ4` operator and adds new unit tests to ensure invalid input shapes are properly rejected. The main focus is on improving error handling and making the code more robust against malformed inputs. **Shape inference and validation improvements:** * Added checks to ensure all relevant input tensors (`A`, `B`, and `B_shape`) have shapes before proceeding with shape inference in `matmulQ4ShapeInference` (`contrib_defs.cc`). * Improved validation for the `B` input to require it to be a 1-D tensor with a known, non-negative size, and for the `B_shape` input to require it to be a 1-D int64 tensor of length 2 (`contrib_defs.cc`). * Added a check to ensure the `B_shape` initializer contains exactly two int64 values before using its data (`contrib_defs.cc`). * Fixed the validation for the packed `B` tensor to correctly check its size against the expected pack size, removing an incorrect logical condition (`contrib_defs.cc`). **Unit test additions:** * Added new negative tests in `matmul_fpq4_test.cc` to verify that invalid input shapes for `B` and `B_shape` are properly rejected, including cases for scalar and short initializers.
This pull request enhances the handling of optional inputs for quantized operators, ensuring correctness when optional zero point inputs are omitted and improving code robustness. The most important changes are grouped below: **Core Functionality Improvements:** * Added a new utility function `SetOptionalInput` in `s8_to_u8.cc` to correctly set optional input slots in a node, updating both input definitions and input argument counts, and handling omitted inputs gracefully. [[1]](diffhunk://#diff-f1ed8d1a3ceb3229c24831a98c511aa862d535a8ba3c486bd7ca47a2244b4fe4R8-R32) [[2]](diffhunk://#diff-b6b2ee740008dfb1616c212a5fe976a30a489e83e7da3f5389d63487025fc0f7R78-R79) * Updated logic in both `TryConvertDynamicQuantizeLSTM` and `ConvertS8WeightToU8` to use `SetOptionalInput` when assigning new initializers for zero point inputs, ensuring proper handling of optional inputs. [[1]](diffhunk://#diff-9082e1a8710cae2bf43315ecca905dc65fb8e13956287506b7f71134c777803bL140-R141) [[2]](diffhunk://#diff-9082e1a8710cae2bf43315ecca905dc65fb8e13956287506b7f71134c777803bL151-R153) [[3]](diffhunk://#diff-f1ed8d1a3ceb3229c24831a98c511aa862d535a8ba3c486bd7ca47a2244b4fe4L51-R77) **Bug Fixes and Robustness:** * Modified checks for zero point input existence to require both non-null and `.Exists()`, preventing errors when optional inputs are omitted. [[1]](diffhunk://#diff-9082e1a8710cae2bf43315ecca905dc65fb8e13956287506b7f71134c777803bL76-R76) [[2]](diffhunk://#diff-9082e1a8710cae2bf43315ecca905dc65fb8e13956287506b7f71134c777803bL88-R90) [[3]](diffhunk://#diff-f1ed8d1a3ceb3229c24831a98c511aa862d535a8ba3c486bd7ca47a2244b4fe4L29-R54) **Testing Improvements:** * Added a new unit test `MatMulIntegerOmittedZeroPoints` to verify that the transformer correctly handles omitted zero point inputs, fills in optional input slots, and generates the expected uint8 initializer.
This pull request introduces stricter validation and error handling for initializers with in-memory external data references in ONNX Runtime's graph handling. The main goal is to ensure that all such references are properly registered and that their data matches expectations, preventing invalid model states and improving robustness. Additionally, new tests are added to verify these behaviors. **Validation and Error Handling Improvements:** * Added a new `ValidateInMemoryInitializers` method to the `Graph` class, which checks that all in-memory external data initializers have corresponding `OrtValue` objects with matching data, and integrated this validation into the graph transformation process. [[1]](diffhunk://#diff-aaea1507ec81a94c72a1fa72ce320df712156b665f7798573be3f7e439bb4c37R1579-R1583) [[2]](diffhunk://#diff-e231a92b40d89409cc8e82436be0a15bc87ef95c93b303b9feaeab6e50c8835cR4000-R4023) [[3]](diffhunk://#diff-3e2227e1225091e8b74c02688e23b21630d1393dd395e15966558901538dd2c7R1549-R1551) * Introduced a helper function `GetValidatedInMemoryInitializer` in `graph_utils.cc` to enforce that in-memory external data initializers are registered and their data matches, replacing ad-hoc checks in various code paths. * Updated `MakeInitializerCopyIfNotExist` and `ConvertInMemoryDataToInline` to use the new validation helper, ensuring consistent and early detection of invalid initializer states. [[1]](diffhunk://#diff-0791c3ebdddb6f4be85d07b707d494551597574eb4f198b0a476d6602c7e2d8bR495-L496) [[2]](diffhunk://#diff-0791c3ebdddb6f4be85d07b707d494551597574eb4f198b0a476d6602c7e2d8bR530) **Testing Enhancements:** * Added the `RejectsUnregisteredInMemoryInitializerCopy` test to verify that the system correctly rejects initializers with arbitrary or unregistered in-memory references, both during validation and when attempting to copy such initializers.
This pull request adds a validation step to the CUDA `GatherElements` kernel to ensure the number of output elements does not exceed the supported limit, preventing potential overflows or runtime errors. It also introduces a corresponding utility function and unit tests for this validation. **CUDA GatherElements Kernel Improvements:** * Added a call to `ValidateGatherElementsElementCount` in `gather_elements.cc` to check that the number of output elements does not exceed `INT32_MAX`, ensuring CUDA kernel compatibility and preventing overflows. * Implemented the inline function `ValidateGatherElementsElementCount` in `gather_elements.h` to encapsulate the element count validation logic. * Included `<limits>` in `gather_elements.h` to support numeric boundary checks. **Testing:** * Added unit tests in `gather_elements_op_test.cc` to verify that `ValidateGatherElementsElementCount` correctly accepts valid counts and rejects counts exceeding `INT32_MAX`.
This pull request strengthens input validation and error handling for subgraph classes in the ONNX Runtime transformers codebase. It introduces additional checks for required input/output shapes and dimensions, ensuring that missing or malformed shapes are detected early with clear error messages. The changes also add corresponding unit tests to verify the new validation logic. **Validation and Error Handling Improvements:** * Added checks to ensure subgraph output names are not empty in `subgraph_base.cc`, and that required tensor shapes (such as logits and past state shapes) are not null before proceeding with parameter extraction or validation. This applies to GPT, T5, and Whisper subgraph classes (`subgraph_base.cc`, `subgraph_gpt.cc`, `subgraph_t5_decoder.cc`, `subgraph_t5_encoder.cc`, `subgraph_whisper_decoder.cc`, `subgraph_whisper_encoder.cc`). [[1]](diffhunk://#diff-36f9a180d24088d6336e1f42bbec6d0fe37210f4953c167f9a34ae1e9882a8d7R65-R66) [[2]](diffhunk://#diff-36f9a180d24088d6336e1f42bbec6d0fe37210f4953c167f9a34ae1e9882a8d7R145-R147) [[3]](diffhunk://#diff-1c3c36abe873809b84c5b9b0c4620cdd9acbac51c55cee11b2925440fd62f8b3R169-R170) [[4]](diffhunk://#diff-97b23192a59f74e96db55e7eef910270f2987285fa4b734285e69e9467569c28R54-R57) [[5]](diffhunk://#diff-97b23192a59f74e96db55e7eef910270f2987285fa4b734285e69e9467569c28R100) [[6]](diffhunk://#diff-97b23192a59f74e96db55e7eef910270f2987285fa4b734285e69e9467569c28R109-R110) [[7]](diffhunk://#diff-bb4a85473116e8ebd8b731818e28cc2bf969f5ca52f7090633c181b104e2f769R105) [[8]](diffhunk://#diff-6161e160fc3b48fb67829162a627991d3d2b9acb662d685123ab56f84d334b00R52) [[9]](diffhunk://#diff-6161e160fc3b48fb67829162a627991d3d2b9acb662d685123ab56f84d334b00R98) [[10]](diffhunk://#diff-6161e160fc3b48fb67829162a627991d3d2b9acb662d685123ab56f84d334b00R108-R109) [[11]](diffhunk://#diff-1c7fc3b879a5909572c7a602415544b6b8a37c15efdec3b1b417bdb3dc47a491L52-R52) [[12]](diffhunk://#diff-1c7fc3b879a5909572c7a602415544b6b8a37c15efdec3b1b417bdb3dc47a491R65) * Improved validation of input dimensions, such as ensuring decoder subgraph input tensors have the expected number of dimensions for T5 and Whisper models (`subgraph_t5_decoder.cc`, `subgraph_whisper_decoder.cc`). [[1]](diffhunk://#diff-97b23192a59f74e96db55e7eef910270f2987285fa4b734285e69e9467569c28R109-R110) [[2]](diffhunk://#diff-6161e160fc3b48fb67829162a627991d3d2b9acb662d685123ab56f84d334b00R108-R109) **Unit Testing:** * Added new unit tests in `beam_search_test.cc` to verify that missing logits or past state shapes are correctly rejected, and that error messages are descriptive. This includes helper code to load test models and manipulate subgraph outputs for testing. [[1]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R6-R9) [[2]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R18) [[3]](diffhunk://#diff-782b3e352d8957cd1226d1b87f2127c58efd5ba7a6472528afbdd6baa8232198R108-R175) **Bug Fixes:** * Fixed an error message in `subgraph_whisper_encoder.cc` to reference the correct input index for `decoder_input_ids`. These changes improve robustness by ensuring that subgraph classes fail early and clearly when required inputs are missing or malformed, and they are now covered by dedicated unit tests.
### Description <!-- Describe your changes. --> Enable WebGPU CI for WebGPU plugin EP release branches (`plugin-ep-webgpu/rel-*`). ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Additional test coverage for release branch changes. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated daily backmerge from ORT main to ovep-develop. No conflicts detected. Do NOT squash or rebase - use merge commit only.