initial commit
This commit is contained in:
huangweifeng
+1
@@ -0,0 +1 @@
|
||||
f217402897f40ebba457e2421bc0a4702771968e
|
||||
Vendored
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) Microsoft Corporation
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
# Privacy
|
||||
|
||||
## Data Collection
|
||||
The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
|
||||
|
||||
***
|
||||
|
||||
### Private Builds
|
||||
No data collection is performed when using your private builds built from source code.
|
||||
|
||||
### Official Builds
|
||||
ONNX Runtime does not maintain any independent telemetry collection mechanisms outside of what is provided by the platforms it supports. However, where applicable, ONNX Runtime will take advantage of platform-supported telemetry systems to collect trace events with the goal of improving product quality.
|
||||
|
||||
Currently telemetry is only implemented for Windows builds and is turned **ON** by default in the official builds distributed in their respective package management repositories ([see here](../README.md#binaries)). This may be expanded to cover other platforms in the future. Data collection is implemented via 'Platform Telemetry' per vendor platform providers (see [telemetry.h](../onnxruntime/core/platform/telemetry.h)).
|
||||
|
||||
#### Technical Details
|
||||
The Windows provider uses the [TraceLogging](https://docs.microsoft.com/en-us/windows/win32/tracelogging/trace-logging-about) API for its implementation. This enables ONNX Runtime trace events to be collected by the operating system, and based on user consent, this data may be periodically sent to Microsoft servers following GDPR and privacy regulations for anonymity and data access controls.
|
||||
|
||||
Windows ML and onnxruntime C APIs allow Trace Logging to be turned on/off (see [API pages](../README.md#api-documentation) for details).
|
||||
For information on how to enable and disable telemetry, see [C API: Telemetry](./C_API.md#telemetry).
|
||||
There are equivalent APIs in the C#, Python, and Java language bindings as well.
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
<p align="center"><img width="50%" src="docs/images/ONNX_Runtime_logo_dark.png" /></p>
|
||||
|
||||
**ONNX Runtime is a cross-platform inference and training machine-learning accelerator**.
|
||||
|
||||
**ONNX Runtime inference** can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. [Learn more →](https://www.onnxruntime.ai/docs/#onnx-runtime-for-inferencing)
|
||||
|
||||
**ONNX Runtime training** can accelerate the model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition for existing PyTorch training scripts. [Learn more →](https://www.onnxruntime.ai/docs/#onnx-runtime-for-training)
|
||||
|
||||
## Get Started & Resources
|
||||
|
||||
* **General Information**: [onnxruntime.ai](https://onnxruntime.ai)
|
||||
|
||||
* **Usage documentation and tutorials**: [onnxruntime.ai/docs](https://onnxruntime.ai/docs)
|
||||
|
||||
* **YouTube video tutorials**: [youtube.com/@ONNXRuntime](https://www.youtube.com/@ONNXRuntime)
|
||||
|
||||
* [**Upcoming Release Roadmap**](https://onnxruntime.ai/roadmap)
|
||||
|
||||
* **Companion sample repositories**:
|
||||
- ONNX Runtime Inferencing: [microsoft/onnxruntime-inference-examples](https://github.com/microsoft/onnxruntime-inference-examples)
|
||||
- ONNX Runtime Training: [microsoft/onnxruntime-training-examples](https://github.com/microsoft/onnxruntime-training-examples)
|
||||
|
||||
## Builtin Pipeline Status
|
||||
|
||||
|System|Inference|Training|
|
||||
|---|---|---|
|
||||
|Windows|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=9)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=218)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=47)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=228)||
|
||||
|Linux|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=11)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=64)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=12)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=45)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=55)|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=86)<br>[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=84)|
|
||||
|Mac|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=13)||
|
||||
|Android|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=53)||
|
||||
|iOS|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=134)||
|
||||
|Web|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=161)||
|
||||
|Other|[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=187&repoName=microsoft%2Fonnxruntime)||
|
||||
|
||||
This project is tested with [BrowserStack](https://www.browserstack.com/home).
|
||||
|
||||
## Third-party Pipeline Status
|
||||
|
||||
|System|Inference|Training|
|
||||
|---|---|---|
|
||||
|Linux|[](https://github.com/Ascend/onnxruntime/actions/workflows/build-and-test.yaml)||
|
||||
|
||||
## Releases
|
||||
|
||||
The current release and past releases can be found here: https://github.com/microsoft/onnxruntime/releases.
|
||||
|
||||
For details on the upcoming release, including release dates, announcements, features, and guidance on submitting feature requests, please visit the release roadmap: https://onnxruntime.ai/roadmap.
|
||||
|
||||
## Data/Telemetry
|
||||
|
||||
Windows distributions of this project may collect usage data and send it to Microsoft to help improve our products and services. See the [privacy statement](docs/Privacy.md) for more details.
|
||||
|
||||
## Contributions and Feedback
|
||||
|
||||
We welcome contributions! Please see the [contribution guidelines](CONTRIBUTING.md).
|
||||
|
||||
For feature requests or bug reports, please file a [GitHub Issue](https://github.com/Microsoft/onnxruntime/issues).
|
||||
|
||||
For general discussion or questions, please use [GitHub Discussions](https://github.com/microsoft/onnxruntime/discussions).
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
|
||||
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
|
||||
or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the [MIT License](LICENSE).
|
||||
+6156
File diff suppressed because it is too large
Load Diff
+1
@@ -0,0 +1 @@
|
||||
1.22.0
|
||||
@@ -0,0 +1,10 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
// CustomOpContext defines an interface allowing a custom op to access ep-specific resources.
|
||||
struct CustomOpContext {
|
||||
CustomOpContext() = default;
|
||||
virtual ~CustomOpContext() {};
|
||||
};
|
||||
@@ -0,0 +1,14 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
enum ResourceOffset {
|
||||
cpu_resource_offset = 0,
|
||||
cuda_resource_offset = 10000,
|
||||
dml_resource_offset = 20000,
|
||||
rocm_resource_offset = 30000,
|
||||
// offsets for other ort eps
|
||||
custom_ep_resource_offset = 10000000,
|
||||
// offsets for customized eps
|
||||
};
|
||||
@@ -0,0 +1,19 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#include "onnxruntime_c_api.h"
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/**
|
||||
* \param use_arena zero: false. non-zero: true.
|
||||
*/
|
||||
ORT_EXPORT
|
||||
ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_CPU, _In_ OrtSessionOptions* options, int use_arena)
|
||||
ORT_ALL_ARGS_NONNULL;
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
+6195
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,535 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <stdint.h>
|
||||
#include <cmath>
|
||||
#include <cstring>
|
||||
#include <limits>
|
||||
|
||||
namespace onnxruntime_float16 {
|
||||
|
||||
namespace detail {
|
||||
|
||||
enum class endian {
|
||||
#if defined(_WIN32)
|
||||
little = 0,
|
||||
big = 1,
|
||||
native = little,
|
||||
#elif defined(__GNUC__) || defined(__clang__)
|
||||
little = __ORDER_LITTLE_ENDIAN__,
|
||||
big = __ORDER_BIG_ENDIAN__,
|
||||
native = __BYTE_ORDER__,
|
||||
#else
|
||||
#error onnxruntime_float16::detail::endian is not implemented in this environment.
|
||||
#endif
|
||||
};
|
||||
|
||||
static_assert(
|
||||
endian::native == endian::little || endian::native == endian::big,
|
||||
"Only little-endian or big-endian native byte orders are supported.");
|
||||
|
||||
} // namespace detail
|
||||
|
||||
/// <summary>
|
||||
/// Shared implementation between public and internal classes. CRTP pattern.
|
||||
/// </summary>
|
||||
template <class Derived>
|
||||
struct Float16Impl {
|
||||
protected:
|
||||
/// <summary>
|
||||
/// Converts from float to uint16_t float16 representation
|
||||
/// </summary>
|
||||
/// <param name="v"></param>
|
||||
/// <returns></returns>
|
||||
constexpr static uint16_t ToUint16Impl(float v) noexcept;
|
||||
|
||||
/// <summary>
|
||||
/// Converts float16 to float
|
||||
/// </summary>
|
||||
/// <returns>float representation of float16 value</returns>
|
||||
float ToFloatImpl() const noexcept;
|
||||
|
||||
/// <summary>
|
||||
/// Creates an instance that represents absolute value.
|
||||
/// </summary>
|
||||
/// <returns>Absolute value</returns>
|
||||
uint16_t AbsImpl() const noexcept {
|
||||
return static_cast<uint16_t>(val & ~kSignMask);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a new instance with the sign flipped.
|
||||
/// </summary>
|
||||
/// <returns>Flipped sign instance</returns>
|
||||
uint16_t NegateImpl() const noexcept {
|
||||
return IsNaN() ? val : static_cast<uint16_t>(val ^ kSignMask);
|
||||
}
|
||||
|
||||
public:
|
||||
// uint16_t special values
|
||||
static constexpr uint16_t kSignMask = 0x8000U;
|
||||
static constexpr uint16_t kBiasedExponentMask = 0x7C00U;
|
||||
static constexpr uint16_t kPositiveInfinityBits = 0x7C00U;
|
||||
static constexpr uint16_t kNegativeInfinityBits = 0xFC00U;
|
||||
static constexpr uint16_t kPositiveQNaNBits = 0x7E00U;
|
||||
static constexpr uint16_t kNegativeQNaNBits = 0xFE00U;
|
||||
static constexpr uint16_t kMaxValueBits = 0x7BFFU; // Largest normal number
|
||||
static constexpr uint16_t kOneBits = 0x3C00U;
|
||||
static constexpr uint16_t kMinusOneBits = 0xBC00U;
|
||||
|
||||
uint16_t val{0};
|
||||
|
||||
Float16Impl() = default;
|
||||
|
||||
/// <summary>
|
||||
/// Checks if the value is negative
|
||||
/// </summary>
|
||||
/// <returns>true if negative</returns>
|
||||
bool IsNegative() const noexcept {
|
||||
return static_cast<int16_t>(val) < 0;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is NaN
|
||||
/// </summary>
|
||||
/// <returns>true if NaN</returns>
|
||||
bool IsNaN() const noexcept {
|
||||
return AbsImpl() > kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is finite
|
||||
/// </summary>
|
||||
/// <returns>true if finite</returns>
|
||||
bool IsFinite() const noexcept {
|
||||
return AbsImpl() < kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value represents positive infinity.
|
||||
/// </summary>
|
||||
/// <returns>true if positive infinity</returns>
|
||||
bool IsPositiveInfinity() const noexcept {
|
||||
return val == kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value represents negative infinity
|
||||
/// </summary>
|
||||
/// <returns>true if negative infinity</returns>
|
||||
bool IsNegativeInfinity() const noexcept {
|
||||
return val == kNegativeInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is either positive or negative infinity.
|
||||
/// </summary>
|
||||
/// <returns>True if absolute value is infinity</returns>
|
||||
bool IsInfinity() const noexcept {
|
||||
return AbsImpl() == kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is NaN or zero. Useful for comparisons.
|
||||
/// </summary>
|
||||
/// <returns>True if NaN or zero.</returns>
|
||||
bool IsNaNOrZero() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs == 0 || abs > kPositiveInfinityBits);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is normal (not zero, subnormal, infinite, or NaN).
|
||||
/// </summary>
|
||||
/// <returns>True if so</returns>
|
||||
bool IsNormal() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs < kPositiveInfinityBits) // is finite
|
||||
&& (abs != 0) // is not zero
|
||||
&& ((abs & kBiasedExponentMask) != 0); // is not subnormal (has a non-zero exponent)
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is subnormal (denormal).
|
||||
/// </summary>
|
||||
/// <returns>True if so</returns>
|
||||
bool IsSubnormal() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs < kPositiveInfinityBits) // is finite
|
||||
&& (abs != 0) // is not zero
|
||||
&& ((abs & kBiasedExponentMask) == 0); // is subnormal (has a zero exponent)
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates an instance that represents absolute value.
|
||||
/// </summary>
|
||||
/// <returns>Absolute value</returns>
|
||||
Derived Abs() const noexcept { return Derived::FromBits(AbsImpl()); }
|
||||
|
||||
/// <summary>
|
||||
/// Creates a new instance with the sign flipped.
|
||||
/// </summary>
|
||||
/// <returns>Flipped sign instance</returns>
|
||||
Derived Negate() const noexcept { return Derived::FromBits(NegateImpl()); }
|
||||
|
||||
/// <summary>
|
||||
/// IEEE defines that positive and negative zero are equal, this gives us a quick equality check
|
||||
/// for two values by or'ing the private bits together and stripping the sign. They are both zero,
|
||||
/// and therefore equivalent, if the resulting value is still zero.
|
||||
/// </summary>
|
||||
/// <param name="lhs">first value</param>
|
||||
/// <param name="rhs">second value</param>
|
||||
/// <returns>True if both arguments represent zero</returns>
|
||||
static bool AreZero(const Float16Impl& lhs, const Float16Impl& rhs) noexcept {
|
||||
return static_cast<uint16_t>((lhs.val | rhs.val) & ~kSignMask) == 0;
|
||||
}
|
||||
|
||||
bool operator==(const Float16Impl& rhs) const noexcept {
|
||||
if (IsNaN() || rhs.IsNaN()) {
|
||||
// IEEE defines that NaN is not equal to anything, including itself.
|
||||
return false;
|
||||
}
|
||||
return val == rhs.val;
|
||||
}
|
||||
|
||||
bool operator!=(const Float16Impl& rhs) const noexcept { return !(*this == rhs); }
|
||||
|
||||
bool operator<(const Float16Impl& rhs) const noexcept {
|
||||
if (IsNaN() || rhs.IsNaN()) {
|
||||
// IEEE defines that NaN is unordered with respect to everything, including itself.
|
||||
return false;
|
||||
}
|
||||
|
||||
const bool left_is_negative = IsNegative();
|
||||
if (left_is_negative != rhs.IsNegative()) {
|
||||
// When the signs of left and right differ, we know that left is less than right if it is
|
||||
// the negative value. The exception to this is if both values are zero, in which case IEEE
|
||||
// says they should be equal, even if the signs differ.
|
||||
return left_is_negative && !AreZero(*this, rhs);
|
||||
}
|
||||
return (val != rhs.val) && ((val < rhs.val) ^ left_is_negative);
|
||||
}
|
||||
};
|
||||
|
||||
// The following Float16_t conversions are based on the code from
|
||||
// Eigen library.
|
||||
|
||||
// The conversion routines are Copyright (c) Fabian Giesen, 2016.
|
||||
// The original license follows:
|
||||
//
|
||||
// Copyright (c) Fabian Giesen, 2016
|
||||
// All rights reserved.
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted.
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
namespace detail {
|
||||
union float32_bits {
|
||||
unsigned int u;
|
||||
float f;
|
||||
};
|
||||
} // namespace detail
|
||||
|
||||
template <class Derived>
|
||||
inline constexpr uint16_t Float16Impl<Derived>::ToUint16Impl(float v) noexcept {
|
||||
detail::float32_bits f{};
|
||||
f.f = v;
|
||||
|
||||
constexpr detail::float32_bits f32infty = {255 << 23};
|
||||
constexpr detail::float32_bits f16max = {(127 + 16) << 23};
|
||||
constexpr detail::float32_bits denorm_magic = {((127 - 15) + (23 - 10) + 1) << 23};
|
||||
constexpr unsigned int sign_mask = 0x80000000u;
|
||||
uint16_t val = static_cast<uint16_t>(0x0u);
|
||||
|
||||
unsigned int sign = f.u & sign_mask;
|
||||
f.u ^= sign;
|
||||
|
||||
// NOTE all the integer compares in this function can be safely
|
||||
// compiled into signed compares since all operands are below
|
||||
// 0x80000000. Important if you want fast straight SSE2 code
|
||||
// (since there's no unsigned PCMPGTD).
|
||||
|
||||
if (f.u >= f16max.u) { // result is Inf or NaN (all exponent bits set)
|
||||
val = (f.u > f32infty.u) ? 0x7e00 : 0x7c00; // NaN->qNaN and Inf->Inf
|
||||
} else { // (De)normalized number or zero
|
||||
if (f.u < (113 << 23)) { // resulting FP16 is subnormal or zero
|
||||
// use a magic value to align our 10 mantissa bits at the bottom of
|
||||
// the float. as long as FP addition is round-to-nearest-even this
|
||||
// just works.
|
||||
f.f += denorm_magic.f;
|
||||
|
||||
// and one integer subtract of the bias later, we have our final float!
|
||||
val = static_cast<uint16_t>(f.u - denorm_magic.u);
|
||||
} else {
|
||||
unsigned int mant_odd = (f.u >> 13) & 1; // resulting mantissa is odd
|
||||
|
||||
// update exponent, rounding bias part 1
|
||||
// Equivalent to `f.u += ((unsigned int)(15 - 127) << 23) + 0xfff`, but
|
||||
// without arithmetic overflow.
|
||||
f.u += 0xc8000fffU;
|
||||
// rounding bias part 2
|
||||
f.u += mant_odd;
|
||||
// take the bits!
|
||||
val = static_cast<uint16_t>(f.u >> 13);
|
||||
}
|
||||
}
|
||||
|
||||
val |= static_cast<uint16_t>(sign >> 16);
|
||||
return val;
|
||||
}
|
||||
|
||||
template <class Derived>
|
||||
inline float Float16Impl<Derived>::ToFloatImpl() const noexcept {
|
||||
constexpr detail::float32_bits magic = {113 << 23};
|
||||
constexpr unsigned int shifted_exp = 0x7c00 << 13; // exponent mask after shift
|
||||
detail::float32_bits o{};
|
||||
|
||||
o.u = (val & 0x7fff) << 13; // exponent/mantissa bits
|
||||
unsigned int exp = shifted_exp & o.u; // just the exponent
|
||||
o.u += (127 - 15) << 23; // exponent adjust
|
||||
|
||||
// handle exponent special cases
|
||||
if (exp == shifted_exp) { // Inf/NaN?
|
||||
o.u += (128 - 16) << 23; // extra exp adjust
|
||||
} else if (exp == 0) { // Zero/Denormal?
|
||||
o.u += 1 << 23; // extra exp adjust
|
||||
o.f -= magic.f; // re-normalize
|
||||
}
|
||||
|
||||
// Attempt to workaround the Internal Compiler Error on ARM64
|
||||
// for bitwise | operator, including std::bitset
|
||||
#if (defined _MSC_VER) && (defined _M_ARM || defined _M_ARM64 || defined _M_ARM64EC)
|
||||
if (IsNegative()) {
|
||||
return -o.f;
|
||||
}
|
||||
#else
|
||||
// original code:
|
||||
o.u |= (val & 0x8000U) << 16U; // sign bit
|
||||
#endif
|
||||
return o.f;
|
||||
}
|
||||
|
||||
/// Shared implementation between public and internal classes. CRTP pattern.
|
||||
template <class Derived>
|
||||
struct BFloat16Impl {
|
||||
protected:
|
||||
/// <summary>
|
||||
/// Converts from float to uint16_t float16 representation
|
||||
/// </summary>
|
||||
/// <param name="v"></param>
|
||||
/// <returns></returns>
|
||||
static uint16_t ToUint16Impl(float v) noexcept;
|
||||
|
||||
/// <summary>
|
||||
/// Converts bfloat16 to float
|
||||
/// </summary>
|
||||
/// <returns>float representation of bfloat16 value</returns>
|
||||
float ToFloatImpl() const noexcept;
|
||||
|
||||
/// <summary>
|
||||
/// Creates an instance that represents absolute value.
|
||||
/// </summary>
|
||||
/// <returns>Absolute value</returns>
|
||||
uint16_t AbsImpl() const noexcept {
|
||||
return static_cast<uint16_t>(val & ~kSignMask);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a new instance with the sign flipped.
|
||||
/// </summary>
|
||||
/// <returns>Flipped sign instance</returns>
|
||||
uint16_t NegateImpl() const noexcept {
|
||||
return IsNaN() ? val : static_cast<uint16_t>(val ^ kSignMask);
|
||||
}
|
||||
|
||||
public:
|
||||
// uint16_t special values
|
||||
static constexpr uint16_t kSignMask = 0x8000U;
|
||||
static constexpr uint16_t kBiasedExponentMask = 0x7F80U;
|
||||
static constexpr uint16_t kPositiveInfinityBits = 0x7F80U;
|
||||
static constexpr uint16_t kNegativeInfinityBits = 0xFF80U;
|
||||
static constexpr uint16_t kPositiveQNaNBits = 0x7FC1U;
|
||||
static constexpr uint16_t kNegativeQNaNBits = 0xFFC1U;
|
||||
static constexpr uint16_t kMaxValueBits = 0x7F7FU;
|
||||
static constexpr uint16_t kRoundToNearest = 0x7FFFU;
|
||||
static constexpr uint16_t kOneBits = 0x3F80U;
|
||||
static constexpr uint16_t kMinusOneBits = 0xBF80U;
|
||||
|
||||
uint16_t val{0};
|
||||
|
||||
BFloat16Impl() = default;
|
||||
|
||||
/// <summary>
|
||||
/// Checks if the value is negative
|
||||
/// </summary>
|
||||
/// <returns>true if negative</returns>
|
||||
bool IsNegative() const noexcept {
|
||||
return static_cast<int16_t>(val) < 0;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is NaN
|
||||
/// </summary>
|
||||
/// <returns>true if NaN</returns>
|
||||
bool IsNaN() const noexcept {
|
||||
return AbsImpl() > kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is finite
|
||||
/// </summary>
|
||||
/// <returns>true if finite</returns>
|
||||
bool IsFinite() const noexcept {
|
||||
return AbsImpl() < kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value represents positive infinity.
|
||||
/// </summary>
|
||||
/// <returns>true if positive infinity</returns>
|
||||
bool IsPositiveInfinity() const noexcept {
|
||||
return val == kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value represents negative infinity
|
||||
/// </summary>
|
||||
/// <returns>true if negative infinity</returns>
|
||||
bool IsNegativeInfinity() const noexcept {
|
||||
return val == kNegativeInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is either positive or negative infinity.
|
||||
/// </summary>
|
||||
/// <returns>True if absolute value is infinity</returns>
|
||||
bool IsInfinity() const noexcept {
|
||||
return AbsImpl() == kPositiveInfinityBits;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is NaN or zero. Useful for comparisons.
|
||||
/// </summary>
|
||||
/// <returns>True if NaN or zero.</returns>
|
||||
bool IsNaNOrZero() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs == 0 || abs > kPositiveInfinityBits);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is normal (not zero, subnormal, infinite, or NaN).
|
||||
/// </summary>
|
||||
/// <returns>True if so</returns>
|
||||
bool IsNormal() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs < kPositiveInfinityBits) // is finite
|
||||
&& (abs != 0) // is not zero
|
||||
&& ((abs & kBiasedExponentMask) != 0); // is not subnormal (has a non-zero exponent)
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tests if the value is subnormal (denormal).
|
||||
/// </summary>
|
||||
/// <returns>True if so</returns>
|
||||
bool IsSubnormal() const noexcept {
|
||||
auto abs = AbsImpl();
|
||||
return (abs < kPositiveInfinityBits) // is finite
|
||||
&& (abs != 0) // is not zero
|
||||
&& ((abs & kBiasedExponentMask) == 0); // is subnormal (has a zero exponent)
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates an instance that represents absolute value.
|
||||
/// </summary>
|
||||
/// <returns>Absolute value</returns>
|
||||
Derived Abs() const noexcept { return Derived::FromBits(AbsImpl()); }
|
||||
|
||||
/// <summary>
|
||||
/// Creates a new instance with the sign flipped.
|
||||
/// </summary>
|
||||
/// <returns>Flipped sign instance</returns>
|
||||
Derived Negate() const noexcept { return Derived::FromBits(NegateImpl()); }
|
||||
|
||||
/// <summary>
|
||||
/// IEEE defines that positive and negative zero are equal, this gives us a quick equality check
|
||||
/// for two values by or'ing the private bits together and stripping the sign. They are both zero,
|
||||
/// and therefore equivalent, if the resulting value is still zero.
|
||||
/// </summary>
|
||||
/// <param name="lhs">first value</param>
|
||||
/// <param name="rhs">second value</param>
|
||||
/// <returns>True if both arguments represent zero</returns>
|
||||
static bool AreZero(const BFloat16Impl& lhs, const BFloat16Impl& rhs) noexcept {
|
||||
// IEEE defines that positive and negative zero are equal, this gives us a quick equality check
|
||||
// for two values by or'ing the private bits together and stripping the sign. They are both zero,
|
||||
// and therefore equivalent, if the resulting value is still zero.
|
||||
return static_cast<uint16_t>((lhs.val | rhs.val) & ~kSignMask) == 0;
|
||||
}
|
||||
};
|
||||
|
||||
template <class Derived>
|
||||
inline uint16_t BFloat16Impl<Derived>::ToUint16Impl(float v) noexcept {
|
||||
uint16_t result;
|
||||
if (std::isnan(v)) {
|
||||
result = kPositiveQNaNBits;
|
||||
} else {
|
||||
auto get_msb_half = [](float fl) {
|
||||
uint16_t result;
|
||||
#ifdef __cpp_if_constexpr
|
||||
if constexpr (detail::endian::native == detail::endian::little) {
|
||||
#else
|
||||
if (detail::endian::native == detail::endian::little) {
|
||||
#endif
|
||||
std::memcpy(&result, reinterpret_cast<char*>(&fl) + sizeof(uint16_t), sizeof(uint16_t));
|
||||
} else {
|
||||
std::memcpy(&result, &fl, sizeof(uint16_t));
|
||||
}
|
||||
return result;
|
||||
};
|
||||
|
||||
uint16_t upper_bits = get_msb_half(v);
|
||||
union {
|
||||
uint32_t U32;
|
||||
float F32;
|
||||
};
|
||||
F32 = v;
|
||||
U32 += (upper_bits & 1) + kRoundToNearest;
|
||||
result = get_msb_half(F32);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
template <class Derived>
|
||||
inline float BFloat16Impl<Derived>::ToFloatImpl() const noexcept {
|
||||
if (IsNaN()) {
|
||||
return std::numeric_limits<float>::quiet_NaN();
|
||||
}
|
||||
float result;
|
||||
char* const first = reinterpret_cast<char*>(&result);
|
||||
char* const second = first + sizeof(uint16_t);
|
||||
#ifdef __cpp_if_constexpr
|
||||
if constexpr (detail::endian::native == detail::endian::little) {
|
||||
#else
|
||||
if (detail::endian::native == detail::endian::little) {
|
||||
#endif
|
||||
std::memset(first, 0, sizeof(uint16_t));
|
||||
std::memcpy(second, &val, sizeof(uint16_t));
|
||||
} else {
|
||||
std::memcpy(first, &val, sizeof(uint16_t));
|
||||
std::memset(second, 0, sizeof(uint16_t));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
} // namespace onnxruntime_float16
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,54 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
/*
|
||||
* This file defines RunOptions Config Keys and format of the Config Values.
|
||||
*
|
||||
* The Naming Convention for a RunOptions Config Key,
|
||||
* "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
|
||||
* Such as "ep.cuda.use_arena"
|
||||
* The Config Key cannot be empty
|
||||
* The maximum length of the Config Key is 128
|
||||
*
|
||||
* The string format of a RunOptions Config Value is defined individually for each Config.
|
||||
* The maximum length of the Config Value is 1024
|
||||
*/
|
||||
|
||||
// Key for enabling shrinkages of user listed device memory arenas.
|
||||
// Expects a list of semi-colon separated key value pairs separated by colon in the following format:
|
||||
// "device_0:device_id_0;device_1:device_id_1"
|
||||
// No white-spaces allowed in the provided list string.
|
||||
// Currently, the only supported devices are : "cpu", "gpu" (case sensitive).
|
||||
// If "cpu" is included in the list, DisableCpuMemArena() API must not be called (i.e.) arena for cpu should be enabled.
|
||||
// Example usage: "cpu:0;gpu:0" (or) "gpu:0"
|
||||
// By default, the value for this key is empty (i.e.) no memory arenas are shrunk
|
||||
static const char* const kOrtRunOptionsConfigEnableMemoryArenaShrinkage = "memory.enable_memory_arena_shrinkage";
|
||||
|
||||
// Set to '1' to not synchronize execution providers with CPU at the end of session run.
|
||||
// Per default it will be set to '0'
|
||||
// Taking CUDA EP as an example, it omit triggering cudaStreamSynchronize on the compute stream.
|
||||
static const char* const kOrtRunOptionsConfigDisableSynchronizeExecutionProviders = "disable_synchronize_execution_providers";
|
||||
|
||||
// Set HTP performance mode for QNN HTP backend before session run.
|
||||
// options for HTP performance mode: "burst", "balanced", "default", "high_performance",
|
||||
// "high_power_saver", "low_balanced", "extreme_power_saver", "low_power_saver", "power_saver",
|
||||
// "sustained_high_performance". Default to "default".
|
||||
static const char* const kOrtRunOptionsConfigQnnPerfMode = "qnn.htp_perf_mode";
|
||||
|
||||
// Set HTP performance mode for QNN HTP backend post session run.
|
||||
static const char* const kOrtRunOptionsConfigQnnPerfModePostRun = "qnn.htp_perf_mode_post_run";
|
||||
|
||||
// Set RPC control latency for QNN HTP backend
|
||||
static const char* const kOrtRunOptionsConfigQnnRpcControlLatency = "qnn.rpc_control_latency";
|
||||
|
||||
// Set QNN Lora Config File for apply Lora in QNN context binary
|
||||
static const char* const kOrtRunOptionsConfigQnnLoraConfig = "qnn.lora_config";
|
||||
|
||||
// Set graph annotation id for CUDA EP. Use with enable_cuda_graph=true.
|
||||
// The value should be an integer. If the value is not set, the default value is 0 and
|
||||
// ORT session only captures one cuda graph before another capture is requested.
|
||||
// If the value is set to -1, cuda graph capture/replay is disabled in that run.
|
||||
// User are not expected to set the value to 0 as it is reserved for internal use.
|
||||
static const char* const kOrtRunOptionsConfigCudaGraphAnnotation = "gpu_graph_id";
|
||||
+366
@@ -0,0 +1,366 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
/*
|
||||
* This file defines SessionOptions Config Keys and format of the Config Values.
|
||||
*
|
||||
* The Naming Convention for a SessionOptions Config Key,
|
||||
* "[Area][.[SubArea1].[SubArea2]...].[Keyname]"
|
||||
* Such as "ep.cuda.use_arena"
|
||||
* The Config Key cannot be empty
|
||||
* The maximum length of the Config Key is 1024
|
||||
*
|
||||
* The string format of a SessionOptions Config Value is defined individually for each Config.
|
||||
* The maximum length of the Config Value is 2048
|
||||
*/
|
||||
|
||||
// Key for disable PrePacking,
|
||||
// If the config value is set to "1" then the prepacking is disabled, otherwise prepacking is enabled (default value)
|
||||
static const char* const kOrtSessionOptionsConfigDisablePrepacking = "session.disable_prepacking";
|
||||
|
||||
// A value of "1" means allocators registered in the env will be used. "0" means the allocators created in the session
|
||||
// will be used. Use this to override the usage of env allocators on a per session level.
|
||||
static const char* const kOrtSessionOptionsConfigUseEnvAllocators = "session.use_env_allocators";
|
||||
|
||||
// Set to 'ORT' (case sensitive) to load an ORT format model.
|
||||
// If unset, model type will default to ONNX unless inferred from filename ('.ort' == ORT format) or bytes to be ORT
|
||||
static const char* const kOrtSessionOptionsConfigLoadModelFormat = "session.load_model_format";
|
||||
|
||||
// Set to 'ORT' (case sensitive) to save optimized model in ORT format when SessionOptions.optimized_model_path is set.
|
||||
// If unset, format will default to ONNX unless optimized_model_filepath ends in '.ort'.
|
||||
static const char* const kOrtSessionOptionsConfigSaveModelFormat = "session.save_model_format";
|
||||
|
||||
// If a value is "1", flush-to-zero and denormal-as-zero are applied. The default is "0".
|
||||
// When multiple sessions are created, a main thread doesn't override changes from succeeding session options,
|
||||
// but threads in session thread pools follow option changes.
|
||||
// When ORT runs with OpenMP, the same rule is applied, i.e. the first session option to flush-to-zero and
|
||||
// denormal-as-zero is only applied to global OpenMP thread pool, which doesn't support per-session thread pool.
|
||||
// Note that an alternative way not using this option at runtime is to train and export a model without denormals
|
||||
// and that's recommended because turning this option on may hurt model accuracy.
|
||||
static const char* const kOrtSessionOptionsConfigSetDenormalAsZero = "session.set_denormal_as_zero";
|
||||
|
||||
// It controls to run quantization model in QDQ (QuantizelinearDeQuantizelinear) format or not.
|
||||
// "0": enable. ORT does fusion logic for QDQ format.
|
||||
// "1": disable. ORT doesn't do fusion logic for QDQ format.
|
||||
// Its default value is "0" unless the DirectML execution provider is registered, in which case it defaults to "1".
|
||||
static const char* const kOrtSessionOptionsDisableQuantQDQ = "session.disable_quant_qdq";
|
||||
|
||||
// It controls whether to enable Double QDQ remover and Identical Children Consolidation
|
||||
// "0": not to disable. ORT does remove the middle 2 Nodes from a Q->(QD->Q)->QD pairs
|
||||
// "1": disable. ORT doesn't remove the middle 2 Nodes from a Q->(QD->Q)->QD pairs
|
||||
// Its default value is "0"
|
||||
static const char* const kOrtSessionOptionsDisableDoubleQDQRemover = "session.disable_double_qdq_remover";
|
||||
|
||||
// If set to "1", enables the removal of QuantizeLinear/DequantizeLinear node pairs once all QDQ handling has been
|
||||
// completed. e.g. If after all QDQ handling has completed and we have -> FloatOp -> Q -> DQ -> FloatOp -> the
|
||||
// Q -> DQ could potentially be removed. This will provide a performance benefit by avoiding going from float to
|
||||
// 8-bit and back to float, but could impact accuracy. The impact on accuracy will be model specific and depend on
|
||||
// other factors like whether the model was created using Quantization Aware Training or Post Training Quantization.
|
||||
// As such, it's best to test to determine if enabling this works well for your scenario.
|
||||
// The default value is "0"
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsEnableQuantQDQCleanup = "session.enable_quant_qdq_cleanup";
|
||||
|
||||
// Enable or disable gelu approximation in graph optimization. "0": disable; "1": enable. The default is "0".
|
||||
// GeluApproximation has side effects which may change the inference results. It is disabled by default due to this.
|
||||
static const char* const kOrtSessionOptionsEnableGeluApproximation = "optimization.enable_gelu_approximation";
|
||||
|
||||
// This setting controls whether to enable AheadOfTime function inlining.
|
||||
// AOT function inlining examines the graph and attempts to inline as many locally defined functions in the model
|
||||
// as possible with the help of enabled execution providers.
|
||||
// This can reduce the number of function calls and improve performance because it is done before
|
||||
// Level1 optimizers and constant folding. However, under some circumstances, when the EPs are not available,
|
||||
// one can disable the AOT inlining, produce an optimized model and postpone AOT until run time.
|
||||
// "0": enable; "1": disable.
|
||||
// Its default value is "0".
|
||||
static const char* const kOrtSessionOptionsDisableAheadOfTimeFunctionInlining = "session.disable_aot_function_inlining";
|
||||
|
||||
#ifdef ENABLE_TRAINING
|
||||
// Specifies a path of the file containing a list of memory optimization configurations.
|
||||
// The value should be a string indicating the file path of the config file.
|
||||
// The content of the config file is a JSON struct like this:
|
||||
// [
|
||||
// "Gelu+Cast+:1:0",
|
||||
// "Dropout+:1:1"
|
||||
// ]
|
||||
// Taking the example of "Gelu+Cast+:1:0",
|
||||
// > "Gelu+Cast+" is the subgraph string, a valid "subgraph string" should be one subgraph representation
|
||||
// output by ORT graph transformations.
|
||||
// > "1" is "optimization strategy", valid values: 0 - disabled, 1 - recompute.
|
||||
// > "0" is "number of subgraph to apply" which is used to control how many subgraphs to apply optimization,
|
||||
// to avoid "oversaving" the memory.
|
||||
static const char* const kOrtSessionOptionsMemoryOptimizerApplyConfig = "optimization.memory_optimizer_config";
|
||||
|
||||
// Specifies the config for detecting subgraphs for memory footprint reduction.
|
||||
// The value should be a string contains int separated using commas. The default value is "0:0".
|
||||
static const char* const kOrtSessionOptionsMemoryOptimizerProbeConfig = "optimization.enable_memory_probe_recompute_config";
|
||||
#endif
|
||||
|
||||
// This setting if set should contain a comma separated list of optimizers names that should be disabled.
|
||||
// Optimizers may take time to execute and affect model loading time. If you feel that a specific optimizer
|
||||
// does not provider runtime benefits, but affects your model loading time you may disable it using this config
|
||||
// entry. This option is not enabled in ORT_MINIMAL_BUILD build.
|
||||
// A list of optimizes is available in onnxruntime/core/optimizer/graph_transformer_utils.cc
|
||||
//
|
||||
// Default is an empty string which means no optimizers are disabled.
|
||||
static const char* const kOrtSessionOptionsDisableSpecifiedOptimizers = "optimization.disable_specified_optimizers";
|
||||
|
||||
// Enable or disable using device allocator for allocating initialized tensor memory. "1": enable; "0": disable. The default is "0".
|
||||
// Using device allocators means the memory allocation is made using malloc/new.
|
||||
static const char* const kOrtSessionOptionsUseDeviceAllocatorForInitializers = "session.use_device_allocator_for_initializers";
|
||||
|
||||
// Configure whether to allow the inter_op/intra_op threads spinning a number of times before blocking
|
||||
// "0": thread will block if found no job to run
|
||||
// "1": default, thread will spin a number of times before blocking
|
||||
static const char* const kOrtSessionOptionsConfigAllowInterOpSpinning = "session.inter_op.allow_spinning";
|
||||
static const char* const kOrtSessionOptionsConfigAllowIntraOpSpinning = "session.intra_op.allow_spinning";
|
||||
|
||||
// Key for using model bytes directly for ORT format
|
||||
// If a session is created using an input byte array contains the ORT format model data,
|
||||
// By default we will copy the model bytes at the time of session creation to ensure the model bytes
|
||||
// buffer is valid.
|
||||
// Setting this option to "1" will disable copy the model bytes, and use the model bytes directly. The caller
|
||||
// has to guarantee that the model bytes are valid until the ORT session using the model bytes is destroyed.
|
||||
static const char* const kOrtSessionOptionsConfigUseORTModelBytesDirectly = "session.use_ort_model_bytes_directly";
|
||||
|
||||
/// <summary>
|
||||
/// Key for using the ORT format model flatbuffer bytes directly for initializers.
|
||||
/// This avoids copying the bytes and reduces peak memory usage during model loading and initialization.
|
||||
/// Requires `session.use_ort_model_bytes_directly` to be true.
|
||||
/// If set, the flatbuffer bytes provided when creating the InferenceSession MUST remain valid for the entire
|
||||
/// duration of the InferenceSession.
|
||||
/// </summary>
|
||||
static const char* const kOrtSessionOptionsConfigUseORTModelBytesForInitializers =
|
||||
"session.use_ort_model_bytes_for_initializers";
|
||||
|
||||
// This should only be specified when exporting an ORT format model for use on a different platform.
|
||||
// If the ORT format model will be used on ARM platforms set to "1". For other platforms set to "0"
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsQDQIsInt8Allowed = "session.qdqisint8allowed";
|
||||
|
||||
// x64 SSE4.1/AVX2/AVX512(with no VNNI) has overflow problem with quantizied matrix multiplication with U8S8.
|
||||
// To avoid this we need to use slower U8U8 matrix multiplication instead. This option, if
|
||||
// turned on, use slower U8U8 matrix multiplications. Only effective with AVX2 or AVX512
|
||||
// platforms.
|
||||
static const char* const kOrtSessionOptionsAvx2PrecisionMode = "session.x64quantprecision";
|
||||
|
||||
// Specifies how minimal build graph optimizations are handled in a full build.
|
||||
// These optimizations are at the extended level or higher.
|
||||
// Possible values and their effects are:
|
||||
// "save": Save runtime optimizations when saving an ORT format model.
|
||||
// "apply": Only apply optimizations available in a minimal build.
|
||||
// ""/<unspecified>: Apply optimizations available in a full build.
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsConfigMinimalBuildOptimizations =
|
||||
"optimization.minimal_build_optimizations";
|
||||
|
||||
// Note: The options specific to an EP should be specified prior to appending that EP to the session options object in
|
||||
// order for them to take effect.
|
||||
|
||||
// Specifies a list of stop op types. Nodes of a type in the stop op types and nodes downstream from them will not be
|
||||
// run by the NNAPI EP.
|
||||
// The value should be a ","-delimited list of op types. For example, "Add,Sub".
|
||||
// If not specified, the default set of stop ops is used. To specify an empty stop ops types list and disable stop op
|
||||
// exclusion, set the value to "".
|
||||
static const char* const kOrtSessionOptionsConfigNnapiEpPartitioningStopOps = "ep.nnapi.partitioning_stop_ops";
|
||||
|
||||
// Enabling dynamic block-sizing for multithreading.
|
||||
// With a positive value, thread pool will split a task of N iterations to blocks of size starting from:
|
||||
// N / (num_of_threads * dynamic_block_base)
|
||||
// As execution progresses, the size will decrease according to the diminishing residual of N,
|
||||
// meaning the task will be distributed in smaller granularity for better parallelism.
|
||||
// For some models, it helps to reduce the variance of E2E inference latency and boost performance.
|
||||
// The feature will not function by default, specify any positive integer, e.g. "4", to enable it.
|
||||
// Available since version 1.11.
|
||||
static const char* const kOrtSessionOptionsConfigDynamicBlockBase = "session.dynamic_block_base";
|
||||
|
||||
// This option allows to decrease CPU usage between infrequent
|
||||
// requests and forces any TP threads spinning stop immediately when the last of
|
||||
// concurrent Run() call returns.
|
||||
// Spinning is restarted on the next Run() call.
|
||||
// Applies only to internal thread-pools
|
||||
static const char* const kOrtSessionOptionsConfigForceSpinningStop = "session.force_spinning_stop";
|
||||
|
||||
// "1": all inconsistencies encountered during shape and type inference
|
||||
// will result in failures.
|
||||
// "0": in some cases warnings will be logged but processing will continue. The default.
|
||||
// May be useful to expose bugs in models.
|
||||
static const char* const kOrtSessionOptionsConfigStrictShapeTypeInference = "session.strict_shape_type_inference";
|
||||
|
||||
// "1": every model using a more recent opset than the latest released one will fail
|
||||
// "0": the model may or may not work if onnxruntime cannot find an implementation, this option
|
||||
// is used for development purpose.
|
||||
static const char* const kOrtSessionOptionsConfigStrictAllowReleasedOpsetsOnly = "session.allow_released_opsets_only";
|
||||
|
||||
// The file saves configuration for partitioning node among logic streams
|
||||
static const char* const kNodePartitionConfigFile = "session.node_partition_config_file";
|
||||
|
||||
// This Option allows setting affinities for intra op threads.
|
||||
// Affinity string follows format:
|
||||
// logical_processor_id,logical_processor_id;logical_processor_id,logical_processor_id
|
||||
// Semicolon isolates configurations among threads, while comma split processors where ith thread expected to attach to.
|
||||
// e.g.1,2,3;4,5
|
||||
// specifies affinities for two threads, with the 1st thread attach to the 1st, 2nd, and 3rd processor, and 2nd thread to the 4th and 5th.
|
||||
// To ease the configuration, an "interval" is also allowed:
|
||||
// e.g. 1-8;8-16;17-24
|
||||
// orders that the 1st thread runs on first eight processors, 2nd thread runs on next eight processors, and so forth.
|
||||
// Note:
|
||||
// 1. Once set, the number of thread affinities must equal to intra_op_num_threads - 1, since ort does not set affinity on the main thread which
|
||||
// is started and managed by the calling app;
|
||||
// 2. For windows, ort will infer the group id from a logical processor id, for example, assuming there are two groups with each has 64 logical processors,
|
||||
// an id of 64 will be inferred as the last processor of the 1st group, while 65 will be interpreted as the 1st processor of the second group.
|
||||
// Hence 64-65 is an invalid configuration, because a windows thread cannot be attached to processors across group boundary.
|
||||
static const char* const kOrtSessionOptionsConfigIntraOpThreadAffinities = "session.intra_op_thread_affinities";
|
||||
|
||||
// This option will dump out the model to assist debugging any issues with layout transformation,
|
||||
// and is primarily intended for developer usage. It is only relevant if an execution provider that requests
|
||||
// NHWC layout is enabled such as NNAPI, XNNPACK or QNN.
|
||||
//
|
||||
// Default is off. Set to "1" to enable.
|
||||
//
|
||||
// If modified by layout transformation the model will be dumped after these steps:
|
||||
// 1) insertion of the layout transformation Transpose nodes
|
||||
// 2) after those are optimized using the transpose optimizer,
|
||||
// 3) after the L1 transformers are applied to the updated graph.
|
||||
// The model will be saved to filename post_layout_transform_step_<step_number>.onnx.
|
||||
static const char* const kDebugLayoutTransformation = "session.debug_layout_transformation";
|
||||
|
||||
// Graph nodes that are not supported by the execution providers (EPs) explicitly added to the session are
|
||||
// assigned (i.e., "fallback") to the CPU EP by default.
|
||||
//
|
||||
// This option allows the user to disable the fallback of unsupported graph nodes to the CPU EP.
|
||||
// If this option is set to "1", session creation will fail if the execution providers other than the CPU EP cannot
|
||||
// fully support all of the nodes in the graph.
|
||||
//
|
||||
// It is invalid to set this option and explicitly add the CPU EP to the session. In this case, session creation
|
||||
// will also fail with an error.
|
||||
//
|
||||
// Option values:
|
||||
// - "0": CPU EP fallback is not disabled. [DEFAULT]
|
||||
// - "1": CPU EP fallback is disabled.
|
||||
static const char* const kOrtSessionOptionsDisableCPUEPFallback = "session.disable_cpu_ep_fallback";
|
||||
|
||||
// Use this config when serializing a large model after optimization to specify an external initializers file
|
||||
static const char* const kOrtSessionOptionsOptimizedModelExternalInitializersFileName =
|
||||
"session.optimized_model_external_initializers_file_name";
|
||||
|
||||
// Use this config to control the minimum size of the initializer when externalizing it during serialization
|
||||
static const char* const kOrtSessionOptionsOptimizedModelExternalInitializersMinSizeInBytes =
|
||||
"session.optimized_model_external_initializers_min_size_in_bytes";
|
||||
|
||||
// When loading model from memory buffer and the model has external initializers
|
||||
// Use this config to set the external data file folder path
|
||||
// All external data files should be in the same folder
|
||||
static const char* const kOrtSessionOptionsModelExternalInitializersFileFolderPath =
|
||||
"session.model_external_initializers_file_folder_path";
|
||||
|
||||
// Use this config when saving pre-packed constant initializers to an external data file.
|
||||
// This allows you to memory map pre-packed initializers on model load and leave it to
|
||||
// to the OS the amount of memory consumed by the pre-packed initializers. Otherwise,
|
||||
// pre-packed data resides on the heap.
|
||||
//
|
||||
// - "0": Default is not save pre-packed initializers to a data file.
|
||||
// - "1": Save pre-packed constant initializers to an external data file.
|
||||
// Sample usage: sess_options.add_session_config_entry(kOrtSessionOptionsSavePrePackedConstantInitializers, "1")
|
||||
static const char* const kOrtSessionOptionsSavePrePackedConstantInitializers =
|
||||
"session.save_external_prepacked_constant_initializers";
|
||||
|
||||
// Use this config when you want to collect memory stats for each node in the graph.
|
||||
// The file format is a CSV file with the following columns:
|
||||
// The file will be created if it does not exist, and will be overwritten if it does.
|
||||
//
|
||||
// The content of the file can be used to estimate memory requirements at run time including
|
||||
// the temporary allocations. This operation is preferably done on a CPU device, as the model may exceed
|
||||
// device memory limits in constrained environments. When enabling this option, it is important to disable
|
||||
// memory patterns, as they tend to allocate large blocks to avoid fragmentation and accommodate needs of multiple
|
||||
// kernels. Memory patterns may make it difficult to allocate on a device with limited memory.
|
||||
//
|
||||
// The collected stats then can be used to partition the graph among the devices in a way that only the
|
||||
// required memory is allocated on each device.
|
||||
//
|
||||
// node_name, initializers_memory, dynamic_outputs_sizes, temp_allocations_size
|
||||
//
|
||||
// - "full path to file": there is not a default for this option. If the file can not be opened for writing, an error will be returned.
|
||||
static const char* const kOrtSessionOptionsCollectNodeMemoryStatsToFile = "session.collect_node_memory_stats_to_file";
|
||||
|
||||
/// This is a composite CSV setting formatted as "memory limit in kb,file name for collected stats"
|
||||
/// "limit > 0": enables Capacity Aware Partitioning for Cuda EP. `limit` is optional and when absent
|
||||
/// the provider may attempt to figure out the memory available automatically.
|
||||
/// The setting with no limit is expected to look like: ",file name for collected stats"
|
||||
/// The EP will place nodes on device "file name" :
|
||||
/// this file is expected to be found at the same folder with the model. The file contains
|
||||
/// pre-recorded stats collected when running with kOrtSessionOptionsCollectNodeMemoryStatsToFile enforce (see above)
|
||||
static const char* const kOrtSessionOptionsResourceCudaPartitioningSettings =
|
||||
"session.resource_cuda_partitioning_settings";
|
||||
|
||||
// Enable EP context feature to dump the partitioned graph which includes the EP context into Onnx file.
|
||||
// The dumped Onnx model with EP context can be used for future inference to avoid the EP graph partitioning/compile overhead.
|
||||
// "0": disable. (default)
|
||||
// "1": enable.
|
||||
static const char* const kOrtSessionOptionEpContextEnable = "ep.context_enable";
|
||||
|
||||
// Specify the file path for the Onnx model which has EP context.
|
||||
// Default to original_file_name_ctx.onnx if not specified
|
||||
// Folder is not a valid option
|
||||
static const char* const kOrtSessionOptionEpContextFilePath = "ep.context_file_path";
|
||||
|
||||
// Flag to specify whether to dump the EP context into the Onnx model.
|
||||
// "0": dump the EP context into separate file, keep the file name in the Onnx model. (default).
|
||||
// "1": dump the EP context into the Onnx model.
|
||||
static const char* const kOrtSessionOptionEpContextEmbedMode = "ep.context_embed_mode";
|
||||
|
||||
// Specify the EPContext node name prefix to make it unique
|
||||
// in case user need to merge/connect multiple EPContext nodes in one model
|
||||
static const char* const kOrtSessionOptionEpContextNodeNamePrefix = "ep.context_node_name_prefix";
|
||||
|
||||
// Share EP related resources across sessions
|
||||
static const char* const kOrtSessionOptionShareEpContexts = "ep.share_ep_contexts";
|
||||
|
||||
// Stop to share EP related resources across sessions from then on
|
||||
static const char* const kOrtSessionOptionStopShareEpContexts = "ep.stop_share_ep_contexts";
|
||||
|
||||
// Used only for context model generation.
|
||||
// This configuration is used when some nodes are partitioned on the CPU EP and those nodes have external initializers.
|
||||
// When generating the EP context model, the new model should not rely on the old external data file used by the source ONNX model.
|
||||
// Use this setting when dumping the EP context model with an external initializers file.
|
||||
// If specified, all initializers will be placed inside the external data file.
|
||||
// Otherwise, all initializers will be embedded inside the generated ONNX file.
|
||||
// By default, this option is not set, meaning all initializers will be included within the ONNX file.
|
||||
static const char* const kOrtSessionOptionsEpContextModelExternalInitializersFileName =
|
||||
"ep.context_model_external_initializers_file_name";
|
||||
|
||||
// Gemm fastmath mode provides fp32 gemm acceleration with bfloat16 based matmul.
|
||||
// Option values:
|
||||
// - "0": Gemm FastMath mode is not enabled. [DEFAULT]
|
||||
// - "1": Gemm FastMath mode is enabled.
|
||||
static const char* const kOrtSessionOptionsMlasGemmFastMathArm64Bfloat16 = "mlas.enable_gemm_fastmath_arm64_bfloat16";
|
||||
|
||||
// When converting DQ + MatMul -> MatMulNBits, the accuracy level of the MatMulNBits is controlled by this option.
|
||||
// Refer to MatMulNBits op schema for more details.
|
||||
// If not provided, default is 4.
|
||||
static const char* const kOrtSessionOptionsQDQMatMulNBitsAccuracyLevel = "session.qdq_matmulnbits_accuracy_level";
|
||||
|
||||
// THIS OPTION IS NOT A REGULAR SESSION OPTION SINCE IT CAN BE MODIFIED AT ANY TIME
|
||||
// Meant to be used with SetEpDynamicOptions
|
||||
// Specify the type of workload for this session.
|
||||
// “Default”: OS determines the scheduling priority and processor performance to service this workload. [Default]
|
||||
// “Efficient”: OS treats this workload is efficiency oriented with low scheduling priority and efficient processor performance.
|
||||
static const char* const kOrtEpDynamicOptionsWorkloadType = "ep.dynamic.workload_type";
|
||||
|
||||
// Disables model compilation during session initialization.
|
||||
//
|
||||
// If this option is set to "1", inference session creation will fail with error code ORT_MODEL_REQUIRES_COMPILATION
|
||||
// if compilation is required to run the model on any Execution Provider added to the session.
|
||||
// Only the following kinds of models are valid when this option is set to "1":
|
||||
// - Pre-compiled models that have EPContext nodes for the compiling Execution Providers in the session.
|
||||
// - Non-compiled models that run only on non-compiling Execution Providers, like CPU EP.
|
||||
//
|
||||
// See \href https://onnxruntime.ai/docs/execution-providers/EP-Context-Design.html for details about
|
||||
// compiled models with EPContext nodes.
|
||||
//
|
||||
// Option values:
|
||||
// - "0": EP compile is not disabled. [DEFAULT]
|
||||
// - "1": EP compile is disabled.
|
||||
static const char* const kOrtSessionOptionsDisableModelCompile = "session.disable_model_compile";
|
||||
@@ -0,0 +1,18 @@
|
||||
// Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
// Licensed under the MIT License.
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
namespace onnxruntime {
|
||||
|
||||
// data types for execution provider options
|
||||
|
||||
using ProviderOptions = std::unordered_map<std::string, std::string>;
|
||||
using ProviderOptionsVector = std::vector<ProviderOptions>;
|
||||
using ProviderOptionsMap = std::unordered_map<std::string, ProviderOptions>;
|
||||
|
||||
} // namespace onnxruntime
|
||||
@@ -0,0 +1,26 @@
|
||||
|
||||
####### Expanded from @PACKAGE_INIT@ by configure_package_config_file() #######
|
||||
####### Any changes to this file will be overwritten by the next CMake run ####
|
||||
####### The input file was PROJECT_CONFIG_FILE ########
|
||||
|
||||
get_filename_component(PACKAGE_PREFIX_DIR "${CMAKE_CURRENT_LIST_DIR}/../../../" ABSOLUTE)
|
||||
|
||||
macro(set_and_check _var _file)
|
||||
set(${_var} "${_file}")
|
||||
if(NOT EXISTS "${_file}")
|
||||
message(FATAL_ERROR "File or directory ${_file} referenced by variable ${_var} does not exist !")
|
||||
endif()
|
||||
endmacro()
|
||||
|
||||
macro(check_required_components _NAME)
|
||||
foreach(comp ${${_NAME}_FIND_COMPONENTS})
|
||||
if(NOT ${_NAME}_${comp}_FOUND)
|
||||
if(${_NAME}_FIND_REQUIRED_${comp})
|
||||
set(${_NAME}_FOUND FALSE)
|
||||
endif()
|
||||
endif()
|
||||
endforeach()
|
||||
endmacro()
|
||||
|
||||
####################################################################################
|
||||
include("${CMAKE_CURRENT_LIST_DIR}/onnxruntimeTargets.cmake")
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# This is a basic version file for the Config-mode of find_package().
|
||||
# It is used by write_basic_package_version_file() as input file for configure_file()
|
||||
# to create a version-file which can be installed along a config.cmake file.
|
||||
#
|
||||
# The created file sets PACKAGE_VERSION_EXACT if the current version string and
|
||||
# the requested version string are exactly the same and it sets
|
||||
# PACKAGE_VERSION_COMPATIBLE if the current version is >= requested version,
|
||||
# but only if the requested major version is the same as the current one.
|
||||
# The variable CVF_VERSION must be set before calling configure_file().
|
||||
|
||||
|
||||
set(PACKAGE_VERSION "1.22.0")
|
||||
|
||||
if(PACKAGE_VERSION VERSION_LESS PACKAGE_FIND_VERSION)
|
||||
set(PACKAGE_VERSION_COMPATIBLE FALSE)
|
||||
else()
|
||||
|
||||
if("1.22.0" MATCHES "^([0-9]+)\\.")
|
||||
set(CVF_VERSION_MAJOR "${CMAKE_MATCH_1}")
|
||||
if(NOT CVF_VERSION_MAJOR VERSION_EQUAL 0)
|
||||
string(REGEX REPLACE "^0+" "" CVF_VERSION_MAJOR "${CVF_VERSION_MAJOR}")
|
||||
endif()
|
||||
else()
|
||||
set(CVF_VERSION_MAJOR "1.22.0")
|
||||
endif()
|
||||
|
||||
if(PACKAGE_FIND_VERSION_RANGE)
|
||||
# both endpoints of the range must have the expected major version
|
||||
math (EXPR CVF_VERSION_MAJOR_NEXT "${CVF_VERSION_MAJOR} + 1")
|
||||
if (NOT PACKAGE_FIND_VERSION_MIN_MAJOR STREQUAL CVF_VERSION_MAJOR
|
||||
OR ((PACKAGE_FIND_VERSION_RANGE_MAX STREQUAL "INCLUDE" AND NOT PACKAGE_FIND_VERSION_MAX_MAJOR STREQUAL CVF_VERSION_MAJOR)
|
||||
OR (PACKAGE_FIND_VERSION_RANGE_MAX STREQUAL "EXCLUDE" AND NOT PACKAGE_FIND_VERSION_MAX VERSION_LESS_EQUAL CVF_VERSION_MAJOR_NEXT)))
|
||||
set(PACKAGE_VERSION_COMPATIBLE FALSE)
|
||||
elseif(PACKAGE_FIND_VERSION_MIN_MAJOR STREQUAL CVF_VERSION_MAJOR
|
||||
AND ((PACKAGE_FIND_VERSION_RANGE_MAX STREQUAL "INCLUDE" AND PACKAGE_VERSION VERSION_LESS_EQUAL PACKAGE_FIND_VERSION_MAX)
|
||||
OR (PACKAGE_FIND_VERSION_RANGE_MAX STREQUAL "EXCLUDE" AND PACKAGE_VERSION VERSION_LESS PACKAGE_FIND_VERSION_MAX)))
|
||||
set(PACKAGE_VERSION_COMPATIBLE TRUE)
|
||||
else()
|
||||
set(PACKAGE_VERSION_COMPATIBLE FALSE)
|
||||
endif()
|
||||
else()
|
||||
if(PACKAGE_FIND_VERSION_MAJOR STREQUAL CVF_VERSION_MAJOR)
|
||||
set(PACKAGE_VERSION_COMPATIBLE TRUE)
|
||||
else()
|
||||
set(PACKAGE_VERSION_COMPATIBLE FALSE)
|
||||
endif()
|
||||
|
||||
if(PACKAGE_FIND_VERSION STREQUAL PACKAGE_VERSION)
|
||||
set(PACKAGE_VERSION_EXACT TRUE)
|
||||
endif()
|
||||
endif()
|
||||
endif()
|
||||
|
||||
|
||||
# if the installed or the using project don't have CMAKE_SIZEOF_VOID_P set, ignore it:
|
||||
if("${CMAKE_SIZEOF_VOID_P}" STREQUAL "" OR "8" STREQUAL "")
|
||||
return()
|
||||
endif()
|
||||
|
||||
# check that the installed version has the same 32/64bit-ness as the one which is currently searching:
|
||||
if(NOT CMAKE_SIZEOF_VOID_P STREQUAL "8")
|
||||
math(EXPR installedBits "8 * 8")
|
||||
set(PACKAGE_VERSION "${PACKAGE_VERSION} (${installedBits}bit)")
|
||||
set(PACKAGE_VERSION_UNSUITABLE TRUE)
|
||||
endif()
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
#----------------------------------------------------------------
|
||||
# Generated CMake target import file for configuration "Release".
|
||||
#----------------------------------------------------------------
|
||||
|
||||
# Commands may need to know the format version.
|
||||
set(CMAKE_IMPORT_FILE_VERSION 1)
|
||||
|
||||
# Import target "onnxruntime::onnxruntime" for configuration "Release"
|
||||
set_property(TARGET onnxruntime::onnxruntime APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE)
|
||||
set_target_properties(onnxruntime::onnxruntime PROPERTIES
|
||||
IMPORTED_LOCATION_RELEASE "${_IMPORT_PREFIX}/lib64/libonnxruntime.so.1.22.0"
|
||||
IMPORTED_SONAME_RELEASE "libonnxruntime.so.1"
|
||||
)
|
||||
|
||||
list(APPEND _cmake_import_check_targets onnxruntime::onnxruntime )
|
||||
list(APPEND _cmake_import_check_files_for_onnxruntime::onnxruntime "${_IMPORT_PREFIX}/lib64/libonnxruntime.so.1.22.0" )
|
||||
|
||||
# Commands beyond this point should not need to know the version.
|
||||
set(CMAKE_IMPORT_FILE_VERSION)
|
||||
@@ -0,0 +1,106 @@
|
||||
# Generated by CMake
|
||||
|
||||
if("${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION}" LESS 2.8)
|
||||
message(FATAL_ERROR "CMake >= 2.8.3 required")
|
||||
endif()
|
||||
if(CMAKE_VERSION VERSION_LESS "2.8.3")
|
||||
message(FATAL_ERROR "CMake >= 2.8.3 required")
|
||||
endif()
|
||||
cmake_policy(PUSH)
|
||||
cmake_policy(VERSION 2.8.3...3.29)
|
||||
#----------------------------------------------------------------
|
||||
# Generated CMake target import file.
|
||||
#----------------------------------------------------------------
|
||||
|
||||
# Commands may need to know the format version.
|
||||
set(CMAKE_IMPORT_FILE_VERSION 1)
|
||||
|
||||
# Protect against multiple inclusion, which would fail when already imported targets are added once more.
|
||||
set(_cmake_targets_defined "")
|
||||
set(_cmake_targets_not_defined "")
|
||||
set(_cmake_expected_targets "")
|
||||
foreach(_cmake_expected_target IN ITEMS onnxruntime::onnxruntime)
|
||||
list(APPEND _cmake_expected_targets "${_cmake_expected_target}")
|
||||
if(TARGET "${_cmake_expected_target}")
|
||||
list(APPEND _cmake_targets_defined "${_cmake_expected_target}")
|
||||
else()
|
||||
list(APPEND _cmake_targets_not_defined "${_cmake_expected_target}")
|
||||
endif()
|
||||
endforeach()
|
||||
unset(_cmake_expected_target)
|
||||
if(_cmake_targets_defined STREQUAL _cmake_expected_targets)
|
||||
unset(_cmake_targets_defined)
|
||||
unset(_cmake_targets_not_defined)
|
||||
unset(_cmake_expected_targets)
|
||||
unset(CMAKE_IMPORT_FILE_VERSION)
|
||||
cmake_policy(POP)
|
||||
return()
|
||||
endif()
|
||||
if(NOT _cmake_targets_defined STREQUAL "")
|
||||
string(REPLACE ";" ", " _cmake_targets_defined_text "${_cmake_targets_defined}")
|
||||
string(REPLACE ";" ", " _cmake_targets_not_defined_text "${_cmake_targets_not_defined}")
|
||||
message(FATAL_ERROR "Some (but not all) targets in this export set were already defined.\nTargets Defined: ${_cmake_targets_defined_text}\nTargets not yet defined: ${_cmake_targets_not_defined_text}\n")
|
||||
endif()
|
||||
unset(_cmake_targets_defined)
|
||||
unset(_cmake_targets_not_defined)
|
||||
unset(_cmake_expected_targets)
|
||||
|
||||
|
||||
# Compute the installation prefix relative to this file.
|
||||
get_filename_component(_IMPORT_PREFIX "${CMAKE_CURRENT_LIST_FILE}" PATH)
|
||||
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
|
||||
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
|
||||
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
|
||||
if(_IMPORT_PREFIX STREQUAL "/")
|
||||
set(_IMPORT_PREFIX "")
|
||||
endif()
|
||||
|
||||
# Create imported target onnxruntime::onnxruntime
|
||||
add_library(onnxruntime::onnxruntime SHARED IMPORTED)
|
||||
|
||||
set_target_properties(onnxruntime::onnxruntime PROPERTIES
|
||||
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include/onnxruntime"
|
||||
)
|
||||
|
||||
# Load information for each installed configuration.
|
||||
file(GLOB _cmake_config_files "${CMAKE_CURRENT_LIST_DIR}/onnxruntimeTargets-*.cmake")
|
||||
foreach(_cmake_config_file IN LISTS _cmake_config_files)
|
||||
include("${_cmake_config_file}")
|
||||
endforeach()
|
||||
unset(_cmake_config_file)
|
||||
unset(_cmake_config_files)
|
||||
|
||||
# Cleanup temporary variables.
|
||||
set(_IMPORT_PREFIX)
|
||||
|
||||
# Loop over all imported files and verify that they actually exist
|
||||
foreach(_cmake_target IN LISTS _cmake_import_check_targets)
|
||||
if(CMAKE_VERSION VERSION_LESS "3.28"
|
||||
OR NOT DEFINED _cmake_import_check_xcframework_for_${_cmake_target}
|
||||
OR NOT IS_DIRECTORY "${_cmake_import_check_xcframework_for_${_cmake_target}}")
|
||||
foreach(_cmake_file IN LISTS "_cmake_import_check_files_for_${_cmake_target}")
|
||||
if(NOT EXISTS "${_cmake_file}")
|
||||
message(FATAL_ERROR "The imported target \"${_cmake_target}\" references the file
|
||||
\"${_cmake_file}\"
|
||||
but this file does not exist. Possible reasons include:
|
||||
* The file was deleted, renamed, or moved to another location.
|
||||
* An install or uninstall procedure did not complete successfully.
|
||||
* The installation package was faulty and contained
|
||||
\"${CMAKE_CURRENT_LIST_FILE}\"
|
||||
but not all the files it references.
|
||||
")
|
||||
endif()
|
||||
endforeach()
|
||||
endif()
|
||||
unset(_cmake_file)
|
||||
unset("_cmake_import_check_files_for_${_cmake_target}")
|
||||
endforeach()
|
||||
unset(_cmake_target)
|
||||
unset(_cmake_import_check_targets)
|
||||
|
||||
# This file does not depend on other imported targets which have
|
||||
# been exported from the same project but in a separate export set.
|
||||
|
||||
# Commands beyond this point should not need to know the version.
|
||||
set(CMAKE_IMPORT_FILE_VERSION)
|
||||
cmake_policy(POP)
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,13 @@
|
||||
prefix=/usr/local
|
||||
bindir=${prefix}/bin
|
||||
mandir=${prefix}/share/man
|
||||
docdir=${prefix}/share/doc/onnxruntime
|
||||
libdir=${prefix}/lib64
|
||||
includedir=${prefix}/include/onnxruntime
|
||||
|
||||
Name: onnxruntime
|
||||
Description: ONNX runtime
|
||||
URL: https://github.com/microsoft/onnxruntime
|
||||
Version: 1.22.0
|
||||
Libs: -L${libdir} -lonnxruntime
|
||||
Cflags: -I${includedir}
|
||||
Reference in New Issue
Block a user