「AI 創新需要開源生態系，而 ONNX 確保了 frameworks 之間的互通性，ONNC 的目標是將所有的 DLA ASIC 快速、簡單地與 ONNX 連結起來，確保所有的 DLA ASIC 都可以在 ONNX 上執行。」
由於 AI 的應用層面越來越廣，數百種新 AI 晶片將在不久的未來大量出現，目前市場上卻沒有一個能完整支援各家 DLA 的開源編譯器。根據統計，在 2018 年會有超過 34 家 IC 與 IP 廠商提供各式各樣的 AI 晶片與 deep learning accelerator (DLA) ASICs，因此急需一個開源的編譯器來支援各種不同的 AI 晶片。
成立於 2013 年的 Skymizer 一直都專注在 compiler 和 machine learning 領域，看到這個趨勢之後，建立了基於 ONNX 的編譯器 Open Nerual Network Compiler – ONNC，將所有的 AI 晶片與 ONNX 連結起來。
ONNX 是開放類神經網路交換格式，目前常見的 AI framework 如Caffe2、PyTorch…等等各有各的支持者，格式間互通性差，因此訓練出來的深度學習模型無法套用到別的 framework 上，但有了 ONNX，開發者可在不同格式之間輕鬆轉換。而透過 ONNX 支援多平台的特性，ONNC 可以支援各種不同的 AI frameworks，如 Caffe、Caffe2 與 PyTorch，幫助 DLA ASIC 廠商在短時間內就可以支援各種 AI frameworks，提升性能並縮短開發時間。
由於 AI 的應用層面越來越廣，數百種新 AI 晶片將在不久的未來大量出現，目前市場上卻沒有一個能完整支援各家 DLA 的開源編譯器。根據統計，在 2018 年會有超過 34 家 IC 與 IP 廠商提供各式各樣的 AI 晶片與 deep learning accelerator (DLA) ASICs，因此急需一個開源的編譯器來支援各種不同的 AI 晶片。
成立於 2013 年的 Skymizer 一直都專注在 compiler 和 machine learning 領域，看到這個趨勢之後，建立了基於 ONNX 的編譯器 Open Nerual Network Compiler – ONNC，將所有的 AI 晶片與 ONNX 連結起來。
ONNX 是開放類神經網路交換格式，目前常見的 AI framework 如Caffe2、PyTorch…等等各有各的支持者，格式間互通性差，因此訓練出來的深度學習模型無法套用到別的 framework 上，但有了 ONNX，開發者可在不同格式之間輕鬆轉換。而透過 ONNX 支援多平台的特性，ONNC 可以支援各種不同的 AI frameworks，如 Caffe、Caffe2 與 PyTorch，幫助 DLA ASIC 廠商在短時間內就可以支援各種 AI frameworks，提升性能並縮短開發時間。
ONNC features:
Easy Backend Integration
ONNC is integrated with the LLVM bitcode runtime and compiler. If a DLA already supports the LLVM compiler, it can be connected to ONNC seamlessly. This helps most CPUs, GPUs, and DSPs ported to ONNC in a very short time. On the other hand, if a DLA has unique computation features and is not compatible to LLVM, ONNC also provides a modular framework to speed up the compiler development. DLA vendors can quickly customize an ONNC backend from a so called “vanilla” backend, which already provides some necessary optimization algorithms.
Reusable Compiler Optimizations
Two of ONNC’s contributions are dividing the AI compilation into several clear phases and giving the corresponding APIs for algorithm development.
There are five phases carefully defined, each of which is focused on a particular compiler problem: IR building, partitioning, scheduling, allocation, and code emitting.
ONNC also provides a series of optimization algorithms ready for use. They are general and reusable, including tensor selection, tensor liveness analysis, linear scan local memory allocation, etc. ONNC’s pass manager is flexible and similar to LLVM’s. AI researchers and engineers who are familiar with LLVM can intuitively contribute their general or target-specific optimization algorithms to ONNC.
Status and Future Work
Project ONNC is not mature yet.
Still, we have a lot to do and need the community together to make it better. In this preview release, we carefully designed the software architecture so as to simplify the future development and
maintenance.
The items we have done include:
- Clear compilation phases and the corresponding APIs.
- Well-defined and extensible intermediate representation (IR) of target-specific instructions.
- A pass manager that supports automatic scheduling according to the dependency claimed by each pass.
- A Sophon backend that supports BITMAIN AI ASIC.
- Shim library such as ADT, Diagnostics, JSON, etc.
Project ONNC follows the “release early, release often” principle,
so the next release should come soon by the end of August.
We plan to finish the following items.
- An x86 backend that enables the execution of AI models on an x86 machine. That backend should support both JIT and interpreter way of execution.
- Improvements of memory allocation based on liveness analysis.
JOIN US
To keep up with the latest development or make suggestions, please join the ONNC mailing list. Also, please head over to the official website https://onnc.ai/ and GitHub https://repo.onnc.ai for more information.
留言