AISystem
1.0.0
The text course content is being updated one after another. Take the time to continue to update AISys as much as possible. I hope you will encourage and participate! ! !
Text courses are open source in AISys, series of video hosting B and oil pipes, PPT is open source in github, welcome to use! ! !
This open source course is called AI System (AISys) in English and is called AI System in Chinese.
This open source course mainly discusses and learns the system design of artificial intelligence and deep learning with you, and the entire system revolves around ZOMI's accumulated, sorts out and builds the full stack of AI systems in work. I hope to discuss and research with all good friends who are concerned about AI open source courses and promote learning and discussion together.
The course mainly includes the following five modules:
| Tutorial content | Introduction | address |
|---|---|---|
| Overview of AI System Full Stack | AI basics and AI systems overview of full stack overview of AI systems, as well as the systematic design and methodology of deep learning systems, are mainly about a comprehensive understanding of AI training and inference of full stack architecture content. | [Slides] |
| AI chips and architecture | As the hardware architecture of AI, it mainly refers to AI chips, which is very hard-core here. From the chip foundation of CPU and GPU to the principles, design and application scenarios of AI chips, the design of AI chips not only considers the acceleration of AI computing, but also requires full consideration of AI application algorithms, AI frameworks and other middleware, rather than staying at yelling at Nvidia and CUDA every day. In fact, chips are difficult to use. | [Slides] |
| AI Programming and Computing Architecture | The Advanced Edition introduces AI programming and computing architecture, and will consider compiler issues that need to be considered in designing modern machine learning systems, especially intermediate expressions and even back-end optimization. | [Slides] |
| AI reasoning system and engine | In practice, the reasoning system and engine are used to explain too many principles and the body is too weak and easy to digest. It still has to return to the essence of the business so that industries and enterprises can truly apply it. The reasoning system involves some core algorithms and things to pay attention to. | [Slides] |
| Core technologies of AI framework | Introducing the core technologies of the AI framework, first introduce the automatic differentiation that any AI framework cannot do without. After the automatic differentiation function, the graph and operator representing the neural network will be generated. Then, the optimization of the front-end of the AI framework is introduced, and the key technologies of distributed training of large models in the AI framework that have been popular recently. | [Slides] |
This course is mainly designed for undergraduate seniors, master's and doctoral students, and AI system practitioners to help everyone:
Completely understand the computer system architecture of AI, and understand the system design under the complete life cycle of AI through practical problems and cases.
Introduce research work that combines cutting-edge system architecture and AI, and understand mainstream frameworks, platforms and tools to understand AI systems.
| serial number | name | Specific content |
|---|---|---|
| 1 | AI System | Combining algorithms, frameworks and architectures to form an AI system |
| serial number | name | Specific content |
|---|---|---|
| 1 | AI computing system | Computation mode and computing architecture of AI technologies such as neural networks |
| 2 | AI chip basics | Basic principles of chip architecture such as CPU, GPU, NPU, etc. |
| 3 | Graphics Processor GPU | The basic principles of GPU, the architectural development of Nvidia GPU |
| 4 | Nvidia GPU details | In-depth analysis of Tensor Core and NVLink of NVIDIA GPU |
| 5 | Foreign AI processors | Core principles of dedicated AI processors such as Google and Tesla |
| 6 | Domestic AI processors | Core principles of special AI processors such as Cambrian and Suiyuan Technology |
| 7 | 10 years of gold for AI chips | Summary of the programming model and development of AI chips |
| serial number | name | Specific content |
|---|---|---|
| 1 | Traditional compilers | Traditional compilers GCC and LLVM, LLVM detailed architecture |
| 2 | AI compiler | AI compiler development and architecture definition, future challenges and thinking |
| 3 | Front-end optimization | Front-end optimization of AI compiler (operator fusion, memory optimization, etc.) |
| 4 | Backend optimization | Back-end optimization of AI compiler (Kernel optimization, AutoTuning) |
| 5 | polyhedron | Waiting for update... |
| 6 | PyTorch2.0 | The most important new feature of PyTorch2.0: Compilation technology stack |
| serial number | name | Specific content |
|---|---|---|
| 1 | Inference system | Overall introduction to the inference system, and the inference engine architecture sorting |
| 2 | Lightweight network | Introduction to SOTA models such as lightweight backbone networks, MobileNet, etc. |
| 3 | Model compression | Model compression set, quantization, distillation, pruning and binarization |
| 4 | Conversion & Optimization | After training of AI framework, the model is transformed and the calculation graph is optimized. |
| 5 | Kernel Optimization | Kernel layer and operator layer optimization, operator, memory, and scheduling optimization |
| serial number | name | Specific content |
|---|---|---|
| 1 | AI Framework Basics | The role, development, and programming paradigm of AI framework |
| 2 | Automatic differentiation | Implementation method and principle of automatic differentiation |
| 3 | Calculation diagram | The concept of computing graphs, graph optimization, graph execution, control flow expression |
This warehouse has reached the crazy 10G (ZOMI provides all production processes and high-definition pictures intact). If you want git clone, it will be very slow, so it is recommended to go to Releases · chenzomi12/AISystem to download the content you need first
I really hope you will also participate in this open source course. B station will leave a message to ZOMI!
Everyone is welcome to find bugs or errata during use and submit the code PR directly to the open source community!
Everyone is welcome to find bugs or errata during use and submit PR directly to the open source community!
Please respect the efforts of open source and ZOMI. Please reprint and indicate the source when quoting PPT content!
