초록 ▼

A neural processing unit (NPU) is described. The NPU includes an NPU direct memory access (NDMA) core. The NDMA core includes a read engine having a read buffer. The NDMA core also includes a write engine having a write buffer. The NPU also includes a controller. The controller is configured to dire

A neural processing unit (NPU) is described. The NPU includes an NPU direct memory access (NDMA) core. The NDMA core includes a read engine having a read buffer. The NDMA core also includes a write engine having a write buffer. The NPU also includes a controller. The controller is configured to direct the NDMA core to perform hardware memory bandwidth optimization for reading/writing NDMA data in the read buffer and/or NDMA data in the write buffer. The NDMA core is also configured to transparently combine NDMA transaction requests for a data stripe to increase local access to available tensors in artificial neural networks.

대표청구항 ▼

1. A neural processing unit (NPU), comprising: an NPU direct memory access (NDMA) core comprising: a read engine having a read buffer; anda write engine having a write buffer; anda controller configured to direct the NDMA core to perform hardware memory bandwidth optimization for reading/writing NDM

1. A neural processing unit (NPU), comprising: an NPU direct memory access (NDMA) core comprising: a read engine having a read buffer; anda write engine having a write buffer; anda controller configured to direct the NDMA core to perform hardware memory bandwidth optimization for reading/writing NDMA data in the read buffer and/or NDMA data in the write buffer, the NDMA core configured to transparently combine NDMA transaction requests for a data stripe. 2. The NPU of claim 1, in which the controller is configured to direct the NDMA core to pre-fetch deep learning bandwidth compression (DLBC) metadata when NDMA data is compressed by a DLBC block of a network on chip (NoC) coupled between an external memory and the NDMA core. 3. The NPU of claim 1, in which the controller is configured to direct the NDMA core to combine blocks of NDMA data from scattered memory locations of an external memory as a contiguous address space for a read client. 4. The NPU of claim 3, in which the controller is configured to direct the NDMA core to split NDMA read and/or NDMA write transactions that cross a minimum access length (MAL) boundary of the external memory. 5. The NPU of claim 1, further comprising a read client coupled to an interface of the read engine of the NDMA core and a write client coupled to an interface of the write engine of the NDMA core. 6. The NPU of claim 1, further comprising: a read arbiter coupled to the read engine;a write arbiter coupled to the write engine; andan external memory coupled to the read arbiter and the write arbiter. 7. The NPU of claim 6, further comprising a bus bridge coupled between the external memory and the read arbiter and the write arbiter. 8. The NPU of claim 7, further comprising a network on chip (NoC) coupled between the external memory and the bus bridge. 9. The NPU of claim 1, further comprising: a write client coupled to a first memory interface of the NDMA core; anda read client coupled to a second memory interface of the NDMA core. 10. The NPU of claim 9, in which the write client and the read client comprise a client buffer used to store DMA data of the NDMA core. 11. A method for hardware-based memory bandwidth optimization of a neural processing unit (NPU) direct memory access (NDMA) in artificial neural networks, comprising: programming configuration registers of a neural processing unit (NPU) direct memory access (NDMA) core for a read client and/or a write client;transparently combining NDMA transaction requests from the read client and/or the write client as a single NDMA transaction request; andstreaming data blocks of data stripes of the single NDMA transaction request, the data blocks being streamed to/from an external memory of the NDMA core and to/from the read client and/or the write client. 12. The method of claim 11, in which the transparently combining NDMA transaction requests comprises combining blocks of NDMA data from scattered memory locations of the external memory as a contiguous address space for the read client. 13. The method of claim 11, in which the transparently combining NDMA transaction requests further comprises concatenating the data blocks of the data stripes using an increased transaction size. 14. The method of claim 13, further comprising re-concatenating the data blocks of the data stripes at interleaved locations of a read buffer of the read client and/or a write buffer of the write client. 15. The method of claim 11, in which the transparently combining NDMA transaction requests further comprises delaying arbitration for an NDMA channel until a predetermined number of the data blocks are read and/or written to/from the read client and/or the write client. 16. An artificial neural network for hardware-based memory bandwidth optimization of a neural processing unit (NPU) direct memory access (NDMA), the artificial neural network comprising: means for programming configuration registers of a neural processing unit (NPU) direct memory access (NDMA) core for a read client and/or a write client;means for transparently combining NDMA transaction requests from the read client and/or the write client as a single NDMA transaction request; andmeans for streaming data blocks of data stripes of the single NDMA transaction request, the data blocks of data stripes being streamed to/from an external memory of the NDMA core and to/from the read client and/or the write client. 17. The artificial neural network of claim 16, in which the means for transparently combining NDMA transaction requests comprises means for combining blocks of NDMA data from scattered memory locations of the external memory as a contiguous address space for the read client. 18. The artificial neural network of claim 16, in which the means for transparently combining NDMA transaction requests further comprises means for concatenating data blocks of data stripes using an increased transaction size. 19. The artificial neural network of claim 18, further comprising means for re-concatenating data blocks of data stripes at interleaved locations of a read buffer of the read client and/or a write buffer of the write client. 20. The artificial neural network of claim 16, in which the means for transparently combining NDMA transaction requests further comprises means for delaying arbitration for an NDMA channel until a predetermined number of data blocks are read and/or written to/from the read client and/or the write client. 21. A non-transitory computer-readable medium having program code recorded thereon for hardware-based memory bandwidth optimization of a neural processing unit (NPU) direct memory access (NDMA), the program code being executed by a processor and comprising: program code to program configuration registers of a neural processing unit (NPU) direct memory access (NDMA) core for a read client and/or a write client;program code to transparently combine NDMA transaction requests from the read client and/or the write client as a single NDMA transaction request; andprogram code to stream data blocks of data stripes of the single NDMA transaction request, data blocks of data stripes being streamed to/from an external memory of the NDMA core and to/from the read client and/or the write client. 22. The non-transitory computer-readable medium of claim 21, in which the program code to transparently combine NDMA transaction requests comprises program code to combine blocks of NDMA data from scattered memory locations of the external memory as a contiguous address space for the read client. 23. The non-transitory computer-readable medium of claim 21, in which the program code to transparently combine NDMA transaction requests further comprises program code to concatenate the data blocks of the data stripes using an increased transaction size. 24. The non-transitory computer-readable medium of claim 23, further comprising program code to re-concatenate the data blocks of the data stripes at interleaved locations of a read buffer of the read client and/or a write buffer of the write client. 25. The non-transitory computer-readable medium of claim 21, in which the program code to transparently combine NDMA transaction requests further comprises program code to delay arbitration for an NDMA channel until a predetermined number of the data blocks are read and/or written to/from the read client and/or the write client.