Performing an allreduce operation on a plurality of compute nodes of a parallel computer
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06F-015/80
G06F-009/30
G06F-009/302
출원번호
US-0124745
(2008-05-21)
등록번호
US-8484440
(2013-07-09)
발명자
/ 주소
Faraj, Ahmad
출원인 / 주소
International Business Machines Corporation
대리인 / 주소
Biggers & Ohanian LLP
인용정보
피인용 횟수 :
0인용 특허 :
69
초록▼
Methods, apparatus, and products are disclosed for performing an allreduce operation on a plurality of compute nodes of a parallel computer, each node including at least two processing cores, that include: establishing, for each node, a plurality of logical rings, each ring including a different set
Methods, apparatus, and products are disclosed for performing an allreduce operation on a plurality of compute nodes of a parallel computer, each node including at least two processing cores, that include: establishing, for each node, a plurality of logical rings, each ring including a different set of at least one core on that node, each ring including the cores on at least two of the nodes; iteratively for each node: assigning each core of that node to one of the rings established for that node to which the core has not previously been assigned, and performing, for each ring for that node, a global allreduce operation using contribution data for the cores assigned to that ring or any global allreduce results from previous global allreduce operations, yielding current global allreduce results for each core; and performing, for each node, a local allreduce operation using the global allreduce results.
대표청구항▼
1. A method of performing an allreduce operation on a plurality of compute nodes of a parallel computer, the compute nodes connected through a data communication network, each compute node comprising at least two processing cores, each processing core having contribution data for the allreduce opera
1. A method of performing an allreduce operation on a plurality of compute nodes of a parallel computer, the compute nodes connected through a data communication network, each compute node comprising at least two processing cores, each processing core having contribution data for the allreduce operation, the method comprising: establishing, for each compute node, a plurality of logical rings, each logical ring including a different set of at least one processing core on that compute node, each logical ring including the processing cores on at least two of the compute nodes;iteratively, for each compute node, until each processing core of that compute node is assigned to each logical ring established for that compute node: assigning each processing core of that compute node to one of the logical rings established for that compute node to which the processing core has not previously been assigned during the allreduce operation, and performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node; andperforming, for each compute node, a local allreduce operation using the global allreduce results for each processing core on that compute node. 2. The method of claim 1 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 3. The method of claim 1 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each direction along each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 4. The method of claim 1 wherein performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node further comprises: performing, for each logical ring, a reduce scatter operation using the contribution data for each processing core included in that logical ring or any of the global allreduce results from the previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding a reduce scatter result for each processing core included in that logical ring; andperforming, for each logical ring, an allgather operation using the reduce scatter result for each processing core included in that logical ring, yielding the global allreduce result for each processing core in that logical ring. 5. The method of claim 1 wherein the global allreduce operations performed for logical rings during each iteration occur in parallel. 6. The method of claim 1 wherein the plurality of compute nodes are connected through a plurality of data communications networks, at least one of the data communications networks optimized for point to point operations, and at least one data communications networks optimized for collective operations. 7. A parallel computer for performing an allreduce operation on a plurality of compute nodes of a parallel computer, the compute nodes connected through a data communication network, each compute node comprising at least two processing cores, each processing core having contribution data for the allreduce operation, the parallel computer comprising computer memory operatively coupled to the processing cores of the parallel computer, the computer memory having disposed within it computer program instructions capable of: establishing, for each compute node, a plurality of logical rings, each logical ring including a different set of at least one processing core on that compute node, each logical ring including the processing cores on at least two of the compute nodes;iteratively, for each compute node, until each processing core of that compute node is assigned to each logical ring established for that compute node: assigning each processing core of that compute node to one of the logical rings established for that compute node to which the processing core has not previously been assigned during the allreduce operation, and performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node; andperforming, for each compute node, a local allreduce operation using the global allreduce results for each processing core on that compute node. 8. The parallel computer of claim 7 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 9. The parallel computer of claim 7 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each direction along each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 10. The parallel computer of claim 7 wherein performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node further comprises: performing, for each logical ring, a reduce scatter operation using the contribution data for each processing core included in that logical ring or any of the global allreduce results from the previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding a reduce scatter result for each processing core included in that logical ring; andperforming, for each logical ring, an allgather operation using the reduce scatter result for each processing core included in that logical ring, yielding the global allreduce result for each processing core in that logical ring. 11. The parallel computer of claim 7 wherein the global allreduce operations performed for logical rings during each iteration occur in parallel. 12. The parallel computer of claim 7 wherein the plurality of compute nodes are connected through a plurality of data communications networks, at least one of the data communications networks optimized for point to point operations, and at least one data communications networks optimized for collective operations. 13. A computer program product for performing an allreduce operation on a plurality of compute nodes of a parallel computer, the compute nodes connected through a data communication network, each compute node comprising at least two processing cores, each processing core having contribution data for the allreduce operation, the computer program product disposed upon a computer readable medium, wherein the computer readable medium is not a signal, the computer program product comprising computer program instructions capable of: establishing, for each compute node, a plurality of logical rings, each logical ring including a different set of at least one processing core on that compute node, each logical ring including the processing cores on at least two of the compute nodes;iteratively, for each compute node, until each processing core of that compute node is assigned to each logical ring established for that compute node: assigning each processing core of that compute node to one of the logical rings established for that compute node to which the processing core has not previously been assigned during the allreduce operation, and performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node; andperforming, for each compute node, a local allreduce operation using the global allreduce results for each processing core on that compute node. 14. The computer program product of claim 13 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 15. The computer program product of claim 13 wherein establishing, for each compute node, a plurality of logical rings further comprises establishing, for each direction along each axis of each dimension of the data communications network, a logical ring that includes at least one processing core from each compute node along that axis of that dimension of the data communications network. 16. The computer program product of claim 13 wherein performing, for each logical ring for that compute node, a global allreduce operation using the contribution data for the processing cores assigned to that logical ring or any global allreduce results from previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding current global allreduce results for each processing core on that compute node further comprises: performing, for each logical ring, a reduce scatter operation using the contribution data for each processing core included in that logical ring or any of the global allreduce results from the previous global allreduce operations in which the processing cores assigned to that logical ring may have participated, yielding a reduce scatter result for each processing core included in that logical ring; andperforming, for each logical ring, an allgather operation using the reduce scatter result for each processing core included in that logical ring, yielding the global allreduce result for each processing core in that logical ring. 17. The computer program product of claim 13 wherein the global allreduce operations performed for logical rings during each iteration occur in parallel. 18. The computer program product of claim 13 wherein the plurality of compute nodes are connected through a plurality of data communications networks, at least one of the data communications networks optimized for point to point operations, and at least one data communications networks optimized for collective operations.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (69)
Scott Steven L. ; Pribnow Richard D. ; Logghe Peter G. ; Kunkel Daniel L. ; Schwoerer Gerald A., Adaptive congestion control mechanism for modular computer networks.
Archer, Charles J.; Inglett, Todd A.; Ratterman, Joseph D.; Smith, Brian E., Configuring compute nodes of a parallel computer in an operational group into a plurality of independent non-overlapping collective networks.
Kato Sadayuki,JPX ; Ishihata Hiroaki,JPX ; Horie Takeshi,JPX ; Inano Satoshi,JPX ; Shimizu Toshiyuki,JPX, Data gathering/scattering system for a plurality of processors in a parallel computer.
Connor, Patrick L.; McVay, Robert G., Direct memory access transfer reduction method and apparatus to overlay data on to scatter gather descriptors for bus-mastering I/O controllers.
Michael Olivier, Dynamically matching users for group communications based on a threshold degree of matching of sender and recipient predetermined acceptance criteria.
Cypher Robert E. (Los Gatos CA) Sanz Jorge L. C. (Los Gatos CA), Hierarchical interconnection network architecture for parallel processing, having interconnections between bit-addressib.
Flaig Charles M. (Pasadena CA) Seitz Charles L. (San Luis Rey CA), Inter-computer message routing system with each computer having separate routinng automata for each dimension of the net.
Blumrich, Matthias A.; Chen, Dong; Chiu, George L.; Cipolla, Thomas M.; Coteus, Paul W.; Gara, Alan G.; Giampapa, Mark E.; Heidelberger, Philip; Kopcsay, Gerard V.; Mok, Lawrence S.; Takken, Todd E., Massively parallel supercomputer.
Carmichael Richard D. ; Ward Joel M. ; Winchell Michael A., Method and apparatus for controlling (N+I) I/O channels with (N) data managers in a homogenous software programmable en.
Rangarajan, Vijay; Maniyar, Shyamsundar N.; Eatherton, William N., Method and apparatus for storing tree data structures among and within multiple memory channels.
Rangarajan,Vijay; Maniyar,Shyamsundar N.; Eatherton,William N., Method and apparatus for storing tree data structures among and within multiple memory channels.
Birrittella Mark S. ; Kessler Richard E. ; Oberlin Steven M. ; Passint Randal S. ; Thorson Greg, Multiprocessor computer system with interleaved processing element nodes.
Krishnamoorthy Ashok V. (11188 Caminito Rodar San Diego CA 92126) Kiamilev Fouad (c/o UNC Charlotte ; Dept. of EE ; Smith Hall Room 332 Charlotte NC 28223), Packet-switched self-routing multistage interconnection network having contention-free fanout, low-loss routing, and fan.
Yasuda Yoshiko,JPX ; Tanaka Teruo,JPX, Parallel computer system using properties of messages to route them through an interconnect network and to select virtua.
Wilkinson Paul Amba ; Dieffenderfer James Warren ; Kogge Peter Michael ; Schoonover Nicholas Jerome, Partitioning of processing elements in a SIMD/MIMD array processor.
VanHuben Gary Alan ; Blake Michael A. ; Mak Pak-kin, SMP clusters with remote resource managers for distributing work to other clusters while reducing bus traffic to a minimum.
Padmanabha I. Venkitakrishnan ; Gopalakrishnan Janakiraman ; Tsen-Gong Jim Hsu ; Rajendra Kumar, Scalable system control unit for distributed shared memory multi-processor systems.
Kil, David H.; Pottschmidt, David B., System and method for automatic generation of a hierarchical tree network and the use of two complementary learning algorithms, optimized for each leaf of the hierarchical tree network.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.