On-disk file format for a serverless distributed file system
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04L-029/00
출원번호
US-0014030
(2004-12-16)
등록번호
US-7454612
(2008-11-18)
발명자
/ 주소
Bolosky,William J.
Cermak,Gerald
Adya,Atul
Douceur,John R.
출원인 / 주소
Microsoft Corporation
대리인 / 주소
Lee & Hayes, PLLC
인용정보
피인용 횟수 :
27인용 특허 :
100
초록▼
A file format for a serverless distributed file system is composed of two parts: a primary data stream and a metadata stream. The data stream contains a file that is divided into multiple blocks. Each block is encrypted using a hash of the block as the encryption key. The metadata stream contains a
A file format for a serverless distributed file system is composed of two parts: a primary data stream and a metadata stream. The data stream contains a file that is divided into multiple blocks. Each block is encrypted using a hash of the block as the encryption key. The metadata stream contains a header, a structure for indexing the encrypted blocks in the primary data stream, and some user information. The indexing structure defines leaf nodes for each of the blocks. Each leaf node consists of an access value used for decryption of the associated block and a verification value used to verify the encrypted block independently of other blocks. In one implementation, the access value is formed by hashing the file block and encrypting the resultant hash value using a randomly generated key. The key is then encrypted using the user's key as the encryption key. The verification value is formed by hashing the associated encrypted block using a one-way hash function. The file format supports verification of individual file blocks without knowledge of the randomly generated key or any user keys. To verify a block of the file, the file system traverses the tree to the appropriate leaf node associated with a target block to be verified. The file system hashes the target block and if the hash matches the access value contained in the leaf node, the block is authentic.
대표청구항▼
The invention claimed is: 1. A method implemented at least in part by a machine comprising: segmenting a sparse file into multiple blocks; differentiating non-data blocks in the sparse file that contain no substantive content from data blocks in the sparse file that contain substantive data; creati
The invention claimed is: 1. A method implemented at least in part by a machine comprising: segmenting a sparse file into multiple blocks; differentiating non-data blocks in the sparse file that contain no substantive content from data blocks in the sparse file that contain substantive data; creating an indexing structure to index individual blocks; and deallocating storage of both the non-data blocks and portions of the indexing structure that reference the non-data blocks that contain no substantive content, wherein the sparse file and the indexinci structure are reduced in size. 2. A method as recited in claim 1, further comprising storing the data blocks in a data stream and the indexing structure in a metadata stream. 3. A method as recited in claim 1, further comprising: computing a hash of each of the data blocks to produce block hash values; and encrypting the data blocks using their corresponding block hash values as encryption keys to produce encrypted data blocks. 4. A method as recited in claim 3, wherein the indexing structure contains first leaf nodes for each corresponding data blocks and second leaf nodes for each corresponding non-data block, the first leaf nodes containing an access value for use in decrypting the encrypted data blocks and a verification value for use in verifying individual encrypted data block independently of other encrypted data blocks. 5. One or more computer readable storage media comprising computer-executable instructions that, when executed, perform the method as recited in claim 1. 6. A method comprising: segmenting a sparse file into multiple blocks, the sparse file containing at least one non-data block that contains no substantive data; differentiating the non-data blocks from data blocks of the sparse file that contain substantive data; computing hashes of each of the data blocks to produce block hash values; encrypting the data blocks using their corresponding block hash values as encryption keys to produce encrypted data blocks; creating an indexing structure to index individual blocks, the indexing structure containing first leaf nodes for each corresponding encrypted data block and second leaf nodes for each corresponding non-data block, the first leaf nodes containing an access value formed by encrypting the block hash value for the corresponding encrypted block using an access key and a verification value formed by hashing the corresponding encrypted block; and setting the second leaf nodes to a first binary value. 7. A method as recited in claim 6, further comprising storing the encrypted blocks in a data stream and the indexing structure in a metadata stream. 8. A method as recited in claim 7, further comprising deallocating portions of the metadata stream that hold the second leaf nodes. 9. One or more computer readable storage media comprising computer-executable instructions that, when executed, perform the method as recited in claim 6. 10. One or more computer readable storage media comprising computer-executable instructions that, when executed, direct a computing device to: segment a sparse file into multiple blocks, the sparse file containing at least one non-data block that contains no substantive data; differentiate the non-data blocks from data blocks of the sparse file that contain substantive data; compute hashes of each of the data blocks to produce block hash values; encrypt the data blocks using their corresponding block hash values as encryption keys to produce encrypted data blocks; creating an indexing structure to index the non-data blocks and the encrypted data blocks; and deallocate portions of the indexing structure that reference the non-data blocks. 11. A component in a device in a distributed file system in which files are stored across multiple distributed computers, the component comprising: a segmenting module to divide a sparse file into multiple blocks, the sparse file containing at least one non-data block that contains no substantive data; a control module to differentiate the non-data blocks from data blocks of the sparse file that contain substantive data; a hash module to hash each of the data blocks to produce block hash values; a cryptographic engine to encrypt the data blocks using their corresponding block hash values as encryption keys to produce encrypted blocks; and an index builder to create an indexing structure to index individual blocks, the indexing structure containing first leaf nodes for each corresponding encrypted block and second leaf nodes for each corresponding non-data block, the first leaf nodes containing an access value formed by encrypting the block hash value for the corresponding encrypted block using an access key and a verification value formed by hashing the corresponding encrypted block, the second leaf nodes being set to a first binary value. 12. A component as recited in claim 11, wherein the encrypted blocks are stored in a data stream and the indexing structure is stored in a metadata stream. 13. A component as recited in claim 12, wherein the control module deallocates portions of the metadata stream that hold the second leaf nodes. 14. A method implemented at least in part by a machine comprising: segmenting a sparse file into multiple blocks; differentiating non-data blocks in the sparse file that contain no substantive content from data blocks in the sparse file that contain substantive data; creating an indexing structure to index individual blocks; deallocating storage of both the non-data blocks and portions of the indexing structure that reference the non-data blocks that contain no substantive data, wherein the sparse file is reduced in size; computing a hash of each of the data blocks to produce block hash values; and encrypting the data blocks using their corresponding block hash values as encryption keys to produce encrypted data blocks. 15. A method as recited in claim 14, further comprising storing the data blocks in a data stream and the indexing structure in a metadata stream. 16. A method as recited in claim 14, wherein the indexing structure contains first leaf nodes for each corresponding data blocks and second leaf nodes for each corresponding non-data block, the first leaf nodes containing an access value for use in decrypting the encrypted data blocks and a verification value for use in verifying individual encrypted data block independently of other encrypted data blocks. 17. One or more computer readable storage media comprising computer-executable instructions that, when executed, perform the method as recited in claim 14.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (100)
Schmidt, Rene W.; Muller, Hans E.; Violet, Scott R., Apparatus and method for incremental updating of archive files.
Hirayama Hideaki,JPX ; Shirakihara Toshio,JPX, Checkpointing computer system having duplicated files for executing process and method for managing the duplicated files.
Drummond Shattuck Reed ; Peter Earnshaw Heymann ; Steven Mark Mushero ; Kevin Benard Jones ; Jeffrey Todd Oberlander ; Dan Banay, Computer-based communication system and method using metadata defining a control-structure.
Govindarajan Rangaprasad ; Edwards Russ ; Bannister Cecil ; Gostanian ; Jr. Raffi, Data processing system and method for providing personal information in a communication network.
Arnold Kenneth C. R. C. ; Waldo James H. ; Riggs Roger ; Wollrath Ann M. ; Jones Peter, Database system employing polymorphic entry and entry matching.
Berbec Robert R. ; Brady John L. ; Caffrey James M. ; Crimi Joanne T. ; Fenaroli Arthur P. ; Iatridis Matthew C. ; Puchkoff Gary S. ; Sanchez Roberto J. ; Whalen Madeline R. ; Williams Joseph A., Dynamic server-managed access control for a distributed file system.
Terada Hiroaki,JPX ; Nishikawa Hiroaki,JPX ; Yoshida Shin-ichi,JPX ; Hine Shunji,JPX ; Nishikawa Youichiro,JPX ; Hara Shuji,JPX ; Shima Kenji,JPX ; Inaoka Yoshie,JPX ; Yamasaki Tetsuo,JPX, File management apparatus permitting access to portions of a file by specifying a data structure identifier and data ele.
Alam Salim ; Bhalerao Vinayak A. ; Wu Charles ; Hu George ; Ferrell John I., File object synchronization between a desktop computer and a mobile device.
Alger Jeffrey H. ; Bennett John G. ; Marshall David A. ; Shutt David R., Generating and compressing universally unique identifiers (UUIDs) using counter having high-order bit to low-order bit.
Steven R. Soltis ; Matthew T. O'Keefe ; Thomas M. Ruwart ; Gerald A. Houlder ; James A. Coomes ; Michael H. Miller ; Edward A. Soltis ; Raymond W. Gilson ; Kenneth W. Preslan, Global file system and data storage device locks.
East Jeffrey A. ; Lingelbach Albert L. ; Lindell Steven J. ; Graefe Goetz ; Zastera Craig G. ; Agarwal Sameet H., Linked data structure integrity verification system which verifies actual node information with expected node information stored in a table.
Douceur,John R.; Theimer,Marvin M.; Adya,Atul; Bolosky,William J., Locating potentially identical objects across multiple computers based on stochastic partitioning of workload.
Douceur,John R.; Theimer,Marvin M.; Adya,Atul; Bolosky,William J., Locating potentially identical objects across multiple computers based on stochastic partitioning of workload.
Sudama Ram (Concord MA) Magid David L. (Worcester MA) Ouellette Kenneth W. (Groton MA), Mechanism for locating without search discrete application resources known by common name only in a distributed network.
Coates, Joshua L.; Bozeman, Patrick E.; Patterson, David A., Method and apparatus for accessing remote storage in a distributed storage cluster architecture.
Cohen, Marc L.; Cooper, Michael R.; Nogay, Patrick Edward; Vanderwiele, Mark Wayne, Method and apparatus for compacting a metadatas stream in a data processing system.
Cooper Michael R. ; Hamzy Mark Joseph ; Nogay Patrick Edward ; Vanderwiele Mark Wayne, Method and apparatus for creating metadata streams with embedded device information.
O'Brien,Michael R.; Beckhardt,Steven R.; Lorenson,Elizabeth A.; Conmy,Douglas W., Method and apparatus for encrypting electronic messages composed using abbreviated address books.
Stevenson, David James; Gray, Andrew Hunter; Duncan, Robert James; Chisholm, Alastair Hugh; Serra, Vanessa; Tinto, Colin, Method and apparatus for fetching sparsely indexed MIB tables in managed network systems.
Logan Keith W. ; Misra Pradyumna K. ; Leach Paul J. ; Van Dyke Clifford P. ; Straube Dave D. ; Miller Arnold S., Method and apparatus for representing and applying network topological data.
Gramlich Wayne C. (Sunnyvale CA) Tirfing Soren J. (Palo Alto CA), Method and apparatus for the naming of database component files to avoid duplication of files.
Garay Juan Alberto ; Gennaro Rosario ; Jutla Charanjit Singh ; Rabin Tal D., Method and apparatus for the secure distributed storage and retrieval of information.
Cohen, Marc L.; Cooper, Michael R.; Nogay, Patrick Edward; Vanderwiele, Mark Wayne, Method and apparatus in a data processing system for generating metadata streams with per page data.
Sarit Mukherjee ; Ibrahim Kamel ; Prasant Mohapatra, Multimedia file systems using file managers located on clients for managing network attached storage devices.
Schmuck Frank B. ; Zlotek Anthony J. ; Shmueli Boaz,ILX ; Mandler Benjamin,ILX ; Yehudai Zvi Yosef,ILX ; Kish William A., Parallel file system with method using tokens for locking modes.
Falls Patrick T.,GBX ; Wightman Andy T.,GBX, Processes and apparatuses for generating file correspondency through replication and synchronization between target and.
Knudsen Helge (Oakville CAX) Chong Daniel T. (Woodbridge CAX) Yaffe John (Mississauga CAX) Taugher James E. (Mississauga CAX) Robertson Michael (Mississauga CAX) Plazak Zbigniew (Etobicoke CAX), Programmable computer with automatic translation between source and object code.
Jones, Michael B.; Draves, Jr., Richard P.; Rosu, Daniela; Rosu, Marcel-Catalin, Providing predictable scheduling of programs using a repeating precomputed schedule.
Matyas, Jr.,Stephen Michael; Peyravian,Mohammad; Roginsky,Allen Leonid; Zunic,Nevenko, Secure data storage and retrieval in a client-server environment.
Matyas, Jr., Stephen Michael; Peyravian, Mohammad; Roginsky, Allen Leonid; Zunic, Nevenko, Secure data storage and retrieval with key management and user authentication.
Carter John B. ; Davis Scott H. ; Dietterich Daniel J. ; Frank Steven J. ; Lee Hsin H., Shared client-side web caching using globally addressable memory.
William J. Bolosky ; John R. Douceur ; Scott M. Cutshall ; Richard F. Rashid ; Nathan P. Myhrvold ; David A. Goebel, Single instance store for file systems.
Tevis Gregory J. (Tucson AZ) Waldo Ellen J. (Tucson AZ), Storage management of a first file system using a second file system containing surrogate files and catalog management i.
Carter John B. ; Davis Scott H. ; Dietterich Daniel J. ; Frank Steven J. ; Phillips Robert S. ; Woods John ; Porter David ; Lee Hsin H., System and method for providing highly available data storage using globally addressable memory.
Vainstein, Klimenty; Nath, Satyajit; Ouye, Michael Michio, Method and apparatus for transitioning between states of security policies used to secure electronic documents.
Vainstein, Klimenty; Nath, Satyajit; Ouye, Michael Michio, Method and apparatus for transitioning between states of security policies used to secure electronic documents.
Huang, Weiqing; Supramaniam, Senthilvasan; Vainstein, Klimenty, Method and system for implementing changes to security policies in a distributed security system.
Garcia, Denis Jacques Paul; Ouye, Michael Michio; Rossmann, Alain; Crocker, Steven Toye; Gilbertson, Eric; Huang, Weiqing; Humpich, Serge; Vainstein, Klimenty; Ryan, Nicholas Michael, Methods and systems for providing access control to secured data.
Garcia, Denis Jacques Paul; Ouye, Michael Michio; Rossmann, Alain; Crocker, Steven Toye; Gilbertson, Eric; Huang, Weiqing; Humpich, Serge; Vainstein, Klimenty; Ryan, Nicholas Michael, Methods and systems for providing access control to secured data.
Garcia, Denis Jacques Paul; Ouye, Michael Michio; Rossmann, Alain; Crocker, Steven Toye; Gilbertson, Eric; Huang, Weiqing; Humpich, Serge; Vainstein, Klimenty; Ryan, Nicholas Michael, Methods and systems for providing access control to secured data.
Garcia, Denis Jacques Paul; Ouye, Michael Michio; Rossmann, Alain; Crocker, Steven Toye; Gilbertson, Eric; Huang, Weiqing; Humpich, Serge; Vainstein, Klimenty; Ryan, Nicholas Michael, Methods and systems for providing access control to secured data.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.