Systems and methods for encoding and playing back video at different frame rates using enhancement layers
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
H04N-007/173
H04N-021/2343
H04N-021/258
H04N-021/6587
H04N-021/854
출원번호
US-0679469
(2015-04-06)
등록번호
US-9866878
(2018-01-09)
발명자
/ 주소
van der Schaar, Auke Sjoerd
Funnell, John
출원인 / 주소
Sonic IP, Inc.
대리인 / 주소
KPPB LLP
인용정보
피인용 횟수 :
4인용 특허 :
220
초록▼
Systems and methods in accordance with embodiments of this invention provide for encoding and playing back video at different frame rates using enhancement layers. In a number of embodiments, video is encoded using temporal scalability involving the creation of a base layer at a first frame rate and
Systems and methods in accordance with embodiments of this invention provide for encoding and playing back video at different frame rates using enhancement layers. In a number of embodiments, video is encoded using temporal scalability involving the creation of a base layer at a first frame rate and an enhancement layer including additional frames enabling playback at a second higher frame rate. The second higher frame rate can also be referred to as an enhanced frame rate. In a number of embodiments, the base and enhancement layers are stored in one or more container files that contain metadata describing the enhancement layer. Based on the capabilities of a playback device, it can select the particular frame rate at which to playback encoded video.
대표청구항▼
1. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process comprising: obtain video comprising a merged plurality of access units with temporal scalability, each access unit in the merged
1. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process comprising: obtain video comprising a merged plurality of access units with temporal scalability, each access unit in the merged plurality of access units associated with a particular layer of a plurality of temporal layers based on a temporal identifier of the access unit that identifies the particular layer, wherein the temporal layers include a base layer and one or more enhancement layers;obtain an ordered sequence of access units ordered based on a decoding order for decoding the video and generating metadata specifying an ordered sequence of temporal identifiers ordered based on, for each access unit in the ordered sequence of access units, identifying and retaining the temporal identifier associated with the access unit;separate, based on the temporal identifier of each access unit in the ordered sequence of access units, the ordered sequence of access units into (1) a base layer of access units sequenced in decode order to provide a base frame rate and (2) at least one enhancement layer with additional access units ordered in decode order to provide an enhanced frame rate, wherein at least some of the access units in the enhancement layer are encoded based upon dependencies to access units in the base layer, and wherein access units in the base layer are encoded without dependencies upon access units in the enhancement layer; andstoring the base layer in a first container file; andstoring the enhancement layer, including the generated metadata specifying the ordered sequence of temporal identifiers, in a second container file separate from the first container file, wherein the metadata is used to recombine in decode order the access units stored in the second container file with access units stored in the first container file for playback at the enhanced frame rate. 2. The non-transitory machine readable medium of claim 1, wherein the temporal identifier of an access unit is stored in an initial set of bits of the access unit. 3. The non-transitory machine readable medium of claim 1, further comprising inserting a temporal identifier for a set of access units of the plurality of access units of the video. 4. The non-transitory machine readable medium of claim 1, further comprising inserting a metadata tag providing information regarding the layers available and the frame rate of each layer. 5. The non-transitory machine readable medium of claim 1, further comprising: receiving a request for streaming of the video;determining playback capabilities of a device requesting the video;streaming different sets of layers of the video based on the playback capabilities of the device, wherein (i) the base layer is streamed to a playback device capable of playing back the base layer and (ii) the base layer and at least one enhancement layer is streamed to a playback device capable of streaming the plurality of layers. 6. The non-transitory machine readable medium of claim 5, wherein streaming the different sets of layers comprises merging the different layers using the metadata that specifies the sequence for combining access units from the base layer with access units from the enhancement layer. 7. The non-transitory machine readable medium of claim 1, further comprising: receiving a request for downloading of the video from a playback device;providing the encoded video to the playback device. 8. The non-transitory machine readable medium of claim 1, wherein the frame rate of the at least one enhancement layer is an integer multiple of the base layer. 9. The non-transitory machine readable medium of claim 1, further comprising storing each layer as a separate stream. 10. The non-transitory machine readable medium of claim 1, further comprising storing each layer in a separate container file. 11. A source encoder, comprising: a processor configured by a source encoder application to:obtain video comprising a merged plurality of access units with temporal scalability, each access unit in the merged plurality of access units associated with a particular layer of a plurality of temporal layers based on a temporal identifier of the access unit that identifies the particular layer, wherein the temporal layers include a base layer and one or more enhancement layer;obtain an ordered sequence of access units ordered based on a decoding order for decoding the video and generating metadata specifying an ordered sequence of temporal identifiers ordered based on, for each access unit in the ordered sequence of access units, identifying and retaining the temporal identifier associated with the access unit;separate, based on the temporal identifier of each access unit in the ordered sequence of access units, the ordered sequence of access units into (1) a base layer of access units sequenced in decode order to provide a base frame rate and (2) at least one enhancement layer with additional access units ordered in decode order to provide an enhanced frame rate, wherein at least some of the access units in the enhancement layer are encoded based upon dependencies to access units in the base layer, and wherein access units in the base layer are encoded without dependencies upon access units in the enhancement layer;storing the base layer in a first container file; andstoring the enhancement layer, including the generated metadata specifying the ordered sequence of temporal identifiers, in a second container file separate from the first container file, wherein the metadata is used to recombine in decoding order the access units stored in the second container file with access units stored in the first container file for playback at the enhanced frame rate. 12. The source encoder of claim 11, wherein the temporal identifier of an access unit is stored in a header of the access unit. 13. The source encoder of claim 11, further comprising inserting a temporal identifier for a set of access units of the plurality of access units of the video. 14. The source encoder of claim 11, further comprising inserting a metadata tag providing information regarding the layers available and the frame rate of each layer. 15. A playback device configured to playback video comprising: a processor configured to communicate with a memory, where the memory contains a client application;wherein the client application configures the processor to:obtain a first container file comprising a base layer of access units ordered in decode order for playback at a base frame rate;obtain, for a device capable of playing back an enhancement layer, a second container file comprising (1) at least one enhancement layer with additional access units ordered in decode order for playback at an enhanced frame rate and (2) metadata specifying an ordered sequence of temporal identifiers for combining, in decode order, access units from the base layer with access units from the at least one enhancement layer for playback at the enhanced frame rate, wherein each access unit includes a temporal identifier identifying a particular layer associated with the access unit;combine the base layer of access units with the at least one enhancement layer based on the metadata obtained from the second container file to obtain a combined sequence of access units ordered in a decoding order at the enhanced frame rate;wherein at least some of the access units in the at least enhancement layer stored in the second container file are decoded based upon dependencies to access units in the base layer stored in the first container file, and wherein access units in the base layer that are stored in the first container file are decoded without dependencies upon access units in the enhancement layer. 16. The playback device of claim 15, wherein the client application further configures the processor to: request streaming of the video to a playback server and provide information regarding playback capabilities of the playback device to the playback server;receive the first container file comprising a first stream comprising the base layer of the video and the second container file comprising a second stream comprising the at least one enhancement layer of the video. 17. The playback device of claim 15, wherein the client application further configures the processor to: request downloading of the video to a content distribution server and provide playback capabilities of the playback device to the content distribution server;receive the video from the content distribution server, wherein a first device receives the first container file comprising the base layer when the first device is capable of playing back the video at the base frame rate and a second device receives the first container file and the second container file comprising the base layer and at least one enhancement layer when the second device is capable of playing back the video at the enhanced frame rate. 18. The playback device of claim 15, wherein the client application further configures the processor to: retrieve a metadata tag from the video identifying the layers available and the frame rate of each layer;determining a set of layers to playback based on the information in the metadata tag. 19. The playback device of claim 15, wherein the client application further configures the processor to: identify the temporal identifier of an access unit by parsing an initial set of bytes of the access unit.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (220)
Galligan, Francis, Adaptive video streaming using time-to-offset mapping.
Boroczky, Lilla; Ngai, Agnes Y.; Westermann, Edward F., Adaptively encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel.
Branstad Mark W. (Rochester MN) Byrn Jonathan W. (Rochester MN) Delp Gary S. (Rochester MN) Leichty Philip L. (Rochester MN) Lynch Jeffrey J. (Apex NC) Plotz Kevin G. (Byron MN) Sendelbach Lee A. (Ro, Apparatus and method for segmentation and time synchronization of the transmission of multimedia data.
Carmel, Sharon; Gill, Dror; Shoham, Tamar, Apparatus and methods for re-compression having a monotonic relationship between extent of compression and quality of compressed image.
Nonomura Tomoyuki,JPX ; Inoue Mitsuhiro,JPX ; Uesaka Yasushi,JPX ; Kozuka Masayuki,JPX, Apparatus for generating sub-picture units for subtitles and storage medium storing sub-picture unit generation program.
Kollmyer,Brad; Baker,Brian; Shapiro,Eric; Kollmyer,Aric; Rutman,Mike; MacLean,Duncan; Robertson,Dan; Taylor,Neal; Hunsche,Dick; Walker,Amanda, Apparatus, system and method for selectively encrypting different portions of data sent over a network.
Walsh Bruce E. ; Herdrich John ; Smith William ; Vrabel Mark E. ; Borghesani Philip ; Hagberg Christine G. ; Champagne Karen, Computer based video system.
Beitel Bradley J. (Woodside CA) Bishop Mark S. (Austin TX) Burns Nancy A. (Austin TX) Deacon John J. (Austin TX) Gordon Robert D. (Sunnyvale CA) Haug Charles L. (Santa Cruz CA) Smith Kenneth B. (Palo, Computer-based, audio/visual creation and presentation system and method.
Beitel Bradley J. (Woodside CA) Bishop Mark S. (Austin TX) Deacon John J. (Austin TX) Gordon Robert D. (Sunnyvale CA) Smith Kenneth B. (Palo Alto CA) Walling Lonnie S. (Ben Lomond CA) Wilkes Michael , Computer-based, audio/visual creation and presentation system and method.
Adams Robert (Lake Oswego OR) Williams David M. (Portland OR) Richardson John (Portland OR) Perry Burt (Beaverton OR), Content programmer control of video and data display using associated data.
Boroczky, Lilla; Ngai, Agnes Y.; Westermann, Edward F., Control strategy for dynamically encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel.
Katayama Hironobu,JPX ; Nogami Hiroaki,JPX ; Shiraishi Kenichi,JPX, Digital recording and reproducing apparatus which multiplexes and records HDTV, SDTV and trick play data together on a magnetic tape.
Grab, Eric William; Russell, Chris; Chan, Francis Yee-Dug; Kiefer, Michael George, Federated digital rights management scheme including trusted systems.
Lyons Paul Wallace ; Beltz John Prickett ; Acampora Alfonse Anthony, Flow control, latency control, and bitrate conversions in a timing correction and frame synchronization apparatus.
Pensak David A. ; Cristy John J. ; Singles Steven J., Information security architecture for encrypting documents for remote access while maintaining access control.
Ando,Hideo; Kikuchi,Shinichi; Taira,Kazuhiko; Ito,Yuji, Information storage system capable of recording and playing back a plurality of still pictures.
Rossum David P. ; Guzewicz Michael ; Crawford Robert S. ; Williams Matthew F. ; Ruffcorn Donald F., Method and apparatus for formatting digital audio data.
Kwon, O-hoon; Ha, Ho-jin; Zhang, Guanhua; Choi, Hyung-tak; Kim, Sun-bal; Keum, Ji-Eun; Im, Sung-bin, Method and apparatus for providing trick play service.
Donald F. Gordon ; Ted Ludvig, Method and apparatus for simultaneously producing compressed play and trick play bitstreams from a video frame sequence.
Boyce Jill MacDonald ; Lane Frank Anton, Method and apparatus for the selection of data for use in VTR trick playback operation in a system using intra-coded vid.
Suzuki Hideharu (Tokyo JPX) Ishibashi Yutaka (Aichi JPX) Nishimura Kazutoshi (Tokyo JPX), Method and apparatus for visual search control of digital video files.
Woo,Je Hak; Lee,Hwan Chul; Cho,Sang Young; Jeong,Seong Ho; Ha,Young Soo; Shin,Seog Kyoon; Kim,Seong Il, Method and system for the information protection of digital content.
Carmel Sharon,ILX ; Daboosh Tzur,ILX ; Reifman Eli,ILX ; Shani Naftali,ILX ITX 62917, Method and system of building and transmitting a data file for real time play of multimedia, particularly animation, and.
DeWitt Frederick J. (Kirkland WA) Stewart Peter B. (Redmond WA), Method and system using pathnames to specify and access audio data at fidelity levels other than the level at which the.
Kim,Kwang Su, Method for creating caption-based search information of moving picture data, searching and repeating playback of moving picture data based on said search information, and reproduction apparatus using said method.
Yoshida Masayuki (Yokohama JPX) Nakanishi Keiko (Tokyo JPX), Method of commonly using font information for outputting information in a system having a plurality of information proce.
Foehr,Oliver H.; Dunietz,Jerry; Leonov,Mikhail V.; Brown,David C.; Daniels,Simon, Methods for providing an accurate visual rendition of a text element formatted with an unavailable font.
Morris James K. (Arlington VA) Probst Robert E. (Reston VA) Utley Robert A. (Randallstown MD), Multimedia complex form creation, display and editing method apparatus.
Dinallo Chris A. (Boca Raton FL) Koval Michael J. (Boca Raton FL) Lawton William W. (Boca Raton FL) Paulat ; Jr. Martin J. (Delray Beach FL) Tyler John G. (Boynton Beach FL) Winters Scott L. (Plantat, Personal computer with generalized data streaming apparatus for multimedia devices.
Okubo, Masafumi; Oashi, Masahiro; Ohto, Hidetaka, Playback device, playback method, and computer-readable recording medium for ensuring stable application execution in synchronism with video data playback.
Pantos, Roger; Biderman, David; May, Jr., William; Flick, Christopher; Bushell, John Samuel; Calhoun, John Kevin, Playlists for real-time or near real-time streaming.
Gutfreund, Steven; Nicol, John R.; Rush, Kimberly; Martin, Christopher M.; Paschetto, James, Real time media journaler with a timing event coordinator.
Tanaka Yasuyuki,JPX ; Fujii Akio,JPX ; Kato Masato,JPX, Recording apparatus for newly recording a second encoded data train on a recording medium on which an encoded data train is recorded.
Nogami Hiroaki,JPX ; Katayama Hironobu,JPX, Recording apparatus which adaptively records amounts of low resolution video inversely to amounts of high resolution video which has the same program content.
Kollmyer,Brad; Baker,Brian A.; Shapiro,Eric Bradley; Kollmyer,Aric; Rutman,Mike; MacLean,Charles Duncan; Robertson,Dan; Taylor,Neal; Hunsche,Dick; Walker,Amanda, Selectively encrypting different portions of data sent over a network.
Soroushian, Kourosh; Salmonsen, Dan; Landeros, Porfirio; Priyadarshi, Shaiwal; Gravel, Francois, Singular, collective and automated creation of a media guide for online content.
Goldmark Peter C. (Stamford CT) Dolan Donald T. (Ridgefield CT) Desai Arvind C. (Stamford CT) Hollywood John M. (Old Greenwich CT), Still picture program video recording composing and playback method and system.
Zintel, William Michael; Christian, Brian S.; Christian, Bradford A., Synchronization of controlled device state using state table and eventing in data-driven remote device control model.
Abbott Michael J. ; Close Paul ; Smith Kevin P., System and method for media stream synchronization with a base atom index file and an auxiliary atom index file.
Dockter Michael J. (Hollister CA) Haug Charles L. (Santa Cruz CA) Seppi Kevin D. (Austin TX), System and method for synchronization of multimedia streams.
Chen Huey-Shiang ; Chen Mon-Song ; Huang Shiow-Laang ; Song Deyang, System for just-in-time retrieval of multimedia files over computer networks by transmitting data packets at transmissio.
Braness, Jason; van der Schaar, Auke Sjoerd; Soroushian, Kourosh, Systems and methods for adaptive bitrate streaming of media stored in matroska container files using hypertext transfer protocol.
Soroushian, Kourosh; Douglas, Scott; Bramwell, Steve; Braness, Jason; Shivadas, Abhishek, Systems and methods for encoding media including subtitles for adaptive bitrate streaming.
Braness, Jason; van der Schaar, Auke Sjoerd; Soroushian, Kourosh, Systems and methods for encoding source media in matroska container files for adaptive bitrate streaming using hypertext transfer protocol.
Braness, Jason; Wallin, Evan; Ferreira, Ederson, Systems and methods for performing adaptive bitrate streaming using automatically generated top level index files.
Braness, Jason; Soroushian, Kourosh; van der Schaar, Auke Sjoerd, Systems and methods for performing smooth visual search of media encoded for adaptive bitrate streaming via hypertext transfer protocol using trick play streams.
Kiefer, Michael George; Grab, Eric William; Braness, Jason, Systems and methods for playing back alternative streams of protected content protected using common cryptographic information.
Kiefer, Michael George; Grab, Eric William; Braness, Jason, Systems and methods for playing back alternative streams of protected content protected using common cryptographic information.
Kiefer, Michael George; Grab, Eric William; Braness, Jason, Systems and methods for protecting alternative streams in adaptive bitrate streaming systems.
Ginter Karl L. ; Shear Victor H. ; Sibert W. Olin ; Spahn Francis J. ; Van Wie David M., Systems and methods for secure transaction management and electronic rights protection.
Eerenberg, Onno; Rijckaert, Albert M. A.; Bruls, Wilhelmus H. A., Trick play signal generation for a digital video recorder using retrieved intra-encoded pictures and generated inter-encoded pictures.
Sakamoto Hideki,JPX ; Kawaguchi Tomoaki,JPX ; Suzuki Hideharu,JPX ; Mori Tatsuo,JPX ; Kimiyama Hiroyuki,JPX ; Nishimura Kazutoshi,JPX ; Itoh Kouichi,JPX ; Maruyama Mitsuru,JPX ; Sugiyama Hiroshi,JPX , Video data playback system using effective scheme for producing coded video data for fast playback mode.
Branstad Mark W. (Rochester MN) Byrn Jonathan W. (Rochester MN) Delp Gary S. (Rochester MN) Leichty Phillip L. (Rochester MN) Lynch Jeffrey J. (Apex NC) Plotz Kevin G. (Byron MN) Sendelbach Lee A. (R, apparatus and method for segmentation and time synchronization of the transmission of multimedia data.
Kiefer, Michael George; Grab, Eric William; Braness, Jason, Playback devices and methods for playing back alternative streams of content protected using a common set of cryptographic keys.
Kiefer, Michael George; Grab, Eric William; Braness, Jason, Systems and methods for playing back alternative streams of protected content protected using common cryptographic information.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.