A video visual and audio query system for quickly identifying video within a large known corpus of videos being played on any screen or display. In one embodiment, the system can record via a mobile phone camera and microphone a live video clip from the TV and transcode it into a sequence of frame-s
A video visual and audio query system for quickly identifying video within a large known corpus of videos being played on any screen or display. In one embodiment, the system can record via a mobile phone camera and microphone a live video clip from the TV and transcode it into a sequence of frame-signatures. The signatures representative of the clips can then be matched against the signatures of the TV content in a corpus across a network to identify the correct TV show or movie.
대표청구항▼
1. A method, comprising: receiving, by a computing device, content comprising a media clip including audio and video;analyzing, by the computing device, at least a portion of the content, said analyzing including preparing query frames of the content by selecting representative frames in the content
1. A method, comprising: receiving, by a computing device, content comprising a media clip including audio and video;analyzing, by the computing device, at least a portion of the content, said analyzing including preparing query frames of the content by selecting representative frames in the content and selecting at least one of the audio and video in the content to generate a signature for a search query;generating, by the computing device, the signature, the signature including a set of bits identifying the content, wherein the set of bits are determined from a plurality of regions of a video frame of the representative frames in the content, a bit of the set of bits associated with a region of the plurality of regions, the bit being assigned a first value if a predetermined half of the region is brighter than the other half of the region, and the bit being assigned a second value if the predetermined half of the region is not brighter than the other half of the region; andreceiving video search results resulting from a comparison of the generated signature in a search query against a search corpus to identify the media clip in the search corpus. 2. The method of claim 1, wherein the receiving by the computing device occurs while the media clip is playing on a different device. 3. The method of claim 1, wherein the content is video and the analyzing including detecting an active play screen area by analyzing intensity edges of the at least a portion of the content. 4. The method of claim 1, wherein the comparison includes randomly selecting multiple collections of a fixed number of bits and comparing the multiple collections against a search corpus. 5. The method of claim 1, wherein the predetermined half of the region is a right half or a left half of the region. 6. The method of claim 1, wherein the predetermined half of the region is a top half or a bottom half of the region. 7. The method of claim 1, wherein the at least a portion of the content is a spectrogram of the audio. 8. The method of claim 7, wherein the analyzing further includes creating a signature from audio in the at least a portion of the content, each bit therein being associated with a change in energy in a frequency band of the spectrogram. 9. The method of claim 8, wherein each bit is assigned a first value if the change in energy represents an increase in energy, and each bit is assigned a second value if the change in energy represents a decrease in energy. 10. A computer-readable storage device embodied with instructions to cause, when executed, a computing device to: receive content comprising a query frame including video content and audio content;analyze the query frame;generate a signature of the video content of the query frame wherein each bit of the signature is associated with a brightness of a region of the frame and each bit is assigned a first value if a predetermined half of the region is brighter than the other half of the region, and each bit is assigned a second value if the predetermined half of the region is not brighter than the other half of the region; anduse the signature of the audio content and the signature of the video content in a search query including randomly selecting multiple collections of a sub-set of bits in the signature of the video content and comparing the multiple collections against a search corpus. 11. A method comprising: receiving, by a first computing device from a second computing device, a signature of a portion of a media clip, the signature representing a recording of a video playing on a screen of a different device which is captured by the second computing device, the signature generated from selected query frames of either or both of audio content or video content in the media clip;using, by a first computing device, the signature in a search query by comparing the signature against a search corpus to reduce a set of result candidates for the search query the comparing including a coarse direct comparison of the signature to ground truth signatures and, if needed, a second comparison to an index of the search corpus, the comparing to identify the video playing on the screen of the different device; and outputting results of the search query from the first computing device to the second computing device. 12. A method of claim 11, wherein the search query matches the signature to a corpus of ground truth signatures specifying an identity of the selected query frames as a video. 13. A method of claim 11, wherein the query frame is created by determining boundaries of the screen by looking for lines that have an intensity variance indicating one or more of the lines is not dark along an inward edge relative to the screen and an intensity variance indicating the one or more lines are dark along an external edge relative to the screen. 14. A method of claim 13, wherein the determining of the boundaries includes filtering out lines that cut through a continuous object. 15. A method of claim 13, wherein the determining of the boundaries includes selecting lines located within the video. 16. A computing device comprising a processor configured to: receive a recording of a video playing on a screen of a different device, the recording including a video portion and an audio portion;determine boundaries of the screen in video portion of the recording by determining lines that have an intensity variance indicating one or more of the lines is not dark along an inward edge relative to the screen and an intensity variance indicating the one or more lines are dark along an external edge relative to the screen;extract the video within the determined boundaries from the recording;extract the audio portion from the recording;generate a signature for the audio portion and a bit wise signature for the video portion and use the signatures in one or more search queries to identify the recording. 17. A method comprising: receiving a recording of a video playing on a screen of a different device, the recording including a video portion and an audio portion;determining boundaries of the screen in the video portion of the recording, the determining of the boundaries includes looking for lines that have an intensity variance indicating one or more of the lines is not dark along an inward edge relative to the screen and an intensity variance indicating the one or more lines are dark along an external edge relative to the screen; andextracting the video within the determined boundaries from the recording;extracting the audio portion from the recording;generating a signature for the extracted audio portion and a bit wise signature for the extracted video portion andusing the extracted video in a search query identify the content as part of a broadcast media content.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (79)
Chang, Shih-Fu; Chen, William; Meng, Horace J.; Sundaram, Hari; Zhong, Di, Algorithms and system for object-oriented content-based video search.
Snyder Patricia D ; Pawlicki Thaddeus Francis ; Gaborski Roger Stephen, Apparatus and method for signal dependent noise estimation and reduction in digital images.
Delgo, Lior; Sharon, Eitan; Brandt, Achiezer; Borenstein, Eran; Moshe, Asael, Apparatus and software system for and method of performing a visual-relevance-rank subsequent search.
Delgo, Lior; Sharon, Eitan; Brandt, Achiezer; Borenstein, Eran; Moshe, Asael, Apparatus and software system for and method of performing a visual-relevance-rank subsequent search.
Yokomitsu, Sumio; Sotodate, Hiromichi; Yan, Hailin; Lee, Chak Joo, Color correction device, color correction method, dynamic camera color correction device, and video search device using the same.
Lo Jim C. K. (San Jose CA) Lung James C. Y. (San Jose CA), Fast vertical scan-conversion and filling method and apparatus for outline font character generation in dot matrix devic.
Cavallerano,Alan P.; Shen,Richard C.; Ciacci,Claudio, Method and device for detecting an event in a program of a video and/or audio signal and for providing the program to a display upon detection of the event.
Thomas William L. (Arapahoe County CO) Sletten Steven J. (Aurora CO) Mathews ; Jr. John W. (Pine CO) Swinehart Jeffrey C. (Arapahoe County CO) Fellinger Michael W. (Boulder CO) Hershey John E. (Bould, Method, apparatus and system for recognizing broadcast segments.
Gosse, David B.; Feindel, Tym D.; Kim, Jungho; Nutzman, Justin R.; Nutzman, Jason P.; Winters, Michael T.; Gosse, Jennifer L., Method, device and software for querying and presenting search results.
Jonathan T. Foote ; Lynn Wilcox, Methods and apparatuses for segmenting an audio-visual recording using image similarity searching and audio speaker recognition.
Broder,Andrei Z; Carmel,David; Herscovici,Michael; Soffer,Aya; Zien,Jason, System, method and computer program product for performing unstructured information management and automatic text analysis, including a search operator functioning as a Weighted AND (WAND).
Koichi Morita JP; Mitsuo Asai JP; Tomoyuki Ishida JP; Takeshi Onodera JP; Yoshihiro Takiyasu JP, Video searching method and apparatus, video information producing method, and storage medium for storing processing program thereof.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.