Systems and methods for selective retention and editing of images captured by mobile image capture device
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G06T-003/40
G06T-011/60
H04N-005/232
출원번호
US-0984628
(2015-12-30)
등록번호
US-9836819
(2017-12-05)
발명자
/ 주소
Donsbach, Aaron Michael
Vanik, Benjamin
Clapper, Jon Gabriel
Lentz, Alison
Lovejoy, Joshua Denali
Fritz, III, Robert Douglas
Duleba, Krzysztof
Zhang, Li
Payne, Juston
Fortuna, Emily Anne
Bialynicka-Birula, Iwona
Aguera-Arcas, Blaise
Ramage, Daniel
McMahan, Hugh Brendan
Lange, Oliver Fritz
Holbrook, Jess
출원인 / 주소
Google LLC
대리인 / 주소
Dority & Manning, P.A.
인용정보
피인용 횟수 :
1인용 특허 :
81
초록▼
The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. The mobile image capture device is operable to input an image into at least one neural network and to receive at least one
The present disclosure provides an image capture, curation, and editing system that includes a resource-efficient mobile image capture device that continuously captures images. The mobile image capture device is operable to input an image into at least one neural network and to receive at least one descriptor of the desirability of a scene depicted by the image as an output of the at least one neural network. The mobile image capture device is operable to determine, based at least in part on the at least one descriptor of the desirability of the scene of the image, whether to store a second copy of such image and/or one or more contemporaneously captured images in a non-volatile memory of the mobile image capture device or to discard a first copy of such image from a temporary image buffer without storing the second copy of such image in the non-volatile memory.
대표청구항▼
1. A system for selective retention and editing of images, the system comprising: a continuous capture mobile image capture device designed for resource efficiency, the mobile image capture device comprising a network interface, a power source, an image sensor, at least one processor, and a memory t
1. A system for selective retention and editing of images, the system comprising: a continuous capture mobile image capture device designed for resource efficiency, the mobile image capture device comprising a network interface, a power source, an image sensor, at least one processor, and a memory that stores at least one neural network usable to determine a desirability of a scene depicted by an image, the memory comprising at least a temporary image buffer and a non-volatile memory, the mobile image capture device operable to: capture a first image that depicts a scene;maintain a first copy of the first image in the temporary image buffer;input the first image into the at least one neural network;receive at least one descriptor of the desirability of the scene depicted by the first image as an output of the at least one neural network;determine, based at least in part on the at least one descriptor of the desirability of the scene of the first image, whether to store a second copy of the first image in the non-volatile memory of the mobile image capture device or to discard the first copy of the first image from the temporary image buffer without storing a second copy of the first image in the non-volatile memory; andtransmit at least one of a plurality of images stored in the non-volatile memory to a user computing device. 2. The system of claim 1, further comprising the user computing device, the user computing device operable to: selectively communicatively connect to the mobile image capture device;select one or more of the plurality of images stored in the non-volatile memory of the user computing device for transfer;receive from the mobile image capture device the selected one or more images; andprovide a user interface that enables a user to edit the one or more images received from the mobile image capture device. 3. The system of claim 1, wherein the at least one neural network comprises at least one convolutional neural network. 4. The system of claim 1, wherein: the at least one neural network comprises a plurality of deep neural networks that respectively analyze a plurality of different characteristics of each input image;the mobile image capture device is operable to input the first image into at least one of the plurality of deep neural networks; andeach of the plurality of deep neural networks outputs at least one annotation for each image input into such deep neural network, the at least one annotation for each input image indicative of the desirability of the scene depicted by such image. 5. The system of claim 4, wherein the mobile image capture device periodically receives one or more updated versions of one or more of the plurality of deep neural networks from one or more of the user computing device and a server computing device, the one or more updated versions of the one or more of the plurality of deep neural networks trained on additional data. 6. The system of claim 1, wherein the mobile image capture device is operable to: maintain data sufficient to generate a high resolution copy of the first image in a temporary raw data buffer;input a low resolution copy of the first image into the at least one neural network; anddetermine, based at least in part on the at least one descriptor of the desirability of the scene of the first image, whether to store a high resolution copy of the first image in the non-volatile memory of the mobile image capture device or to discard the first copy of the first image from the temporary image buffer without storing a high resolution copy of the first image in the non-volatile memory. 7. The system of claim 1, further comprising the user computing device, wherein the mobile image capture device further comprises a wide angle lens; and wherein the user computing device is operable to dewarp the one or more images received from the mobile image capture device to obtain one or more narrow angle images. 8. The system of claim 1, wherein: the mobile image capture device is operable to transmit an excitement signal that is wirelessly receivable by one or more other mobile image capture devices that are located proximate to the mobile image capture device; andthe excitement signal indicates that the scene depicted by the first image has a large desirability. 9. The system of claim 8, wherein, in response to receipt of the excitement signal, the one or more other mobile image capture devices perform at least one of the following: increase a probability that the other mobile image capture device will store a copy of a recently captured image in the non-volatile memory;attempt to identify a match for one or more embeddings included in the excitement signal within the recently captured image; andincrease one or more of a frame rate or a resolution of an image sensor of the other mobile image capture device. 10. The system of claim 1, wherein to communicatively connect: the user computing device is operable to visually display a pattern; andthe mobile image capture device is operable to: capture imagery of the pattern; andin response to capturing the imagery of the pattern, communicatively connect to the user computing device. 11. The system of claim 1, further comprising the user computing device, the user computing device operable to: analyze at least one descriptor of the desirability of the scene depicted by each of the plurality of images stored in the non-volatile memory of the mobile image capture device; anddetermine, based at least in part on the at least one descriptor of the desirability of the scene depicted by each of the plurality of images stored in the non-volatile memory of the mobile image capture device, whether to request transfer of such image. 12. The system of claim 1, wherein: the mobile image capture device is operable to periodically provide a capture visualization to the user computing device for display by the user computing device;display of the capture visualization by the user computing device is indicative of present operation of the mobile image capture device; andthe capture visualization is based at least in part on one or more images recently captured by the mobile image capture device. 13. The system of claim 1, further comprising the user computing device, wherein: the mobile image capture device is operable to perform face recognition with respect to captured images to recognize one or more faces depicted in the captured images; andthe user computing device is operable to: enable the user to select one or more of the recognized faces as having a heightened importance to the user; andcommunicate information to the mobile image capture device that enables the mobile image capture device to increase a probability of storing subsequently captured images which depict the one or more selected faces. 14. The system of claim 1, further comprising: the user computing device; anda server computing device for image synchronization, storage, and sharing, the user computing device selectively communicatively coupleable to the server computing device;wherein the user computing device uploads to the server computing device only images that are requested for upload to the server computing device during or as a result of a user-controlled editing session. 15. The system of claim 1, further comprising the user computing device, and wherein: the mobile image capture device is operable to perform face recognition with respect to captured images to recognize one or more faces depicted in the captured images; andthe user computing device is operable to: enable the user to share the one or more images received from the mobile image capture device with one or more contacts;determine one or more relationships between the one or more contacts with whom the one or more images are shared and the one or more faces depicted in the shared images; andsuggest, with respect to subsequent images received from the mobile image capture device in which the one or more faces are recognized, that the user share such subsequent images with particular ones of the one or more contacts based at least in part on the one or more relationships. 16. A system for selective retention and editing of images, the system comprising: a continuous capture mobile image capture device designed for resource efficiency, the mobile image capture device comprising a network interface, a power source, an image sensor, at least one processor, a memory that includes at least a temporary image buffer and a non-volatile memory, and a scene analyzer, the mobile image capture device operable to: capture a plurality of images that respectively depict a plurality of scenes;maintain a first copy of each of the plurality of images in the temporary image buffer;determine, by the scene analyzer, at least one descriptor of the desirability of the scene depicted by each image;determine, based at least in part on the at least one descriptor of the desirability of the scene of each image, whether to store a second copy of such image in the non-volatile memory of the mobile image capture device or to discard the first copy of such image from the temporary image buffer without storing a second copy of such image in the non-volatile memory; anda user computing device, the user computing device comprising at least one processor, a memory, and a transfer controller, the user computing device operable to: selectively communicatively connect to the mobile image capture device;determine, by the transfer controller based at least in part on the at least one descriptor of the desirability of the scene depicted by each image stored in the non-volatile memory of the mobile image capture device, whether to request transfer of such image from the mobile image capture device; andreceive one or more images requested for transfer from the mobile image capture device. 17. The system of claim 16, wherein: the scene analyzer comprises a plurality of deep neural networks that respectively analyze a plurality of different characteristics of an input image;the mobile image capture device is operable to input each image into at least one of the plurality of deep neural networks;each of the plurality of deep neural networks outputs at least one annotation for each input image, the at least one annotation for each input image indicative of the desirability of the scene depicted by such image; andthe transfer controller determines whether to request transfer of each image stored in the non-volatile memory of the mobile image capture device based at least in part on the one or more annotations provided for such image by the at least one of the plurality of deep neural networks into which such image was input. 18. The system of claim 17, wherein the user computing device determines a recommended composition for at least one of the transferred images based at least in part on the at least one annotation associated with such image, the recommended composition including only a portion of the at least one of the transferred images. 19. The system of claim 16, wherein the transfer controller performs an optimization algorithm to select which of the images stored in the non-volatile memory of the mobile image capture device to request for transfer, the optimization algorithm having a plurality of associated objectives, the plurality of associated objectives including at least a diversity of settings and a diversity of depicted persons. 20. A method to selectively retain and transfer images, the method comprising: capturing, by a mobile image capture device, a first image that depicts a scene;maintaining, by the mobile image capture device, a first copy of the first image in a temporary image buffer of the mobile image capture device;inputting, by the mobile image capture device, the first image into at least one neural network to determine a desirability of the scene depicted by the first image, the at least one neural network stored in a memory of the mobile image capture device;receiving, by the mobile image capture device, at least one descriptor of the desirability of the scene depicted by the first image as an output of the neural network;determining, by the mobile image capture device based at least in part on the at least one descriptor of the desirability of the scene, whether to store a second copy of the first image in the non-volatile memory of the mobile image capture device or to discard the first copy of the first image from the temporary image buffer without storing a second copy of the first image in the non-volatile memory; andtransferring, by the mobile image capture device, one or more of a plurality of images stored in the non-volatile memory of the mobile image capture device to a user computing device.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (81)
Wexler, Yonatan; Shashua, Amnon, Apparatus for adjusting image capture settings.
Kuberka, Cheryl J.; Barnum, David C.; Williams, Frances C.; Border, John N.; Johnson, Kenneth A., Camera configurable for autonomous self-learning operation.
Bernardi Bryan D. (Rochester NY) McIntyre Dale F. (Honeoye Falls NY) Dunsmore Clay A. (Fairport NY) Wolcott Dana W. (Honeoye Falls NY), Camera on-board voice recognition.
Kuchta Daniel W. (Brockport NY) Sucy Peter J. (Hamlin NY), Electronic still camera providing multi-format storage of full and reduced resolution images.
Ejima, Satoshi; Nozaki, Hirotake; Hiraide, Fumio, Image processing apparatus having image selection function, and recording medium having image selection function program.
Strub, Henry B.; Burgess, David A.; Johnson, Kimberly H.; Cohen, Jonathan R.; Reed, David P.; Aiello, G. Roberto, Low attention recording unit for use by vigorously active recorder.
Ostojic, Bojana; Glein, Christopher A; Gibson, Mark R.; Vong, William H; Flora, William T; Alton, Benjamin N; Newell, Mark S, Media user interface gallery control.
Schaffer, James David; Ali, Walid; Eshelman, Larry J.; Cohen-Bacrie, Claude; Lagrange, Jean-Michel; Levrier, Claire; Villain, Nicholas; Entrekin, Robert R., Method and apparatus for automatically developing a high performance classifier for producing medically meaningful descriptors in medical diagnosis imaging.
Lee, Il Yong; Kim, Sung Hyun; Kim, Lag Young; Hong, Yun Pyo; Byun, Seong Chan, Method for processing image data in portable electronic device, and portable electronic device having camera thereof.
Steinberg, Eran; Prilutsky, Yury; Corcoran, Peter; Bigioi, Petronel, Perfecting of digital image capture parameters within acquisition devices using face detection.
Tedesco, Daniel E.; Jorasch, James A.; Gelman, Geoffrey M.; Walker, Jay S.; Tulley, Stephen C.; O'Neil, Vincent M.; Alderucci, Dean P., System for image analysis in a network that is structured with multiple layers and differentially weighted neurons.
Isogai, Kuniaki; Kawamura, Takashi; Kawabata, Akihiro, Video analysis apparatus and method for calculating interpersonal relationship evaluation value using video analysis.
Lin,Yun Ting; Gutta,Srinivas; Brodsky,Tomas; Philomin,Vasanth, Video monitoring system employing hierarchical hidden markov model (HMM) event learning and classification.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.