Temperature compensation for structured light depth imaging system
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G01B-011/22
G06T-007/00
G06K-009/52
G06K-009/62
G06T-007/60
G06K-009/66
H04N-005/225
출원번호
US-0251966
(2016-08-30)
등록번호
US-10089738
(2018-10-02)
발명자
/ 주소
Zhao, Jian
Price, Raymond Kirk
Bleyer, Michael
Demandolx, Denis
출원인 / 주소
Microsoft Technology Licensing, LLC
대리인 / 주소
Arent Fox LLP
인용정보
피인용 횟수 :
0인용 특허 :
4
초록▼
Disclosed are an apparatus and a method of compensating temperature shifts of a structured light pattern for a depth imaging system. In some embodiments, a depth imaging device includes a light source, an imaging sensor and a processor. The light source emits light corresponding to a pattern. A temp
Disclosed are an apparatus and a method of compensating temperature shifts of a structured light pattern for a depth imaging system. In some embodiments, a depth imaging device includes a light source, an imaging sensor and a processor. The light source emits light corresponding to a pattern. A temperature drift of the light source can cause a shift of the pattern. The imaging sensor receives the light reflected by environment in front of the depth imaging device and generates a depth map including a plurality of pixel values corresponding to depths of the environment relative to the depth imaging device. The processor estimates the shift of the pattern based on a polynomial model depending on the temperature drift of the light source. The processor further adjusts the depth map based on the shift of the pattern.
대표청구항▼
1. A depth imaging device, comprising: a light source configured to emit light corresponding to a predetermined pattern;an imaging sensor configured to: receive the light as reflected by an environment of the depth imaging device;generate a depth map including a plurality of pixel values correspondi
1. A depth imaging device, comprising: a light source configured to emit light corresponding to a predetermined pattern;an imaging sensor configured to: receive the light as reflected by an environment of the depth imaging device;generate a depth map including a plurality of pixel values corresponding to depths of the environment relative to the depth imaging device;a temperature sensor configured to measure a temperature drift from a reference temperature of one or more of the light source, an optical component of the depth imaging device, or the environment of the depth imaging device; anda processor configured to: estimate a shift of the pattern by using a polynomial model depending on the temperature drift; andadjust the depth map based on the shift of the pattern. 2. The depth imagine device of claim 1, wherein the light source is a structured light source configured to emit light corresponding to a known pattern. 3. The depth imaging device of claim 1, wherein the pattern is a speckle pattern corresponding to a reference image including a plurality of dots, each of the dots having known coordinates in the reference image. 4. The depth imagine device of claim 1, wherein the polynomial model is used to compensate for shifts in the pattern for generating an accurate depth map. 5. The depth imagine device of claim 1, where wherein the polynomial model includes a first polynomial and a second polynomial, the first and second polynomials being functions of a plurality of variables, the variables including a temperature drift from a reference temperature and positional coordinates in the reference image. 6. The device of claim 5, wherein the pattern includes a plurality of objects and the first and second polynomials estimate shifts of the objects of the pattern due to the temperature drift. 7. The depth imaging device of claim 5, wherein each of the first and second polynomials includes a plurality of cubic terms to model interactions between the positional coordinates in the reference image and the temperature drift, each of the cubic terms including a product of multiplying three variables. 8. The depth imaging device of claim 5, wherein each of the first and second polynomials is a weighted combination of terms depending on the temperature drift, a first positional coordinate in the reference image, or a second positional coordinate in the reference image; and wherein the processor is further configured to train weight coefficients of the first and second polynomials using a training data set, the training data set including values of the positional coordinates in the reference image, values of the variables of the first and second polynomials associated with temperature drift, and corresponding known values of shifted positional coordinates in shifted images that correspond to shifts of the pattern due to the temperature drift. 9. The depth imaging device of claim 1, wherein the pattern includes a plurality of objects; and wherein the polynomial model includes a plurality of function sets, each of the function sets is predictive of shifts of one of the plurality of objects of the pattern due to the temperature drift, each of the function sets including a function predictive of a shift along a first direction and another function predictive of a shift along a second direction. 10. The depth imaging device of claim 9, wherein each function of the function sets includes a plurality of quadratic terms to model interactions between positional coordinates of a reference image and the temperature drift, each of the quadratic terms including a product of multiplying two variables. 11. The depth imaging device of claim 9, wherein each function of the function sets is a weighted combination of terms depending on the temperature drift, a first coordinate in a reference image, or a second coordinate in the reference image; and wherein weight coefficients of each function are trained based on a plurality of known shifts of the pattern due to the temperature drift. 12. The depth imaging device of claim 9, wherein the plurality of function sets are predictive of shifts of a subset of sampling objects within the pattern; and wherein the polynomial model estimates a shift of a remaining object using one function set that corresponds to a sampling object within the subset, the sampling object in proximity to the remaining object in a reference image. 13. A method of compensating for temperature shifts of a structured light pattern of a depth imaging system, the method comprising: emitting light corresponding to a known pattern by a structured light source, the known pattern including a plurality of objects that have shifts in the pattern due to a temperature drift from a reference temperature of one or more of the structured light source, an optical component of the depth imaging device, or an ambient temperature of the environment of the depth imaging device ; and;receiving the light as reflected by an environment of the depth imaging device;generating, based on the received light, a depth map including a plurality of pixel values corresponding to depths of objects in the environment relative to the depth imaging device;predicting the temperature shifts of the plurality of objects of the pattern based on a polynomial model; andadjusting the depth map based on the temperature shifts of the objects of the patterns. 14. The method of claim 13, further comprising: identifying a location of an object in the environment by using the depth map that has been adjusted. 15. The method of claim 13, further comprising: training the polynomial model by optimizing model parameters of the polynomial model using a training data set, the training data set including known shifts of the plurality of objects of the pattern due to temperature drifts and corresponding temperature drifts from a reference temperature. 16. The method of claim 13, wherein the polynomial model includes a first polynomial predictive of shifts in a first direction and a second polynomial predictive of shifts in a second direction, each of the first and second polynomials including a plurality of cubic terms, each of the cubic terms including a product of multiplying three variables that include a temperature drift from a reference temperature, a x coordinate in a reference image corresponding to the pattern, or a y coordinate in the reference image; and wherein predicting the shifts of the objects comprises: predicting the temperature shifts of the objects of the pattern by applying the first and second polynomials to each of the plurality of objects of the pattern. 17. The method of claim 13, wherein the polynomial model includes a plurality of polynomial sets, each of the polynomial sets predictive of shifts of one of the objects of the pattern due to temperature drift, each of the polynomial sets including a polynomial to estimate a shift along a first direction and another polynomial to estimate a shift along a second direction; and wherein predicting the shifts of the objects comprises: predicting the temperature shifts of the objects of the pattern by applying each polynomial set of the plurality of polynomial sets to a corresponding object of the plurality of objects of the pattern. 18. A head-mounted display device comprising: a display configured to output an image to an eye of a user;a light source to emit light corresponding to a pattern including a plurality of features, wherein a temperature drift from a reference temperature of one or more of the light source, an optical component of the head-mounted display device, or an ambient temperature of the environment of the head-mounted display device causes a shift of the pattern;an imaging sensor to receive the light as reflected by the environment of the head-mounted display device and that, when in operation, generates a depth map including a plurality of pixel values corresponding to depths of the environment relative to the head-mounted display device; anda processor that, when in operation, estimates the shifts of the features of the pattern by using a polynomial model depending on the temperature drift and adjusts the depth map based on the shifts of the features.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (4)
Silverbrook, Kia; Walmsley, Simon Robert, Adaptive tracking of dots in optical storage system using ink dots.
Hornback,Bert; Harwood,Doug; Boyd,W. Eric; Carlson,Randy, Imaging device with multiple fields of view incorporating memory-based temperature compensation of an uncooled focal plane array.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.