Method and device for detecting lane elements to plan the drive path of autonomous vehicle by using a horizontal filter mask, wherein the lane elements are unit regions including pixels of lanes in an input image
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06K-009/00
G06T-007/73
G08G-001/16
G08G-001/04
출원번호
16263123
(2019-01-31)
등록번호
10373004
(2019-08-06)
발명자
/ 주소
Kim, Kye-Hyeon
Kim, Yongjoong
Kim, Insu
Kim, Hak-Kyoung
Nam, Woonhyun
Boo, SukHoon
Sung, Myungchul
Yeo, Donghun
Ryu, Wooju
Jang, Taewoong
Jeong, Kyungjoong
Je, Hongmo
Cho, Hojin
출원인 / 주소
StradVision, Inc.
대리인 / 주소
Husch Blackwell LLP
인용정보
피인용 횟수 :
0인용 특허 :
0
초록▼
A method for detecting lane elements, which are unit regions including pixels of lanes in an input image, to plan the drive path of an autonomous vehicle by using a horizontal filter mask is provided. The method includes steps of: a computing device acquiring a segmentation score map from a CNN usin
A method for detecting lane elements, which are unit regions including pixels of lanes in an input image, to plan the drive path of an autonomous vehicle by using a horizontal filter mask is provided. The method includes steps of: a computing device acquiring a segmentation score map from a CNN using the input image; instructing a post-processing module, capable of performing data processing at an output end of the CNN, to generate a magnitude map by using the segmentation score map and the horizontal filter mask; instructing the post-processing module to determine each of lane element candidates per each of rows of the segmentation score map by referring to values of the magnitude map; and instructing the post-processing module to apply estimation operations to each of the lane element candidates per each of the rows, to thereby detect each of the lane elements.
대표청구항▼
1. A method for detecting one or more lane elements, which are one or more unit regions including pixels of one or more lanes in at least one input image, by using at least one horizontal filter mask, comprising steps of: (a) a computing device, if a CNN (convolutional neural network) generates at l
1. A method for detecting one or more lane elements, which are one or more unit regions including pixels of one or more lanes in at least one input image, by using at least one horizontal filter mask, comprising steps of: (a) a computing device, if a CNN (convolutional neural network) generates at least one segmentation score map by using the input image, acquiring the segmentation score map;(b) the computing device instructing at least one post-processing module, capable of performing data processing at an output end of the CNN, to generate at least one magnitude map by using (i) the segmentation score map and (ii) the horizontal filter mask including a plurality of filtering parameters;(c) the computing device instructing the post-processing module to determine each of one or more lane element candidates per each of rows of the segmentation score map by referring to values of the magnitude map; and(d) the computing device instructing the post-processing module to apply one or more estimation operations to each of the lane element candidates per each of the rows, to thereby detect each of the lane elements. 2. The method of claim 1, wherein, at the step of (b), the computing device instructs the post-processing module to generate the magnitude map by converting values of the segmentation score map through the horizontal filter mask. 3. The method of claim 2, wherein, assuming that coordinates of values included in a k-th row on the segmentation score map whose size is m×n are (x, k), where x is an integer varying from 1 to m, and k is selected among integers from 1 to n, the computing device instructs the post-processing module to generate the magnitude map which has a same size with the segmentation score map by calculating values of (x, k) on the magnitude map, wherein (i) if x is from 2 to m−1, each of the values of (x, k) on the magnitude map is each of sums of elements of each of element-wise products generated by multiplying values of (x−1, k), (x, k), and (x+1, k) on the segmentation score map and values of [−1, 0, 1] included in the horizontal filter mask, respectively, (ii) if x equals to 1, a value of (x, k) on the magnitude map is a sum of elements of each of element-wise products generated by multiplying values of (x, k) and (x+1, k) on the segmentation score map and values of [0, 1], included in the horizontal filter mask, respectively, and (iii) if x equals to m, a value of (x, k) on the magnitude map is a sum of elements of each of element-wise product generated by multiplying values of (x−1, k) and (x, k) on the segmentation score map and values of [−1, 0], included in the horizontal filter mask, respectively. 4. The method of claim 1, wherein, at the step of (c), the computing device instructs the post-processing module to detect each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates per each of the rows by referring to each of absolute values and each of signs of each of the values of the magnitude map corresponding to each of the rows of the segmentation score map, to thereby determine each of the lane element candidates per each of the rows. 5. The method of claim 4, wherein the computing device instructs the post-processing module to determine each of extended lane element candidates per each of the rows by referring to each of adjusted left boundary coordinates and each of adjusted right boundary coordinates whose distances from each of centers of each of the lane element candidates per each of the rows are determined as multiples of distances from each of the centers thereof to each of the initial left boundary coordinates and to each of the initial right boundary coordinates respectively. 6. The method of claim 1, wherein, at the step of (c), the computing device instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the step of (d), the computing device instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that (i) each of reference coordinates of each of the lane element candidates per each of the rows is determined as each of coordinates whose value is largest among each of values, on the segmentation map, of each of the lane element candidates per each of the rows larger than a first threshold, and (ii) each of final left boundary coordinates is determined as each of coordinates, whose value is close to a second threshold within a predetermined range, on the left side of each of the reference coordinates, and each of final right boundary coordinates is determined as each of coordinates, whose value is close to a third threshold within a predetermined range, on the right side of each of the reference coordinates. 7. The method of claim 6, wherein the computing device instructs the post-processing module (i) to determine each of reference coordinate candidates whose value is larger than the first threshold among each of the values of each of the lane element candidates per each of the rows of the segmentation score map and (ii) to detect each largest value among each of values of each of the reference coordinate candidates by repeating a process of eliminating each of some values, which is smaller than either its left one or right one, among each of the values of each of the reference coordinate candidates, to thereby determine each of coordinates corresponding to said each largest value as each of the reference coordinates of each of the lane element candidates per each of the rows. 8. The method of claim 1, wherein, at the step of (c), the computing device instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the step of (d), the computing device instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that (i) each sample width per each of the rows of the segmentation score map is determined by referring to information on a predetermined lane width, (ii) each of sub groups, whose width corresponds to said each sample width, is generated by using each of values of each of the lane element candidates per each of the rows on the segmentation score map, (iii) each difference is calculated between each of averages of each of values included in each of the sub groups and that of values, on the segmentation score map, which are not included in said each of the sub groups, to thereby determine each of representative sub groups per each of the rows, and (iv) each of final left boundary coordinates is determined as each of coordinates whose value is close to a threshold value within a predetermined range in a left part of each of the representative sub groups, and each of final right boundary coordinates is determined as each of coordinates whose value is close to the threshold value within a predetermined range in a right part of each of the representative sub groups. 9. The method of claim 1, wherein, at the step of (c), the computing device instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the step of (d), the computing device instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that each maximum sum subarray is determined where each of sums of deviation values of each of the lane element candidates per each of the rows is largest, and each of final left boundary coordinates and each of final right boundary coordinates are determined respectively as each of leftmost coordinates and each of rightmost coordinates in said each maximum sum subarray. 10. The method of claim 1, wherein, at the step of (c), the computing device instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the step of (d), the computing device instructs the processing module to detect each of final left boundary coordinates and each of final right boundary coordinates of each of the lane element candidates per each of the rows by applying the estimation operations to the segmentation score map, and refer to each of the final left boundary coordinates and each of the final right boundary coordinates of each of the lane element candidates per each of the rows, to thereby generate each of the lane elements. 11. A computing device for detecting one or more lane elements, which are one or more unit regions including pixels of one or more lanes in at least one input image, by using at least one horizontal filter mask, comprising: at least one memory that stores instructions; andat least one processor configured to execute the instructions to: perform processes of (I) if the segmentation score map which has been generated through a CNN by using the input image is acquired, instructing at least one post-processing module, capable of performing data processing at an output end of the CNN, to generate at least one magnitude map by using (i) the segmentation score map and (ii) the horizontal filter mask including a plurality of filtering parameters, (II) instructing the post-processing module to determine each of one or more lane element candidates per each of rows of the segmentation score map by referring to values of the magnitude map, and (III) instructing the post-processing module to apply one or more estimation operations to each of the lane element candidates per each of the rows, to thereby detect each of the lane elements. 12. The computing device of claim 11, wherein, at the process of (I), the processor instructs the post-processing module to generate the magnitude map by converting values of the segmentation score map through the horizontal filter mask. 13. The computing device of claim 12, wherein, assuming that coordinates of values included in a k-th row on the segmentation score map whose size is m×n are (x, k), where x is an integer varying from 1 to m, and k is selected among integers from 1 to n, the processor instructs the post-processing module to generate the magnitude map which has a same size with the segmentation score map by calculating values of (x, k) on the magnitude map, wherein (i) if x is from 2 to m−1, each of the values of (x, k) on the magnitude map is each of sums of elements of each of element-wise products generated by multiplying values of (x−1, k), (x, k), and (x+1, k) on the segmentation score map and values of [−1, 0, 1] included in the horizontal filter mask, respectively, (ii) if x equals to 1, a value of (x, k) on the magnitude map is a sum of elements of each of element-wise products generated by multiplying values of (x, k) and (x+1, k) on the segmentation score map and values of [0, 1], included in the horizontal filter mask, respectively, and (iii) if x equals to m, a value of (x, k) on the magnitude map is a sum of elements of each of element-wise product generated by multiplying values of (x−1, k) and (x, k) on the segmentation score map and values of [−1, 0], included in the horizontal filter mask, respectively. 14. The computing device of claim 11, wherein, at the process of (II), the processor instructs the post-processing module to detect each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates per each of the rows by referring to each of absolute values and each of signs of each of the values of the magnitude map corresponding to each of the rows of the segmentation score map, to thereby determine each of the lane element candidates per each of the rows. 15. The computing device of claim 14, wherein the processor instructs the post-processing module to determine each of extended lane element candidates per each of the rows by referring to each of adjusted left boundary coordinates and each of adjusted right boundary coordinates whose distances from each of centers of each of the lane element candidates per each of the rows are determined as multiples of distances from each of the centers thereof to each of the initial left boundary coordinates and to each of the initial right boundary coordinates respectively. 16. The computing device of claim 11, wherein, at the process of (II), the processor instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the process of (III), the processor instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that (i) each of reference coordinates of each of the lane element candidates per each of the rows is determined as each of coordinates whose value is largest among each of values, on the segmentation map, of each of the lane element candidates per each of the rows larger than a first threshold, and (ii) each of final left boundary coordinates is determined as each of coordinates, whose value is close to a second threshold within a predetermined range, on the left side of each of the reference coordinates, and each of final right boundary coordinates is determined as each of coordinates, whose value is close to a third threshold within a predetermined range, on the right side of each of the reference coordinates. 17. The computing device of claim 16, wherein the processor instructs the post-processing module (i) to determine each of reference coordinate candidates whose value is larger than the first threshold among each of the values of each of the lane element candidates per each of the rows of the segmentation score map and (ii) to detect each largest value among each of values of each of the reference coordinate candidates by repeating a process of eliminating each of some values, which is smaller than either its left one or right one, among each of the values of each of the reference coordinate candidates, to thereby determine each of coordinates corresponding to said each largest value as each of the reference coordinates of each of the lane element candidates per each of the rows. 18. The computing device of claim 11, wherein, at the process of (II), the processor instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the process of (III), the processor instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that (i) each sample width per each of the rows of the segmentation score map is determined by referring to information on a predetermined lane width, (ii) each of sub groups, whose width corresponds to said each sample width, is generated by using each of values of each of the lane element candidates per each of the rows on the segmentation score map, (iii) each difference is calculated between each of averages of each of values included in each of the sub groups and that of values, on the segmentation score map, which are not included in said each of the sub groups, to thereby determine each of representative sub groups per each of the rows, and (iv) each of final left boundary coordinates is determined as each of coordinates whose value is close to a threshold value within a predetermined range in a left part of each of the representative sub groups, and each of final right boundary coordinates is determined as each of coordinates whose value is close to the threshold value within a predetermined range in a right part of each of the representative sub groups. 19. The computing device of claim 11, wherein, at the process of (II), the processor instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the process of (III), the processor instructs the post-processing module to apply the estimation operations to each of the lane element candidates per each of the rows, wherein the estimation operations are performed such that each maximum sum subarray is determined where each of sums of deviation values of each of the lane element candidates per each of the rows is largest, and each of final left boundary coordinates and each of final right boundary coordinates are determined respectively as each of leftmost coordinates and each of rightmost coordinates in said each maximum sum subarray. 20. The computing device of claim 11, wherein, at the process of (II), the processor instructs the post-processing module to determine each of the lane element candidates per each of the rows by detecting each of initial left boundary coordinates and each of initial right boundary coordinates of each of the lane element candidates, and wherein, at the process of (III), the processor instructs the processing module to detect each of final left boundary coordinates and each of final right boundary coordinates of each of the lane element candidates per each of the rows by applying the estimation operations to the segmentation score map, and refer to each of the final left boundary coordinates and each of the final right boundary coordinates of each of the lane element candidates per each of the rows, to thereby generate each of the lane elements.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.