Explainable AI (xAI) platform for computational pathology
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G16H-030/40
G06T-007/00
G06F-018/2431
G06V-010/778
G06V-020/69
출원번호
17843309
(2022-06-17)
등록번호
11842488
(2023-12-12)
발명자
/ 주소
Tosun, Akif Burak
Chennubhotla, Srinivas Chakra
Fine, Jeffrey Louis
출원인 / 주소
SpIntellx, Inc.
대리인 / 주소
Goodwin Procter LLP
인용정보
피인용 횟수 :
0인용 특허 :
0
초록▼
Pathologists are adopting digital pathology for diagnosis, using whole slide images (WSIs). Explainable AI (xAI) is a new approach to AI that can reveal underlying reasons for its results. As such, xAI can promote safety, reliability, and accountability of machine learning for critical tasks such as
Pathologists are adopting digital pathology for diagnosis, using whole slide images (WSIs). Explainable AI (xAI) is a new approach to AI that can reveal underlying reasons for its results. As such, xAI can promote safety, reliability, and accountability of machine learning for critical tasks such as pathology diagnosis. HistoMapr provides intelligent xAI guides for pathologists to improve the efficiency and accuracy of pathological diagnoses. HistoMapr can previews entire pathology cases' WSIs, identifies key diagnostic regions of interest (ROIs), determines one or more conditions associated with each ROI, provisionally labels each ROI with the identified conditions, and can triages them. The ROIs are presented to the pathologist in an interactive, explainable fashion for rapid interpretation. The pathologist can be in control and can access xAI analysis via a “why?” interface. HistoMapr can track the pathologist's decisions and assemble a pathology report using suggested, standardized terminology.
대표청구항▼
1. A system for performing explainable pathological analysis of medical images, the system comprising: a first processor; anda first memory in electrical communication with the first processor, and comprising instructions that, when executed by a processing unit that comprises the first processor or
1. A system for performing explainable pathological analysis of medical images, the system comprising: a first processor; anda first memory in electrical communication with the first processor, and comprising instructions that, when executed by a processing unit that comprises the first processor or a second processor, and that is in electronic communication with a memory module that comprises the first memory or a second memory, program the processing unit to:for a region of interest (ROI) in a whole slide image (WSI) of a tissue, identify features of a plurality of feature types, wherein at least one feature type is at least partially indicative of a pathological condition of the tissue within the ROI;operate as a classifier, trained to classify an image using features of the plurality of feature types into one of a plurality of classes of tissue conditions, to: (i) classify the ROI into a class within the plurality of classes, and (ii) designate to the ROI a label indicating a tissue condition associated with the class and with the tissue in the ROI;store explanatory information about the designation of the label, the explanatory information comprising information about the identified features; anddisplay: (i) at least a portion of the WSI with boundary of the ROI highlighted, (ii) the label designated to the ROI, and (iii) a user interface (UI) comprising: (a) a first UI element for providing to a user access to the stored explanatory information, and (b) one or more additional UI elements enabling the user to provide feedback on the designated label. 2. The system of claim 1, wherein: the tissue comprises breast tissue; andthe plurality of classes of tissue conditions comprises two or more of: invasive carcinoma, ductal carcinoma in situ (DCIS), high-risk benign, low-risk benign, atypical ductal hyperplasia (ADH), flat epithelial atypia (FEA), columnar cell change (CCC), and normal duct. 3. The system of claim 1, wherein: the tissue comprises lung tissue; andthe plurality of classes of tissue conditions comprises: idiopathic pulmonary fibrosis (IPF) and normal. 4. The system of claim 1, wherein: the tissue comprises brain tissue; andthe plurality of classes of tissue conditions comprises: classical cellular tumor and proneural cellular tumor. 5. The system of claim 1, wherein a feature type is cytological features or architectural features (AFs). 6. The system of claim 5, wherein a feature of the feature type cytological features is of one of the subtypes: nuclear size, nuclear shape, nuclear morphology, or nuclear texture. 7. The system of claim 5, a feature of the feature type architectural features is of one of the subtypes: an architectural feature based on a color of a group of superpixels in the ROI (AF-C); (ii) an architectural feature based on a cytological phenotype of nuclei in the ROI (AF-N); or (iii) a combined architectural feature (AF-CN) based on both a color of a group of superpixels in the ROI and a cytological phenotype of nuclei in the ROI. 8. The system of claim 5, a feature of the feature type architectural features is of one of the subtypes: nuclear arrangement, stromal cellularity, epithelial patterns in ducts, epithelial patterns in glands, cell cobblestoning, stromal density, or hyperplasticity. 9. The system of claim 1, wherein the information about the features comprises one or more of: a total number of features types that were detected in the ROI and that correspond to the tissue condition indicated by the label;a count of features of a particular feature type that were detected in the ROI;a measured density of features of the particular feature type in the ROI; ora strength of the particular feature type in indicating the tissue condition. 10. The system of claim 1, wherein the explanatory information comprises a confidence score computed by the classifier in designating the label, wherein the confidence score is based on one or more of: a total number of feature types that were detected in the ROI and that correspond to the tissue condition indicated by the label;for a first feature type: (i) a strength of the first feature type in indicating the tissue condition, or (ii) a count of features of the first feature type that were detected in the ROI; oranother total number of features types that were detected in the ROI but that correspond to a tissue condition different from the condition associated with the label. 11. The system of claim 1, wherein the instructions further program the processing unit to: in response to the user interacting with the first UI element:generate explanatory description using a standard pathology vocabulary and the stored explanatory information; anddisplay the explanatory description in an overlay window, a side panel, or a page. 12. The system of claim 11, wherein the instructions further program the processing unit to: highlight in the ROI, features of a particular feature type, that at least partially indicates the tissue condition indicated by the label, using a color designated to the feature type; anddisplay the highlighted ROI in the overlay window, the side panel, or the page. 13. The system of claim 1, wherein: the instructions further program the processing unit to:repeat the identify, designate, and store operations for a plurality of different ROIs; andprior to the display operation, (i) compute a respective risk metric for each of the ROIs, the risk metric of an ROI being based on: (a) designated label of the ROI, or (b) a confidence score for the ROI, and (ii) sequencing the ROIs according to the respective risk metrics thereof; andto perform the display operation, the instructions program the processing unit to:display in one panel: (i) at least a portion of the WSI with boundary of the ROI having the highest risk metric highlighted, (ii) the label designated to that ROI, and (iii) a user interface (UI) providing to the user access to the stored explanation for the designated label of that ROI; anddisplay in another panel thumbnails of the sequence of ROIs. 14. The system of claim 1, wherein the instructions further program the processing unit to: obtain the whole slide image (WSI); andidentify the ROI in the WSI, wherein to identify the ROI, the instructions program the processing unit to: (i) mark in the WSI, superpixels of at least two types, one type corresponding to hematoxylin stained tissue and another type corresponding to eosin stained tissue; and (ii) mark segments of pixels of a first type to define an enclosed region as the ROI. 15. The system of claim 14, wherein the instructions further program the processing unit to: identify a plurality of ROIs in the WSI. 16. The system of claim 1, wherein the instructions further program the processing unit to: update a training dataset for the classifier wherein, to update the training dataset, the instructions program the processing unit to:receive from the user via the one or more additional UI elements feedback for the label designated to the ROI, the feedback indicating correctness of the designated label; andstore a portion of the WSI associated with the ROI and the designated label in a training dataset. 17. The system of claim 1, wherein the classifier is selected from a group consisting of: a decision tree, a random forest, a support vector machine, an artificial neural network, and a logistic regression based classifier. 18. A system for distributing cases among a group of pathologists, the system comprising: a first processor; anda first memory in electrical communication with the first processor, and comprising instructions that, when executed by a processing unit that comprises the first processor or a second processor, and that is in electronic communication with a memory module that comprises the first memory or a second memory, program the processing unit to:for each one of a plurality of cases, process a corresponding whole slide image (WSI) of a tissue wherein, to process the WSI, the instructions program the processing unit to:identify one or more regions of interest (ROIs) in the WSI, each ROI having designated thereto a respective diagnostic label indicating a condition of a tissue in the ROI;for each ROI, compute a respective confidence score for the respective designation;compute for the WSI: (i) a severity score based on the respective diagnostic labels designated to the one or more ROIs in the WSI; and (ii) a confidence level based on the respective confidence scores for the one or more ROIs;store as explanatory information, the severity score, the confidence level, and the respective confidence scores;if the severity score is at or above a specified threshold severity score, transmit the WSI to a rush pathologist in the group of pathologists;otherwise, if the confidence level is at or below a specified threshold confidence level, transmit the WSI to a subspecialist in the group of pathologists; andotherwise, transmit the case to a generalist pathologist in the group of pathologists. 19. The system of claim 18, wherein to transmit the case to a general pathologist, the instructions program the processing unit to: select the general pathologists from a pool of general pathologists within the group of pathologists such that upon transmitting the case to the selected general pathologist maintains a balanced workload for the pool. 20. The system of claim 18, wherein the instructions further program the processing unit to: designate the respective diagnostic label to at least one ROI in at least one WSI wherein, to designate the respective diagnostic label to a particular ROI, the instructions program the processing unit to:operate as a classifier, trained to classify an image, using features of a plurality of feature types that are identified in the image, into one of a plurality of classes of tissue conditions, to: (i) classify the ROI into a class within the plurality of classes; and (ii) designate to the ROI a label indicating a tissue condition associated with the class. 21. The system of claim 18, wherein: for at least one ROI in at least one WSI, the respective diagnostic label was provided by a prior reviewer; andthe group of pathologists represents a group of subsequent reviewers. 22. The system of claim 18, wherein the instructions further program the processing unit to: in response to a user requesting, via a UI element, an explanation for the transmission of a particular WSI:generate an explanatory description using a standard pathology vocabulary and the stored explanatory information; anddisplay the explanatory description. 23. The system of claim 22, wherein the instructions further program the processing unit to: select from the particular WSI, an ROI for which the designated label indicates a severe condition or the confidence score is at or below a specified threshold confidence score;highlight in the ROI, features of a particular feature type, that at least partially indicates the tissue condition indicated by the label designated to the ROI, using a color designated to the feature type; anddisplay the highlighted ROI along with the explanatory description. 24. The system of claim 18, wherein the explanatory information comprises, for a first ROI in a first WSI, one or more of: a total number of features types that were detected in the first ROI and that correspond to the tissue condition indicated by the label designated to the first ROI;a count of features of a particular feature type that were detected in the first ROI;a measured density of features of the particular feature type in the first ROI; ora strength of the particular feature type in indicating a corresponding tissue condition. 25. The system of claim 18, wherein the confidence score for a first ROI in a first WSI is based on one or more of: a total number of feature types that were detected in the first ROI and that correspond to the tissue condition indicated by the label designated to the first ROI;for a first feature type: (i) a strength of the first feature type in indicating a corresponding tissue condition, or (ii) a count of features of the first feature type that were detected in the ROI; oranother total number of features types that were detected in the first ROI but that correspond to a tissue condition different from the condition associated with the label designated to the first ROI. 26. A system for ground truth labeling of images used for training a classifier, the system comprising: a first processor; anda first memory in electrical communication with the first processor, and comprising instructions that, when executed by a processing unit that comprises the first processor or a second processor, and that is in electronic communication with a memory module that comprises the first memory or a second memory, program the processing unit to:obtain a whole slide image (WSI) of a tissue;identify one or more regions of interest (ROIs) in the WSI, wherein identification of an ROI comprises: (i) marking in the WSI, superpixels of at least two types, one type corresponding to hematoxylin stained tissue and another type corresponding to eosin stained tissue; and (ii) marking segments of pixels of a first type to define an enclosed region as the ROI;display, in a sequence, one or more ROIs; andfor each ROI:display one or more UI elements, a first UI element providing or affirming a respective ground-truth label to be assigned to the ROI; andin response to the user interacting using the first UI element, designate the respective ground-truth label to the ROI and storing the ROI in a training corpus. 27. The system of claim 26, wherein: the first UI element indicates the user's agreement with a provided suggestion; andthe instructions further program the processing unit to, for each ROI in at least a subset of the one or more ROIs:identify features of a plurality of feature types, wherein at least one feature type is at least partially indicative of a pathological condition of the tissue within the ROI;operate as a classifier, trained to classify an image using features of a plurality of feature types into one of a plurality of classes of tissue conditions, to:(i) classify the ROI into a class within the plurality of classes;(ii) designate to the ROI a suggested label indicating a tissue condition associated with the class; and(iii) store explanatory information about the designation of the suggested label, the explanatory information comprising information about the identified features; anddisplay the suggested label as the provided suggestion. 28. The system of claim 27, wherein the instructions further program the processing unit to, in response to a user requesting, via a UI element, an explanation for the suggested label for a particular ROI: generate an explanatory description using a standard pathology vocabulary and the stored explanatory information; anddisplay the explanatory description. 29. The system of claim 28, wherein the instructions further program the processing unit to: highlight in the particular ROI, features of a particular feature type, that at least partially indicates the tissue condition indicated by the label designated to the ROI, using a color designated to the feature type; anddisplay the highlighted ROI along with the explanatory description. 30. The system of claim 28, wherein the explanatory information comprises, for the particular ROI, one or more of: a total number of features types that were detected in the particular ROI and that correspond to the tissue condition indicated by the suggested label designated to the particular ROI;a count of features of a particular feature type that were detected in the particular ROI;a measured density of features of the particular feature type in the particular ROI; ora strength of the particular feature type in indicating a corresponding tissue condition. 31. The system of claim 28, wherein the confidence score for the particular ROI is based on one or more of: a total number of feature types that were detected in the particular ROI and that correspond to the tissue condition indicated by the suggested label designated to the particular ROI;for a first feature type: (i) a strength of the first feature type in indicating a corresponding tissue condition, or (ii) a count of features of the first feature type that were detected in the particular ROI; oranother total number of features types that were detected in the particular ROI but that correspond to a tissue condition different from the condition associated with the suggested label designated to the particular ROI.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.