Software tool for training and testing a knowledge base
원문보기
IPC분류정보
국가/구분
United States(US) Patent
등록
국제특허분류(IPC7판)
G06F-017/00
G06N-005/04
출원번호
UP-0843937
(2007-08-23)
등록번호
US-7756810
(2010-08-02)
발명자
/ 주소
Nelken, Yoram
Hod, Ronen
Navot, Amir
Michelson, Sam
Jessee, Randy
Shacham, Tsachy
Lanin, Beth
Hajaj, Nissan
Margalit, Avi
Magdalen, Josemina
Cohen, Dani
출원인 / 주소
International Business Machines Corporation
대리인 / 주소
Gates & Cooper LLP
인용정보
피인용 횟수 :
5인용 특허 :
310
초록▼
A software tool for creating, training and testing a knowledge base of a computerized customer relationship management system is disclosed. The software tool includes corpus editing processes for displaying and editing text-based corpus items, and assigning selected categories to individual corpus i
A software tool for creating, training and testing a knowledge base of a computerized customer relationship management system is disclosed. The software tool includes corpus editing processes for displaying and editing text-based corpus items, and assigning selected categories to individual corpus items. Knowledge base construction processes construct a knowledge base by analyzing a first subset of the corpus items, and testing processes test the knowledge base on a second subset of the corpus items. Reporting processes generate reports containing indicia representative of the testing results, which may be utilized to edit the corpus items and retrain the knowledge base so as to improve performance.
대표청구항▼
What is claimed is: 1. A computer-implemented method for training and testing a knowledge base of a computerized customer relationship management system, comprising: collecting, in a computer, one or more corpus items into a corpus, wherein the corpus items comprise electronic communications from c
What is claimed is: 1. A computer-implemented method for training and testing a knowledge base of a computerized customer relationship management system, comprising: collecting, in a computer, one or more corpus items into a corpus, wherein the corpus items comprise electronic communications from customers; assigning, in the computer, a category from a set of predefined categories to each of the corpus items in the corpus; building, in the computer, a knowledge base of the computerized customer relationship management system by performing natural language and semantic analysis of a first subset of the corpus items in the corpus; testing, in the computer, the knowledge base of the computerized customer relationship management system on a second subset of the corpus items in the corpus by classifying each of the corpus items of the second subset into at least one of the predefined categories using information contained in the knowledge base of the computerized customer relationship management system; and generating and displaying, in the computer, a report based on results produced by the testing step to a user of the computerized customer relationship management system to gauge performance of the computerized customer relationship management system using the knowledge base, so that appropriate adjustments are made to improve the performance of the computerized customer relationship management system using the knowledge base. 2. The method of claim 1, wherein the step of testing the knowledge base includes calculating a set of scores for each corpus item in the second subset, each score from the calculated set of scores being associated with a corresponding category and being representative of a confidence that the corpus item belongs to the corresponding category. 3. The method of claim 1, wherein the step of generating and displaying a report includes generating a report relating to a single selected category. 4. The method of claim 1, wherein the step of generating and displaying a report includes generating a cumulative report relating to a plurality of categories. 5. The method of claim 1, wherein the step of generating and displaying a report includes: receiving user input specifying one of a precision value, a recall value, false positive rate, false negative rate, automation ratio or a cost ratio; and calculating and displaying, for a selected category, a match score based on the user input. 6. The method of claim 1, wherein the step of generating and displaying a report includes: receiving user input specifying a match score; and calculating and displaying, for a selected category, a precision value and a recall value based on the user input. 7. The method of claim 1, wherein the step of generating and displaying a report includes calculating precision as a function of recall and causing a graph to be displayed depicting the relationship between precision and recall. 8. The method of claim 1, wherein the step of generating and displaying a report includes generating and displaying a graph depicting cumulative success over time, the graph showing, for a plurality of groups of corpus items each having a common time parameter, the fraction of corpus items in the group that were appropriately classified. 9. The method of claim 1, wherein the step of generating and displaying a report includes generating and displaying a report showing, for each of a plurality of pairs of categories, a percentage of corpus items initially assigned to a first category of the pair of categories that were erroneously classified into a second category of the pair of categories. 10. The method of claim 1, wherein the step of generating and displaying a report includes generating and displaying a scoring report showing, for a selected category, match scores for each corpus item in the second subset, the match scores being representative of the relevance of the selected category to the corpus item. 11. The method of claim 1, wherein the first and second subsets of corpus items are selected in accordance with user input. 12. The method of claim 1, wherein the steps of use building and testing the knowledge base include using a modeling engine to analyze and classify corpus items. 13. The method of claim 1, wherein the step of generating and displaying a report includes selecting a report from a plurality of available reports in response to user input. 14. The method of claim 1, wherein the corpus items comprise customer communications received from one or more external sources. 15. The method of claim 1, wherein the corpus items include structured and unstructured information. 16. A device embodying instructions that, when executed by a computer, result in the computer performing a computer-implemented method for training and testing a knowledge base of a computerized customer relationship management system, comprising: collecting, in a computer, one or more corpus items into a corpus, wherein the corpus items comprise electronic communications from one or more customers; assigning, in the computer, a category from a set of predefined categories to each of the corpus items in the corpus; building, in the computer, a knowledge base of the computerized customer relationship management system by performing natural language and semantic analysis of a first subset of the corpus items in the corpus; testing, in the computer, the knowledge base of the computerized customer relationship management system on a second subset of the corpus items in the corpus by classifying each of the corpus items of the second subset into at least one of the predefined categories using information contained in the knowledge base of the computerized customer relationship management system; and generating and displaying, in the computer, a report based on results produced by the testing step to a user of the computerized customer relationship management system to gauge performance of the computerized customer relationship management system using the knowledge base, so that appropriate adjustments are made to improve the performance of the computerized customer relationship management system using the knowledge base.
연구과제 타임라인
LOADING...
LOADING...
LOADING...
LOADING...
LOADING...
이 특허에 인용된 특허 (310)
Vajk Hugo (New Port Richey FL) Stephens William (Redondo Beach CA), ATM/POS based electronic mail system.
Chandra Sashikanth ; Gawlick Dieter ; Jain Sandeep ; Rizvi Hasan ; Shah Premal ; To Alvin, Apparatus and method for message queuing in a database system.
Braden-Harder Lisa ; Corston Simon H. ; Dolan William B. ; Vanderwende Lucy H., Apparatus and methods for an information retrieval system that employs natural language processing of search results to.
Paulo S. Tubel ; Lynn B. Hales ; Randy A. Ynchausti ; Donald G. Foot, Jr., Application of adaptive object-oriented optimization software to an automatic optimization oilfield hydrocarbon production management system.
Brim Ted E. (Grand Haven MI) Gautraud Michael G. (Muskegon MI) Mowers David L. (Muskegon MI) Dudzinski Gary (Bolingbrook IL), Bowling statistics display system.
Geshwind, David Michael, Class of methods for improving perceived efficiency of end-user interactive access of a large database such as the world-wide web via a communication network such as "The Internet".
Masand Brij M. (Medford MA) Smith Stephen J. (Lynnfield MA), Classification of data records by comparison of records to a training database using probability weights.
Hanzek, Joe J., Communication schema of online system and method of status inquiry and tracking related to orders for consumer product having specific configurations.
Koski Robert E. (Sarasota FL) Henderson Kenneth R. (Sarasota FL) Barlow Christopher R. (Sarasota FL), Computer system including means for decision support scheduling.
Pant Sangam ; Andre David L. ; Watson Gray ; Green Richard M. ; Schiegg Michael J., Computer system with user-controlled relevance ranking of search results.
Bates Cary Lee ; Day Paul Reuben, Computer system, program product and method of creating aliases and accessing storage locations in a computer system with same.
Bradley John ; Hewett Frederick A. ; Sinclair Bruce D., Computer-based information access method and apparatus to permit SQL-based manipulation of programming language-specif.
Farrell Joel A. (Endicott NY) Record Stephen E. (Ridgefield CT) Wade Brian K. (Apalachin NY), Controlled scheduling of program threads in a multitasking operating system.
Onishi Satoshi,JPX ; Kubo Yukihiro,JPX ; Kimura Kozue,JPX ; Nishida Osamu,JPX, Conversational sentence translation apparatus allowing the user to freely input a sentence to be translated.
Vipin Kewal Ramani ; Rasiklal Punjalal Shah ; Ramesh Ramachandran ; Piero Patrone Bonissone ; Yu-To Chen ; Phillip Edward Steen ; John Andrew Johnson, Diagnostic system with learning capabilities.
Tsourikov Valery M. ; Batchilo Leonid S. ; Sovpel Igor V.,BYX, Document semantic analysis/selection with knowledge creativity capability utilizing subject-action-object (SAO) structures.
Noble William B. (Santa Monica CA) Patel Bhadra K. (Anaheim CA) Wang Jenny K. (Cerritos CA), Federated information management (FIM) system and method for providing data site filtering and translation for heterogen.
Balogh Aristotle (Bowie MD) Blejer Hatte (Alexandria VA) Chen Eugene (Arlington VA) Flank Sharon (Washington DC) Iannacone Carmen (Fairfax VA) Maloney John (Upper Marlboro MD) Martin Patrick (Arlingt, Generation, archiving, and retrieval of digital images with evoked suggestion-set captions and natural language interfac.
Paik Woojin ; Liddy Elizabeth D. ; Liddy Jennifer Heverin ; Niles Ian Harcourt ; Allen Eileen E., Information extraction system and method using concept relation concept (CRC) triples.
Paik Woojin ; Liddy Elizabeth D. ; Liddy Jennifer Heverin ; Niles Ian Harcourt ; Allen Eileen E., Information extraction system and method using concept-relation-concept (CRC) triples.
Appelt, Douglas E.; Arnold, James Frederick; Bear, John S.; Hobbs, Jerry Robert; Israel, David J.; Kameyama, Megumi; Martin, David L.; Myers, Karen Louise; Ravichandran, Gopalan; Stickel, Mark Edward, Information retrieval by natural language querying.
Horvitz Eric ; Breese John S. ; Heckerman David E. ; Hobson Samuel D. ; Hovel David O. ; Klein Adrian C. ; Rommelse Jacobus A.,NLX ; Shaw Gregory L., Intelligent user assistance facility.
Andreshak Joseph Charles (Mahopac NY) Daggett Gregg H. (Yorktown Heights NY) Karat John (Greenwich CT) Lucassen John (New York NY) Levy Stephen Eric (Valhalla NY) Mack Robert Lawrence (Yorktown Heigh, Interactive computer system recognizing spoken commands.
Friedes Albert (East Brunswick NJ) Leighton Diane R. (Marlboro NJ) Sahni Paramdeep S. (Marlboro NJ) Zahray Walter P. (Matawan NJ), International priority calling system with callback features.
Nasr Hatem N. (Edina MN) Sadjadi Firooz A. (St. Anthony MN) Bazakos Michael E. (Bloomington MN) Amehdi Hossien (Edina MN), Knowledge and model based adaptive signal processor.
Harris Larry R. (Concord MA) Hill Jeffrey M. (Watertown MA) Marcott Dayton (Watertown MA) Rochford Timothy F. (East Greenwich RI), Knowledge base management system.
Erman Lee D. (Palo Alto CA) Clancey William J. (Palo Alto CA) London Philip E. (Cupertino CA) Scott A. Carlisle (Palo Alto CA) Bennett James S. (Palo Alto CA) Lark Jay S. (Palo Alto CA), Knowledge engineering tool.
Mason Raymond I. ; Dixon Michael D. ; Beaman Thomas G. ; Ferraro Robert P. ; Vanduyn Robert M. ; des Rivieres James J.,CAX, Machine control using register construct.
Scott E. Sikora ; Nakul M. Lele ; Rajnish Tahiliani ; Ben L. Yip ; Henry Mingkeung Yeung ; Timothy Merrell, Method and apparatus for allocating mixed transaction type messages to resources via an integrated queuing mechanism.
Kanno: Yuji (Kawasaki JPX) Nagao Kenji (Kawasaki JPX) Udea Kenichi (Tokyo JPX), Method and apparatus for analyzing the semantics and syntax of a sentence or a phrase.
Amado Carlos Armando (444 Brickell Avenue #51-111 Miami FL 33131-2400), Method and apparatus for applying if-then-else rules to data sets in a relational data base and generating from the resu.
Carus Alwin B. (Newton MA) Wiesner Michael (West Roxbury MA) Haque Ateeque R. (Medford MA), Method and apparatus for automated search and retrieval process.
Kudoh Masato,JPX ; Tanaka Midori,JPX ; Koseki Yoshiyuki,JPX, Method and apparatus for cataloging and displaying e-mail using a classification rule preparing means and providing cat.
Jennings ; Jr. Ralph E. (Madison AL) Haley ; Jr. Benjamin P. (Madison AL) Holland R. D. (Somerville AL) Cecil Deborah K. (Madison AL) Tassone Anthony E. (Madison AL), Method and apparatus for enabling control of task execution.
Buchsbaum Adam Louis ; Giancarlo Raffaele,ITX ; Westbrook Jeffery Rex, Method and apparatus for generating deterministic approximate weighted finite-state automata.
Kloth, Raymond J.; Edsall, Thomas J.; Fine, Michael; Dutt, Dinesh G., Method and apparatus for implementing a quality of service policy in a data communications network.
Zahariev Manuel,CAX, Method and apparatus for managing electronic documents by alerting a subscriber at a destination other than the primary destination.
Beck Christopher Clemmett Macleod ; Berke Jonathan Michael ; Johnstone Joel A ; Mitchell Robin Marie ; Powers James Karl ; Sidell Mark Franklin ; Knuff Charles Dazler, Method and apparatus for supporting diverse interaction paths within a multimedia communication center.
Bergeron Larry E. (72 Coventry La. Trumbull CT 06611) Chamberlin David B. (11 Hunter Ridge Rd. Monroe CT 06468) Jachmann Emil F. (34 Harbor View Pl. Stratford CT 06497) Su Jy-Hong J. (14 Hunters La. , Method and apparatus for the generation of reports.
Johnson William J. (Flower Mound TX) Weber Owen W. (Coppell TX), Method and system for prioritization of email items by selectively associating priority attribute with at least one and.
Fitch Blake G. (New Rochelle NY) Giampapa Mark E. (Irvington NY), Method and system for process scheduling from within a current context and switching contexts only when the next schedul.
Tadashi Takahashi ; Takeshi Kumazawa JP; Hirokatsu Araki ; Tsuyoshi Hirao ; Hitoshi Matsumoto, Method and system for sorting and forwarding electronic messages and other data.
Scannell Niamh C. (Mountain View CA) Dawson Stuart D. (Reading GB2) Redmond Anthony J. (Dublin IEX) Himbaut Serge (Antibes FRX) Bares Pascale (Villeneuve Loubet FRX) Clark Alison (Morpeth GB2), Method and system for sorting and prioritizing electronic mail messages.
Billheimer, D. Dean; Booker, Andrew James; Condliff, Michelle Keim; Greaves, Mark Thomas; Holt, Fredrick Baden; Kao, Anne Shu-Wan; Pierce, Daniel John; Poteet, Stephen Robert; Wu, Yuan-Jye, Method and system for text mining using multidimensional subspaces.
van Zuijlen Job M. (Utrecht NLX), Method and system for the representation of multiple analyses in dependency grammar and parser for generating such repre.
Schroeder, Myron E.; Blaschke, Marilyn W.; Zetlmeisl, Michael J.; Fischer, David M.; Tacchi, Kenneth J., Method for chemical addition utilizing adaptive optimization.
Chapman William (Scottsdale AZ) Chang Gwo-Jer (Mesa AZ) Hwang Emerald (Tempe AZ) Holt James (Del Valle TX) Howington Lee (Austin TX) Chalmers James (Pflugerville TX), Method for describing measured results.
Masato Asami JP; Youichi Yamamoto JP; Tomoto Shimizu JP; Yoshito Kamegi JP, Method for querying a database in which a query statement is issued to a database management system for which data types can be defined.
Brett Louis Lindsley ; Uri Dayan IL; Moshe Tarrab IL, Method, device and article of manufacture for implementing a real-time task scheduling accelerator.
McCown Patricia M. (Cresskill NJ) Conway Timothy J. (Highland Park NJ) Jessen Karl M. (Bayonne NJ), Methods and apparatus for monitoring system performance.
Lamb, Christopher H.; Petrack, Scott B.; Slaughter, III, Frank G.; Toga, James E., Methods and apparatus for providing communications services between connectionless and connection-oriented networks.
Agrawal Rakesh ; Chakrabarti Soumen ; Dom Byron Edward ; Raghavan Prabhakar, Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values.
Rischar Charles M. (Chardon OH) Rohn David R. (Willoughby Hills OH) Johnston David A. (Mentor OH) Husted Raymond (Mentor OH), Multitasking industrial controller.
de Hita Carolina Rubio,BEX ; Akker David van den,BEX ; Govaers Erik C. E.,BEX ; Platteau Frank M. J.,BEX ; Deun Kurt Van,BEX ; Macpherson Melissa ; de Bie Peter,BEX ; Laviolette Sophie,BEX, Natural language information retrieval system and method.
Kirsch Steven T. ; Chang William I., Performing automated document collection and selection by providing a meta-index with meta-index values indentifying co.
Joseph L. Hellerstein ; Nikolaus Haus, Predictive model-based measurement acquisition employing a predictive model operating on a manager system and a managed system.
Nunberg Geoffrey D. (San Francisco CA) Stansbury H. Tayloe (Mountain View CA) Abbott Curtis (Menlo Park CA) Smith Brian C. (La Honda CA), Processing natural language text using autonomous punctuational structure.
Narayanan C. Murali (Wheaton IL) Zee Benjamin (Oak Park IL), Programmable memory-based arbitration system for implementing fixed and flexible priority arrangements.
Komissarchik, Edward; McPartlan, Kevin; O'Brien, Lauren; Sorensen, Gary Lee, Real-time transaction routing augmented with forecast data and agent schedules.
Theimer Marvin M. (Mountain View CA) Spreitzer Michael J. (Tracy CA) Weiser Mark D. (Palo Alto CA) Goldstein Richard J. (San Francisco CA) Terry Douglas B. (San Carlos CA) Schilit William N. (Palo Al, Selective delivery of electronic messages in a multiple computer system based on context and environment of a user.
Rakesh Agrawal ; Andreas Arning DE; Roland Seiffert DE; Ramakrishnan Srikant, Self-adaptive method and system for providing a user-preferred ranking order of object sets.
Cronshaw ; David ; Shemer ; Jack E. ; Turner ; William D. ; Hartke ; Dav id ; Keddy ; James R. ; DuVall ; Wilbur E. ; Sterling ; Warren M., Soft display word processing system with multiple autonomous processors.
Bahl Lalit R. (Amawalk NY) Bellegarda Jerome R. (Goldens Bridge NY) De Souza Peter V. (Mahopac Falls NY) Gopalakrishnan Ponani S. (Croton-on-Hudson NY) Nadas Arthur J. (Rock Tavern NY) Nahamoo David , Speech coding apparatus having speaker dependent prototypes generated from nonuser reference data.
Potts ; Sr. George W. (Wellesley MA) Breen Patrick (Medford MA) Paushter Robert (Watertown MA), System and method for converting sentence elements unrecognizable by a computer system into base language elements recog.
Ruben Ernesto Ortega ; Dwayne Edward Bowman, System and method for correcting spelling errors in search queries using both matching and non-matching search terms.
Sheth Amit ; Avant David ; Bertram Clemens, System and method for creating a semantic web and its applications in browsing, searching, profiling, personalization and advertising.
Ponting Keith Michael,GBX ; Series Robert William,GBX, System and method for ordering data in a computer system in accordance with an input data sequence.
Bardy Gust H., System and method for providing normalized voice feedback from an individual patient in an automated collection and analysis patient care system.
Brady James Thomas ; Finney Damon W. ; Hartung Michael Howard ; Ko Michael Anthony ; Mendelsohn Noah R. ; Menon Jaishankar Moothedath ; Nowlen David R., System and method for queuing of tasks in a multiprocessing system.
Bateman Thomas Howard,CAX ; Kierstead Bruce Edward,CAX ; Noble William Alexander,CAX ; Curry Timothy Lee,CAX ; Lockett John Alan,CAX ; Mersereau Laurie Edward,CAX ; Ouellette Robert James,CAX, System for coordinating communications via customer contact channel changing system using call centre for setting up the.
Records Roger M. (Kent WA) Taylor John A. (Mercer Island WA) Shontz William D. (Renton WA) McKenzie William A. (Federal Way WA), System for creating a single electronic checklist in response to multiple faults.
Tracz William J. (Owego NY) Newton Eric C. (Odenton MD), System for identifying and linking domain information using a parsing process to identify keywords and phrases.
Rivette Kevin G. ; Rappaport Irving S. ; Hohmann Luke ; Puglia David ; Jackson Adam ; Rabb ; Jr. Charles ; Smith David W. ; Park Brian ; Thornthwaite Warren ; Navarrete Jorge A., System, method, and computer program product for patent-centric and group-oriented data processing.
Mohri Mehryar ; Pereira Fernando Carlos Neves ; Riley Michael Dennis, Systems and methods for determinization and minimization a finite state transducer for speech recognition.
Hoffman Roy L. (Pine Island MN) Kempke William G. (Rochester MN) McCullough John W. (Atlanta GA) Soltis Frank G. (Rochester MN) Turner Richard T. (Rochester MN), Task handling apparatus.
Kieckhafer Roger M. (Ellicott City MD) Finn Alan M. (Amston CT) Walter Chris J. (Columbia MD), Task scheduler for a fault tolerant multiple node processing system.
Horvitz Eric ; Heckerman David E. ; Dumais Susan T. ; Sahami Mehran ; Platt John C., Technique which utilizes a probabilistic classifier to detect "junk" e-mail by automatically updating a training and re-training the classifier based on the updated training set.
Stephane Herman Maes, Text independent speaker recognition with simultaneous speech recognition for transparent command ambiguity resolution and continuous access control.
Aust Harald,DEX ; Brode Holger W.,DEX ; Schroer Olaf,DEX ; Marschner Jens F.,DEX ; Marti Del Olmo Erique,DEX ; Mehlan Ralf,DEX, Transaction system based on a bidirectional speech channel by status graph building and problem detection for a human us.
Liddy Elizabeth D. ; Paik Woojin ; McKenna Mary E. ; Weiner Michael L. ; Yu Edmund S. ; Diamond Theodore G. ; Balakrishnan Bhaskaran ; Snyder David L., User interface and other enhancements for natural language information retrieval system and method.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.