초록 ▼

Language shift points in a computer document written in a plurality of natural languages are determined. An interval is defined on and moved through a text document in a computer memory, the interval contains a portion of the text in the document. As the interval is moved through the document for ea

Language shift points in a computer document written in a plurality of natural languages are determined. An interval is defined on and moved through a text document in a computer memory, the interval contains a portion of the text in the document. As the interval is moved through the document for each position of the interval, a probability that the text in the interval is written in each of a plurality of candidate languages is determined for the position. For the first position of the interval, generally the beginning of the document, a first candidate language is classified as the current language if it has the highest probability of all the candidate languages within the interval. A language shift point in the document is identified where the relative probability of a second candidate language is higher than the current language at a new position of the interval. At this point, the second candidate language is classified as the current language in the document after the language shift point. The process continues to identify other language shift points in the document.

대표청구항 ▼

[ We claim:] [1.] A method for detecting language shift points in a computer document written in a plurality of natural languages, comprising the steps of:moving an interval through a text document in a computer memory, the interval containing a plurality of words in the document;for each position o

[ We claim:] [1.] A method for detecting language shift points in a computer document written in a plurality of natural languages, comprising the steps of:moving an interval through a text document in a computer memory, the interval containing a plurality of words in the document;for each position of the interval, determining a probability that text in the interval is written in each of a plurality of candidate languages according to a respective number of matches of words in the interval with words in each of a plurality of word lists of a few common words selected from each respective candidate language;for a first position of the interval, classifying a first candidate language having the highest probability as the current language within the interval;finding a language shift point in the document where the probability that a second candidate language is higher than the current language for a new position of the interval; andclassifying the second candidate language as the current language in the document after the language shift point.