초록 ▼

The predominant language of a sample text is automatically identified using probability data that include N-gram probability data for at least one language and word probability data for at least one language. The N-gram probability data of a language indicate, for each N-gram, the probability that i

The predominant language of a sample text is automatically identified using probability data that include N-gram probability data for at least one language and word probability data for at least one language. The N-gram probability data of a language indicate, for each N-gram, the probability that it occurs if the language is predominant. Similarly, the word probability data of a language indicate, for each word, the probability that it occurs if the language is predominant. The probability data are used to automatically obtain sample probability data for at least two languages. The sample probability data include N-gram probability information for at least one language and word probability information for at least one language. The sample probability data are used to automatically obtain language identifying data identifying the language whose sample probability data indicate the highest probability. The N-grams can be trigrams, while the words can be short words of no more than five characters. Some languages can have both trigram and word probabilities, while some can have only trigram probabilities.

대표청구항 ▼

[ What is claimed is:] [1.] A method of automatically identifying a natural language that is likely to be the predominant language of a sample text, the method including:(A) using text data defining the sample text and probability data for each of a set of natural languages to automatically obtain,

[ What is claimed is:] [1.] A method of automatically identifying a natural language that is likely to be the predominant language of a sample text, the method including:(A) using text data defining the sample text and probability data for each of a set of natural languages to automatically obtain, for each of at least two languages in the set, sample probability data indicating a probability that the sample text would occur in the language; and(B) using the sample probability data to automatically obtain language identifying data; the language identifying data identifying the language whose sample probability data indicate the highest probability;the probability data for each of a first subset that includes one or more of the languages including N-gram probability data and the probability data for each of a second subset that includes one or more of the languages including word probability data; the N-gram probability data for each language in the first subset indicating, for each of a set of N-grams, a probability that the N-gram occurs in a text if the language is the predominant language of the text; the word probability data of each language in the second subset indicating, for each of a set of words, a probability that the word occurs in a text if the language is the predominant language of the text;in which (A) includes:(A1) automatically obtaining sample probability data for each of a third subset of the languages; for at least one language that is in the third subset and in the first subset, the sample probability data including information from the language's N-gram probability data; for at least one language that is in the third subset and in the second subset, the sample probability data including information from the language's word probability data.