There are several reasons why Utah Compose's scoring engine may not flag all errors in a student’s essay or may incorrectly flag an error.

English is a complex, evolving language and usage varies across contexts

The English language has rules related to grammar, syntax, and usage. While rules provide guidelines for forming words, constructing sentences, and conveying meaning, for understanding and communicating in English, there are often exceptions, idiomatic expressions, and variations across contexts. Be mindful of the dynamic and ever-evolving nature of the English language when interpreting errors and feedback from the scoring engine.

Computers don’t “read” text the way humans do

In general, the scoring engine conforms to specific rules. An example is the rule that all proper nouns are capitalized. While this seems like a straightforward rule, it is not always clear to the scoring engine which words in student writing are proper nouns. Many common nouns are used for names or nicknames, such as “rocky,” “hunter,” “bear,” “hope,” “sugar,” and even “apple,” to name a few. Humans learn to apply context clues to determine whether words like these are common or proper nouns. Since the scoring engine does not “read” words in context the way humans do, it may not correctly differentiate between common and proper nouns in context.

Applying language rules can produce false positives

The scoring engine may generate a false positive by incorrectly flagging an “error.” For example, it can be difficult for the scoring engine to distinguish run-on sentences from longer and more complex sentences.

False positives can also result when the engine flags words it does not recognize. Oftentimes these are misspelled words, but they can be words that are not used frequently (see next section). When the scoring engine identifies a word that appears to be misspelled, it suggests similar words, which may not be appropriate in the event the flagged word was a false positive. For example, homophone errors, like using “bare” for “bear” and vice versa, are not always easily distinguished because some of those pairs can occur in similar contexts. If you have ever used the spelling and grammar check in Microsoft Word, you will see the same kinds of false positives. The scoring algorithm tries to limit the number of false positives, which can result in some true errors being overlooked. The Utah Compose team is constantly refining the grammar and spelling rules to give students optimal feedback.

Some words in the English language are used infrequently

The scoring engine recognizes words based on its lexicon, which includes thousands of words. These include Tier 1 words (common, everyday words) and most Tier 2 words (academic words used across multiple contexts). However, many Tier 3 words (academic words used within a particular domain) are not included in the lexicon because they are used infrequently. If the scoring engine does not recognize a word, it may be marked as misspelled, or another word may be suggested. Teachers can add Tier 3 words to their prompts so that the scoring engine does not flag them in student writing. These words are gathered periodically and added to the lexicon.

Utah Compose is designed to support teaching and learning of writing

Utah Compose was designed as a tool to support teachers and students. Utah Compose allows students increased writing practice opportunities needed to improve writing quality, while allowing teachers to focus their feedback efforts. As part of a formative assessment process, it allows students and teachers to monitor growth in writing performance. Utah Compose automated scores and feedback are intended to support students’ writing practice and knowledge of the writing quality criteria.