Performance Checks are Ai-Media’s unique method of qualifying our agents’ output. When an agent needs their output reviewed, we perform a detailed analysis to gather data based on three key categories:
- Accuracy measures how truthful the transcribed text is to the media.
- Presentation measures how the captions/transcript appear to the viewer.
- Style measures whether the output adheres to Ai-Media standards and requirements.
These checks help to determine what work is available for you to claim on the personalised dashboard. We perform these checks when you’re doing really well (Passing QA), when you’re having a rough patch (Failing QA), and periodically throughout your Ai-Media career. We may also send feedback via email based on the results of a Performance Check.
Accuracy
Accuracy is about how accurate and truthful the output is when compared to the audible media. Misunderstood content, missing or added content, unverified terms and spelling errors impact the accuracy of transcribed text.
Overuse of inaudible and unknown labels can impact the accuracy of a transcript when the audio is actually discernable or the term is able to be verified by a search engine. Missing or ineffective descriptive labels also impact accuracy. Missing punctuation can sometimes impact the accuracy of a transcript if sentence structure changes and affects the viewer’s understanding of the audible media.
Some errors carry more weight than others – it’s all about the impact on the person reading the transcribed text. For example, if the transcribed text says “I can do it” when the speaker says “I can’t do it”, the viewer would leave with a completely incorrect understanding of the media.
Presentation
Presentation is all about punctuation, capitalisation, and speaker labels – things that make the transcribed text easier to read and comprehend. Unnecessary content can also impact the presentation of a transcript. Like with Accuracy, the weighting of errors depends on the impact the Presentation errors have on the viewer.
Style
Style measures whether you’ve followed the guidelines that Ai-Media or the customer have provided. You should always be following Ai-Media’s transcription guidelines found in the Style Guide. The only exception to these rules is when there are special instructions or supporting documents attached to the task details page on Community Portal. To meet minimum quality requirements, your task will have zero Style errors. Any errors in Style will impact your aggregate performance check score and could result in you seeing a smaller variety of tasks on the personalised dashboard, even if your work has high scores in Accuracy and Presentation.
Errors
You should always endeavour to submit work that has no errors in any category. Although, we do understand the element of human error. This is why there are maximum permissible errors allowed at each quality level. However, the table below should only be used as a general guideline. Some errors carry more or less weight than others. For example, errors that change the meaning of the audible media are scored more strictly.
For example, Jane is able to see Standard level tasks on her personalised dashboard. She submits a transcript that is found to contain 14 Accuracy errors and 20 presentation errors. The performance check done on the task determines that the transcript meets Standard quality requirements.
Jane’s next performance check determines the transcript she submitted contains 10 Accuracy errors and 18 Presentation errors. The Accuracy score qualifies for High quality in that category, but the Presentation score remains at the Standard level. This means the task is only meeting Standard quality requirements.
Jane focuses on improving the Accuracy and Presentation of her work and her next performance check determines her submitted transcript contains 9 Accuracy errors and 8 Presentation errors. The Accuracy and Presentation scores qualify for High quality.
Jane’s next performance check determines her submitted transcript contains 5 Accuracy errors and 7 Presentation errors. The transcript meets Very High quality requirements! Go Jane! If she consistently submits work at this quality level, she may start to notice a greater variety of work on her personalised dashboard.
The above scenarios assume that all transcripts have approximately 1,000 words. As an example, if a transcript contains 500 words and contains 11 Accuracy and 10 Presentation errors, it is considered Standard quality. If a transcript contains 3,000 words and contains 20 Accuracy errors and 14 Presentation errors, it is considered Very High quality.