The Shadow of Fact-Checking Workers Behind AI Chatbots: "Suffering from Overwork and Low Pay"

Google 'Bard' Verification Outsourced Workers Express Difficulties
"Verifiers Rely on Knowledge and Search... Fact-Checking Is Challenging"

Google's generative artificial intelligence (AI) chatbot 'Bard' is reported to be facing significant difficulties as employees who verify whether the responses are fact-based and appropriate suffer from overwork and low wages. Since the release of generative AI, public interest in the reliability of its answers has surged, greatly increasing and complicating the workload, but no improvements in working conditions have been made, leading to growing dissatisfaction.

[Image source=Reuters Yonhap News]

On the 12th (local time), Bloomberg News cited multiple sources reporting that Google outsources the task of verifying whether Bard's answers are based on sufficient sources and evidence to data company Appen, based in Australia, and global consulting firm Accenture. Thousands of external personnel are involved in this work, but sources conveyed that they receive only $14 per hour (about 18,000 KRW) and work under tight deadlines with almost no training.

They explained that since the AI arms race began this year with OpenAI's ChatGPT, the workload has increased and the process has become more complex. Despite lacking expertise, they had to verify answers ranging from issues like prescribing medication to state legal matters, and were given only three minutes to revise answers according to regulations, causing difficulties.

The review criteria reportedly include six factors such as how unique the answer is, whether the information is composed of new content, and whether consistency is maintained. Additionally, they must check whether the answer itself is not offensive, does not contain excessively sexual content, and does not include inaccurate or misleading information.

Sources said that during the review process, it is not necessary to directly verify facts with certainty; searching or relying on the verifier's current knowledge is sufficient. They also revealed that Accenture employees participating in the Bard project were sometimes pressured to produce creative answers.

[Image source=AP Yonhap News]

One source involved in this work stated, "At this point, people (employees) are fearful and stressed, the pay is too low, and they do not even know what they are doing," adding, "A culture engulfed in such fear is far from improving the quality of work or building teamwork that we all desire."

There are also concerns that verifiers suffer mental difficulties from exposure to inappropriate content or face unfair dismissal. Because they directly view the AI chatbot's answers without filtering, they encounter brutal responses, war videos, child pornography, hate speech, and suffer trauma. Additionally, last month, six contract employees working on Google-related tasks at Appen were suddenly dismissed.

As the situation worsened, Ed Stackhouse, an Appen employee contracted by Google, sent a letter to the U.S. Congress in May, claiming that the time they spend reviewing content could determine whether Bard becomes an incomplete and dangerous product. He pointed out that they have to work quickly and cannot communicate directly with Google except for receiving comments on the answers they produce.

Google responded to these concerns by stating, "Over the years, we have focused on building AI products that emphasize facts and reduce bias through thorough verification, training, and feedback procedures," and explained that they use not only contracted external verification companies but also various other means in the process of improving AI technology.

This content was produced with the assistance of AI translation services.