Skilled Human Oversight – IGEA Trust & Safety

Skilled Human Oversight | Enhancing safety
with skilled human moderators.

Skilled Human Oversight | Enhancing safety with skilled human moderators.

With more than 60% of adult video game players regularly playing with others online, the video game industry takes seriously its goal of fostering positive, safe interactions for players. The industry uses a multidisciplinary approach to protect players online. Advanced moderation technologies can work hand-in-hand with teams of trained individuals who help moderate content and conduct in gaming environments. The industry invests in trained individuals to review and analyze flagged content and conduct and to determine appropriate next steps. Video game companies customize their approach to human moderation and advanced technologies depending on what is most appropriate for their games, services and player community.

Following incidents of disruptive behavior or inappropriate conduct, the company determines the appropriate response and next steps. Responses to disruptive behavior may include written warnings, restrictions on the use of online communications or other features, temporary suspension from a game or platform or a permanent ban from a game or platform. When appropriate, the video game industry will report the matter to law enforcement.

Many companies also train and collaborate with gamers themselves to help oversee individual game communities and promote a supportive culture.

Activision Blizzard

All Activision Blizzard teams utilize a combination of human moderation and tools to address reports of misconduct in a timely manner or prevent them all together.

Given the tremendous scale of Activision’s Call of Duty franchise, its anti-toxicity team employs a variety of tools to expand the scope of their reach. Among them: text-based filtering across 14 languages for in-game text (chat and usernames); a robust in-game player reporting system; a real-time voice chat moderation system (in select markets) powered by Modulate called ToxMod, an AI-powered voice chat moderation technology, to address toxic speech. They also leverage an AI-powered content moderation and filtering platform that shields players from online harm, including cyberbullying, abuse, hate speech, threats and child exploitation.

Microsoft

Xbox players are protected from unwanted content and conduct through a combination of technical means—including AI-based tools and automated content filters—and highly trained professional staff who review user reports and investigate allegations of misconduct.

The Xbox Moderation team serves as the company’s critical first responders. This team reviews user reports and items flagged by automated technology, often in real-time, and takes action as needed to protect player experiences. Microsoft also leverages community managers and Xbox Ambassadors, encouraging them to report any issues they observe in individual Xbox game communities.

The Xbox Investigation team looks more deeply into issues when more serious allegations are made. It also engages real-world help for users in crisis through third-party partnerships. This team is also responsible for appeals, where users who have received enforcement actions can challenge the determination and have their cases reinvestigated.

As Microsoft continues to invest in artificial intelligence (AI) and automation for content moderation, the goal is not only efficiency, but also to reduce harm to its human-staffed teams by reducing their repetitive exposure to harmful content. Despite advances in AI and automation, there likely will continue to be an ongoing need for skilled human oversight, so Microsoft supports its safety teams with care plans and wellness and resiliency training to ensure it is protecting those who protect players.

PlayStation

Sony Interactive Entertainment

Sony Interactive Entertainment’s (SIE) automated solutions are supplemented by its trained global moderation teams, which support players in all languages 24/7/365 where PlayStation Network services are available. SIE makes it easy for players to report unwanted interactions through the PlayStation Network interface. When the moderation team receives a report, they carefully review it and decide on next steps, which may include corrective action ranging from written warnings to console suspensions, in order to protect players and ensure a joyful play experience online. SIE ensures transparency by communicating its decisions to those involved and by providing an appeals process for permanent suspensions, where applicable.

Roblox

Moderation:
We have a stringent safety and monitoring system integrated into Roblox to promote civility and ensure the safety of our users. These systems are designed to enforce our policies, protect users’ personal information, and abide by local laws. We leverage human moderators and automated systems to proactively identify behaviours that may violate our policies.

We have a large, expertly trained team with thousands of professionals dedicated to protecting our users and monitoring 24/7 for inappropriate content. Our Safety team strives to take swift action and respond within minutes to address any content or behaviour that violates our Terms of Use or Community Standards.

Content submitted by creators automatically goes through a multi-step review process before appearing on the platform, using a combination of review by a large team of human moderators and machine detection.

We scan experiences at multiple stages: at the time of publishing and again when they are updated on Roblox. Experiences are evaluated by a suite of tools that identify problematic content that violates our standards. A human review team continuously operates to evaluate flagged experiences.

We filter text chat on the platform to block inappropriate content, such as discriminatory speech, bullying, extremism, violence, sexual content, etc., as well as personal information and offsite links. We update our filters continuously and filter text in multiple languages. We apply stricter filtering policies to users 12 years old and younger. For example, words that are unknown to Roblox will be filtered for our younger users. You can learn more about our safety tools and controls here.