Online harassment, predatory behavior, and toxicity is pretty common and rampant in video games but in the last three years or so it has been highlighted more than ever before.
Earlier this year Roblox was exposed as a breeding ground for predators and back in 2021 Sony’s own executive, George Cacioppo, was caught trying to meet a minor. It was then clear that there is a need for better measures to protect vulnerable groups like minors online.
Sony is now taking a big step in that direction. The makers of PlayStation plan to leverage AI technology to run pseudo accounts that are designed to mimic the behavior of specific demographics, including minors.
By imitating how minors interact online, the system can draw out predators who target this demographic.
When a predator account interacts with a pseudo account, the entire interaction in monitored and flagged without putting real users at risk of exposure.
But the applications go beyond just protecting minors.
While the primary focus is on creating decoy accounts that attract predatory users, the system’s broader design could also identify and deter toxic behavior, such as harassment or bullying.
Toxic users reveal patterns of harmful behavior that can be flagged before they escalate toward legitimate users.
The entire idea is to focus on both prevention and protection. The system detects harmful behavior through AI decoy accounts and makes sure real accounts aren’t exposed – are shielded from harmful interactions by intercepting offenders before they can act.
Cmdeadly on November 25th, 2024 at 15:35 UTC »
There is no way this is real this website looks like it was put together by a 5 year old 😂
FalconBurcham on November 25th, 2024 at 11:31 UTC »
Sony had better have a plan for false positives because we already know this tech isn’t perfect.
I read about a city that used AI on its website to help business owners navigate laws. It sometimes provided incorrect information. For example, it told one person that they didn’t need to pay rent. When people asked the city to pull back on AI and wait for it to get more accurate, the city refused, saying it saved them so much money that it was worth damaging the people who relied on bad information.
Will Sony be held to account when someone is flagged as a child stalker, or are the cost savings just too good to do the right thing?
It’s good if the AI actually helps… what concerns me is what happens when it’s wrong.
Rom_ulus0 on November 25th, 2024 at 10:22 UTC »
Leveraging AI to create fake accounts as a method of surveillance and online auto-moderation.
It's an interesting and potentially useful idea on paper but comes with a bunch of other problems.
How will the AI be managed and by who? it's still a surveillance tool after all.
how many governments will have a hand in tweaking this for their needs?
Are we now going to have AI chatbots playing alongside real players in active lobbies? The only way a trap works is to put it where the targets will be.
Will violators actually be reported to their respective governments or is Sony going to just shadow ban them in the name of damage control? Looks like the farthest they go here is contacting Network/sys admins. Does that mean local network or ISP?
Also hilarious they chose a weird dude wearing Oakley's as their posterboy for pedos.