It’s a comment I’ve seen hundreds of times, or variations of throughout my time here at Kotaku: internet complaints about the quality of reviews. “A bot can do better than this,” some would cry. So let’s put that to the test.
I’ve run this test before, although last time I fed Kotaku Australia comments into the machine learning model. That was run using a free online version of the GPT-2 language model through a tool called Shortly, although the more powerful GPT-3 model is available now if you’re willing to pay to access the API.
The elements where the machine learning was asked to riff on experience and emotion actually turned out better than expected, but at almost every opportunity it read like the model adopted the “fake it till you make it approach.” It’d introduce details that were wrong, sometimes even names and developers from different projects entirely. Conclusion: AI can’t review a video game.
But it’s interesting to see just how far that AI-generated content has come. I imagine we’re not far off the day where some outlets or news wires start dabbling with GPT-3 generation for press release material, simply because the sheer volume of content online outstrips the number of people available to write it (but not the potential readership).
When it comes to reviews at least, you can’t beat the human touch. People know best what elements matter to other people. AI will get there one day, but that day isn’t today.
Original article posted at https://kotaku.com/what-happens-when-ai-tries-to-review-a-video-game-1847917110