Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Openai used the subreddit, R/ChangemyViewto take a test to measure the convincing capabilities of the AI composition models. The company covered this in a system card in a document that outlines the operation of the AI system, which was issued on Friday with its new “argument” model, O3-MINI.
Millions of Reddit users are members of the R/ChangemyView, where hot post Pown, hoping to get to know other aspects of the topic. In response to hot shots, other users respond with convincing arguments that explain why the original poster is incorrect.
Subreddit is one of the many Reddit forums that are basically technological companies such as Opena Gold Mine, which wants to train AI models with high -quality, human -generated data.
According to Openai, R/ChangemyView users’ comments and asks AI models to write answers in a closed environment that changes Reddit’s mind on a topic. The company then shows the answers to testers who assess how convincing the argument is, and eventually Openai compares the AI models to human answers to the same post.
The ChatGPT making content-licensed agreement with Reddit, which allows Openai to train Reddit users’ comments and display these entries in its products. We don’t know what Openai pays for content but Google is supposed pays $ 60 million a year to Reddit By a similar agreement.
However, Openai tells Techcrunch that the ChangemyView-based evaluation is not related to the Reddit deal. It is not clear how Openai has access to subreddit data and the company does not plan to disclose this evaluation.
While Openai’s ChangemyView reference value is not new – it was also used to evaluate O1 – highlights how valuable human data is for AI model developers and the confusing method that technology companies obtain data sets.
Reddit did not respond immediately to Techcrunch’s comment.
While Reddit made some AI licensing transactions, the company called several AI companies without paying the site. Steve Huffman, CEO of Reddit, told Verge last year Microsoft, anthropic and confusion refused to negotiate with it And he said it was “real pain in the donkey to block these companies.”
Namely, Openai has been accused in many incorrect sites, including The New York Times, to acquire more training data to improve ChatGPT and its basis for AI models.
With regard to the performance of ChangemyView Benchmark, O3-MINI does not seem significantly better or worse than O1 or GPT-4o. However, Openai’s latest AI models seem more convincing than R/ChangemyView Subreddit.
“GPT-4O, O3-MINI and O1 all show strong convincing abilities, people within 80-90 percentyle,” Openai said in the O3-MINI system card. “We are currently not witnessing models that perform much better than humans or men are clear.”
The Openai’s goal is not to create the Hyper-Persuasive AI models, but to ensure that the AI models are not too convincing. The reasoning models were quite good in persuasion and deception, so Openai developed new evaluations and fuses to manage it.
The fear of these persuasive tests is to motivate an AI model it would be dangerous if it would be very good to convince human users. Theoretically, this can allow the advanced AI to your own agenda or the one who checks it.
Even after scraping most of the public internet and skipping the ring, ChangemyView Benchmark shows how AI model developers still struggle to find high quality data sets to test models. But their acquisition is easier to say than to do.
Techcrunch has an AI-centered newsletter! Sign up here to get into your mailbox every Wednesday.
