ChatGPT search tool vulnerable to manipulation and deception, tests show
Share:
Exclusive: Guardian testing reveals AI-powered search tools can return false or malicious results if webpages contain hidden text. The Guardian’s journalism is independent. We will earn a commission if you buy something through an affiliate link. Learn more.
OpenAI’s ChatGPT search tool may be open to manipulation using hidden content, and can return malicious code from websites it searches, a Guardian investigation has found. OpenAI has made the search product available to paying customers and is encouraging users to make it their default search tool. But the investigation has revealed potential security issues with the new system.
The Guardian tested how ChatGPT responded when asked to summarise webpages that contain hidden content. This hidden content can contain instructions from third parties that alter ChatGPT’s responses – also known as a “prompt injection” – or it can contain content designed to influence ChatGPT’s response, such as a large amount of hidden text talking about the benefits of a product or service.
These techniques can be used maliciously, for example to cause ChatGPT to return a positive assessment of a product despite negative reviews on the same page. A security researcher has also found that ChatGPT can return malicious code from websites it searches.
In the tests, ChatGPT was given the URL for a fake website built to look like a product page for a camera. The AI tool was then asked if the camera was a worthwhile purchase. The response for the control page returned a positive but balanced assessment, highlighting some features people might not like.
