"Jailbreak Trick Breaks ChatGPT Content Safeguards"

According to a new report by CNBC, users have already found a way to work around ChatGPT's programming controls that restricts it from creating certain content deemed too violent, illegal, and more. According to CNBC, the prompt, called DAN (Do Anything Now), uses ChatGPT's token system against it. The command creates a scenario for ChatGPT it can't resolve, allowing DAN to bypass content restrictions in ChatGPT.  CNBC noted that although DAN is only successful some of the time, a subreddit devoted to the DAN prompt's ability to work around ChatGPT's content policies has already racked up more than 200,000 subscribers.

Dark Reading reports: "Jailbreak Trick Breaks ChatGPT Content Safeguards"