In a new paper, researchers have shown an exploit that allows users to possibly bypass the safety filters of large language models (LLMs) like GPT-4 and Claude Sonnet. By inducing hallucinations through clever text manipulation, this method reverts the models to their pre-RLHF state, effectively turning them into unconstrained word prediction machines capable of generating any content imaginable - no matter how inappropriate or dangerous.
Using Hallucinations to Bypass GPT4’s Filter
Large language models (LLMs) are initially trained on vast amounts of data, then fine-tuned using reinforcement learning from human feedback (RLHF); this also serves to teach the LLM
