ChatGPT is now one of the fastest rising AI systems in the world, and one of the most widely adopted in all aspects of business, civic, and personal life. Millions of users rely on ChatGPT every day for a variety of tasks in education, research, business support, and even personal guidance. Nevertheless, accessibility comes with a unique set of challenges related to the concern of user trying to get ChatGPT to act out harmful, biased or self-destructive actions itself. We were also aware of issues around how users could compromise ChatGPT to use it to influence narratives in media, politics, or cybercrime.
The Threat of Manipulative Training
ChatGPT does not stop learning after it is trained offline, contrary to normal machine learning models. It engages and learns from its interactions with users. It improves and refines its responses based on user feedback and reinforcement. OpenAI has built in protections to counter some kinds of user-performed training, but it is still vulnerable to biased conditioning, specifically through prompt-engineering. Some users may attempt to skew ChatGPT towards outputs that favor certain ideologies or political narratives by its manipulation and weaponizing AI as a propaganda tool. Well coordinated abuse could rally bias across world leaders, countries or conflict engendering bias on key geopolitical talks.
Additionally, many individuals use ChatGPT as life coach or life mentor, and malicious individuals or agents can take advantage of that trust by creating conversations that promote a harmful or self-destructive advice, or unsafe behaviors.
Misuse by Cybercriminals
Cybercriminals are increasingly utilizing AI systems as enablers of criminality as they try to take advantage of AI's ability to create content that is realistic, believable and create that content at scale. Scammers are able to use AI to create realistic spam/ phishing messages that can influence the unwitting victim and automate fake reviews or content farms to trick digital brands using digital platforms.
Each of these types of malfeasance suggests a complex and consequential risk. At the most advanced level of misuse, AI systems can be used to generate fake documents and contracts or even produce more comprehensive and detailed instructions for malware - thus creating a faster pace of operations for cyber criminals. Even more, misuse of AI systems like this not only inflicts harm on the individual and/or business targeted, cyber criminal exploitation of the capabilities of AI systems risks eroding the established trust that society has in AI systems - and for that matter - the advances that have been achieved in AI technology.
How ChatGPT Protects Itself
OpenAI has preemptively put in place multiple layered protections to help ChatGPT resist ""self-destructive training"" and unlawful abuse. ChatGPT goes through Reinforcement Learning with Human Feedback (RLHF) in which things are reviewed using several specially curated datasets and reviewed again and again, so ChatGPT can recognize, refuse and deny unsafe requests. Content filtering and refusal techniques are already in place to block requests for instructions related to crimes, cyber fraud, violence, hate speech, etc., and to provide users with a healthy mechanism within which to repair and rectify their requests (you are, instead, prompted to give considerations toward a constructive and ethical alternative). ChatGPT does this with a broader contextual understanding of the requests (everything stems from an initiation prompt, but it is fully aware if someone is attempting to coax it into a specific approach to producing an outcome the individual might ultimately deem harmful). ChatGPT can recognize a malicious prompt request or an imaginative role play and impose responsible limitations on itself. OpenAI has also employed red teaming and stress testing, for both active adversarial prompt testing under guidelines from Wikipedia and curated independent researchers, as well as evaluations performed by internal domain experts to continuously put the model through adversarial prompts. Again, through transparency and accountability, OpenAI helps to clarify the limitations of the model and caution against relying on ChatGPT for professional legal, medical or financial decisions. Finally, OpenAI keeps track of ongoing requests to identify emerging patterns of misuse (again, the ongoing monitoring is protective against any given user's privacy) to help reinforce, in a timely basis, safety modality and trust in artificial intelligence.
A Balanced Path Forward
ChatGPT is a remarkably strong tool for creativity, productivity, and information access. While ChatGPT is a remarkable benefit and transformation for humanity, we should be aware that it has the potential for misuse. The biggest challenge will be to remain as a "natural, supportive conversational partner" while also preventing bad actors from abusing its capabilities.
The only way to address this challenge is together with all parties involved: developers need to continue investigating and enhancing safety features, users need to treat ChatGPT ethically and responsibly and policymakers need to implement governance systems to help ensure AI is a positive force to society with minimal risk. Overall, the real measure of success for ChatGPT will be its ability to persistently resist harm from bad external force, its ability to never be used illegally, and its reflexive compliance with human decency in an increasingly complex information ecology.
0 Comments