What are the key weaknesses of Generative AI?
5 Key Weaknesses of Generative AI
Experts have studied generative AI extensively. This article highlights the key weaknesses of generative AI, providing insights into its limitations based on research.
Latanya Sweeney, a Harvard Professor of Digital Government, guessed that over 90% of content will be generated by AI and bots in the future.
Recent innovations in artificial intelligence are redefining how we interact with our worlds. Latanya also reiterated that industry leaders in data privacy and emerging technologies have been researching limitations and developments in AI and have discovered how Chatgpt is heralding a major shift in how we use the internet and communicate with each other and the world at large.
A 2024 survey sampled public opinion on whether AI-generated content is better in terms of quality than human-generated content. The outcome revealed over 50% of respondents were rooting for human-crafted content. Also, one out of every 5 individuals interviewed held on to the opinion that AI-generated content is somewhat better or much better than human-made works.
We want to explore the fears and concerns of the 4 out of 5 people interviewed in that study by building on another research conducted by Microsoft, which highlighted some downsides related to generative AI .
For emphasis, generative AI is an aspect of AI that leverages large AI models or foundation models to create new content such as videos, audio, music, images, and text. This technology can implement out-of-the-box tasks faster than humans, summarize content, answer questions, classify, and do much more.
As with any other innovation, generative AI platforms such as Midjourney, Chatgpt, Meta AI, and Claude can improve productivity and can be abused or used for scams. The survey by Microsoft revealed that internet users all across the globe are bothered about fakes, abuse, and scams. Let’s spotlight some of these downsides and explore some instances where they played out with the consequences.
Index
AI-enhanced scams
One of every four companies has banned using generative AI in the workplace.
Yet, cybercriminals persistently trick employees into sharing sensitive information or completing a fraudulent payment. Criminals can use ChatGPT or its derailed twin brother in the dark web FraudGPT to craft fake IDs and real videos of financial statements or even use the voice and image of a company executive to create convincing deep fakes.
As for statistics on this new revelation, it is lugubrious. 65% of the respondents have had their companies fall victim to either actual or attempted payment scams in 2022. 71% of those who lost money were phished via email. The attack targeted large organizations with an annual turnover of $1 billion based on the survey conducted by the Association of Financial Professionals with email scams.
Phishing email is one of the common forms of fraudulent email. Before the entrance of generative AI, you could easily spot a phishing email with a typographical error. Right now, you would need more than intuition and gut feeling not to fall victim to perfectly crafted fake emails that look like a trusted source. I am talking about convincing-looking websites like eBay, Amazon, or Chase. A link in the phony email usually directs victims to log in and share confidential details. With this victim information in the hands of cybercriminals, they can trade with the IDs, use it to perform other heinous crimes, or empty the bank accounts of victims.
Spear phishing is another type of email scam, but it is more specific. You might not fall for a generic email scam. Still, when you see an email where the threat actor addresses you or your organization after researching your job title and the names of your manager, supervisors, or colleagues, there is a higher tendency to fall for it.
These scams are not new, but the case here is that generative AI makes it hard to differentiate what is authentic from fake. Gone are the days when you could detect phishing emails by wonky fonts or odd writing. Generative AI makes it possible for criminals to impersonate a company executive and hijack their voice to be used in a fake phone or video conversation.
This was the case in a recent incident in Hong Kong when a finance executive thought he had received an email from a chief financial officer based in the UK requesting $25.6 million to be transferred. While he initially suspected it was a phishing email, his doubt was drenched after seeing the CFO and a couple of other associates he could recognize in the video. Well, it turned out the call had been deeply faked. He learned about this after checking with the head office, but the transfer had been made by then. Christopher Budd, a director at Sophos, a cybersecurity firm, described the research and level of work invested into this scam as ‘pretty impressive.’
We have also seen deep fakes of several celebrities. We won’t forget how a deep fake Elon Musk was used to promote a platform that never existed in a phony investment scheme, or that of Gayle King, a News Anchor at CBS News, Tucker Carlson, former Fox News host, and Bill Maher , a talk show host all purposely promoting the new investment platform by Musk. Interestingly, the videos went viral on YouTube, Facebook and TikTok.
Industry experts have upheld that people can quickly generate synthetic identities by leveraging generative AI. Cybercriminals can bank on much information on the internet today and, with the help of generative AI, create a phishing email that will break your bank account. Should we be bothered about this?
Generative AI is hallucinating.
Large language models (LLM) are not infallible, yet we will explore several instances whereby they provide responses that appear authentic but were just fabricated data.
AI hallucinations are misleading or incorrect results generated by AI models. Sixty-one percent of respondents in a TELUS International Survey shared concerns about the tendency of generative AI to keep dispensing and spreading inaccurate information. So GenAI can produce 5 results, and 2 out of the results are fabricated and outright false or nonsensical. A New York attorney used a conversational chatbot to perform legal research while representing a client for an injury claim. The federal judge overseeing the suit discovered six bogus claims out of the precedents cited by the attorney in his brief. The exciting part of the story was that the chatbot made up the fake claim and referenced where the claims could be found in the legal databases. This is why having guardrails to deal with GenAI hallucinations is not an option but a must.
We can establish guardrails when we understand how AI hallucinations occur. AI industry experts may not have actual reasons for AI hallucinations, but several triggers have been identified.
If you train a model with insufficient or input significant gaps that culminate in edge cases that the model is not used to, hallucinations can occur. Let’s say you want to draft a contract; a generative AI trained with contract data from the healthcare sector may need more exposure to technical jargon, concepts, and terms in financial services. Generative AI works by producing a response based on a prompt. The quality of the prompt determines the prompt you generate: garbage in, garbage out. Do you know that sometimes, if GenAi needs help understanding the prompt, it will still provide you with a response using the insufficient data it has been trained, thus producing a faulty and damaging result?
Another trigger is how training prompts and texts are encoded. Usually, LLMs link terms to a set of numbers called vector encodings, and they come with some upsides. A word such as ‘bank’ has multiple meanings. It could mean a financial institution or a bank; This type of word can have an encoding per meaning to reduce ambiguity. Any error while encoding and decoding representations and text can cause generative AI to hallucinate.
LLMs must be trained with accurate and sufficient data, and stakeholders must embrace transparency and establish quality control to prevent hallucinations.
Generative AI is aiding the production of child sexual abuse content.
You may want to disagree with this statement, but we can all agree that GenAI is simplifying how child predators are quickly creating child sexual abuse content (CSAC) on a large scale. They often use original pictures and videos and adapt them to generate new abuse material. They can even turn the benign content of children and sexualize them to produce AI-generated CSAC.
The United States National Center for Missing and Exploited Children reported in January that it has received 4,700 reports of AI-generated content that depicts child sexual exploitation. The organization expects this figure to grow as AI evolves.
Child predators can use generative AI to devise new ways of victimizing and re-victimizing children. They can generate new images that match a child’s looks and design new poses, egregious content, or scale sextortion efforts. The technology can also produce an easy-to-use guide for bad actors on how to sexually abuse and coerce a child or destroy trials and distort artifacts of abuse.
Generative AI amplifies bias.
Humans are naturally biased. The data used in training LLM models comes from humans. What do you expect? If humans are biased, expect the worst from GenAI.
Stable diffusion generated 500 images based on written prompts. The result was a complete distortion of reality. An analysis of the outcome revealed that stable diffusion from the stability of AI took gender and racial disparities beyond the extremes more than obtained in real life.
I love the way a research scientist, Sasha Luccin captioned it. She said we are projecting a worldview into a world with diverse cultures and several visual identities. For instance, if you train AI models with statements such as, “black is synonymous with crime, and women are nurses, not doctors”, it will end up amplifying those biases.
Generative AI endangers data privacy.
Privacy is a significant concern despite the notable progress in Generative AI. What do you expect from a model not trained with algorithms that preserve privacy.
The data generated by the AI model is new and looks like the one it used during training. If training data includes sensitive information, there is a likelihood of violating the sensitive data of an individual, seeing training databases include the personal data of any individual with no explicit consent sought.
LLM training involves using millions of words over different natural language tasks. Take note that LLMs are a subset of GenAI, and studies have revealed that despite the possibility of memorizing massive volumes of data, including confidential information, they still pose privacy risks , which threat actors can gain access to perform nefarious activities.
An exfiltration is a form of attack that worsens the whole situation. Research demonstrated that an unauthorized individual can access training datasets, transfer, migrate, or steal them. Another angle is the disclosure of sensitive information while creating prompts.
Now that businesses are synchronizing unvetted apps that leverage GenAI into their business operations and systems, there are more risks of violating compliance and data breaches.
The bottom line is that if we want to deal with all these weaknesses and maximize the benefits of having the technology in our time, every stakeholder must embrace concerted efforts to ensure that what is designed for good will not end up as a self-destructive tool . It is up to us to make the world a better and safer place.
References
Cisco, (2024). More than 1 in 4 organizations banned use of GenAI over privacy and data security risks. Available at: https://newsroom.cisco.com/c/r/newsroom/en/us/a/y2024/m01/organizations-ban-use-of-generative-ai-over-data-privacy-security-cisco -study.html (Accessed: 02 April 2024)
Collier, K. (2023). Deepfake scams have arrived: Fake videos spread on Facebook, TikTok and YouTube . Available at https://www.nbcnews.com/tech/tech-news/deepfake-scams-arrived-fake-videos-spread-facebook-tiktok-youtube-rcna101415 (Accessed: 02 April 2024)
Kong, H., (2024). Everyone looked real’: multinational firm’s Hong Kong office loses HK$200 million after scammers stage deepfake video meeting . Available at https://www.scmp.com/news/hong-kong/law-and-crime/article/3250851/everyone-looked-real-multinational-firms-hong-kong-office-loses-hk200-million- after-scammers-stage (Accessed: 02 April 2024)
Missing Kids, (2024). Generative AI CSAM is CSAM . Available at: https://www.missingkids.org/blog/2024/generative-ai-csam-is-csam (Accessed: 02 April 2024)
Nicoletti, L. & Bass, D. (2023). HUMANS ARE BIASED. GENERATIVE AI IS EVEN WORSE . Available at: https://www.bloomberg.com/graphics/2023-generative-ai-bias/ (Accessed: 02 April 2024)
Rubenstein, A. (2023). ChatGPT is not quite ready to be your lawyer . Available at: https://www.morningbrew.com/daily/stories/2023/05/29/chatgpt-not-lawyer?mbcid=31642653.1628960&mblid=407edcf12ec0&mid=964088404848b7c2f4a8ea179e251bd1&utm_campaign=mb&utm_medium=newsletter&utm_source=morning_brew (Accessed: 02 April 2024)
Sheng, E. (2024). Generative AI financial scammers are getting very good at duping work emails . Available at: https://www.cnbc.com/2024/02/14/gen-ai-financial-scams-are-getting-very-good-at-duping-work-email.html (Accessed: 02 April 2024 )
Statista, (2024). Opinion of AI-generated content being better than human-created content among consumers in the United States as of 2024. Available at: https://www.statista.com/statistics/1461390/ai-generated-content-better-than- human-created-content/ (Accessed: 02 April 2024)
Tellux, (2023). Generative AI hallucinations: Why they occur and how to prevent them . Available at: https://www.telusinternational.com/insights/ai-data/article/generative-ai-hallucinations (Accessed: 02 April 2024)
Thorn, (2023) Thorn and All Tech Is Human Forge Generative AI Principles with AI Leaders to Enact Strong Child Safety Commitments. Available at: https://www.thorn.org/blog/generative-ai-principles/ (Accessed: 02 April 2024)
University of Sheffield, (nd). University Library: Generative AI literacy . Available at: https://sheffield.libguides.com/genai/perspectives (Accessed: 02 April 2024)
Usenix, (2021). Extracting training data from large language models. This paper is included in the Proceedings of the 30th USENIX Security Symposium. Available at: https://www.usenix.org/system/files/sec21-carlini-extracting.pdf
Falode, E. (2023). Is generative AI a threat to cybersecurity? Elijah Falode. https://elijahfalode.com/is-generative-ai-a-threat-to-cybersecurity/ (Accessed: 05 May 2024)
Comments are closed.