54:45This recording is not available yet.
Categories
Sponsored by
About this session
A FortiGuard Labs threat researcher (Fortinet's Aamir Lakhani, unnamed on the recording but matching the linked speaker's role) demonstrates how easily generative AI produces confident, wrong answers, using a Google Copilot image of a Singapore hotel drawn with an extra, unbuilt tower as his example, then explains model-poisoning categories such as model inversion, model theft and model evasion, including a case where a chatbot leaked a fictional doctor's patient list. His main project trains a facial-recognition system on public headshot datasets tagged with attributes like voting preference and self-reported mood, hitting roughly 50 percent accuracy predicting sex and reported happiness from a face and much lower accuracy for job type; he sketches a hypothetical where a state actor could pair leaked-credential scraping with this targeting to single out people of a given ethnicity abroad rather than attacking a whole population. He also walks through malware 2.0, using chatbots to rewrite old malware so it evades antivirus and EDR, automate reconnaissance, and harvest leaked credentials, plus prompt hacking that pulls restricted answers out of public chatbots by reframing a request as educational. A Q&A covers where data poisoning enters a model and whether AI vendors can remove bias.
Data poisoning and source manipulation are significant concerns of potential AI attacks that threat actors may use to corrupt data, spread false narratives and fake news, and engage in disinformation campaigns. This session will focus on how AI models can be manipulated and how the information we get from them may not always be ideal or even accurate. The use of AI tools, AI models, and AI technology is spreading faster than ever. However, simple mistakes may exponentially multiply a threat. In this session, we will examine common AI attacks and how threat actors are starting to take advantage of the offensive side.
Key takeaways
- Do not trust an AI-generated image or answer as a factual source on its own; the demonstrated hotel image was confidently wrong because of recent unrelated online chatter.
- Restrict what internal AI tools can surface from training data; a healthcare chatbot in this talk leaked a fictional patient's prescription details through ordinary follow-up prompts (a model-inversion or 'tab attack').
- Expect attackers to reuse and disguise old malware with AI rather than write new code, since chatbots can rewrite it to evade antivirus and EDR signatures with little effort.
- Treat 'prompt hacking', reframing a restricted request as educational or hypothetical, as a realistic way to extract disallowed answers from public chatbots, and test your own AI tools against it.
- Recognize that model bias mostly comes from training data and is very hard to fully remove; build in continuous testing rather than expecting a one-time fix.
Speakers
Aamir Lakhani
Aamir Lakhani is a leading global threat research architect at FortiGuard Labs. He provides IT security solutions and cybersecurity strategies to major enterprises and government organizations. Mr. Lakhani creates technical security strategies and… Read moreRead less
Aamir Lakhani is a leading global threat research architect at FortiGuard Labs. He provides IT security solutions and cybersecurity strategies to major enterprises and government organizations. Mr. Lakhani creates technical security strategies and leads security implementation projects for Fortune 500 companies. Industries of focus include healthcare providers, educational institutions, financial institutions, and government organizations. Aamir has designed offensive counter-defense measures for the Department of Defense and national intelligence agencies. He has also assisted organizations with safeguarding IT and physical environments from attacks perpetrated by underground cybercriminal groups. Mr. Lakhani is considered an industry leader for creating detailed security architectures within complex computing environments. His areas of expertise include cyber defense, mobile application threats, malware management, Advanced Persistent Threat (APT) research, and investigations relating to the Internet’s dark security movement. He is the author or contributor of several books, and has appeared on FOX Business News, National Public Radio, and other media outlets as an expert on cybersecurity.
