Fake Faces, Real Damage: The Corporate Risk of AI-Powered Manipulation
Security professionals are rapidly confronting a new reality: artificial intelligence (AI) and big data, while excellent tools for improving productivity and business operations, are equally lowering the barriers for sophisticated attacks by a wide range of threat groups. From hostile nation-states to issue-motivated groups to cybercriminals, these technologies are enabling attacks that are more personalized, scalable, and harder to detect.
The widespread availability of our personal data—from what we post on social media to the massive resale of information gathered by data brokers from both our devices and our online activity—has made open-source data the key ingredient for highly effective AI-driven deception and disruption and enabled the creation of deepfakes.
How Our Data Is Collected
It’s simple to see how our publicly shared social media and online data can be collected, but many people are unaware that many portable devices, such as mobile phones and smartwatches, run applications that are actively selling our personal data to international data brokers. The scale of this is huge, with an estimated 4,000 data brokering companies all competing within a growing data market that reached $294.27 billion in 2025 and is projected to expand to $419.72 billion by 2030.
These brokers compile information from multiple sources and on-sell it at relatively low prices. Some apps, for example, transmit a user’s location every 15 minutes. When combined with other big data sources, including the increasing number of databases that have been hacked or leaked, this information can be used to build a detailed pattern of life: where a person lives and works, where his or her children go to school, and other highly personal insights.
It is now relatively easy to set up a fake recruitment company, backed by a shiny website, employees on LinkedIn, and a strong social media presence. This strategy can be used to target individuals with generous job offers in order to collect detailed resumes with a wealth of professional and personal information. These fake companies also make offers of paid surveys or ask for a reference to help a colleague get their dream job—offers or requests designed to gather additional data for later targeted attacks.
AI-Driven Deception in Action
In early 2024, a finance employee unknowingly transferred $25 million after participating in a Zoom meeting where all the other participants—including the company’s CFO—were AI-generated deepfakes.
Unfortunately, we’ve also seen deepfake technology used to exploit teenagers and adults. Fake profiles and AI-generated videos have been used to coerce individuals into sharing intimate images, which are then used for sextortion. Tragically, this has led to numerous suicides around the world.
Criminals are also using social media photos and deepfake software to create fake explicit content that appears to feature real individuals. These fake videos and images are used for blackmail, intimidation, or revenge—motivations that have proven particularly effective against people in sensitive professions like teachers, police officers, or politicians, as well as those in conservative communities.
We have seen the use of deepfakes, AI, and open-source data used to infiltrate the workplace. In 2023, the U.S. State Department issue an advisory warning that North Korean operatives linked to the Reconnaissance General Bureau had secured remote jobs in Western tech and cryptocurrency firms. Using open-source platforms like GitHub and LinkedIn, they scraped real identities and employed generative AI to create synthetic voices and video avatars to pass remote employment interviews. These deepfakes frequently passed background checks, allowing operatives to exfiltrate source code, deploy malware, and steal intellectual property.
Several likely scenarios are playing out in corporate environments outside the public view. These include AI-enhanced phishing, where open-source data is combined with social engineered information and AI to craft hyper-targeted phishing emails.
Consider this real-world inspired scenario. You receive an email that reads:
“Hi, it’s Chris from next door. I’ve just hit and badly hurt a cat and I think it might be Cleo – I’ve attached a photo. Can you confirm if it’s her?”
Naturally, you panic and click.
The details feel authentic because they are.
- Your neighbor’s name came from a local Facebook group.
- Your pet’s name was found in an Instagram post.
- The photo shows a cat that looks like Cleo, with a street background that was AI-generated from Google Maps and shows imagery of your suburb.
You’re relieved it isn’t Cleo, but meanwhile, malware silently infects your device and possibly your corporate network. You mention the scare at lunch, but none of your colleagues flag it for the security team. Why? Because employees are trained to look for spelling mistakes and suspicious email addresses, not emotionally manipulative and highly realistic phishing attacks.
Deepfakes as Insider Threat Tools
AI can now be used by disgruntled insiders or issue-motivated groups to create fake videos or audio recordings that appear to show employees engaged in damaging behavior. These could include:
- Audio of an employee in a discriminatory or bullying workplace conversation
- A photo showing an employee at a protest that conflicts with his or her employer’s values
- A video of an employee using drugs or behaving violently at a private event
Are corporate investigators equipped to question the authenticity of such “evidence”? Tools to clone voices, fake images, and generate realistic videos are now widely available and low cost, and that require no advanced technical skill.
What Can Corporate Security Teams Do?
While companies involved in defense or cyber will closely monitor national and subnational threat actors, it is critical that, regardless of the sector, they must and can take certain steps to protect against the growing diversity of threat vectors and actors.
Deepfake detection. Certain tools help investigators to verify the likelihood that evidence has been created using deepfake technology, but these are not flawless and continue to develop. Ensuring verification of critical information by using trusted information is key. For example, investigators can verify that a certain individual was physically in the workplace through access-control data, or they can use the company’s cyber logs to confirm when and where an individual was logged on.
Although investigators already know the importance of the chain of evidence, this becomes critical with deepfakes. Determining where and how a recording was made or a picture was taken will be a key factor in helping to determine its authenticity.
Internal investigation teams should update their investigation policies and procedures to reflect the possibility of key evidence being fake and ensure that additional checks and confirmation are mandatory.
Employee training. Traditional security awareness training is no longer enough. All staff training must now cover:
- Awareness of emotionally manipulative or urgent communications
- Hyper-targeted phishing emails
- Deepfakes—including voice, photography, video, and online meetings
Employees must:
- Verify unusual or urgent requests—even from senior figures—via trusted channels. Trusted channels should be systems and applications that are difficult for external parties to access. Examples are internal company collaboration systems and workflow tools.
- Report strange or unusual content, even from personal emails.
- Limit what they share online and implement strong privacy settings on social media.
Collaboration with targeted staff. Key staff (such as executives, executive assistants, finance, IT, and HR) need updated standard operating procedures and verification methods, including a focus on internal controls.
- Financial safeguards. Update procedures and policies to account for new threats. Two examples are verifying changes to payee details via known account managers using trusted phone numbers and requiring two-factor confirmation for high-value payments.
- Device security. Corporate devices should enforce strong passwords, auto-lock, encryption and multifactor authentication, and they should disable geotagging or ad tracking. High-risk apps should be banned.
- Remote hiring. Fully remote workers should undergo enhanced vetting and receive limited access until verification is complete.
- Crisis planning. The crisis plan should include consideration and response to the release of deepfake videos, photos, or audio from key executives. Consider how to quickly respond with correct information and to minimize potential share market impacts and reputational damage.
AI and big data are here to stay, and they bring incredible benefits. But with these advancements come new and evolving threats. As security professionals, we must continuously update our training, adapt our procedures, and develop modern controls to keep people safe and maintain trust.
Nick de Bont, CPP, is the chief security officer for Thales Australia and New Zealand. In this role, de Bont leads the corporate security team, overseeing national security for hundreds of complex and classified projects spanning aerospace, defense, digital identity, communications, and cybersecurity. He previously worked at the Commonwealth Bank Group, where he was responsible for security for 13 countries, 50,000 employees, and 1,200 business sites.
