In conversations with clients, a certain assumption comes up regularly — one that concerns me. Companies ask: “How can we use AI for defense?” That is an excellent question — but it is only half the story. The other question I often ask in response is: “Do you know how attackers are already using AI against you today?” The silence on the other end speaks volumes.
📚 Threat context: what phishing is and how to protect against it — the fundamental attack that AI now accelerates and scales.
Artificial intelligence has stopped being exclusively a tool on the defenders’ side. It has become an offensive weapon — accessible, inexpensive, and scalable. Ransomware groups use language models to generate malware. BEC groups clone executives’ voices in real time. Dark web services offer GPT-based “phishing-as-a-service” for a few dozen dollars a month. This shift is not the future — it is happening right now, in December 2025, inside the infrastructure of companies that consider themselves well protected.
This article is a guide to the offensive use of AI by attackers. Without alarmism, with concrete examples, and — more importantly — with a practical defense strategy.
How Has Artificial Intelligence Changed the Cyberthreat Landscape?
For years, cybersecurity operated according to a certain logic of asymmetry: attackers needed to find one vulnerability, while defenders had to protect everything. AI deepens that asymmetry. Attackers have gained tools that radically lower the cost and time required to prepare an attack, while simultaneously and dramatically raising its quality and effectiveness.
Before the era of generative AI, preparing a credible spear-phishing campaign required hours of manual work: researching the target, analyzing their language and style, and crafting a convincing pretext. Today, a language model does this in seconds, based on a LinkedIn profile, a few social media posts, and a handful of public statements. The barrier to entry for cybercriminals has dropped to zero.
The second key mechanism is scalability. Traditional phishing was mass-scale but predictable — the same templates, easy to detect. Today’s AI-based attacks are simultaneously mass-scale and personalized. Each victim receives a message tailored specifically to them, written in perfect language, with references to their specific situation. This is not spam — it is surgical precision at industrial scale.
The third change concerns adaptation. Traditional malware had a fixed signature — antivirus systems could detect and block it. Generative malware can mutate its own code, evade signatures, and adapt its behavior to the environment in which it operates. The cat-and-mouse game between antivirus software and malicious code enters an entirely new level.
It is worth emphasizing that we are talking about technologies already in deployment, not about scenarios from research laboratories. Reports from Europol, the FBI, and CISA from the second half of 2025 consistently identify AI as the primary driver of increased sophistication in social engineering attacks and new malware variants.
📚 Read the complete guide: AI Security: AI w cyberbezpieczeństwie - zagrożenia, obrona, przyszłość
What Are Deepfake Audio and Video — How Do Attackers Impersonate Executives?
Deepfake refers to synthetic media — voice, video, or both — generated by AI based on samples of original recordings. The technology relies on deep learning architectures: Generative Adversarial Network (GAN) neural networks and newer diffusion models that can reproduce a voice or face with alarming fidelity.
What just three years ago required weeks of work and the computing power of a data center can today be accomplished in a few minutes on an ordinary laptop. Voice cloning from a sample shorter than 30 seconds is a standard feature offered by commercial APIs. Tools such as ElevenLabs, Resemble AI, and dozens of their equivalents on the dark web enable real-time speech generation — meaning an attacker can conduct a phone call “in the voice” of an executive with zero delay.
The most prominent case from 2024 involved a company in Hong Kong, where a financial employee transferred 25 million dollars after a video conference in which “the entire board of directors participated” — all faces and voices were deepfakes generated in real time. The victim saw familiar faces, heard familiar voices, and spoke with familiar people — none of whom were real.
In Poland and Central Europe, this type of attack — known as “deepfake BEC” or “CEO vishing fraud” — has been experiencing rapid growth since the beginning of 2025. Attackers primarily target Chief Financial Officers, chief accountants, and employees in payment processing departments. The scenario is always similar: an urgent, confidential transaction, time pressure, the authority of a “superior,” and a request to bypass standard verification procedures.
A critical point to understand is that the entry threshold for attackers is already very low. A dozen seconds of publicly available CEO footage from an industry conference or a media interview is sufficient to create a convincing voice clone. A LinkedIn profile with a short “about me” video or a webinar recording — that is all it takes.
How Does AI Automate Phishing and Spear-Phishing Campaigns?
Traditional phishing operated on a sieve principle: you send millions of messages and catch a fraction of a percent. It was easy to recognize for trained users and easy to filter out by email protection systems. AI changes this game fundamentally.
Automated AI-based spear-phishing operates in several phases. The first is reconnaissance: a language model analyzes the victim’s profile — their LinkedIn, posts on X/Twitter, contributions to industry forums, events they attended, names of their colleagues, projects they led. Based on this, the model builds a detailed psychological and contextual profile.
The second phase is content generation. The model creates a phishing message written precisely in the style of the victim and their organization — with the appropriate level of formality, industry jargon, references to real events and familiar people. The message contains no typos, ungrammatical sentences, or other “red flags” that security awareness training teaches people to recognize.
The third phase is pretext personalization. AI generates a plausible reason for contact — “I am following up on our conversation at the Gartner Security Summit in Berlin” or “You were recommended to me by Mark Smith from team X” — referencing real events from the victim’s life extracted through OSINT.
An example of scale is the WormGPT service, which entered mass use within criminal communities in 2024. This model, specifically fine-tuned without the ethical constraints of standard LLMs, generates phishing messages on demand, in dozens of languages, tailored to the profile of a specific victim. As of December 2025, security analysts identify more than a dozen similar services available on the underground market.
📚 Read the complete guide: Phishing: Phishing — types of attacks, recognition techniques, and organizational protection
What Is Generative Malware and How Does It Bypass Traditional Defenses?
Generative malware is malware that uses AI to dynamically modify its own code, behavior, or structure — in order to evade detection by systems based on signatures and heuristics.
Traditional antivirus systems and EDR solutions rely on pattern recognition: known signatures of malicious code, characteristic instruction sequences, typical behaviors. This approach has worked for decades — but it has a fundamental limitation. If the code is different every time, there is no fixed pattern to detect.
Generative malware solves this problem in several ways. AI-based polymorphism is a technique in which each copy of the malware is functionally identical but differs in code structure — much like two sentences can mean the same thing while being written in different words. An AI model generates new variants in real time, outpacing signature updates.
Metamorphism goes a step further — the malware rewrites not only its code but also its operational logic, making detection difficult even for behavioral systems. Tools such as BlackMamba (described by HYAS Protect in 2023 and widely analyzed in 2024–2025) demonstrate that AI can generate keyloggers and stealers on demand, on the fly, without a repeating pattern.
Another threat class consists of malware with built-in LLM capabilities that adapts its detection-evasion techniques based on signals from the victim’s environment. If it detects an active sandbox, it behaves like an innocuous program. If it detects a production environment, it activates the actual payload. This context-awareness is something that traditional malware was never capable of.
For organizations, this means one thing: an approach based exclusively on signatures is insufficient. What is required is a combination of behavioral analysis, environment isolation, network segmentation, and continuous anomaly monitoring — precisely what modern EDR and XDR platforms with AI modules provide.
How Does AI Support Social Engineering Attacks — Personalization at Mass Scale?
Social engineering has always been the most effective attack vector — not because it is easy, but because it targets the unremovable “software” of every organization: human psychology. AI gives attackers a superpower: the ability to conduct highly personalized social engineering attacks against hundreds or thousands of targets simultaneously.
In conversations with clients, I often describe this as the shift from “a fisherman with a net” to “a sniper with a clone.” A traditional attacker could either carefully prepare an attack against one individual or execute a mass but predictable attack against many people. Today they can do both at once.
AI enables automatic generation of pretexts tailored to the victim’s role. An HR employee will receive a message about an urgent personnel matter. An IT department employee will receive an alert about a critical vulnerability requiring immediate action. A Chief Financial Officer will receive information about a tender “the company submitted a bid for without their knowledge.” Each pretext is different; each is tailored to its recipient.
A new and particularly concerning trend is AI-assisted vishing — automated phone calls conducted by voice bots with cloned voices. The bot holds a natural conversation, responds to the victim’s questions, and adapts its communications based on what it hears. Victims describe these calls as fully convincing — natural speech rhythm, proper pauses, appropriate context.
Supply chain attacks gain a new dimension through AI. A model can analyze the business communications and correspondence style of a business partner (after one email account has been compromised), then generate messages that perfectly mimic the tone and style of that partner — creating convincing fraudulent invoices, requests to change banking details, or demands for urgent payments.
Personalization at mass scale changes the fundamental assumption of cybersecurity awareness training. Employees were taught that phishing is “easy to recognize” — language errors, generic tone, suspicious links. Today’s attacks lack these characteristics. This requires a fundamental shift in approach: instead of teaching employees to recognize the features of a bad message, they must be taught to recognize high-risk situations.
How to Detect Deepfakes — Tools and Verification Techniques
The question of deepfake detection is complex, because detection and generation technology are engaged in an arms race. Every new detection model is soon circumvented by an improved generation model. Nevertheless, effective approaches exist — both technical and procedural.
On the technical tools side: AI-based systems such as Microsoft Video Authenticator, Intel FakeCatcher, and solutions from Sentinel and Pindrop (for audio) analyze artifacts typical of synthetic media. Generative models leave characteristic traces — irregular blinking, unrealistic reflections in the eyes, depth-of-field anomalies, micro-artifacts at skin-to-hair transitions. In audio: micro-intonation anomalies, absence of natural ambient noise, overly perfect diction.
However, a critical caveat: technical tools for deepfake detection are not a panacea. They have significant false-negative rates, especially with the latest generation models. Relying on them exclusively is a mistake.
Procedural verification mechanisms are far more effective. The “out-of-band verification” principle — verifying every non-standard request through an alternative communication channel — is the most important defensive tool. If an executive calls requesting an urgent transfer, the employee should hang up and call back on a known, verified number. Not reply to an email in the same thread, and not use the number provided by the caller.
Organizations should implement identity verification procedures based on contextual questions that only the real person could know — and change these questions regularly. For transactions above a defined threshold, require confirmation through at least two independent channels.
It is also worth implementing a “security code” system — fixed passwords or phrases known only to an authorized pair of individuals, which confirm the authenticity of urgent requests. This is primitive but effective — and its resistance to AI is its strongest feature.
Practical warning signals during audio/video calls:
- Unusual response latency (AI responds too quickly or too slowly)
- Absence of natural slips of the tongue, “hmm,” or filler sounds typical of spontaneous speech
- A request to turn off the camera or poor image quality “due to technical reasons”
- Time pressure and emphasis on bypassing procedures
- A transaction subject that was not previously discussed through official channels
How to Protect Your Organization Against AI-Powered BEC (Business Email Compromise)?
Business Email Compromise is one of the most costly types of cyberattack — according to FBI data, BEC losses in 2024 alone exceeded 3 billion dollars. AI in attackers’ hands increases the effectiveness of BEC by an order of magnitude, because it eliminates the primary warning signals that employees were trained to rely on for their vigilance.
The first line of defense consists of technologies on the email side. Implementing and properly configuring SPF, DKIM, and DMARC is the absolute minimum — it eliminates sender domain spoofing. Secure Email Gateway solutions with AI modules (Microsoft Defender for Office 365, Proofpoint, Mimecast) analyze sender behavior patterns and detect anomalies: a message from the “CEO” who has never before written to this recipient, sent outside business hours, from a different IP address, on a topic the CEO has never communicated about by email.
However, technology is only one layer. Financial procedures are critical. Every transaction above a defined threshold (calibrated to the company’s scale) should require multi-step authorization through at least two independent channels. Any change to the banking details of a payment recipient should require verification directly with the supplier through a known contact number — never through details provided in the message initiating the change.
It is worth implementing a “zero trust for urgent requests” policy — every request marked as “urgent,” “confidential,” or containing pressure to bypass procedures should automatically trigger an extended verification protocol, not a shortened one. Paradoxically, the more “urgent” something is, the more slowly the approval should proceed.
In conversations with clients, I draw attention to one element that is frequently overlooked: employees must have the psychological and organizational permission to stop the process and ask questions — even if the person on the other side is someone important. A culture in which questioning a superior’s directive “through procedures” is rewarded, not penalized, is the foundation of BEC resilience.
How to Train Employees to Recognize AI-Assisted Attacks?
Security awareness training must evolve as rapidly as the attacks it is meant to protect against. A program based on knowledge from 2022 will not prepare an employee for deepfake attacks in 2025.
The first principle is a paradigm shift: instead of teaching “what a bad email looks like,” teach “what situations are by definition risky.” An urgent request for a bank transfer? A risky situation — regardless of how credible the sender appears. A request for login credentials over the phone? A risky situation — without exception. A supplier changing their banking details? A risky situation — always requires direct verification.
Training must include deepfake demonstrations. Employees who have never heard or seen a voice or face clone have no intuitive warning reflex. Showing real examples — both audio deepfakes and video — builds an alertness “muscle memory.” It is worth using deepfakes of the training participants themselves for this purpose (with their consent), as this creates the most lasting impression.
Vishing (voice phishing) simulations should be a regular component of the program. Just as email phishing simulations have become standard practice, “deepfake telephone tests” should become the norm in organizations with elevated risk profiles. Services providing BEC training simulations with synthetic voice are commercially available and increasingly widely used.
Training should be brief, frequent, and contextual — rather than an annual ceremony with a lengthy presentation. Monthly five-minute knowledge “nuggets” with concrete examples of attacks from the preceding weeks are far more effective than a four-hour training session once a year.
A “no-blame” culture is just as important as the content of the training. An employee who clicked on a phishing link should report it immediately — without fear of punishment. A quick incident report can limit the damage to a minimum. A culture in which employees conceal their mistakes out of fear gives attackers weeks before anyone notices the problem.
What Does a Defense Strategy Against AI-Driven Threats Look Like?
Effective defense against AI-driven threats is not a single product or a single system — it is a multilayered strategy combining technology, procedures, and people. The table below presents a map of actions grouped by priority and domain.
| Domain | AI Threat | Countermeasure | Priority |
|---|---|---|---|
| AI-generated spear-phishing | SPF/DKIM/DMARC + AI-powered Secure Email Gateway | Critical | |
| Financial transactions | Deepfake BEC, voice cloning | Dual-channel procedure + authorization threshold | Critical |
| Endpoints | Generative, polymorphic malware | EDR/XDR with behavioral — not signature-based — analysis | Critical |
| Identity and access | AI-assisted credential stuffing | Phishing-resistant MFA (FIDO2 keys), PAM | High |
| Voice communication | Deepfake audio, AI vishing | Out-of-band verification procedure, verification codes | High |
| Video conferencing | Real-time deepfake video | Verification procedure, predefined questions, background checks | High |
| Network | AI-directed lateral movement | Segmentation, AI-powered NDR, Zero Trust Network Access | High |
| Supply chain | AI-personalized attacks on suppliers | Supplier identity verification, banking-details-change policy | Medium |
| Cloud | Automated API scanning and exploitation | CSPM, API Gateway, regular penetration tests | Medium |
| Employee awareness | All of the above vectors | Continuous training, simulations, no-blame culture | Foundational |
The key insight from this table is that none of these countermeasures is new or revolutionary. Most are good practices that should be implemented regardless of AI. The change lies in the urgency — what was “nice to have” is becoming “must have immediately.”
Particular attention should be paid to Zero Trust architecture. A trust model based on verifying every access request, regardless of location and identity, is naturally resistant to many AI attack vectors. An attacker with a compromised account or a deepfake encounters multilayered contextual barriers — access from an unknown device, at unusual hours, to a resource the account has never previously accessed — each of these signals can trigger an alarm and require additional verification.
It is also worth investing in AI-related threat intelligence. Tracking new attack techniques, tools available on the dark web, and the activity of APT groups using AI enables organizations to get ahead of threats. Organizations that know what attacks to expect in the coming quarter are in a significantly better position than those reacting after the fact.
How Does nFlo Detect and Neutralize AI-Assisted Threats?
At nFlo, since the beginning of 2025 we have been conducting intensified operations in the area of detecting and responding to AI-driven threats. Working with more than 200 clients and experience from more than 500 projects gives us unique insight into how these threats evolve in real production environments.
Our approach to AI-powered threats is the principle of “fight AI with AI.” Traditional rule-based and signature-based systems lose against generative malware and dynamic phishing campaigns. For this reason, our SOC services rely on XDR and SIEM platforms equipped with AI models that analyze behavioral anomalies, event correlations, and deviations from the baseline — not merely on known signatures.
Response time, agreed per incident priority and written into the contract, matters more than usual with AI-assisted threats. Automated phishing campaigns and polymorphic malware move fast — the window from first click to data exfiltration or file encryption has shortened from hours to minutes. Every minute of delayed response has a measurable impact on the scope of damage.
In the areas of deepfake and BEC, we conduct dedicated workshops for clients from higher-risk sectors — financial, insurance, and manufacturing. The workshops include demonstrations of live deepfakes, training on verification procedures, and the implementation of process policies tailored to the organization’s structure. Our experience shows that organizations that have implemented “dual-channel” procedures for transactions above a defined threshold have recorded a 90% reduction in successful BEC attacks.
For clients with more complex IT infrastructure, we offer comprehensive AI-based social engineering resilience testing — including controlled vishing campaigns with synthetic voice and spear-phishing simulations generated by LLMs. This allows us to measure the real resilience of an organization and identify specific areas for improvement before real attackers do.
Our goal is not to frighten — it is to prepare. AI threats are real and growing, but they are not insurmountable. Organizations that understand the mechanisms of these attacks and implement appropriate countermeasures drastically reduce their risk. What decides the outcome is not the tooling but the rule that survives a convincing message: anything moving money, granting access or changing credentials gets confirmed through a second, already-known channel.
Want to assess your organization’s resilience against AI threats? Our experts will conduct a risk assessment and show you your exposure to deepfake attacks, AI-phishing, and generative malware. Contact us to schedule a conversation.
The only defence that does not depend on content
Every piece of advice built on spotting the fake fails against this class of attack, because there is nothing left to spot. The voice is right. The writing is fluent and personal. The context matches an actual project. Training people to look harder produces slower decisions and the same outcome.
What survives is a rule indifferent to how convincing the request is: anything that moves money, grants access or changes credentials gets confirmed through a second channel — a number already known, not one supplied in the message — before it is done. It costs a phone call and it is the only control that works equally against a crude phish and a synthesised call from someone whose voice you recognise. Whether the rule is real or merely written down is what a phishing simulation measures.
Related Concepts
Explore the key terms related to this article in our cybersecurity glossary:
- Deepfake — Deepfake refers to synthetic audio or video media generated by AI that impersonates real recordings or individuals.
- Phishing — Phishing is a type of social engineering attack aimed at deceiving a victim into revealing confidential information or taking a harmful action.
- BEC (Business Email Compromise) — BEC is an advanced email fraud in which an attacker impersonates a trusted individual to extort money or data.
- Social Engineering — Social engineering is a set of psychological manipulation techniques used to gain unauthorized access or information.
- Zero Trust — Zero Trust is a security model that assumes no default trust for any user, device, or network.
Learn More
Explore related articles in our knowledge base:
- Deepfake and AI in the Hands of Cybercriminals: How to Protect Your Company Against the New Generation of Fraud?
- Security Culture: How to Turn Employees into a Human Firewall Instead of the Weakest Link?
- How to Effectively Protect Your Company Against Phishing?
- Business Email Compromise — Analysis and Defense Strategy
- Agentic AI Framework: How Autonomous AI Agents Are Transforming Security Testing
Check Our Services
Do you need cybersecurity support? Explore:
- SOC as a Service — 24/7 security monitoring with response times agreed per incident priority in the contract
- Security Audits — comprehensive assessment of resilience against AI threats
- Penetration Testing — identifying vulnerabilities before attackers do
FAQ
Is deepfake audio really good enough to deceive a human?
Yes — modern voice cloning models achieve a level of realism at which even close colleagues of the victim cannot distinguish the clone from the original in a phone conversation. Research from 2024–2025 shows that people correctly identify synthetic voice in fewer than 50% of cases — meaning that intuition is not an effective defense mechanism. The only effective defense consists of verification procedures that are independent of voice assessment.
How can you tell a phishing message generated by AI from a genuine one?
It is increasingly difficult if you are looking for traditional signals (language errors, generic tone). More effective is focusing on context: did this request arrive through the normal channel? Was it announced in advance? Does it require bypassing normal procedures? Is the sender asking for something that would not normally come through email? What is suspicious is not the content — it is the situation.
Do antivirus systems protect against generative malware?
Traditional signature-based antivirus solutions have limited effectiveness against AI-generated polymorphic malware. Modern EDR platforms with behavioral and contextual analysis are significantly more effective, because they do not look for known code patterns but instead detect suspicious process behavior within the system. For organizations with elevated risk, it is necessary to go beyond the endpoint alone and deploy XDR with event correlation across network, cloud, and email sources.
Which sectors are most exposed to AI attacks?
According to 2025 data, the highest risk applies to the financial and insurance sectors (BEC and deepfake targets), the healthcare sector (valuable data, often weaker defenses), public administration (disinformation campaigns and espionage), and manufacturing companies with high IP value. In practice, however, no sector is safe — attackers select their targets based on potential gain and detected vulnerability, not on industry category.
Is implementing defenses against AI threats expensive?
Most of the key controls — SPF/DKIM/DMARC, phishing-resistant MFA, transaction verification procedures — are solutions with a low implementation cost relative to the risk they eliminate. The greatest cost is not technology but the change in organizational processes and culture. Companies that defer these investments risk losses far exceeding the cost of prevention — the average cost of a BEC incident amounts to hundreds of thousands of dollars, and in large organizations many times more.
Sources
- FBI Internet Crime Complaint Center (IC3): 2024 Internet Crime Report — BEC loss data and trends
- Europol: Cybercrime Threat Landscape Report 2025 — analysis of AI’s role in cybercrime
- IBM Security: X-Force Threat Intelligence Index 2025 — trends in malware and phishing
- CISA: AI Cybersecurity Collaboration Playbook 2025 — recommendations for organizations
- Gartner: Top Cybersecurity Trends 2025 — AI in offensive and defensive cybersecurity
- Microsoft Security: Digital Defense Report 2024 — analysis of BEC and deepfake campaigns
- HYAS Protect: BlackMamba: Using AI to Generate Polymorphic Malware (2023, updated 2024)
- Pindrop: Voice Intelligence and Security Report 2025 — synthetic voice detection
Related topics
See also:
