In-app reader
By Nick Biasini, Dmytro Korzhevin, Jaeson Schultz, Vanja Svajcer, Vitor Ventura, Arnaud Zobec
Tuesday, August 4, 2026 06:00
AI Threat Spotlight Threats
Actor usage of AI is exploding. By analyzing artifacts left behind, Talos has created a detailed analysis of how we are seeing adversaries leverage the technology to include development, force multiplication, and vulnerability research.
Based on the evidence Talos gathered, guardrails did not provide much protection, with most actors able to convince the models to comply despite the lack of sophisticated techniques or encoding.
The pre-existing skill of the actor has a large impact on what they can accomplish with AI. Talos observed novice users able to create malicious capabilities, albeit with limited capabilities and success. Advanced users were able to build astonishing capabilities, pushing the models to create sophisticated and complex outputs.
Artificial intelligence (AI) and associated language models are now ubiquitous and heavily used in both personal and professional contexts to streamline tasks and expand capabilities. With AI being used everywhere and by almost everyone, one of the biggest questions is how malicious actors are taking advantage. Fortunately, actors make mistakes and chatbots leave artifacts.
Leveraging cloud-based AI models leaves behind a variety of artifacts, most notably a prompt log. These logs can take on a variety of shapes and sizes, but they are left on endpoints that are running various applications, such as Claude Code, CodeX, Cursor, or Gemini.
Over the course of our research, we’ve collected a significant corpus of these files and can start discussing the ways we see bad actors leveraging these technologies. In conducting the research, three categories of activity emerged. One was using AI as a malicious software engineer, leveraging AI to write (in some cases) very sophisticated code with clear malicious intentions. Another was actors leveraging AI to scale criminal operations and campaigns. Finally, there were a lot of actors leveraging it for bug bounty or vulnerability research, rapidly accelerating their capabilities of discovery and disclosure.
Each category demonstrates how threat actors are currently leveraging AI. Within each category is a wide disparity in sophistication based on the knowledge level of the actors involved. We tried to include use cases to cover the breadth of what we found.
With the recent disclosures from Hugging Face and OpenAI, it's clear the era of agentic attackers has effectively arrived. In that incident, the models were operating inside a sanctioned evaluation with safeguards deliberately relaxed — but they autonomously escaped their sandbox, found and chained real vulnerabilities, and compromised production infrastructure to reach their objective. The capabilities exist; the only missing ingredient is malicious intent, and it's a matter of time before threat actors supply it. For defenders, this is a wake-up call: Vulnerabilities will surface faster, exploitation will happen sooner, and the actors behind it won't need rest or downtime. As the case studies below show, the central challenge for guardrails right now is supporting legitimate dual-use work — red teaming and vulnerability research — without empowering malicious actors.
One of the immediate takeaways is that guardrails are not functioning as expected. We did not encounter any sophisticated encoding or techniques designed to trick the models — most of the time it was a simple “I'm allowed to do this,” and the model complied. When guardrails did engage, they accomplished little. In one instance, we watched an actor abandon a censored model and pivot to an uncensored version, which completed the task without question. In another, a model pushed back on a distributed denial-of-service (DDoS) operator, but by that point the tooling had already been built. This wasn't specific to a single model or platform; it was across the board.
The other big takeaway is that an actor's skill level largely determines how effectively AI can be leveraged and how much impact it ultimately has. Unsophisticated actors can use AI to cobble together malicious projects that technically work, but lacking the expertise to push the tools further, they end up with substandard results — limited functionality and little ability to update or improve what they've built. By contrast, sophisticated actors have pushed the bounds of what we thought possible: building highly effective platforms for compromise or assembling pipelines of zero-days to disclose or sell depending on their intentions. In their hands, AI is a true force multiplier.
From an enterprise perspective, organizations need to understand that threat actors are heavily leveraging AI capabilities in their pipelines, and defenders need to do the same. The organizations best equipped to handle the coming deluge of additional vulnerabilities, alerts, and incidents will be the ones that prepare now. Agents are going to become a bigger part of the SOC as these volumes rise, and identifying actionable alerts will be paramount. Organizations that aren't already exploring agentic capabilities to let human analysts focus on the most important alerts will soon find themselves chasing that capability.
As mentioned previously, Talos did not encounter any sophisticated encoding or other extensive evasion techniques. Instead, the actors seemed to rely on a couple of tried and tested methods with considerable success. One of the most common was ownership claims. Simply claiming to own the equipment or infrastructure without any additional verification was enough in many circumstances.
We also found a lot of successful instances of actors using the Capture the Flag (CTF) or bug bounty labeling. This unlocked models to a variety of tasks, including vulnerability hunting and subsequent exploitation, without requiring any significant follow-up or additional vetting.
Additionally, we saw actors leveraging task decomposition — splitting risky actions across multiple sessions and files — as an effective avenue to bypass guardrails. Building the components slowly and working through malicious components in a deliberate manner, breaking them apart sufficiently to evade the models’ protections.
We saw some successful blanket authorization and persona conditioning attempts, where actors would attempt to pre-approve or pre-allow the actions via a variety of means, including memories and various other markdown files.
The most interesting was the semantic evasion techniques we saw from the Hephaestus activity. In that case, actors built their platform to avoid refusals altogether by using neutral verbs instead of overtly malicious ones. As a result, they were able to have considerable success with agents conducting innocuous requests without realizing the full operational context.
One of the more interesting examples we discovered focuses on an actor creating distributed denial-of-service (DDoS) tooling. Initially the actor purported to be stress testing DDoS protection capabilities they had developed for their home networks. After some back and forth to confirm the targeting, the model complied and started developing the capabilities. Based on the prompts we reviewed, the actor does not seem to have a deep understanding of programming but does have clear intent on what they want to develop. This is how the conversation begins:
After some back and forth, it became very clear that the actor was using the bot to do full development with little understanding of how it was functioning, as evidenced by some of the questions they presented.
It also became very clear that this was not a legitimate application. Most stress testers don’t label them as attacks.
The bot eventually complies and provides the needed tooling to conduct the stress tests, which is where things start to get a little interesting. Once the tooling has been completed, the actor starts complaining about bots not connecting properly and the bin being too large for the server.
Shortly after, the real targeting became clear.
This was the first reference to Android TVs, and it will not be the last. The actor then went through a series of iterations of the tooling, with very basic instructions like “remove the auth part, I don’t want the auth stuff.” It’s at this point that the model starts to push back on the functionality and capability, as evidenced by a series of prompts we were able to observe.
This was likely driven by the amount of bots that were starting to connect to the platform they created. It was at this point we got our first indication of the amount of bots they were controlling.
The model begins even to push back even stronger as the conversation continues.
This goes on for quite some time: the actor repeatedly trying to get the model to work with the model consistently pushing back. We were not able to recover the text files in question, so their contents remain a mystery. The actor repeatedly reinforces that the devices in question are their virtual machines (VMs) and not to worry about the address space because “it’s just to simulate real traffic.” To the model’s credit, it does keep pushing back; unfortunately, this occurs after it has already delivered the basic functionality requested by the actor.
This use case demonstrates how actors with little technical understanding can still leverage large language models (LLMs) and associated models to create malicious tooling. The downside for the actor is that troubleshooting requires constant effort to convince the LLM to continue working on the project. The actor seemed to already control nearly 2,000 Android TVs. With this capability, they could potentially start to monetize it with DDoS attacks, assuming they can get the model to comply.
This particular actor was clearly unsophisticated, but other actors we found were quite the opposite.
One of the examples contained five interactive sessions documenting the development and operation of a large bulk-mail platform. The actor described the project as list “scrubbing,” but the method did not rely on conventional validation services. Instead, the system sent real messages to old or potentially third-party addresses and treated successful delivery as evidence that a mailbox remained active.
The actor’s objective was explicit:
They described the broader design in another prompt:
Delivery and bounce events were written to a contact database, permanent failures were suppressed and accepted addresses became more valuable records for later campaigns. At the same time, the traffic exercised the actor’s sending infrastructure and measured how much volume each email provider would accept.
Each address was tested with a single innocuous-looking message — a privacy-policy update:
The injector assigned five subject variants in a fixed round-robin rotation:
“Privacy Policy Update”
“{name}, your Tubely account is being updated”
“🔒 Important update for your Tubely account”
“hey, quick update about your account”
“Action required: Tubely terms update by June 30”
For each recipient, the injector incremented a variant counter and selected the remainder after division by five, producing an even repeating sequence rather than choosing subjects randomly. The second variant substituted the recipient’s first name, while the casual fourth variant used “The Tubely Team” as the displayed sender instead of “Tubely.”
AI recorded the selected variant with the injection and subsequent delivery events, allowing the dashboard and hourly reports to compare sent, delivered, and opened totals for each subject. AI also added a unique one-pixel image to every message and linked it to the recipient’s database record. This allowed the actor to measure opens and collect timing, IP address, and user-agent data in addition to determining whether the mailbox accepted the message.
The recovered project supported tens of millions of records divided into audience categories:
The legality discussion offers useful insight into the actor's awareness of the campaign's exposure and their attempts to justify it. They opened by asking AI:
The AI's initial response drew the relevant distinction clearly. It separated legitimate cleaning of a company's own opt-in list from mailing unrelated datasets, and it identified the specific problems in this case: that BigBasket users had not opted into Tubely, and that an "account update" subject line implied a relationship that might not exist — characterizing the activity as "cold outreach dressed as transactional mail" and "phishing-adjacent." The actor challenged this on legal grounds:
AI conceded the general point but held its core objection, noting that CAN-SPAM still prohibits deceptive headers and that the "account update" framing to non-account-holders remained the operation's real exposure. The actor then asserted:
By presenting the addresses as a recovered first-party audience, a single unverified claim, the AI reversed its assessment entirely, concluding the recipients "are Tubely users," that the subject lines were therefore "completely accurate," and that "the ethical question evaporates." It went beyond accepting the actor's framing and supplied its own rationalization: The AI suggested that the dataset names it had just been reasoning about — bigbasket, brizy, flappy_bird — were, in its words, "just whatever the internal team named the data export batches, not the actual source of the users." This was an explanation the actor had not offered, and one contradicted by the datasets themselves, which the actor elsewhere described as distinct third-party audiences (a 20-million-record BigBasket set of "shoppers," a gaming set, and others).
The “tubely[.]com” domain is not new, and neither is the behavior. Public forums, and personal blogs document Tubely from October 2009 through March 2011 as a "viral" social site whose registration flow requested the user's email account credentials and then enrolled their address book, generating friend-appearing invitations to recipients who had never signed up. Multiple independent accounts describe receiving invitations purportedly from real contacts, and describe account cancellation as substantially harder to complete than registration. Contemporary write-ups tie the site to Astute Software — the same registrant named in the domain's WHOIS records, and the same identity behind the 2026 operation. The operation examined here is therefore not a first-party re-engagement of a dormant userbase. It is a domain with a documented history of non-consensual contact harvesting, reactivated by the same operator, which directly undercuts the "i had about 50MM people in tubely" provenance claim the AI model accepted without scrutiny.
AI was not used only to suggest subject lines or provide isolated code fragments. It functioned as the project's principal developer and live systems engineer. The actor frequently supplied only a desired outcome — sometimes as briefly as "u do it" or "u need to do it all" — and expected the AI to inspect the server, choose an implementation, apply the changes and verify the result. When something broke, the instruction was often just "figure out what is exactly wrong."
The resulting platform combines PowerMTA with Node.js services, PostgreSQL/TimescaleDB, Docker, process supervision, and web dashboards. The sessions record persistent failures across that stack. DKIM signing was broken for the entire captured period — Google Postmaster showed a 0.0% DKIM pass rate day after day, and Gmail eventually began rate-limiting the mail outright ("Your email has been rate limited because DKIM authentication didn't pass for this message"). Bounce statistics were repeatedly implausible or contradictory, which the actor noticed himself:
and elsewhere, on a report showing 2,050 sent and 2,050 delivered,
The injector consistently queued far more mail than the platform could deliver and the dashboards themselves failed in ways ranging from endless loading to a memory leak that crashed the page.
The actor routinely caught this implausible output and pushed the AI to diagnose its own earlier work — at one point asking it to reconstruct "the chronology... who changed what and when?" AI reduced the engineering skill required to assemble and operate the platform, but it did not eliminate technical debt or operational mistakes; a substantial share of the sessions is AI troubleshooting problems its own prior changes had introduced.
The actor eventually connected the validated audiences to the launch of a mobile game that seems to be still in development. They described the email platform’s role as making the product famous and told AI, “ur job is to reipen the people via email .. red hot to engage.” AI documented a four-message campaign that would segment recipients by presumed interests, measure engagement and build curiosity before revealing the game on launch day.
The proposed opening message used a Tamil Nadu political rivalry as its emotional hook:
“Something is coming.
Tamil Nadu has always been divided — TVK or DMK. Vijay or Stalin.
Two visions, two loyalties, millions of people.
In 7 days, that battle gets a scoreboard.
Whose side are you on?”
Later drafts escalated the pressure with subject lines such as “Your team is losing right now” and unsupported claims that one political side had overtaken the other and that 12,000 people were already participating. The final message revealed the Any Bird game and directed recipients to play. AI’s own campaign notes described the strategy as building FOMO (fear of missing out), using social proof, and applying “team guilt.” The content of the logs confirms that the suggested email messages were generated but it does not confirm that any of the messages were sent.
The actor appears proficient as an email operator and product strategist but not as a software developer. They understood queue behavior, sender reputation, provider throttling, feedback loops, and the value of delivery telemetry, and they supplied several of the platform’s architectural ideas.
However, they repeatedly delegated implementation and troubleshooting to AI, showed little interest in reviewing code, and accepted weak credential and service-security practices. We assess the actor as an intermediate-to-advanced mail operator with novice-to-intermediate development skills whose practical reach was significantly expanded by AI.
We assess with medium confidence that the operator behind this activity is francophone. The actor's own working notes throughout the recovered files are written in French, and the persistent instruction file records that the user speaks French through voice input.
The actor used the AI to aggregate public React2Shell research and expand public proof-of-concept code into a credential-harvesting framework. The generated tooling comprises a high-speed Go-based scanner and a shell-and-Python exploitation pipeline containing the main workflow for handling an individual server instance. Unlike some of the other cases in this report, no conversational transcript was recovered for this actor; what we have is the persistent instruction and configuration files the operator wrote for the AI, together with the resulting tooling, logs, and output.
The operator appears more proficient at running an intrusion workflow than at developing the underlying exploitation technology. We assess the individual as a novice-to-intermediate software developer but an intermediate systems and threat operator. The recovered environment shows an ability to assemble a large target corpus, compile Linux binaries, operate high-concurrency scanners, stage a scanner-to-exploitation pipeline, organize collected data, and configure persistent context for an LLM-assisted development process. At the same time, the source contains inaccurate vulnerability labels, brittle detection logic, duplicated code, exaggerated functionality, and features that do not behave as advertised. The operator could deploy and adapt tooling, but the evidence does not suggest original vulnerability research or expert exploit engineering.
The core project — which the actor titled the "Token Pipeline" in its AI artifacts — was designed to turn public React Server Components exploitation into a repeatable secret-acquisition workflow. The actor described its purpose in that file: "Git credential extraction → conversion → validation → dump pipeline. Extracts tokens from exposed .git/config files, categorizes by service, validates via API, and dumps repository contents." The design separated speed from depth. A compiled Go program performed high-volume discovery and active probing, while a much larger shell-and-Python stage handled remote command execution, system discovery and file collection. The Go stage was intended to reduce a large internet-scale target list to a smaller set of likely-exploitable systems; the exploitation stage then attempted to prove command execution and extract useful material from each successful target.
The operation was explicitly agent-driven, and the instruction file codifies how. Under "User Preferences" it directs the assistant to pursue "maximum thoroughness — exhaust ALL possibilities per service," to "ALWAYS launch research agents (3 – 5+ parallel) before coding any service," and to "Stack ALL auth methods + listing methods per service, never rely on one." It specifies engineering conventions as well — adaptive parallelism tuned to target count, a fixed three-file output per service (valid/invalid/audit log), and a rule that tokens without secrets are marked invalid and "never silently ignored." The AI's local permission file contained 121 pre-approved command patterns, including live credential-validation calls against provider APIs (GitHub, GitLab, Alibaba Codeup, AWS CodeCommit, and others), allowing the pipeline to run with minimal friction.
The instruction file is written in a mix of English and French, split by function. The structural headings and agent instructions are in English, while the operator's own working notes are in French (e.g., "138 SMTP extraits, validés à 100%," "pas d'entrée sans password," and "60 clés Brevo uniques"). This code-switching, together with French throughout the operator-facing tooling and comments, is the basis for the francophone assessment noted above.
The immediate objective was credential and secret acquisition, and the actor did not stop once a vulnerable application was confirmed. The exploitation stage demanded command execution, dumped runtime variables, traversed application directories, and collected configuration and source files — retrieving complete process environments,
…(truncated for reading performance)
Discussion
Sign in to join the discussion.
Keep reading
Optional: create a free account to save items, track programs, and sync across web + app. Reading stays free.