In this episode of Siren Investigates, Siren CEO John Randles sits down with Dr. Sae Schatz, founder and CEO of The Knowledge Forge, to explore what happens when artificial intelligence, information and human cognition collide.
A former director of the US Department of Defense Advanced Distributed Learning Initiative, Sae has spent much of her career studying the intersection of digital technology and human cognition, including how people learn, process information and make decisions.
Their conversation explores the rise of cognitive warfare, AI-enabled scams and cyber threats, the impact of information overload on decision-making, the challenge of establishing truth in an era of deepfakes and synthetic media, and why trusted information could become increasingly important for investigators, organisations and society.
What is cognitive warfare and how is AI changing it?
Information has always played a role in conflict. Propaganda, deception and attempts to influence populations existed long before artificial intelligence.
What has changed is the sophistication and scale at which information can now be combined with technology and our understanding of human behaviour.
Sae describes cognitive warfare as the use of information alongside areas such as artificial intelligence, cognitive science, neuroscience and social science to influence how people perceive information, direct their attention and make decisions.
AI can dramatically increase the scale of these activities.
Content can be generated quickly, bots can amplify particular narratives and messages can increasingly be personalised for specific audiences or even individuals.
Understanding human cognition can make those techniques more effective.
Repeated exposure to information, for example, can make people more likely to perceive something as true. Similarly, overwhelming people with competing explanations can consume attention and make it more difficult to determine what actually happened.
For investigators and intelligence professionals, the challenge is therefore no longer simply identifying whether an individual piece of information is true or false.
They increasingly need to understand the wider information environment: where information originated, how it spread, who amplified it and whether attempts are being made to deliberately influence the people receiving it.
How are AI and information overload affecting human decision-making?
Humans have a limited capacity for attention.
Every decision, notification, system and piece of information competes for part of that capacity. As people become overloaded, their ability to carefully evaluate information can begin to decline.
Sae explains that this matters because modern professionals are operating in an environment where information is being generated at extraordinary speed.
Generative AI adds another layer.
It can transform large datasets into reports, summaries and other outputs almost instantly. But producing more information doesn’t necessarily make it easier to reach better conclusions.
Low-quality AI-generated work can actually transfer the burden from the person creating information to the person receiving it. The recipient then has to determine whether the material is accurate, relevant or trustworthy.
For investigators, that challenge can become particularly significant.
An analyst may already be working across numerous systems while receiving alerts, messages and new intelligence. Adding more information without helping them identify what actually matters can increase cognitive load rather than reduce it.
Sae argues that the emerging challenge is therefore not simply access to information, but trusted wayfinding: helping people navigate enormous quantities of information while preserving their attention for the decisions that matter.
How can investigators identify truth in an age of deepfakes and AI-generated content?
Generative AI has made convincing synthetic content significantly easier and cheaper to produce.
Deepfake video and audio, AI-generated identities and synthetic documents can all be used in fraud, scams and information operations.
But Sae argues that the problem extends beyond identifying individual deepfakes.
The growing volume of synthetic media can begin to affect people’s overall perception of truth.
When almost any image, recording or document could potentially have been generated or manipulated, people may become less willing to trust genuine information as well.
That creates a second problem: authentic evidence can potentially be dismissed simply by claiming that it was artificially generated.
For investigators, provenance therefore becomes increasingly important.
Rather than evaluating a piece of information solely by how convincing it appears, investigators need ways to understand where it came from, how it has been handled and whether its origins can be verified.
Sae discusses technologies such as verifiable credentials and distributed identifiers, which could allow information and assertions to be digitally authenticated and their provenance tracked.
The longer-term objective is what she describes as a trust ecosystem: an environment where people can better understand the source and lineage of information rather than relying solely on appearances.
How can organisations build cognitive resilience against AI-enabled threats?
Defending against AI-enabled threats isn’t solely a technology problem.
Sae argues that organisations also need to protect the cognitive resources of the people responsible for making important decisions.
The first step is awareness.
People need to understand that attention and decision-making capacity are finite. Information overload, constant connectivity and repeated decision-making can reduce a person’s ability to carefully assess what is in front of them.
Organisations can then develop processes that reduce unnecessary cognitive load.
That can mean simplifying workflows, reducing the number of systems employees need to navigate and prioritising the information that genuinely requires attention.
Sae also highlights the importance of creating mental white space: periods where people can step away from continuous streams of information and allow their cognitive resources to recover.
Verification processes are another important part of resilience.
As impersonation and synthetic media become more sophisticated, organisations may need stronger methods for confirming identities, communications and the origins of information.
Ultimately, cognitive resilience means recognising that protecting an organisation isn’t only about securing networks and data.
It also means protecting the ability of people to pay attention, assess information critically and make sound decisions when the information environment itself may be designed to overwhelm or influence them.
Key takeaways
- Cognitive warfare combines information with technology and behavioural science to influence perception, attention and decision-making.
- Generative AI can increase the scale and sophistication of scams, cyber threats and information operations.
- Human attention is finite, making information overload an important consideration for investigators and intelligence analysts.
- Deepfakes and synthetic media create challenges not only for identifying false information, but also for maintaining confidence in genuine information.
- Provenance and verifiable information could become increasingly important as organisations attempt to establish trust in digital content.
- Cognitive resilience requires awareness, verification processes, simpler workflows and protection of people’s limited attentional resources.
- As information becomes easier to generate, trusted wayfinding may become more valuable than simply providing people with more data.
Watch highlights from the episode
What is the Illusory Truth Effect? | Dr Sae Schatz
Will AI Make Us Better Decision Makers?
What is “AI Work Slop”? | Dr. Sae Schatz
How does Generative AI Change the Way Humans Acquire Knowledge? | Dr Sae Schatz