Researchers analysed 179 incidents to map AI memory ‘negative space’

Introduction

A joint effort by the University of Illinois Urbana‑Champaign and Northeastern University examined 179 real‑world incidents reported by 146 users of conversational AI. The goal was to understand what kinds of information people don’t want these systems to retain. From this analysis the team identified a taxonomy of 15 sensitive categories—what they call the AI’s negative space—and built a prototype called FENCE that enforces user‑defined memory rules before data ever reaches persistent AI memory.

---

Mapping the "Negative Space" of AI Memory

The incident dataset revealed that users’ privacy concerns extend far beyond traditional personally identifiable information. The 15‑category taxonomy includes items such as:

1. Personal health details
2. Financial account numbers
3. Confidential work‑related information
4. Private family matters
5. Sensitive political opinions
6. …and nine other nuanced groups identified by participants.

By structuring these concerns into a flat, human‑readable list, the researchers created a shared vocabulary that avoids the complexity of deep hierarchical taxonomies while remaining expressive enough for everyday users.

---

Proactive Governance with FENCE

FENCE operates as a policy‑level filter placed between the user’s utterance and the AI’s long‑term memory store. When a conversation contains information that matches any of the user‑selected categories, the system blocks that snippet from being written to memory. Users can still override the rule on a case‑by‑case basis, allowing personalization when they explicitly consent.

Key design principles:

• Default protection – the chosen categories are excluded unless the user opts in.
• Standing rules – policies persist across sessions, reducing the need for repeated manual deletions.
• Transparency – the system reports which categories were blocked, helping users build trust.

A within‑subject study with 15 participants showed that FENCE was perceived as more usable and trustworthy than conventional post‑hoc deletion tools. Participants reported a clearer mental model of what the AI would remember and why.

---

Why Policy‑Level Memory Governance Matters

Current AI platforms typically rely on reactive mechanisms: users discover that a piece of data was stored and then request its removal. This approach suffers from three major drawbacks:

1. Latency – the data may have already been used to influence future responses.
2. Inconsistency – providers define retention policies, which can change without user notice.
3. Cognitive load – users must track and manage many individual data points.

By shifting the focus to preventing unwanted data from entering memory, FENCE addresses these gaps. The policy layer is stable yet revisable, giving users a durable privacy posture that can evolve as their interaction patterns change.

---

Implications and Next Steps

The study highlights a growing demand for collective standards governing what AI should not remember. The authors suggest that providers should:

• Publish default negative‑space policies and allow user customization.
• Communicate any changes to these policies transparently.
• Adopt taxonomy‑based rule languages that are both interpretable and interoperable across platforms.

If industry adopts such standards, users could enjoy the benefits of personalization while retaining confidence that their most sensitive disclosures stay private.

---

Practical Takeaway

For developers and product teams building conversational agents, the FENCE model offers a concrete blueprint:

1. Define a clear, user‑friendly taxonomy of exclusion categories.
2. Implement a pre‑memory filter that enforces these categories by default.
3. Provide override mechanisms for cases where users explicitly want the data retained.
4. Expose policy decisions in the UI so users understand what is being blocked and why.

Adopting proactive, policy‑driven memory governance can turn privacy from an afterthought into a core feature, fostering trust and broader adoption of AI assistants.