From Privacy by Design to Autonomy by Design

The Missing Control Layer in AI Safety.

Introduction: The red pill of Compliance in AI Governance

In my last post, I talked about the dangerous trend we are watching emerge:

• AI that shapes human cognition in such subtle ways that most users don’t realise, with recent studies showing a decrease on cognitive capacity. And;

• The decadence of Human-on-the-loop safeguards, as we’ve proven ourselves terrible a overriding AI recommedations.

I also introduced the very (very!) basic premise of the aligment problem and how, as critical its research is for AI Safety as a whole, solving it wouldn’t be enough to guarantee that human autonomy is preserved.

But there is something I want to stress here:

It is not that the same people who are researching alignment, don’t recognise the urgency or “don’t care” about implementing control safeguards for Autonomy.

The cold truth is:

They’re already doing way more than most people in Privacy & compliance are even aware of. It’s up to US to start holding our end, from the other side of AI Safety.

As a recap…

AI Safety is broadly divided in two pillars:

1. Alignment: Ensuring AI wants to do what’s best for humans. This mostly concerns ML engineering, with AI companies dedicating entire teams to alignment work.

2. Control: Ensuring humans can regulate and override AI. This is where regulators and policy-makers focus, and where most Compliance and Privacy Professionals sit.

But Privacy Engineering can serve as a bridge between Data Protection / Privacy compliance and Privacy by Design that is smart enough to be applied to LLMs’ interactions with end users, and reduce contestability constrains in traditional ML-based ADM systems.

Here is what keeps me up at night:

• Regulation is contributing to Compliance Theatre, with risk classifications that treat AI as we would a “static” product, and not a rapidly evolving entity.

• Many Privacy Professionals still demonize AI, critizising from the sidelines, instead of using it to stress-test what the end-user experience looks like from a privacy perspective.

• Regulatory / compliance- focused AI Governance is too busy trying to find reactive solutions to avoid liability or fines.

We need to get out of this reactive loop, FAST.

We are not doing enough to mirror the progress of Control to the progress of Alignment. Some may argue that we’re only slowing it down, and it comes down to admitting that: we can’t control what we don’t even try to understand.

Let alone, “govern” it.

Why Autonomy by Design is the Logical Next Step

For years, Privacy by Design (PbD) has been the cornerstone of responsible data governance, ensuring that privacy safeguards are embedded into systems rather than retrofitted as an afterthought. But as AI systems evolve beyond simple data processing into autonomous decision-making agents, PbD alone is not enough to uphold what Art.22 of the GDPR promised.

If we really want to prevent the erosion of human autonomy, we need to build systems that respect it by design.

Autonomy by Design (AbD) should become a governance framework that ensures users retain meaningful control over AI-driven inferences, decisions, and behavioral steering. While PbD focuses on protecting personal data, AbD extends this principle to protecting human agency in AI interactions.

For privacy professionals working in AI governance, integrating Autonomy by Design will become a practical necessity. But…

• how do we actually implement it? Is it even feasible?

• how do we avoid the biggest governance pitfalls, such as automated decision-making opacity and consent fatigue? And,

• why would AI deployers accept this, when they’re barely interested in Pbd as it is?

Bridging Privacy by Design & Autonomy by Design: A Shared Foundation

Privacy by Design focuses on minimizing data collection, giving users control over their data (access rights), and wnsuring secure data processing (encryption, pseudonymization).

Autonomy by Design extends this to AI by:

• Minimizing behavioral profiling & AI-driven nudging (cognitive transparency)

• Giving users control over AI inferences (real-time visibility & override mechanisms).

• Ensuring AI decisions remain contestable (explainability & redress mechanisms)

Privacy controls protect what data AI uses, but autonomy safeguards ensure how AI uses it to shape decision-making and cognition. Both must work together for AI governance to be meaningful in the near future.

The Core Pillars of an Implementable Autonomy by Design Framework

1. Visible AI Profiles: User Access to AI Inferences

AI systems generate inferences about users based on their interactions, often without explicit consent or awareness. These inferences shape content recommendations, search results, hiring decisions, financial assessments, and other high-impact areas.

Providing transparency into these inferences is necessary to ensure users retain control over how AI systems interpret and act upon their data, but also to prevent said inferences to shape users’ decision-making without their knowledge.

Implementation Strategy

• AI Profile Dashboard: A user-facing interface that allows individuals to view how AI categorizes and interprets their behavior. This dashboard should include:

  • Topic Preferences: Example: “You engage most with content on AI governance and privacy” (I would have never guessed!).

  • Behavioral Tendencies: Example: “Your responses suggest a preference for analytical over emotional reasoning.”

  • Content Ranking Factors: Example: “This recommendation was based on past interactions, similar users, and predicted interests.”

• Editable Inference Controls: Users should have the ability to modify or delete AI-generated inferences that misrepresent them.

  • Example: A hiring AI system infers that a candidate lacks leadership skills based on similar profiles. The candidate should be able to contest this inference (for example, providing evidence to the contrary) before it influences hiring outcomes.

Addressing Feasibility and UX Challenges: Prioritizing AI Inferences for User Transparency

• Not all AI inferences require user intervention, just as not all website cookies require explicit consent. Transparency must be designed to avoid cognitive overload while ensuring critical decision-making remains in human hands.

• AI-driven inferences can be categorized by their impact on autonomy, similar to how cookies are classified in privacy regulations:

  • Low-impact inferences (e.g., content recommendations on Spotify or Netflix) function like necessary cookies, enhancing user experience without significantly altering decision-making.

  • High-impact inferences (e.g., behavioral profiling for hiring, credit scoring, or political content curation) act more like third-party tracking cookies, shaping outcomes and influencing choices without user awareness.

• Regulatory precedent already exists for this type of distinction. Just as privacy laws enforce transparency around third-party tracking, AI governance should require that high-stakes inferences (those that materially affect life opportunities, decision-making, or cognitive autonomy) be surfaced to the user in a structured and manageable way.

Addressing Challenges in Automated Decision-Making

• Inference Tracking and Version Control: AI models must maintain logs that document when and how inferences change due to model updates or retraining. This would require implementing version-controlled inference logs that track when a new category of inference is introduced.

• Notification of Major Inference Changes: Users should not be notified of every minor adjustment in their AI profile. Instead, notifications should only be triggered when the AI introduces a new category of inference that may affect outcomes significantly. I think Art.22 remains a good benchmark for this: inferences that produce legal effects on (or similarly affects) the user.

2. Real-Time Consent for New Inferences

AI models continuously refine their understanding of users, leading to evolving inferences that may significantly impact decision-making processes.

Ensuring users are notified of meaningful changes before these inferences influence decisions is a critical safeguard to preserve both their privacy and their autonomy.

Implementation Strategy

• Tiered Consent Mechanisms: Currently being applied specially in the Health research and Banking industries. To prevent excessive interruptions, consent mechanisms should be structured based on the impact of the inference:

  • Passive Mode (Default Setting): AI records new inferences but does not interrupt the user. Instead, users can review these updates in a periodic digest (e.g., “Here’s how AI has adjusted its perception of you this month.”).

  • Active Mode (For High-Impact Inferences): If an AI system generates an inference that alters trust levels, creditworthiness, hiring suitability, or other high-stakes decisions, users should receive a notification with the option to approve or contest the change. Example: “Your trust score in this system has changed. Would you like to review why?”.

Addressing the Challenge of Consent Fatigue

• Batching and Summarizing updates: Instead of prompting users for approval with every minor inference change, AI systems should present summaries at regular intervals (e.g., weekly or monthly reports).

• User-Customized Notification Preferences: Users should be able to specify which types of inference updates require their attention (e.g., only finance-related, political profiling, or health insights).

• Context-Aware Transparency Labels: Instead of requiring users to approve every decision AI makes, the system should display clear explanations at the moment it influences an outcome. Example: “This loan offer is based on your spending behavior over the last six months.”

• Traditional Automated-decision-making (ADM) systems are a pain for contestability.

  • While inference tracking is actually easier with ADM than with LLMs, they don't typically provide contability in real time.

  • Hence, users can challenge decisions after the fact, but often through bureaucratic, slow, or opaque processes (e.g., filing an appeal, requesting a review).

  • Possible fix? Instead of trying to make traditional ADM models more explainable (which is hard), a more feasible solution may be to integrate LLMs as an interface layer.

  • Instead of raw ADM output, give users an LLM-driven explanation of how the decision was made.

  I will also address the different approach needed in AbD when dealing with traditional ADM and in the context of LLMs.

The Technical Foundation: PETs & a Minimum Viable Product for Autonomy

This post provides a first-instance overview of how Privacy and AI Governance teams can integrate Autonomy by Design into their frameworks. However, said practical implementation requires a deeper dive into Privacy-Enhancing Technologies (PETs) and the minimum technical safeguards necessary to make autonomy protections such as the AI profile and Inference notifications functional & UX-friendly.

In my next post, I will outline what a Minimum Viable Product (MVP) for Autonomy by Design should include: The simplest yet effective safeguards that AI systems must implement to ensure users retain control over AI-driven inferences and decision-making. This will cover:

• How AI profiling transparency can be technically achieved, making inferences visible and user-accessible.

• The PETs required to support inference tracking, notifications, and overrides: such as differential privacy, zero-knowledge proofs, and explainability models.

• How we can mirror the approach of alignment researchers, who have defined an MVP for alignment: a minimally functional, continuously improving framework to ensure AI systems behave safely.

What does it actually take to make AI’s decision-making accountable at scale?

What technical safeguards can we push for today, before AI governance standards are set without us?

Traditional ML-based ADM and Modern LLMs pose very different challenges in terms of inference tracking, explainability and contestability. How do we overcome this?

The next post will address these questions, laying out the first functional blueprint for Autonomy by Design: not only for privacy compliance teams, but also for AI developers.

3. The Downsides of Autonomy by Design: The Harsh Reality

While designing Autonomy-first systems will become critical for ensuring human oversight and decision-making power, there are significant barriers to implementation that cannot be ignored.

And here is where I am asking all of you to contest, question and refine this research.

Headache #1: Autonomy by Design Cannot Be Applied Retroactively

AI systems do not "unlearn" the way humans do, even if we give users autonomy now, their past interactions with AI models have already shaped their profile in ways that cannot be fully erased.

AI inference models store patterns rather than direct data, meaning past assumptions cannot always be undone even if an AI profile is reset. The best we can do is allow future inferences to be reset and controlled, but the historical record of how AI learned remains.

However, a soft reset could disregard past inferences and start fresh, and profile customization could enable users to selectively modify or delete AI-generated inferences. Still, this would not erase prior AI learning.

Headache #2: Companies with the Resources to Implement AbD Lack the Motivation

OpenAI, Microsoft, and Google DeepMind have the technical capability to embed autonomy-first design into AI, but doing so would:

• Increase operational complexity and compute costs

• Reduce AI-driven engagement and monetization

• Introduce user friction, which contradicts business incentives

However, as demand for AI transparency grows, AbD could shift from an operational burden to a competitive advantage, just like we’ve seen with PbD. Companies that proactively implement autonomy safeguards may gain early compliance advantages and attract enterprise and government clients in regulated industries (“your burden becomes my burden” kind of logic, just like with GDPR compliance).

AI providers offering inference visibility, override mechanisms, and transparency tools could differentiate themselves in markets where explainability is a strategic priority, such as healthcare, finance, and legal AI applications.

Headache #3: But really,the Only Way Autonomy by Design Gets Implemented? Regulation.

As fancy as I’ve just made the competitive advantage angle sound, major AI companies will not voluntarily add autonomy safeguards unless they are mandated as part of regulatory compliance.

The challenge is that current AI regulations do not explicitly require autonomy safeguards: they focus primarily on safety, transparency, and privacy.

If autonomy governance does not become a compliance requirement, most AI providers will opt for alignment safeguards that prioritize corporate objectives over user control.

But, even if AbD becomes part of the AI regulatory compliance: how do we make sure that legislators get this right? Or at least, with less gray areas than in the AI Act?

Conclusion: Implementing Autonomy by Design in AI Governance is OUR end of AI Safety

Privacy governance was never just about compliance, it was about reclaiming control over data. Now, the fight extends beyond privacy into how AI interprets, influences, and decides for us.

Autonomy by Design is the next necessary evolution, ensuring that AI serves human agency rather than subtly eroding it. The transition is already underway, but the question remains: who will define its standards?

Privacy professionals and AI governance experts, or AI companies optimizing for engagement, persuasion, and cognitive influence?

If privacy professionals do not take the lead, Autonomy will be shaped by the same forces that turned privacy regulation into compliance theater.

Autonomy is as much a legal and regulatory issue as it is an AI engineering challenge. But we must start holding our end.

But we are already positioned in the right rooms: sitting in AI governance boards, shaping Privacy by Design strategies, collaborating with privacy engineers, front-end developers, and UX teams. Now, we need to leverage this position to ensure that autonomy is not another casualty of AI-driven optimization.

Where Privacy Professionals Must Lead the Shift

1-Integrating AI Profiling into Governance Audits
”Privacy audits” on AI systems must assess how AI constructs inferred user profiles and whether those profiles can be challenged or controlled. AI governance must expand to address:

• Does this system allow user overrides?

• Are inference changes trackable?

• Is there transparency in automated decision-making?

Without oversight of AI-driven profiling, privacy compliance will be reduced to a formality while AI systems continue optimizing behavioral influence unchecked.

2- Push for Explainability & User Control in AI Product Design
AI systems cannot function as opaque black boxes. We must advocate for AI Profile Dashboards: interfaces where users can view, challenge, and reset AI inferences about them. Explainability is meaningless unless it is delivered at the user level.

3- Establish Redress Mechanisms for AI-Driven Decisions
In high-stakes applications like hiring, finance, and healthcare, autonomy safeguards must go beyond visibility. AI-driven decisions must be:

• Contestable: Users must have the ability to challenge AI-driven outcomes.

• Overridable: AI models must provide override mechanisms, not just passive explanations.

• Accountable: AI must disclose the reasoning behind high-impact inferences that have legal consequences for the users.

4- Mind Consent Fatigue without sacrificing Control
The failure of Privacy by Design was assuming users would enthusiastically navigate consent panels. The same mistake cannot be repeated. Autonomy by Design must implement tiered transparency, ensuring:

• Minimal friction (“just in time” notices): Users are not interrupted with constant approval requests. But AI must surface inference updates when they impact user experience or decision-making.

• Long-term autonomy: AI-driven personalization must never evolve unchecked in ways that quietly restrict user choice.

Final Consideration

Autonomy by Design may very well become the last defense against AI turning into an unchecked force in shaping human cognition.

If privacy professionals do not take the lead, preservation of human autonomy will be defined by the companies least incentivized to implement it.

We can choose to act now, we can be brave enough to acknowledge that legal mechanisms and compliance checks are not enough to navigate Privacy by Design applied to AI systems… let alone, to help preserve human autonomy.

We can choose the red pill, start learning about PETs and frameworks to help us uphold what Art.22 GDPR promised… or we can choose the blue pill and pretend like compliance audits & risk assessments will be enough to safeguard human autonomy (and, by default, real privacy).

AI alignment is only half the battle. The other half is ensuring humans retain the ability to disagree, override, and challenge AI’s reasoning: before autonomy quietly fades into optimization.

---

References for the main ideas behind “Autonomy By Design”.

1- The paper that truly captures the essence of the Autonomy problem: Catena, E., Tummolini, L., & Santucci, V. G. (2025). Human autonomy with AI in the loop. Philosophical Psychology, 1–28.

2- Why we can’t just expect users “not to engage” in ways that put their Autonomy in danger (just as how we can’t expect users not to feed raw data): Conversational User Interfaces: Explanations and Interactivity Positively Influence Advice Taking From Generative Artificial Intelligence. By Tobias R. Rebholz, Alena Koop, and Mandy Hütter

3- From a technical perspective, several Restack.io pages on how human autonomy is actually essential for the correct functionality of LLMs.

4- How Privacy Enharcing Technologies are already being applied to mitigate privacy challenges posed by AI systems: A Survey and Guideline on Privacy Enhancing Technologies for Collaborative Machine Learning (2022) Elif Ustundag Soykan; Leyli Karacay; Ferhat Karakoc; Emrah Tomur

5- The recent Bridge Privacy Summit held by Privado last week (5th and 6th February) challenged a lot of my previous assumptions. Please see this workshop in particular: Scaling Privacy Reviews with LLMs: Challenges and Breakthroughs.