The Man Who Never Learns Why Doors Stay Closed

This article is about what happens when AI systems draw conclusions about people, and no one tells the people what those conclusions are.

Consider a man, call him David, who applies for a job. He is qualified, experienced, and well-prepared. He does not get an interview. He applies for another. The same result. Over months, he submits dozens of applications. Some receive automated rejections within hours. Others simply disappear into silence. He never learns why.

What David does not know is that an AI screening system, used by most of the companies he applied to, has drawn an inference about him. Perhaps it flagged a gap in his employment history. Perhaps it scored his resume lower because his degree is from an institution the model has learned to associate with lower performance, an association built from historical hiring patterns that reflect biases the model absorbed during training. Perhaps it concluded, from patterns David cannot see and the company cannot fully explain, that he is a higher-risk hire than other candidates.

The inference was drawn. It was acted upon. David was never informed of its existence.

This is not a data problem. David’s data (his resume, his application, his credentials) is accurate. The problem is what the system concluded from that data, combined with patterns drawn from millions of other people’s data. The conclusion is new information about David that David never provided, that no human being may have reviewed, and that David has no mechanism to see, contest, or correct.

This is the inference gap operating at the individual level. And it operates everywhere, not just in hiring.

In insurance, an AI system draws conclusions about a person’s risk profile — from health data, spending patterns, geographic location, or behavioral signals — and adjusts premiums or denies coverage based on inferences the person cannot access.

In lending, an AI system generates a creditworthiness assessment that incorporates variables the applicant may not know are being considered — such as neighborhood, purchasing patterns, and social connections — and produces a score the applicant cannot decompose or challenge.

In education, an AI system assesses a student’s learning trajectory and channels them toward or away from opportunities based on predictions about their future performance — predictions built from patterns that may encode the very inequalities the educational system is supposed to correct.

In each case, the person affected experiences the consequence — the closed door, the higher premium, the denied loan, the narrowed pathway — without knowing that an inference was the cause. The system has reached a conclusion about them. The conclusion has shaped their life. And they have no way to know.

The asymmetry is structural. The person provides the data. The system generates the inference. The person bears the consequence. The system retains the intelligence. At no point in this process does the person have visibility into what was concluded, the right to access it, or a mechanism to contest it.

Legal scholars have named this gap. Sandra Wachter and Brent Mittelstadt argued in 2019 that existing data protection law provides little protection against what they called “high-risk inferences” — conclusions that may be wrong, discriminatory, or consequential for the individual’s opportunities. The GDPR provides robust rights regarding personal data. It provides almost no rights over the conclusions drawn from it.

The governance gap they identified remains open. The question is what mechanism would close it.

In my work on digital sovereignty, I propose a mechanism called inference escrow — treating the conclusions AI systems draw about people as regulated artifacts rather than proprietary outputs. Two levels of protection, suited to different contexts.

The first level is systemic protection, built into the system’s architecture for contexts where the individual is under constraint — a patient trading biometric data for care, a person in financial distress, a worker under continuous AI assessment. The protection comes from the system, not from the person inside it.

The second level is what I describe as a safe deposit box. The conclusions drawn about the person are held under that person’s direct control — like a box to which only they hold the key. They decide who sees what has been concluded about them, when, and for what purpose. The default is reversed: the inference belongs to the person it describes.

Neither level guarantees a different outcome for David. Both guarantee that the conclusion is visible, accountable, and subject to human judgment rather than executed in silence.

That is not a technical proposal. It is a principle that the conclusions AI systems draw about people should be governed by at least the same rigor as the data from which those conclusions are drawn. We have built governance for the input. The output remains ungoverned. The doors keep closing. And the people on the other side never learn why.

This article is drawn from Digital Sovereignty in the Cognitive Age, available at blogs.inspire-aspire.net.