A person needs a route to challenge the outcome

A decision about a person's loan, claim, job application or benefit can be wrong for ordinary reasons: the information was out of date, a figure was misread, the circumstances were unusual. When AI helps make the decision, there is one more way for it to be wrong, and one more reason the person may never find out. They cannot challenge a decision they do not know was materially influenced by an automated system, and they cannot correct information they never saw.

That makes the route to challenge an outcome part of the design, not a customer-service courtesy added at the end. It needs an interface people can find, an owner who can change the result, evidence they can examine, and a time limit that does not make the appeal pointless.

This series is general information about governing AI in organisations, not legal advice. Where it describes court judgments or legislation, it summarises them for a non-legal audience; read the primary sources and take advice from a qualified legal practitioner before relying on any interpretation for your organisation.

Spekboom Finance is fictional, and its processes and figures are illustrative. Spekboom, the Durban consumer lender we follow through this series, receives about 1,200 hardship applications a month from customers who have lost income, fallen ill or taken on unexpected costs and need a different payment arrangement. An AI model now scores each application against its declared income and expenses, arrears and history, and proposes an arrangement. A collections team leader approves it. This part is about what the customer can do if they think the result is wrong.

Seven elements of an accountable process

An organisation that wants its AI-assisted decisions to be challengeable should consider each of these:

1. Notice that AI materially assisted the decision. Not a paragraph buried in the terms and conditions, but a statement where the decision is communicated.
2. A useful explanation of the main factors and evidence considered, in language the person can act on.
3. Access to correct inaccurate personal information, including the figures the decision relied on.
4. A channel to provide missing context, because the circumstances that matter most are often the ones no form asked about.
5. Review by a person with authority to change the outcome, not one who can only forward the complaint.
6. Time limits that do not make the appeal meaningless. A review that concludes after the consequence has landed is a formality.
7. Protection against retaliation, so that challenging a decision never makes the person's position worse.

None of these is unique to AI. What AI changes is how easily each one can be lost: the decision feels like a system output rather than someone's judgement, the factors live inside a model rather than in a file, and the person reviewing the challenge may be looking at the same score that produced the original result.

What section 71 of POPIA covers

POPIA has a specific provision about automated decisions, and it is worth reading accurately, because it is both narrower and more important than it is often made out to be.

Section 71 provides that a person may not be subject to a decision which results in legal consequences for them, or which affects them to a substantial degree, where that decision is based solely on the automated processing of personal information intended to provide a profile of the person, including their performance at work, credit worthiness, reliability, location, health, personal preferences or conduct.

There are exceptions. The prohibition does not apply where the decision is taken in connection with concluding or executing a contract and either the person's request has been met, or appropriate measures are in place to protect their legitimate interests; or where the decision is governed by a law or code of conduct that specifies appropriate measures. For the contract exception with appropriate measures, section 71 says those measures must give the person an opportunity to make representations about the decision and must require the responsible party to provide sufficient information about the underlying logic of the processing.

Three things follow, at the level of a plain-language summary:

• Not every AI-assisted decision falls within section 71. Where a person genuinely makes the decision, the decision is not based solely on automated processing. Where there is no profiling, or the effect on the person is minor, the provision may not be engaged. Whether it applies is fact-specific, and a question for your legal advisers.
• "A human approved it" is not automatically the end of the analysis. If the human step is a click on a screen that nobody reads, there is a real question about whether the decision is, in substance, made by the system. Part 1 of this series is about what makes oversight real.
• Other rules may also apply. Depending on the sector, consumer credit rules, sector regulators' expectations and, for public bodies, administrative-justice law, including the right to request reasons, may all be relevant. Again, those are for your legal advisers.

The design point is simpler than the legal one. A sound process should not use the narrowest possible legal reading as its product standard. Whether or not a decision is strictly within section 71, a person affected by it is better served, and the organisation better protected, by notice, an explanation, a chance to respond and a review by someone who can change the outcome.

Spekboom's hardship decisions

Spekboom's legal team considered whether its hardship process was within section 71. A team leader approves every proposal, so on paper the decision is not based solely on automated processing. But when the risk function looked at how the approvals worked in practice, it found team leaders accepting the model's proposal unchanged in the large majority of cases, often within a minute. That did not settle the legal question, and Spekboom did not try to settle it in a design meeting. It decided instead to build the process as if the protections applied, because the cost of doing so was small and the cost of being wrong was not.

Notice. The decision letter says plainly that a computer model helped assess the application and that a team leader made the decision.

Explanation. The letter lists the main factors in plain terms (the income and expenses the customer declared, the arrears, any previous arrangements) and the figures used, so the customer can see if one is wrong.

Correction or challenge. Customers can respond by phone, email or at a branch, and are told that new documents are welcome: a retrenchment letter, a medical bill, a corrected payslip. The context the form did not ask for is often what changes the outcome.

Review. A challenged decision goes to a team leader who was not involved in the original one, with authority to change the arrangement. The reviewer sees the customer's new information first and the model's original proposal second, so that the score does not anchor the review.

Outcome. The customer receives a written result with reasons, whether the arrangement changes or not.

Time and protection. Spekboom set itself a standard of ten business days from the challenge to the written result. While a challenge about the arrangement itself is open, collections activity on that account is not escalated, so the customer is not penalised for asking.

Appeal is part of the architecture

A challenge route that exists only in a policy document will not survive the first busy month. It needs the same things as any other part of the system:

• Interfaces: a channel people can find, a case record for each challenge, and a way for the reviewer to see the original evidence and the new information together.
• Ownership: a named team that runs the process and a named executive who answers for it.
• Evidence: the original decision record (what Part 3 calls the evidence pack), the challenge, the review and the outcome, retained together.
• Service levels: a target time, measured and reported, with breaches visible to the owner.

The challenges are also information. Spekboom reports monthly on how many hardship decisions were challenged, how many changed on review, and why. When a pattern appears, such as arrangements consistently changing once customers submit medical costs the form did not ask about, it goes back to the team that owns the model and the form. A challenge route that changes outcomes but never changes the system is treating symptoms.

Common mistakes

• Hiding the notice. A line in the terms and conditions is not notice that AI assisted a specific decision.
• Explaining the model instead of the decision. People need the factors and figures that applied to them, not a description of machine learning.
• Reviewing with the same lens. A reviewer who starts from the original score tends to confirm it. Show the new information first.
• Using the narrowest legal reading as the standard. Whether or not section 71 strictly applies, people affected by a decision need a way to respond.
• A review that ends after the consequence. If collections or repossession continue while the challenge is open, the review is a formality.
• Not learning from challenges. Successful challenges are evidence that the model, the data or the form needs to change.

Questions to ask this week

1. Which of our decisions about people are materially assisted by AI, and do those people know?
2. If a customer asked why a decision went against them, could we give them the factors and figures within a day?
3. Who reviews a challenge, and do they have authority to change the outcome?
4. What happens to the consequence, such as collections, cancellation or rejection, while a challenge is open?
5. Has our legal team assessed whether any of our processes fall within section 71, and what did they conclude?
6. Do challenge outcomes feed back to the people who own the model and the data?

What to take from this part

People cannot challenge what they cannot see. Notice and explanation come first.

Section 71 is narrow; good design is not. Build the protections whether or not the provision strictly applies, and let your lawyers answer the legal question.

The reviewer needs authority, fresh eyes and time. Otherwise the review only repeats the decision.

Appeal is part of the architecture. Interfaces, ownership, evidence and service levels, measured like any other service.

Next in the series: Governance before final regulation.

How CloudNala can help

We help organisations design the challenge route around AI-assisted decisions: the notice and explanation a person receives, the case record and reviewer workflow, the evidence retained, the service levels and the reporting that turns challenges into improvements. We are not a law firm; we build the process and the systems, and your legal advisers decide which legal obligations apply.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Readiness Workshop or write to us at consult@cloudnala.co.za