There is a particular kind of confidence that radiates from a screen. A clinician holds a dermatoscope against a patient's skin, captures the image, and a number appears: a probability, a risk score, a clean computational verdict rendered in the universal language of decimals. The machine does not hesitate. It does not say “I am less sure about this one.” It returns the same crisp output whether the skin beneath the lens is the pale, freckled forearm of a redhead from the Scottish Highlands or the deep brown shoulder of a man whose ancestry traces to West Africa. The interface is identical. The confidence is identical. The accuracy, it turns out, is not.
This is the uncomfortable fact at the centre of a slow-building reckoning in one of medicine's most visual specialties. Dermatology was supposed to be the field where artificial intelligence would shine first and brightest. Skin is, after all, the organ you can photograph. A lesion sits on the surface, available to any camera, ready to be classified by a neural network trained on hundreds of thousands of examples. The promise was seductive: democratised expertise, faster triage, melanomas caught months earlier, lives saved in places where a dermatologist is a four-hour drive and a six-month waiting list away. And much of that promise is real. But woven through the optimism is a structural flaw that the field has known about, documented, quantified, and only partially addressed. The machines see darker skin less well. And the institutions deploying them have, for the most part, not told the patients standing on the wrong side of that gap.
The question this raises is not merely technical. It is a question about what we owe people when we ask them to trust a tool we know to be unequal. If an AI diagnostic system is understood by its makers and its deployers to perform worse on darker skin, and it is used on a patient with darker skin who is never told this, has that patient truly consented to anything at all? And what does the principle of health equity, so often invoked and so rarely operationalised, actually demand of the hospital, the clinic, or the national health service that flips the switch?
A Specialty Where Looking Is Everything
To understand why bias in dermatology AI is so consequential, you have to understand the stakes of the underlying diagnosis. Melanoma is the deadliest of the common skin cancers, and it is almost uniquely sensitive to timing. Caught early, while it is still confined to the upper layers of the skin, it is among the most survivable of all cancers. The American Cancer Society puts the five-year survival rate for localised melanoma at around 99 per cent. Allow it to metastasise, to spread to distant organs, and that figure collapses to roughly a third. Few diagnoses in medicine carry such a steep cliff between early and late, between a minor excision under local anaesthetic and a death sentence delivered in instalments.
That cliff does not fall equally across the population. The data on racial disparities in melanoma outcomes is stark and long-established. For the period from 2015 to 2021, the five-year melanoma survival rate among white Americans was about 95 per cent. Among Black Americans, it was roughly 70 per cent. The gap is not driven by biology in any simple sense; melanoma is, in absolute terms, rarer in people with darker skin. It is driven overwhelmingly by stage at diagnosis. One widely cited figure holds that around 39 per cent of Black patients present with regional or distant disease, stage III or stage IV, compared with roughly 15 per cent of white patients. By the time the cancer is found, it has often already moved.
This is the world into which dermatology AI arrives: a specialty where the central task is recognition, where the difference between treatable and fatal is measured in how early something is seen, and where the populations whose cancers are already being caught too late are precisely the populations most likely to be poorly served by a tool that struggles to see them. A technology that performs unequally across skin tones does not enter a level field. It enters a field already tilted, and it risks tilting it further.
The Number That Changed the Conversation
For years, the underperformance of dermatology algorithms on darker skin was suspected, asserted, and worried over, but rarely measured with the kind of rigour that compels institutional attention. The problem was partly circular: to test how an algorithm performs across skin tones, you need a high-quality dataset that spans skin tones, with diagnoses confirmed not by a clinician's guess but by the gold standard of biopsy. Such a dataset did not exist. The very gap in the training data made the gap in performance hard to prove.
That changed in 2022, when a team led by researchers at Stanford, including Roxana Daneshjou and Albert Chiou, published a study in Science Advances built around a resource they had assembled called Diverse Dermatology Images, or DDI. It was, they noted, the first publicly available, expertly curated, pathologically confirmed image set deliberately balanced across the full range of skin tones. The numbers behind it are worth stating plainly, because their plainness is the point. The dataset contained 656 images from 570 patients, sorted by Fitzpatrick skin type: 208 images of the lightest skin, types I and II; 241 of the middle range, types III and IV; and 207 of the darkest skin, types V and VI. Crucially, every lesion had been biopsied, so the truth of each diagnosis was not in question.
When the researchers ran state-of-the-art dermatology algorithms against this honest benchmark, the results were sobering. The models' ability to distinguish malignant from benign lesions, measured by an area under the curve, dropped sharply when confronted with images they had not been built to handle. Performance fell by figures in the region of 27 to 36 per cent relative to the algorithms' own published results, and the decline was concentrated, predictably, on darker skin and on rarer diseases. These were not obscure or amateurish systems. They were among the best in the field, the kind of models that generate excited headlines about machines outperforming doctors. Tested fairly, they faltered exactly where the human cost of faltering is highest.
The study did not end on despair, and this matters for the ethics that follow. When the team fine-tuned the algorithms on the diverse DDI images, the gap narrowed and in places vanished. Models retrained on darker skin not only closed the distance between light and dark performance but, in the case of malignancy detection on dark skin, outperformed the dermatologist raters used for comparison. The lesson was unambiguous. The bias was not an inevitable property of the technology. It was a property of the data, and data can be changed. The disparity was a choice, even if no one had consciously chosen it.
A Structural Problem, Not a Bug
If the DDI study supplied the hard number, a review published in June 2024 in the journal Frontiers in Artificial Intelligence supplied the diagnosis of the disease behind it. Written by Nazma Khatun, Gabriella Spinelli, and Federico Colecchia, the paper set out to map the landscape of technology aimed at reducing health inequality in skin diagnosis for people of colour, and it reached a conclusion that should unsettle anyone who imagines the problem can be patched with a software update.
The authors documented the human cost in unsparing terms, citing evidence that African Americans are around four times more likely to present with stage IV melanoma owing to delayed diagnosis, and approximately 1.5 times more likely to die of the disease than white patients, with five-year survival rates they put at 72.2 per cent against 89.6 per cent. Then they turned to the machinery meant to help. What they found in the training data was not a marginal shortfall but a near-absence. Studies that claimed to include people of colour, they noted, frequently included almost none. One prominent example involved a dataset in which just 2.7 per cent of participants were Fitzpatrick type V and not a single one was type VI, the very darkest category. In another instance, an algorithm reporting impressive accuracy in development correctly diagnosed only a small fraction of cases when tested against predominantly darker skin.
The review's central argument was that this was structural. The underrepresentation of people of colour in dermatology AI was not a discrete error introduced at one point in the pipeline that could be excised by a diligent engineer. It was the downstream consequence of a chain of older inequities: medical curricula that taught skin disease almost exclusively on white skin, research cohorts that skewed white, clinical photography archives accumulated in institutions serving largely white populations, and a development culture that treated representativeness as a nice-to-have rather than a precondition. Without intervention upstream, the authors warned, the systemic underrepresentation could not be solved and would only amplify the disparities already baked into care. The machines were not inventing bias. They were inheriting it, encoding it, and scaling it.
The Datasets That Will Not Say What They Contain
Here the story takes a turn that sharpens the consent question to a fine point. Even if a clinician wanted to know how a given AI tool would perform on a given patient, she frequently could not find out, because the datasets underlying these tools often do not record the one variable that matters most.
Consider the public benchmarks that dominate the field. The ISIC archive and the widely used HAM10000 dataset are foundational resources, the raw material on which a great deal of dermatology AI has been built. Analyses of these collections have repeatedly found them overwhelmingly composed of lighter skin. One assessment of the large ISIC 2020 collection and the related MILK10k set estimated that fewer than one per cent of subjects fell into the darkest Fitzpatrick categories, with the data dominated by lighter types. A benchmark with that composition cannot tell you how a model behaves on dark skin, for the simple reason that dark skin is barely present to be measured. The numbers that look reassuring in a published table describe a population that does not include the patient in front of you.
The deeper problem, surfaced in a study published in npj Digital Medicine in November 2025 by Yingjoy Li, Veronica Rotemberg, Roxana Daneshjou, Jenna Lester, and colleagues, is that many datasets do not document their skin tone composition at all. The team proposed a tool they called a Dataset Nutrition Label, a structured summary of a dataset's contents and limitations modelled loosely on the nutritional information panel on packaged food. Applying it to a large 2024 dataset of more than 400,000 lesion images drawn from total body photography, they found that it contained no skin tone documentation whatsoever. Their label flagged the omission explicitly, cautioning against deploying models trained on the data to assess individuals with darker skin and warning of hidden proxies and underrepresented populations lurking unmeasured within.
Sit with what this means at the bedside. A clinician adopting such a tool cannot consult the label, because there is no label. She cannot reason about her patient population, because the composition of the training data is undisclosed. She inherits a system whose performance on the person in her chair is, in the most literal sense, unknown and unknowable from the documentation provided. The transparency that informed consent presupposes, the idea that someone in the chain knows the relevant facts and can convey them, breaks at the source. You cannot disclose what was never recorded.
The Measuring Stick Is Also Broken
It would be convenient if the tool we use to describe skin tone were itself sound. It is not, and this is more than a pedantic footnote. The Fitzpatrick scale, the six-category system that pervades dermatology and structures nearly every dataset described above, was never designed to measure skin colour. It was devised in the 1970s to predict how skin would respond to ultraviolet light, how readily it would burn and how readily it would tan. It was, in origin, a sunburn classifier built around the responses of lighter skin, later extended to cover darker types almost as an afterthought.
A study published in npj Digital Medicine in December 2025 by Victoria Weir, Veronica Rotemberg, and colleagues compared the Fitzpatrick scale against alternatives, including the more recent Monk Skin Tone scale and objective colorimetry. The Fitzpatrick categories, they found, showed the weakest clustering when mapped against measured colour, meaning each Fitzpatrick band sprawled across a wide and overlapping range of actual skin tones. The scale, the authors observed, does not measure skin tone; it measures photosensitivity, and the two are related but not the same. The Monk scale and objective colour measurement both performed better, with the Monk scale in particular proving more reliable and more capable of revealing genuine differences in how melanoma algorithms perform across tones.
The implication compounds every problem already described. The field has been auditing its own fairness using a ruler with blurred and arbitrary markings, originally manufactured to answer a different question entirely. When a dataset reports its Fitzpatrick distribution, it is offering a coarse, contested proxy and calling it a measurement. The instrument of accountability is itself part of what needs reforming.
What Consent Was Supposed to Mean
Step back from the technical thicket and the ethical architecture comes into focus. Informed consent is one of the load-bearing pillars of modern medicine, the legal and moral mechanism by which a patient's body remains their own even as they hand themselves over to expert care. Its logic is that a competent adult is entitled to the information a reasonable person would want in order to decide whether to accept a proposed course of action, including its material risks and reasonable alternatives. The patient need not become a physician. But they are owed the facts that would matter to a sensible person weighing the choice.
The legal scholarship on whether this doctrine reaches the use of artificial intelligence is, as yet, cautious. In an analysis of AI and the law of informed consent, the legal scholars I. Glenn Cohen and Andrew Slottje concluded in 2024 that current United States law probably does not require a physician to disclose, as a general matter, that an AI system was involved in a diagnosis. The reasoning runs through the materiality standard. Under the patient-centred version of that standard, a risk must be disclosed when a reasonable person would attach significance to it in deciding on treatment. The mere fact that software assisted a clinician, the argument goes, may be no more material than the fact that the clinician consulted a textbook or a colleague.
But Cohen and Slottje also identified the precise circumstance in which the analysis shifts, and it is the circumstance this entire article describes. Algorithmic bias, they noted, can be material, particularly where training data underrepresents a patient's group in a way that predicts poorer performance for that patient specifically. That is not a textbook the clinician happened to read. That is a known, quantified, group-specific reduction in the reliability of the very tool being used to decide whether a mark on someone's skin is cancer. It is difficult to imagine a fact a reasonable patient would more plainly want to know. The commentator Emma Kondrup, writing for Harvard's Petrie-Flom Center in April 2025, pressed the broader worry that informed consent in the age of opaque, evolving algorithms risks becoming symbolic, a signature collected on a form for a process the patient cannot meaningfully evaluate. When the relevant risk is not the inscrutable inner workings of a black box but something as concrete as “this tool was tested mostly on skin lighter than yours and is known to be less accurate on skin like yours,” symbolism is not good enough.
The consent question, then, resolves into something quite sharp. Consent that conceals a material, group-specific disparity is not consent in any meaningful sense. It is the form of consent without its substance, a ritual that produces a signature while withholding the one fact that might have changed the signer's mind. And the cruelty of the arrangement is its distribution. The patients on the wrong side of the accuracy gap are disproportionately those who, in many health systems, have the least access to a specialist second opinion, the fewest resources to seek out alternative assessment, and the least standing to contest a diagnosis that arrives late. The tool performs worst for the people least equipped to notice or to challenge its failure. A disparity in accuracy lands on top of a disparity in power, and the two reinforce each other.
The British Experiment in Doing It Differently
If this all sounds abstract, Britain offers a concrete and instructive case, because the question of deploying biased dermatology AI is not hypothetical there. It is operational. An AI system called DERM, developed by the company Skin Analytics, has been used across a number of NHS England trusts to assess skin lesions, in some configurations taking patients off the urgent cancer pathway without a doctor reviewing every benign result. In May 2025, the National Institute for Health and Care Excellence, the body that judges which technologies the NHS should adopt, issued a conditional recommendation: DERM could be used within the health service over a three-year evidence-generation period while its real-world value was assessed.
What makes the NICE decision notable for the consent debate is what it did about skin tone. Rather than wave the technology through with uniform confidence, NICE built the known uncertainty into the rules of deployment. It specified that for patients with black or brown skin, an additional healthcare professional review would take place during the evidence period, reflecting that the evidence for the tool's accuracy in those groups was less certain. The institution, in other words, did not pretend the disparity away. It acknowledged that it did not yet know how well the machine saw darker skin, and it placed a human safeguard precisely where the machine was least trustworthy.
This is, in one reading, exactly what health equity ought to require: an institution confronting a known performance gap not by hiding it but by compensating for it, allocating extra scrutiny to the patients the technology is most likely to fail. Yet it also lays the underlying problem bare. The British Association of Dermatologists has voiced the longstanding worry that the underrepresentation of darker skin in image datasets could cause AI to perform poorly on those patients, and has noted how few data exist on the technology's effectiveness on dark skin. NICE's safeguard is a tacit admission that the tool is being deployed before that uncertainty is resolved. The human second read is a patch over a gap that more representative data should have closed years earlier. It is a humane response to an inequity that better data collection might have prevented from arising at all.
There is a further, subtler point buried in the NICE arrangement. The extra review for darker skin is a form of institutional disclosure, a recognition encoded in policy that performance differs by skin tone. But the patient sitting in the clinic may never learn why their case is being handled differently, or that it is being handled differently at all. The safeguard protects the patient's body without necessarily informing the patient's mind. It is better than nothing, considerably better, but it is not yet the full transparency that meaningful consent would demand.
What the Regulators See, and What They Do Not Require
Regulators on both sides of the Atlantic have begun, haltingly, to grapple with this. In the United States, the Food and Drug Administration issued draft guidance in January 2025 on the lifecycle management of AI-enabled medical devices, and its expectations now include analysis of performance across demographic subgroups, with attention to race, ethnicity, age, sex, and the equipment used to capture images. On paper, this is the regulator asking precisely the right question: does the device work for everyone it will be used on?
The gap, as ever, lies between the paper and the practice. Analyses of the transparency actually achieved by FDA-reviewed AI devices have found it wanting. One assessment found that demographic reporting in device summaries, while rising, remained low, with fewer than one in five summaries providing race or ethnicity data, and structured subgroup-level performance reporting largely absent. A substantial share of devices reported no clinical study at all, and many reported no performance metrics of any kind. The regulator is asking for the information that would make informed deployment possible. It is frequently not getting it, and it is clearing devices anyway. The result is a market in which a hospital can lawfully acquire an AI dermatology tool whose performance on darker skin is, from the published record, simply unknown.
This is where the chain of responsibility comes into focus, and where it tends to dissolve. The dataset curators did not record skin tone. The developers trained on what was available and reported what regulators minimally required. The regulators cleared the device against a framework that asks for subgroup data but does not reliably compel it. And the deploying institution acquires a tool wrapped in documentation that does not answer the one question equity demands. At each handoff, the relevant fact, “this may not work as well on darker skin,” can slip through a gap in the floorboards, until it reaches a clinician who has no way to retrieve it and a patient who is never told it existed.
What Equity Actually Requires
It is easy to invoke health equity and hard to say what it concretely obliges. The phrase risks becoming a comfortable abstraction, a value affirmed in mission statements and forgotten at procurement. So let us be specific about what it demands of an institution choosing to deploy a dermatology AI system whose performance across skin tones is unequal or unknown.
First, it demands honesty in acquisition. An institution should not deploy a tool whose skin tone performance it cannot characterise, and where the documentation is silent it should treat that silence not as reassurance but as a red flag. The Dataset Nutrition Label proposed by the npj Digital Medicine researchers exists precisely so that absence can be made visible rather than assumed away. An institution that adopts a tool with no skin tone data has not made a neutral choice. It has made a choice to operate in the dark, and it has chosen on behalf of patients who never agreed to be experimented upon.
Second, it demands disclosure to the patient. If a tool is known to perform less accurately on darker skin, the patient with darker skin is owed that information in terms they can understand, alongside the alternatives available to them, including the option of conventional assessment by a clinician. This is not a demand for a lecture on convolutional neural networks. It is a demand for one plain sentence about a material limitation, the kind of sentence consent doctrine has always required for material risks. The legal floor, as Cohen and Slottje observe, may not yet compel this in most jurisdictions. The ethical ceiling plainly does.
Third, it demands compensating safeguards where disparity is known, of the kind NICE built into the DERM deployment, with extra human review allocated to the patients the technology is most likely to fail. Equity is not achieved by treating everyone identically when the tool itself does not. It is achieved by directing additional protection towards those who would otherwise bear the cost of the tool's weakness.
Fourth, and most fundamentally, it demands investment upstream in the data itself. The DDI study proved that the gap is closable, that fine-tuning on diverse, biopsy-confirmed images can erase the disparity and even surpass human performance on darker skin. The disparity persists not because it is technically intractable but because closing it requires deliberate, funded, sustained effort to collect the images that medicine has historically failed to gather. An institution serious about equity does not merely deploy other people's tools more carefully. It contributes to fixing the foundation, because every clinic that collects diverse, well-documented images is widening the path for the next generation of fairer systems.
The Patient Who Was Never Asked
Return, finally, to the screen and its serene confidence. The machine does not know that the skin beneath the lens is dark. It does not know that the dataset it learned from contained almost no one who looked like this patient. It does not know that its certainty is, in this particular case, partly counterfeit. It simply returns its number, clean and unhesitating, and the number carries an authority it has not earned for this person.
The patient, meanwhile, knows none of this either. They were told, perhaps, that an AI system would help assess their skin, and that sounded modern and reassuring, the hospital investing in the future. They were not told that the future had been built mostly out of skin lighter than theirs. They were not offered the sentence that might have prompted them to ask for a second look. They signed, or nodded, or simply did not object, and in the eyes of the institution that counts as consent. It is consent the way a photograph of a meal is dinner: the shape is right, the substance is missing.
The reckoning underway in dermatology AI is often framed as a problem of data, and at one level it is. But beneath the data sits something older and more demanding, a question about what we owe one another when we build tools that see some people more clearly than others. The studies have done their work. The disparity is measured, the mechanism understood, the remedy demonstrated. What remains is a choice about candour, about whether the institutions wielding these systems will speak the plain truth to the patients most at risk of being failed by them, or whether they will let the machine's borrowed confidence stand in for an honesty they were never quite willing to offer. Consent that hides the thing the patient most needs to know is not a contract. It is a performance. And the people watching it most closely, though they may not yet realise it, are the ones it is least designed to protect.
