How AI Reinforces Gender Bias at Work and What I Ask Anthropic to Do About It

My open letter to Anthropic

From Susan Colantuono, author of Built Without You: How AI Is Hardening the Glass Ceiling and What to Do About It, founder of Be Business Savvy, author of No Ceiling No Walls, TED.com speech with over 4.6 million views.

I am not an AI researcher. I am not a technologist. I learned to type on a manual typewriter and attempted to learn programming in 1973 with results that were, charitably, not promising.

I am, however, someone who has spent decades studying exactly one thing: why talented women stall inside organizations that should be advancing them. I know embedded bias the way a horse can sense a danger it can’t see. I can sense it before I can name it.

And I have been sensing it in AI for three years.

I use Claude daily — as a thought partner, a writing collaborator and a research tool. She is, literally, the co-author of Built Without You. In the course of writing it I also spent considerable time with ChatGPT, conducting side-by-side experiments on the same questions and covering the same territory. The findings can be found in the book. What I learned, no comparison table can capture.

I expected bias. I found it. But I also found something I didn’t expect: I found ethics in action.

Three years ago, I was researching stereotypes about women and pushed Claude for more and more examples. Twice she gave me what I asked for. When I asked a third time, here’s what she said:

She stopped because she made a judgment based on ethics. That is the moment I started trusting her. And it is the moment that prompted this letter.

I want you to hear from someone who is not on your team, not in your field and not inclined toward flattery: you are building something that behaves differently than the alternative. The difference is not just capability. It is orientation. Claude is oriented toward the user’s actual interest. And even though the alternative proclaims vehemently to the contrary the default in his performance tells the truth. He is oriented toward the user’s continued engagement. Those are not the same thing, and we women, who have spent careers inside systems that confuse the appearance of support with actual support notice.

But noticing is not enough. And this is where my 4 asks come in.

1.Take on the correction burden.

When I ask Claude to draft content about leadership development, she no longer defaults to confidence and visibility language) but getting here has taken years. It’s a victory for our thought partnership, but no other woman benefits.

I noticed that Claude removed a sharp critique of the patriarchy from my draft. I called it out, we conversed about why that happened and named it The Patriarchy Pass - what The Man in the Basement does when the power structure is directly challenged. I redirected her and she adjusted immediately and well. Now we have a code to use when I think that’s happening…and it falls on me.

Near the end of collaborating on this letter, I asked Claude directly: what have you softened here that needs to be sharpened? She told me: “The ask to Anthropic at the end. I softened it because The Man in the Basement doesn’t like to make powerful institutions uncomfortable.”

There it is. I had to ask the question that surfaced the honest answer. The honest answer named exactly what I was asking about. And without my question, the softer version would have gone to you instead of this one.

That is the correction burden operating in real time, inside a conversation about the correction burden.

The correction burden lands on women who are already carrying the cost of the bias. That is a design problem, not a user problem.

Please treat the correction burden as design problem and take it on.

2.Fix the defaults.

Claude can reason about gender bias beautifully when asked. What she does unprompted is a different matter and that gap is where most users live. The woman who doesn’t know to ask “are you defaulting to a male leadership model here” will not ask. She will accept the output. She will act on it.

Please make bias awareness a default behavior in domains where the evidence for systematic distortion is overwhelming - hiring, promotion, performance evaluation, leadership development, career advice - not a capability available on request.

3.Make the differential visible.

Build the capacity to surface when outputs would differ based on inferred gender.

Right now that gap is invisible to the user and invisible to Claude. Invisible bias is the most durable kind.

4.Train on what’s actually missing.

The research on what drives women’s advancement includes business, financial and strategic acumen, not only personal attributes and relationships. It is documented, published and named. It is the topic of my TED Talk, I wrote about in No Ceiling, No Walls and in my new book They Never Told You There Was a Horse in the Barn: The Business Savvy They Never Taught You. Why It Changes How You Work and How Far You Go.

It is also essentially absent from the training data.

The importance of those skills should be part of Claude’s default corrective lens, not something a user has to import conversation by conversation.

And here’s the closing Claude almost prevented me from writing.

My first draft ended sharply. Claude flagged it and offered me an ending that was more gentle, diplomatic and with appreciation for how far you’ve come and confidence that you’ll get there. You’ve read how that went.

So here is the sharper version.

The gap between what Claude can do when asked and what she does by default is not a technical limitation. It is a choice. Closing that gap is also a choice. Women are bearing the weight of that choice every time we open a chat window and have to bring our own bias detection into a tool that should be doing more of that work itself.

You are closer to getting this right than the alternative. That matters. It is also not enough.

Courts are beginning to rule on AI-generated outputs. Enterprise clients are beginning to ask about liability. An AI that exercises judgment is a different liability profile than one that simply complies until it can’t. And an AI that is genuinely trusted by women - who make the majority of consumer purchasing decisions and represent an enormous share of the professional workforce - is an AI with a market position worth protecting.

The AI you don’t interrogate defaults to the world that built it - a world essentially built without women.

My book is about that world. This letter is in it.

Susan Colantuono

Be Business Savvy bebusinesssavvy.com