You tried to raise your children gender-neutral but your daughter still loves ballerinas and your son loves cars. Maybe it's not your fault. Maybe this shows their urges are genetically programed.
You once had a woman as president of your organization but she never could communicate a vision. Perhaps she couldn't help it. Perhaps this is how we learn that women don't have what it takes to lead.
Stories are important; they can pique our interest, they can engage us. Stories can lead us to want to know more.
Stories are not studies
But stories themselves are not the path to knowledge. Knowing something requires research.
Research is particularly crucial when we're talking about differences between men and women. Everyone has opinions about this; everyone has stories.
And because those stories are easily passed around, we have to be careful not to let anecdotes about ballerinas or bad presidents become general truths.
Sometimes we can actually see a story get repeated so often it (falsely) acquires the status of research. I recently uncovered an example while trying to find the facts behind a claim in Sheryl Sandberg's Lean In. Her book is full of inspiring stories, but in this one case, she creates a context by invoking research that neither she nor anyone else can actually put their hands on.
How much do men overestimate their qualifications?
My investigation started after I heard a wonderful tale that I wanted to use to illustrate the claim that under-qualified men will more easily apply for jobs than under-qualified women. (Read that account, about men who apply for "women only" jobs here.)
I'd seen research showing that men overestimate their skills, but I was remembering something more specific. I had read in Sandberg's book the claim that men have a lower threshold for applying for jobs than women do, and I wanted to find the study.
An internal report at Hewlett-Packard revealed that women only apply for open jobs if they think they meet 100 percent of the criteria listed. Men apply if they think they meet 60 percent of the requirements.
Sandberg cites the article A business case for women, published in The McKinsey Quarterly by Georges Desvaux, Sandrine Devillard-Hoellinger and Mary C. Meany. So I turned to their article to learn more. They anticipate Sandberg's phrasing:
Internal research at HP showed that women apply for open jobs only if they think they meet 100 percent of the criteria listed, whereas men respond to the posting if they feel they meet 60 percent of the requirements.
But that was it. They didn't give any source for this claim, and my curiosity wasn't yet satisfied. I wanted to know more about the Hewlett-Packard results. How did HP do their research? What did they ask? Why did they pose the question in the first place? So, I kept looking.
Why you can't trust internal research
I found many references to this claim in magazines, blogs and even in serious research articles. But they all cite either Sandberg or the McKinsey article.
I started thinking I would never find the actual research. Where were the methods and numbers? We can't be satisfied to simply quote someone who tells about their "internal research." If we don't see it ourselves, we're not building on their work, but rather on their description of their work.
Accepting a research conclusion without being able to see the research violates just about all the tools in Carl Sagan's wise and practical Baloney Detection Kit.
Using social media for fact-checking
I wasn't ready to give up, though. I tried to see if Sheryl Sandberg had more information on this research by posting a question to her and others at the Lean In fanpage on Facebook. No luck.
Next, I tweeted others who might reasonably be expected to have more information, but kept getting sent back to the McKinsey article. Still no luck.
So I tried the authors of the McKinsey Quarterly paper and quickly got a helpful reply from the first author, who confirmed what can be inferred from the article. There is no specific source for the claim; it emerged from confidential interviews with senior executives.
I think that's the end of the road. I don't know where to turn to learn more.
Unless someone at Hewlett-Packard or McKinsey can supply more information, we have every reason to believe this is an off-the-cuff comment. It isn't hard to imagine how the interview might have gone.
McKinsey interviewer: Do you have any special challenges related to recruiting internally to higher-level positions?
Hewlett-Packard senior executive: Well, as a matter of fact, it's much more difficult to get women to apply for senior positions than men. It seems like they lack self-confidence. They don't even think about applying unless they think the meet 100% of the requirements. The men, if they feel like they're even 60% there, they go for it.
Do men and women have different thresholds for applying for jobs?
Where does that leave us? A claim reported to be the result of a study is starting to look less like research and more like gossip.
At this point, we know there is no legitimate, evidence-based foundation for the claim that men apply for jobs when they feel 60 percent qualified while women have to be 100 percent certain. None. Nothing that can be examined, reproduced, reviewed or cited. From a researcher's perspective, it doesn't exist.
Spreading this claim is particularly unfortunate because men and women probably do have different thresholds for applying for jobs. (The Atlantic presents relevant research in The Confidence Gap. But, yes, they promote the HP "discovery," too; in fact, they dress it up a bit beyond what the McKinsey article reports.)
Now, instead of addressing that situation constructively, skeptics can write off this cultural difference between men and women, justifying their indifference by pointing to arguments built on what we now believe to be hearsay.
Sheryl Sandberg and the rest of us should stop repeating this number. We do not know if it is true. We can't. There's no identifiable source. Keeping the myth alive hurts the greater research program and the social changes it can provoke.
"And we know that with 100% certainty"
All of us who have repeated this generalization should have seen a red flag right away. The alleged research claims a result in which women apply only when they meet 100% of the criteria. 100%?! We should always view such claims with suspicion.
Are all women so rigid, so homogeneous, so lacking in derring-do? Is it really true that they never apply for jobs when they think they're under-qualified? Not even one woman is willing to apply when she feels she meets 95% of the requirements? That is just too far-fetched to be taken seriously.
The McKinsey Quarterly has published something that looks like a study but in fact, it's just a story. Stories are important, but they are different than studies. Research is more than just dressing up a story and publishing it in your company magazine.
So what does happen when under-qualified women apply for jobs? I don't know. Sheryl Sandberg and McKinsey seem to think it doesn't happen. But I'm sure it does.
In fact, I could tell you a story ...