So I have a copy of the Rudman paper asserting that not all us feminists are fugly lonely lezzos after all. (My first post on the issue is here.)
My initial response consisted mainly of a rejection of the patriarchal framing of the question itself, and of the misogynistic, heterosexist assumptions underpinning the uncritical characterisation of the stereotypes as “negative”. A reading of the full paper hasn’t disabused me of either of those impressions, but it has added some extra methodological questions.
The most obvious issue with the methodology is this: only young people currently in a heterosexual relationship were included in the sample set. The exclusion of currently-single, bisexual, and gay participants wasn’t noted in the media reports, of course. And the exclusions? Were done after the event. That’s just plain crap science. You set your exclusion criteria BEFORE the trial; otherwise you have no defence against accusations of cherry-picking. You don’t just collect a whole pile of data, sift through it, and then decide which bits you want to accept.
So what they’ve shown right now is not that feminism predicts romantic success, as reported in the media; but that straight young privileged feminists currently in a relationship with a man and volunteering for an experiment on feminism and romance are no more likely to be in dysfunctional relationships than straight young privileged non-feminists currently in a relationship with a man and volunteering for an experiment on feminism and romance. Whoop-de-doo.
The authors try to give a reason for the post hoc exclusion:
Because we wished to avoid memory problems and possible group differences based on sexual orientation, we selected only heterosexual participants who reported being in a current relationship for our focal analyses.
Participants who were not in a current relationship were instructed to think of their most recent past relationship during measures that asked about their relationship and partner ‘s feminist identity and attitudes. As noted, these participants were excluded from the analyses to allow us to focus on current relationship health.
Something to note is that I don’t think these authors are openly or consciously hostile to feminism. Their introduction keeps almost hitting the mark for me, and the discussion raises some useful points about possible future research. I just think, well, that they’re trapped in the matrix like rats in a maze.
Some people wanted to know how the study defined feminist identity. It was by self-report. From the authors:
As a measure of feminism, we combined participants’ identification with feminism with how much they liked feminists and career women.
Participants responded to two items using 6-point scales ranging from 1 (strongly disagree) to 6 (strongly agree). The items were, “I am a feminist” and “My partner is a feminist.” To measure attitudes, they also reported (on four separate scales) how warmly they and their part ners felt t oward feminists and career women on thermometer measures ranging from 1 (very cold) to 10 (very warm). A principal components factor analysis showed that feminist labeling and attitudes toward feminists and career women yielded one factor, so we averaged these three items to form a single index of feminism.
Oh, and attractiveness? It was also measured by self-report:
As a means of assessing attractiveness, participants in both samples also responded to three items, “I consider myself to be attractive,” “People often tell me I am attractive,” and “People compliment me on my looks” on scales ranging from 1 (strongly disagree) to 6 (strongly agree). They also rated their popularity with the other gender. The items were, “It is not difficult for me to get a date,” “I am frequently hit on for sex,” “I seem to be very popular with the opposite sex,” and “I was popular (dating- wise) in high school.”
There are unexplained issues with the recruitment and sampling procedures. How were the subjects recruited? The first part of the study recruited Intro to Psych students fulfilling a mandatory research participation requirement, and the second part publicly recruited participants from “various websites” (details below). There is no information given at all on the information offered to subjects either at recruitment or before the experimental procedure commenced, no attempt made to examine the issue of self-selection bias. Do you think you might get a somewhat biased sample from asking readers of psych research websites whether they’d like to participate in a study on feminism and romantic success?
There also appears to have been little or no notice paid to demographic variables. Aside from collecting data on sexual orientation and current relationship status (used for later exclusion purposes), the only demographic factors collected from the students were race, gender, age, and educational level. No attempt was made to examine whether feminist and non-feminist groups differed in other sociodemographic variables, such as economic privilege, disability, or location of origin. Age and level of education were excluded from the analysis “to avoid multicollinearity and to preserve statistical power.”
Here’s the sampling info:
[Study 1, college students]: Five hundred and thirteen volunteers (298 women, 215 men) participated in exchange for partial credit toward their Introductory Psychology research participation requirement. Participants who were not in a current relationship (129 women, 126 men) or who reported not being exclusively heterosexual (21 women, 14 men) were excluded for the analyses, leaving a sample of 242 volunteers (156 women, 86 men). Of these, 136 (56%) were European American, 60 (25%) were Asian American, 16 (6%) were African American, 17 (6%) were Hispanic American, and the remainder reported another ethnic identity.
[Study 2, website responders]: Four hundred and seventy-one volunteers (327 women, 144 men) were recruited from various web sites (age range = 18 to 65 years). People who were not in a current relationship (91 women, 55 men) or who reported not being exclusively heterosexual (42 women, 15 men) were excluded for the focal analyses, leaving a sample of 289 volunteers (208 women, 81 men) Of these, 209 (72%) were White, 28 (10%) were Asian, 16 (6%) were Hispanic, and the remainder reported another ethnic identity. In addition, 264 (90%) were US residents. Their mean level of education was 14 years (range = 11 to 18 years). Their mean age was 26; SD = 9.00 (women’s M = 26; range = 18″“65; men’s M = 27; range = 18″“55).
Participants were recruited from various websites, including Craig’s List, Social Psychology Network, and Psychology Research on the Net. In addition, requests for participants were posted using Google AdWords and on forums for various Yahoo! and Google Groups.
Another snippet that didn’t make the news?
There was a small but reliable tendency for male feminists to report being homosexual or bisexual, as opposed to heterosexual, r = -.13, p < .01.
[Appeal for assistance: there's a whole section on "suppressor variable effects" that is not well explained and that I don't quite understand. Anyone here familiar with this notion who'd like to take a look at it?]