On the Science of Changing Sex

Machine Learning Transsexual Brains = Garbage In: Garbage Out

Posted in Brain Sex, Science Criticism by Kay Brown on June 8, 2021

If one spends any time reading science papers about transsexuality, one finds good science, mediocre science, poor science, bad science, and bogus science. But here is an example of garbage science. A paper came out last year that baldy stated that using machine learning and brain imaging, they could, somewhat accurately, determine an individual’s gender identity. This sounded like really exciting results. But after reading the paper, I’m calling BULLSHIT! It’s a harsh characterization, I know. But please follow along to see why I had no other choice.

First, let me state that I’m not an expert on Machine Learning and Deep Neural Net coding. But I have, in my capacity as an engineering executive, managed such experts. I’ve also, in my capacity as a Venture Capitalist (VC) technology advisor, conducted due dilligence research on start-up companies developing ML and NN technology. So I have just enough knowledge to be dangerous… that is to say, I know bullshit when I see it. And I see it here.

The bullshit consists of three elements.

The first is that researchers failed to tell us how many of their subjects were in the training set and how many were in the testing set. But first, let my tell you an anecdote about the time I was in the audience at a technical conference where a young researcher was presenting almost unbelievably high classification accuracy from his new computer vision algorithm. Finally, the first question from the audience during the post-presentation Q&A was how many examples were in the training set and how many in the test set? The young man then acknowledged that he had used the training set to test his algorithm. You could hear the visceral disgust sweep across the room at this basic error. Question is, did the authors make the same mistake? They said that 95% of the DATA was used in training and 5% in the “validation” of the model. Umm…. something is not right. There were less than 25 subjects in each category. Five percent of 25 is one. There was no way they could have used different subjects to have gotten a percentage accuracy of classification without having used the same subjects to provide both training and accuracy tests. So, what was the data split? Different parts of the brain scans of the same subjects? Seriously, something is very wrong here. One cannot do that.

The second garbage element is that they knowingly ignored prior science that there is very clear evidence that there are two separate taxons, at least for the Male-To-Female transsexuals, that have notably different brain phenotypes. We know that they knew because they referenced the Guillamon review paper on that very topic. But, since they didn’t bother to identify and segregate the two taxons for separate analysis, they were knowingly conflating the two, which would dilute the signals of both. The basic rule of thumb is never ascribe to conspiracy what can be explained by incompetence. Given the above issue of questionable Machine Learning validation, incompetence may have been the reason. The second possibility is that they knew this conflation was occuring, but felt, for non-scientific reasons, that they wanted this to occur. (I’ve seen this happen in other papers.)

The third garbage element is actually the most egregious. They claim that they identified nine “cardinal” gender related vectors in their study. But did they? I will argue that no they did not. This is where garbage in, garbage out really applies. They used the Bem Sex Role Inventory and cross correlated it with the brain scan data, claiming that the Bem inventory provides a window to gender. Flat out, it does not. It is an inventory of circa 1970s gender stereotypes! The most enraging thing about this is that the authors KNOW that, fully acknowledge that, but decided to use it anyways.

All in all, the Clemens paper is garbage. So the next question is how could such a paper pass peer review? The answer is where it was published. Cerebral Cortex would have reviewers who were experts in the brain science, but NOT sexology nor in machine learning. They just would have looked at the material that was in their field of expertise and allowed the other material to get a pass, unquestioned.

Further Reading:

Silly Stereotypes: Essay on the BEM inventory

Brainstorm: Essay about the Guillamon brain scan review

Reference:

Clemens, B. et. al., “Predictive Pattern Classification Can Distinquish Gender Identity Subtypes From Behavior And Brain Imaging”, Cerebral Cortex, (2020), https://doi.org/10.1093/cercor/bhz272

Tagged with: ,

Comments Off on Machine Learning Transsexual Brains = Garbage In: Garbage Out

%d bloggers like this: