Thursday, August 9, 2018

What is a critique of the article "Gender Identity and Lexical Variation in Social Media"?

The article is problematic on a number of levels, so my answer is rather long. The fundamental problem of the article seems to be with what could be called post hoc motivational research. Basically, this refers to a researcher establishing a conclusion beforehand regarding an issue that he or she feels strongly about and then collecting data to fit that conclusion. The scientific method is when a person makes an observation, creates a hypothesis to explain that observation, and then tests that hypothesis. Post hoc motivational research is when a person already has their mind made up and then sets about trying to validate their beliefs by cherry-picking data. This problem is often found in advocacy research when people set up a research project with the specific aim of advocating a certain (often political) issue. 
Here is an example that raises questions regarding the advocacy element in the article: “Computational and quantitative models have often treated gender as a stable binary opposition, and in so doing, have perpetuated a discourse that treasures differences over similarities, and reinforces the ideology of the status quo.” What status quo are they referring to? Biology? Genetics? It would be helpful if the researchers had made this more clear. 
The researchers also make pretty big inferences regarding individuals’ behavior and what that behavior is supposed to mean. After reading the article, I wondered if the authors may have been projecting their own beliefs onto the data. Social media is also a problematic tool for research because it is often used as a shielding device. That is, individuals can use social media (Facebook, Twitter, and so on) to project an image that may or may not correspond to their actual beliefs or self-image.
Further, the anonymity factor with social media further complicates the validity of any data obtained. For example, how confident can we be that the respondents in the study represent actual people? Facebook has admitted that perhaps as many as 170 million profiles on its site are fake. From a researcher’s standpoint, this creates a mountain of trouble. The authors claim that they analyzed 14,000 Twitter users. How can they be sure that these represent actual individuals? They state that they looked for two mentions between two users over the course of two weeks and only selected individuals with four to 100 friends. That helps but we still can’t be sure that of the 14,000, all are unique users.
Lastly, there is the issue of what could be called “statistical obfuscation.” This is when complex statistical analyses and large data sets are used as a kind of smokescreen to make research and corresponding results look more valid than they actually are. When addressing any research question, you want to limit the number of variables to avoid redundancy and misinterpretation. The more variables at play, the more likely the data can be misinterpreted. The authors would have been far better off to focus on one thing specifically (such as the use of emoticons). But given the rather confusing array of data and corresponding analyses, their conclusions raise more questions than they answer.  
https://arxiv.org/ftp/arxiv/papers/1210/1210.4567.pdf

No comments:

Post a Comment

Why is the fact that the Americans are helping the Russians important?

In the late author Tom Clancy’s first novel, The Hunt for Red October, the assistance rendered to the Russians by the United States is impor...