|
I took a management course in grad school in 2009, and just about the only thing I remember about it was learning about the Hawthorne effect. This was research done at a Chicago factory from 1924-32, in which workers supposedly showed an uptick in productivity while participating in a study examining the effects of lighting on their work. A later study of the results claimed that the lighting had zero effect, and the workers were more productive because they simply liked the idea that researchers were taking an interest in their work. Some later critics found this anecdotal BS, but it was interesting nonetheless.
Tijmen Schep, a Dutch technology critic and privacy designer, recently identified a related effect in the digital realm. Schep calls the phenomenon “social cooling” (a riff on global warming) and describes it as the pattern of altered behavior exhibited when online users know they’re constantly being tracked by Big Data algorithms. And even though algorithms are mathematical, they were built by a human who likely showed some bias in their programming, so no algorithm is totally fair.
 It’s safe to assume that nothing’s private online. Any user who clicks a link or searches on Google has an aggregate “digital reputation” that then leads to profiling. For example, third-party companies privy to a user’s data can accurately determine a user’s addictability, personality traits, economic stability, religion and reading habits simply by combing their data. Schep says that even though a company may claim they don’t sell your data, the patterns and labels they derive from it are legally theirs and they can sell them at will.
Schep worries that a society built entirely on digital reputation could have ill effects. Users may feel pressured to conform, avoid healthy risks to maintain high ratings, and eventually fall into rigid social labels and structures. Mistakes and imperfections are meant to be forgotten, but online records persist indefinitely and may affect a user over the course of their lifetime. Critics are seeing the increasing importance of online reputation as a sort of credit score tailored to social interaction. China is already exploring a variant of a wide-reaching social credit score.
There are countless other examples of the possibility of social engineering through data. In November of last year, Data Alliance, the Big Data arm of advertising company WPP, signed an agreement with music service Spotify to acquire the listening preferences, moods and behaviors of 100 million of Spotify’s users in 60 countries. While the press release for this deal claims that this data will be used to connect with users with targeted ads, one could see this data leveraged by insurance companies to determine a user’s risk of depression and other mood disorders.
Schep identifies a similar effect he calls mathwashing, which more or less defines the bias behind “unbiased” algorithms. (He borrows that term from former Kickstarter data scientist Fred Benenson.) Schep calls for algorithms to be treated ethically as a law rather than a tool, as the design of algorithms is becoming more and more crucial to fair living, and algorithms are increasingly implemented in the justice system through predictive policing and other strategies.
We don’t yet live in a completely quantified society, but it’s easy to see how targeted advertising and Big Data are both nudging it in that direction. I’ll consider that next time I think about clicking on a questionable link.
Image credit: luckey_sun / CC BY-SA 2.0
| 
2
Re: Will Big Data Force Changes In Online Behavior?
Good Answers:
"Almost" Good Answers: