Why ‘online profiling’ is so dangerous

Back when I was at primary school one of the many risks kids faced was being seen near other kids who drew the attention of a teacher and were punished. The thing was that anybody in the area was – by definition – part of the group and would be scooped up and also punished. If you protested and said you’d just been nearby, you were doubly punished for lying.

The school system in action, circa 1970…

The thing was that just about anything invoked punishment – this was a school that punished one kid, I recall, for skipping for joy in jingly sandals. So it was better to keep your head down, keep well away from even the slightest hint of anything, and if the teacher held up four fingers and told you it was five – well, it was five.

I later discovered that this sort of experience was typical of any institution or system where one group has been given total authority over another. It doesn’t take long for the authority group to lose moral compass – to exploit that power for their own benefit and pleasure. When applied to societies – as happened particularly during the world’s ‘totalitarian’ period after the First World War – it provokes deep fear in the disempowered and deep corruption in the beneficiaries of the system.

But it also highlighted a common human cognitive flaw. We tend to see patterns in everything, even where none exist. It’s been postulated the cause is a legacy survival technique; back in hunter-gatherer days, early humans lived longer if they could spot the threat in the long grass from its shadows – and it was better to have a false alarm than to miss something real. So the ‘pattern assignation’ system evolved with a default of ‘on’ and a hyper-alert sensitivity. It also makes processes that create patterns (‘associations’) more convincing for us, cognitively, than processes that do not.

The problem I have is that ‘association’ profiling is not unique to either authoritarian dystopias or primary schools.  The same logic drives the algorithms that major internet companies use to analyse behaviours and serve up targeted ads; if you look for something, you must be interested in it.  And the flaw of the logic is also well evident – you do a Google search for (say) a shower fitting, once, and for the next 8 months you’re barraged with advertisements for plumbing, whether you need any more shower fittings or not.

That’s also how the ‘self-reinforcing bubbles’ work – Facebook, particularly, serves up an increasingly narrow range of content, based on what you appear to want, and it takes quite a bit of effort to break clear of it. People with eclectic interests, it seems, confuse the system.

What worries me – harking back to my school adventures – is the way this profiling can lead to false matches. That isn’t much of a problem when it comes to being barraged with plumbing advertisements. But it is a concern if society turns to its darker side, as actually happened in the totalitarian societies of the twentieth century. Many innocent people were punished for no better reason than inadvertently matching aspects of a profile that labelled them targets of the regime.

The fact that the ‘profiling’ is being done now via what one company or another see you doing online makes no difference to the underlying principle. And it’s too easy for that to go wrong – badly wrong – if society as a whole loses moral compass.

Thoughts?

Copyright © Matthew Wright 2017

Advertisements

8 thoughts on “Why ‘online profiling’ is so dangerous

  1. Oh yeah, and as I see with the ensuing flood of online adds, it costs them little to perform profiling to excess. Digging a grave is backbreaking work. Digging ten graves is a job no one wants. Have a machine that will dig those ten graves while you sip tea? Might as well have it dig a million graves. When it costs almost nothing to search, and little or nothing to falsely accuse … why not do it? Always beware someone who proclaims their moral stock so superior they require no constraints upon their power.

    1. What really worries me about all of this is the way that the internet and social media have been so swiftly hijacked – turned from something with every potential to be a positive and wonderful experience for all to share, into an environment where profit, advertising, scams and malicious software seem to be a constant presence. It says something about human nature, methinks. Sigh…

  2. Very good point! Having spent the last 7 years working with, and around, Google’s search algorithms and PPC executives I know some of the tricks they try on with retargeting and everything else. It’s sophisticated marketing… but also flawed. I’ve taken to just clicking those PPC ads for companies who annoy me (as it costs them money for each click, of course) and then clearing my internet cache regularly.

      1. I think a lot of people don’t know it costs the company money (not a lot – about 60p a time) but, yes, if you don’t like the brand then it’s a protracted way of bringing it down from the outside! MWahahaha etc.

  3. This is such a thought provoking piece! So many great points here. I do find it fascinating about how pattern seeking the human mind is- especially as this pertains to our willingness to believe conspiracy theories. But like you said, this has real world consequences, with self-reinforcing bubbles. In my personal experience for instance, I watch and listen to a range of political content- on the positive side, this has meant I’ve confused Google enough to give me adverts for all sorts of content I’m not interested in- on the negative side… well the same, because I fear it profiles me for certain ads. Then again, this could be my pattern-seeking brain cooking up a conspiracy theory- I’ll never know 😉
    Also, on another note I *shudder* every time you mention your primary school experiences- it sounds like something out of a Roald Dahl novel.

    1. Yeah that primary school wasn’t good – it had gone off the rails on the back of a weak headmaster and a really disturbing deputy head who kept the kids cowed and traumatized as a device for controlling them. I say disturbing because, looking back, he was obviously getting quite a lot of pleasure out of his ability to do so. My parents repeatedly intervened but had little success; as soon as I was in the hands of the school it was all on and, of course, I was doubly punished as a result of their efforts to protect me. Dad kept a dossier with all the details, including minutes of meetings with the headmaster and a wonderful exchange with one teacher who sent his enforcers (a gang of kids) chasing after me, after school – all the way home – trying to drag me back to the school so I could be punished some more. None of them were ever held to account, then or since. Roald Dahl would definitely have had great inspiration! 🙂

Join the discussion!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s