The Hidden Power of the Algorithm

What is an Algorithm?

To start with the basics, let’s look at a definition of an algorithm.  I like this one from Whatis.com?

An algorithm (pronounced AL-go-rith-um) is a procedure or formula for solving a problem. The word derives from the name of the mathematician, Mohammed ibn-Musa al-Khwarizmi, who was part of the royal court in Baghdad and who lived from about 780 to 850. Al-Khwarizmi's work is the likely source for the word algebra as well.

A computer program can be viewed as an elaborate algorithm. In mathematics and computer science, an algorithm usually means a small procedure that solves a recurrent problem.

Another way of looking at it is to say an algorithm is a series of logically ordered instructions, rather like a recipe for baking a cake.  If you are sensible, and use a Delia Smith recipe, you will probably end up with a beautiful cake.  If you are foolish enough to go to a Keith Floyd recipe book, it will probably turn out to be a culinary disaster.

A Mathematical Genius at Work

In itself, an algorithm is neither good nor evil.  As a problem-solving tool it could be effectual or it could be useless; it all depends on the quality of the human mind that creates and programmes the algorithm in the first place it, and the appropriateness of the human response to the algorithm results.

The War on Terror

Thanks to revelations reported in The Guardian by ex American intelligence analyst Edward Snowden, data-gathering techniques by government security organisations have been hitting the headlines across the globe. The US intelligence organisation known as the National Security Agency or NSA has been widely reported to be covertly collecting colossal quantities of data from our phones, computers, and social networks.  Thanks to advances in communications techniques and the ease and cheapness of current-day data storage, it is now physically and economically feasible for them to create, maintain and interrogate these mammoth databases as they search for terrorist conspiracy activities.

The other reason for the existence of these data monsters is that modern computing techniques make it feasible to trawl through them.  It would not be possible for human beings to read all of that information (and it would no doubt be an incredibly boring task, too), but algorithms can search it, analyse it and report back to the government security agencies. These techniques are a major weapon in the state’s armoury for the so-called war against terrorism.  We know that algorithms are deployed by the NSA and GCHQ (the UK equivalent listening centre, General Communications HQ at Cheltenham), but we can only take an educated guess at the rules used by the algorithms, and hope that they are being used wisely.

Big Brother is Watching You

Algorithmic techniques have also been used in recent years to facilitate ‘predictive policing’ and in some instances they have achieved dramatic successes.  The idea here is to harness the power of the algorithm to provide data to enable police to deploy their resources in the right place at the right time.  This is sometimes known as ‘Minority Report Policing’, after the Steven Spielberg  sci-fi movie with Tom Cruise, where "PreCrime", a specialized police department, stops crime before it actually happens, using information provided by three psychics called "precogs". 

In the real world the technique is more correctly known as CRUSH, or ‘Criminal Reduction Utilising Statistical History".  The earliest example of this I could find was ‘Operation Blue Crush’, a pilot Crush operation staged in Memphis, Tennessee back in 2005, and during which 1,200 people were arrested over the course of just three days.  Have a look at this link to find out more: http://www.memphispolice.org/blue%20crush.htm

Algorithms, Quants, and the City

As a reader of this blog you are already well aware of the power of  The Geek.  Did you know that, in the city, highly-paid geeks known as ‘Quants’, or Quantitative Analysts, are employed to create algorithms to formulate trading strategy?  The traditional picture of the stock market trading floor peopled by traders in suits and ties making frantic telephone calls and yelling ‘Sell, sell!’ has now largely been superseded by computer servers running ‘algos’ to predict market fluctuations.

Most financial institutions, including banks and pension funds, now rely on algorithms. Sometimes competing algorithms have been known to clash, and, on occasion they have been blamed for speeding up trading to the extent of destabilizing the market and causing a meltdown.  It’s not only the big rollers in the city that can be affected: the ‘algo’ is also impacting ordinary people by influencing the way their savings and pension funds are invested.

Algorithms in Everyday Life

It doesn’t stop there. The power of the algorithm extends way beyond the rarefied world of investment and trading. Think about dating websites, credit checks, online retailing, retailer loyalty schemes, tailored and targeted discount  vouchers, online insurance quotations and internet search engines: they all use the same principles to analyse our personal interests and our buying habits.

Algorithms are being used extensively now simply because of the explosion in the quantity, quality, and availability of data spawned by the era of global mass communications.  The technique is now so widespread and industrialised that it is commonly known as ‘Data Mining’.

All this is nothing new, of course.  Alan Turing and his team of Bletchley Park code-breakers used algorithms to powerful effect back in the 1940s, and they were instrumental in victory against the Nazis in World War II.  This is a perfect example of an algorithmic tool being harnessed to good effect.  In the wrong hands, and wielded unwisely, there is also the potential for it to become an instrument of the powerful and unscrupulous who seek to dictate, repress, control and censor.