Created by Michael Reilly December 2011

1

Algos everywhere: new policies needed for the invisible processes shaping

our lives?

Summary

Algorithms can be used to develop scaleable processes that produce highly valuable

outputs. There are a growing number and variety of real-world algo applications

including computer-based trading in financial markets, logistic optimisation,

cryptography, retail market intelligence, preventative medicine, online dating and job

matching, large-scale surveillance and investigating paintings with disputed

attribution. Some experts argue that algorithms are a general purpose technology

that will revolutionise our lives. But algorithms are rarely objective and often there is

a lack of transparency regarding their use. Policies on the use of algorithms have not

evolved at the same rate as their development and negative externalities are far from

negligible.

Discussion

An algorithm is a problem-solving process that produces an output from given inputs.

With the invention of enabling general purpose technologies such as the computer

and the internet, algorithms have become pervasive in human life. In business,

algorithms can confer considerable comparative advantages by supporting

economies of scale for very large inputs. For example, large-scale logistical

problems have proved tractable to algorithms. United Parcel Service (UPS) uses

algorithms to optimise the huge number of possible driver routes that are used to

deliver millions of packages each day1. Algorithms are used to improve the efficiency

of air traffic control. Algorithms are also being used to transform the deluge of data

produced by modern retail transactions into highly valuable market intelligence.

Researchers from the Hewlett Packard Labs in Palo Alto are patenting an algorithm

that analyses the sentiments expressed in tweets about films. This algorithm

predicted the box office returns of a sample of films in their opening weekend with 98

per cent accuracy2. In the financial sector, algorithms have supplanted human

traders in equity and FX markets because they can make decisions on whether to

buy and sell financial instruments at the scale of microseconds. Algorithms such as

CrowdForge coordinate human workers through piecework websites like Mechanical

Turk3.

For many years there have been important national security applications for

algorithms that encode and decode information – cryptography is used extensively in

both the private and the public sector. Recent novel applications of algorithms

1 Economist. 2011. Business by numbers. http://www.economist.com/node/9795140.

2 Harkin, J. 2011. How ‘infodemiology’ is quantifying you. Wired.

http://www.wired.co.uk/magazine/archive/2011/06/ideas-bank/james-harkin-infodemiology. 3 Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393.

Created by Michael Reilly December 2011

2

include interpreting fMRI scans so as to quantify the risks of psychosis4, producing

automated sports journalism5, and investigating the attribution and provenance of

paintings6. The Heritage Provider Network has offered a $3m prize to the algorithm

that can best predict when people are likely to be sent to hospital and, therefore,

help to create a better model for preventive medicine7. As much as 40% of single

people in the US are using algorithms to find a partner; and a survey in 2007 found

that 2 per cent of all marriages in the US were facilitated by the EHarmony online

dating website alone8.

The growing number of applications for algorithms has led some experts to argue

that algorithms are a general purpose technology that will revolutionise our lives9.

Implications

Policies on the use of algorithms have not evolved at the same rate as their

development. There have been recent instances of negative externalities associated

with the use of algorithms, which may be related to the lack of transparency

surrounding their application. Algorithms are rarely objective and often have a hidden

agenda.

High-frequency trading in financial markets using algorithms may improve important

outcomes such as liquidity and price efficiency but in conditions of market stress it

can also induce mayhem10. On May 6th 2010 there was a “Flash Crash” in US equity

markets that eliminated approximately $800bn of value in 5 minutes only to regain

almost all of the losses within 30 minutes. This event eroded confidence in stock

markets and was followed by several months of outflows from retail mutual funds in

the US. Algorithms may be creating ‘filter bubbles’ on the World Wide Web that

constrain our access to information with newsfeeds and recommendations that

reinforce rather than challenge our views; this form of ‘algorithmic paternalism’ could

lead to a more homogenous and less resilient society11. The internet is an intrinsic

4 Mourao-Miranda J, Reinders AA, Rocha-Rego V, Lappin J, Rondina J, Morgan C, Morgan KD,

Fearon P, Jones PB, Doody GA, Murray RM, Kapur S, Dazzan P. 2011. Individualized prediction of illness course at the first psychotic episode: a support vector machine MRI study. Psychological Medicine. 5 Lohr, S. 2011. In Case You Wondered, a Real Human Wrote This Column. New York Times.

http://www.nytimes.com/2011/09/11/business/computer-generated-articles-are-gaining-traction.html?_r=1. 6 Ju, A. 2011. Engineering professor uses tools of his trade to count Van Gogh canvas threads.

Cornell University Chronicle online. http://www.news.cornell.edu/stories/March11/JohnsonVanGogh.html. 7 Valentino-Devries, J. 2011. May the best algorithm win. Wall Street Journal.

http://online.wsj.com/article/SB10001424052748704662604576202392747278936.html#ixzz1gRPE2UGU. 8 Bialik, C. 2009. How many marriages started online? Wall Street Journal.

http://blogs.wsj.com/numbersguy/how-many-marriages-started-online-764/. 9 Chazelle, B. 2006. The Algorithm: Idiom of Modern Science.

http://www.cs.princeton.edu/~chazelle/pubs/algorithm.html. 10

Foresight. 2011. The Future of Computer Trading in Financial Markets Working Paper. 11

O’Callaghan, T. 2011. Breaking out of the internet filter bubble. New Scientist. http://www.newscientist.com/blogs/culturelab/2011/06/why-facebook-have-an-important-button.html.

Created by Michael Reilly December 2011

3

part of the economy and there are financial gains to be made from influencing search

engine results through both legal and illegal means. The practice of subverting

Google’s search engine to return a specific result, or ‘Googlebombing’, has persisted

since its inception.

Algorithms may support economies of scale for very large inputs but this can result in

a trade-off with the quality of its outputs. In the words of Albert Einstein “not

everything that can be counted counts, and not everything that counts can be

counted”. This notwithstanding, more sophisticated algorithms are being developed

that can respond to feedback and adapt like living creatures to their environment12.

Algorithms are the centre of a continuing debate on security versus privacy. In

Shenzhen, a surveillance system called ‘Golden Shield’ uses algorithms to fuse the

data provided by CCTV footage, World Wide Web trails, mobile telephone use and

GPS so as to track social unrest13. Much of the technology for this system was

actually provided by Western companies. Facial recognition software was used to

identify participants in the London riots of 2011; and new algorithms are being

developed to recognise specific types of criminal behaviour14.

Algorithms have the potential to be a driver of economic growth. The success of

Google was founded on the application of a ‘PageRank’ algorithm developed by

Larry Page and Sergey Brin at Stanford University in the 1990s, which produced

relevant search results from very large numbers of World Wide Web pages. Google

currently has a market capitalisation of over $200bn. The value of the still nascent

global mobile telephone apps market in 2010 was around $7bn and may increase to

$25bn by 201515. In a globalised world facing rising energy costs and CO2

emissions, algorithms that efficiently solve problems for a given set of inputs could

help to mitigate climate change and ameliorate the impacts on developing world

economic growth. One researcher “imagines a future in which algorithms co-ordinate

an army of human workers, physical sensors and conventional computers”16.

Using algorithms to match people to jobs has the potential to improve the flexibility of

labour markets. Daniel Kahnemann, one of the pioneers of behavioural psychology,

argues that in many instances, including job interviews, algorithms can mitigate

human cognitive biases17. On the other hand, if these algorithms are subjectively

configured they could damage the life chances of individuals and exacerbate

widening social inequality through non-linear feedback effects.

12

Economist. 2007. Of greed and ants. http://www.economist.com/node/9796508. 13

Klein, N. 2008. China’s all-seeing eye. Rolling Stone. http://www.naomiklein.org/articles/2008/05/chinas-all-seeing-eye. 14

Dillow, C. 2011. Smart CCTV System Would Use Algorithm to Zero in on Crime-Like Behaviour. Popular Science. http://www.popsci.com/technology/article/2011-08/new-cctv-system-would-use-behavior-recognition-zero-crimes. 15

Wauters, R. 2011. Report: Mobile App Market Will Be Worth $25 Billion By 2015 – Apple's Share: 20%. Tech Crunch. 16

Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393. 17

Kahnemann, D. 2001. Thinking fast and slow. Farrar, Strauss and Giroux: New York.

Created by Michael Reilly December 2011

4

Early indicators

Flash crashes in financial markets

Internet filter bubbles

Rate of growth of mobile phone apps market

Algorithm competitions

China’s ‘Golden Shield’ system of surveillance

Increase in online dating

Increase in online job matching

Drivers and Inhibitors

Drivers: Increased trade, Data deluge, Consumerism, Growing internet use, Health

costs, Open innovation, Climate Change, Emerging middle class in the developing

world, Computing power, Security, Public disorder, Distrust of experts

Inhibitors: Public mistrust, Activism, Data protection, Deglobalisation, Human

cognitive biases, Political costs of unemployment

Parallels and Precedents

Parallels: Data-intensive science, Evolutionary processes

Precedents: Google, Mobile telephone apps market, Logistic optimisation,

Cryptography