It was claimed (by those who negotiated it) to be not just “historic” but “seismic”. But the G7 tax deal may turn out to have been rather lower on the Richter scale of seismic activity than the politicians and the excited media coverage may have implied.
If you believe the formal announcements, the G7 agreement on tax is all about fairness:
Italy’s prime minister Mario Draghi called it a “historic step towards a fairer and more equitable society for our citizens”
Rishi Sunak, UK chancellor of the exchequer, said the deal would require “the largest multinational tech giants to pay their fair share of tax in the UK”
US treasury secretary, Janet Yellen, said the agreement would “ensure fairness for the middle class and working people in the US and around the world”
During Fair Tax Week, such comments in themselves may be encouraging. But whether the deal lives up to the promise of fairness is somewhat doubtful.
The deal comes in two parts. Pillar 1, applying to the largest 100 multinationals with profit margins greater than 10%, would allow 20% taxation of profits above this 10% threshold to be taxed in the country of operation. Pillar 2 would set a global minimum tax rate of 15%. Pillar 1 is about taxing new economy businesses and the US government was apparently arguing that European nations’ digital tax regimes should be abandoned the moment the G7 agreement was reached. Pillar 2 is aimed at eliminating a race to the bottom in corporate tax rates whereby countries welcome activity (or just brass plate operations) on the basis of low tax regimes.
There are already doubts about whether the deal will last at all. Caroline Lucas MP at a Fair Tax Week event talked about the deal “already being watered down” as we see noise about the UK government seeking exemption for financial services from the scope of the regime. Other countries are reportedly in discussion with their own national champions and considering what the impacts may be.
In any case, the G7 ‘deal’ is in effect nothing more than a proposal to be considered by the rest of the world. There is an ongoing OECD and G20 process on tax and this G7 deal is an attempt to make progress in those discussions, an offer made to the broader group. Whether the other nations in the process will welcome it – and whether it will be seen by them to be fair – seems very much in doubt. The G24, the forum for developing economies seeking to influence the world’s economic institutions (and which includes six of the G20 nations), was blunt in its assessment of the approach: “G-24 is of view that the proposed scope of the Pillar 1 limited to top 100 MNEs will result in smaller distributable residual profit available for market developing countries”. They do not seem any more keen on the Pillar 2 aspects.
But the G7 deal may fail even on its own terms. One of the low-taxed companies that is most often mentioned in European circles as evidence of the need for tax reform is Amazon. It appears to have been deliberately targeted in the French and UK digital taxes. Yet under the G7 agreement, Amazon would be unaffected because its margin – though it has increased steadily in recent years – remains below 6.5%, well beneath the 10% threshold. Amazon thus might face less tax outside the US under the G7 deal than it does at present.
And that 10% profit margin threshold also appears to invite structuring to reduce profits. Most obvious of these would be leverage, removing cashflows from the taxable base by switching them from profits to debt repayments. What is within the taxable base will remain a vital consideration – in this Pillar and also in Pillar 2, where the key question (alongside whether 15% is a fair level of tax) is to what tax base will that 15% rate be applied. Without more clarity on this, the 15% rate is largely meaningless.
As ever with tax, the devil is in the detail. At present, there is simply no detail in what the G7 have announced.
There appear to be as many questions as answers from this G7 ‘deal’. Whether it will amount to a fair proposal frankly seems very much in doubt.
Retail investors have too often tended to be treated badly by financial institutions. The scale of the multi-billion pound restitutions for mis-sold payment protection insurance (PPI) are strong evidence of that, as are the ongoing issues of poor advice regarding the use of pension freedoms. There are many smaller and less publicised ways in which consumers have been exploited. Is upping the ante on fairness the answer to this running sore?
The Financial Conduct Authority (FCA, the UK’s financial regulator) seems to think so. It already expects retail investors – ordinary consumers – to be treated fairly by the financial industry. This is in its Principle 6:
“Customers’ interests: A firm must pay due regard to the interests of its customers and treat them fairly”
The current Consumer Duty consultation – open until the end of July – states its aim as “we want to see firms putting themselves in their customers’ shoes, asking themselves questions such as ‘would I be happy to be treated in the way my firm treats its customers?’, or ‘would I recommend my firm’s products and services to my friends and family?’.” The core question is what outcomes would consumers fairly expect of their interaction with the financial institution, and how the institution will ensure that it consistently delivers on those fair expectations.
Stymied by their weaker bargaining position and asymmetries of information, consumers too often lose out when buying financial services. The FCA concludes that it is not fair for financial institutions to take advantage, and that greater efforts must be made by the industry to protect consumers. The FCA finds that too often institutions do in fact exploit their customers. Its description of what it calls ‘sludge practices’ – delaying tactics that place friction in the way of consumer action – is particularly effective. Sludge ensures that consumers cannot change providers as they might want, making this a far from perfectly competitive market, which in economic theory should be frictionless.
What the FCA is attempting is to set out what it means by fair treatment of consumers with more clarity and bite. Its proposals are relatively simple but if enforced effectively could have profound implications. It puts forward two options for its planned Consumer Principle: either “A firm must act to deliver good outcomes for retail clients” or “A firm must act in the best interests of retail clients”. The first seems the better, though naturally this blog would prefer ‘fair outcomes’ to ‘good’ ones – a move that would have the benefit of not removing all the burden of responsibility from the shoulders of consumers, which the FCA states to be part of its aim.
The chosen Principle would be supported by 3 underpinning behaviours, proposed as requiring financial services firms to:
Take all reasonable steps to avoid causing foreseeable harm to customers.
Take all reasonable steps to enable customers to pursue their financial objectives.
Act in good faith.
None of these seems terribly ambitious nor a particularly high bar to reach, unfortunately. It is not at all clear that delivering these behaviours would result in a delivery of the Consumer Principle. It is therefore this area where I would argue more work is needed if the proposals are to have any practical effect.
The Consumer Principle and behaviours would be supported by 4 outcomes, relating to: Communications; Products and Services; Customer Service; and Price and Value. This is odd as the FCA already has similar, and perhaps better, fairness outcome standards. For all its failings as a regulator (among them not effectively enforcing its rules on TCF, treating customers fairly), the FSA’s 6 TCF consumer outcomes continue to stand up to scrutiny. They are still in place today, and the FCA acknowledges that the new proposals would sit alongside them, though it considers disapplying them when the Consumer Principle applies. Yet these outcomes should remain core to our ambition for a fair financial industry, and perhaps would be a better articulation of what is meant in practice by the Consumer Principle:
Consumers can be confident that they are dealing with firms where the fair treatment of customers is central to the corporate culture.
Products and services marketed and sold in the retail market are designed to meet the needs of identified consumer groups and are targeted accordingly.
Consumers are provided with clear information and are kept appropriately informed before, during and after the point of sale.
Where customers receive advice, the advice is suitable and takes account of their circumstances.
Consumers are provided with products that perform as firms have led them to expect, and the associated service is both of an acceptable standard and as they have been led to expect.
Consumers do not face unreasonable post-sale barriers imposed by firms to change product, switch provider, submit a claim or make a complaint
The one way in which the FCA’s new proposals seem stronger than these is in specifically stating that firms should set prices so that they represent fair value for their target customers. The new proposals are weaker in a number of ways, not least the loss of the expectation that products should perform as consumers have been led to expect, surely a core element of fair consumer protection.
In many ways the proposals feel like an extension of the FCA’s recent Guidance on the fair treatment of vulnerable customers: “We want vulnerable consumers to experience outcomes as good as those for other consumers and receive consistently fair treatment across the firms and sectors we regulate.” Perhaps the conclusion was that expecting vulnerable customers to be treated as well as other consumers did not deliver enough protection, given the financial industry’s track record in relation to consumers as a whole.
Certainly, when one reads some of the statements in this Guidance about what is expected of financial firms, it is hard to see how these expectations should be restricted solely to situations of consumers who are particularly vulnerable. Take for example: “Embed the fair treatment of vulnerable consumers across the workforce. All relevant staff should understand how their role affects the fair treatment of vulnerable consumers.” Or: “Senior leaders should create and champion a firm culture that prioritises the fair treatment of vulnerable consumers. They should ensure that governance, processes and systems support staff to meet the needs of vulnerable customers when carrying out their role.” This is simply an articulation of what is necessary to deliver fairness, and should apply more generally.
One hopes that this will be delivered by the FCA’s current consultation. For the sake of all of us as consumers, we should hope that the bulk of the proposals survive the consultation process without being notably watered down, and that there is an intent properly to enforce the standards. The length of the process leaves scope for such dilution: there will be yet another consultation on detailed rules before the end of this year, and the new rules would be introduced only in July 2022. Let’s be clear: enough time has been spent on this already, with the original discussion paper on a duty of care to consumers dating back to July 2018. The new duty needs to be delivered.
It’s a shame that it takes a regulator’s intervention to deliver greater fairness for customers. A healthy industry should deliver that by its nature – and clearly some financial services businesses already do – but where the odds are so stacked against the ordinary consumer, in terms of asymmetries and complexity, it may be little surprise that the regulator needs to intervene. It is already beyond time for it to do so. Let’s hope change, either on the ground or by the regulator, doesn’t have to wait for yet another 12 months.
The simple aim that “consumers need to be able to trust that the range of products and services they choose from are designed to meet their needs, and offer fair value” should not be too much to ask.
to study the possible impacts of sea level rise and the implications under international law of the partial and complete inundation of state territory, or depopulation thereof, in particular of small island and low-lying states; and
to develop proposals for the progressive development of international law in relation to the possible loss of all or of parts of state territory and maritime zones due to sea level rise, including the impacts on statehood, nationality, and human rights.
Somehow there is something very powerful about the use of clinical legal language in discussing what will be multiple thousands of individual human tragedies. The unjust transition, the unfairness, of sea level rise’s destruction of entire nations comes home very clearly.
As we know, in most cases those affected in this way are among the people least responsible for climate change and least able to finance its consequences. Those who have most benefited from the economic growth that fossil fuel use has powered are unfairly imposing traumatic consequences on those who have gained limited benefits and have limited scope to cope. That is the greatest unfairness, the great injustice, of this transition.
Sea-levels have already risen around 20cm or so since 1880, with the pace increasing from around 1.4mm a year in the 20th century to 3.6mm in more recent years. About two-thirds is due to melting glaciers and ice-sheets, one-third because of the thermal expansion of the water as it heats up. This ratio used to be nearer 50:50 but melting has accelerated in the last decade. Projections from here vary, just as the path of global politics makes it impossible to predict with any certainty the path of future carbon emissions. But the lowest estimates see rises of at least 0.3m by 2100; the more extreme pathway suggests it could be 2.5m. In storms, the rise that is experienced will be still greater.
It is of course not the first time that land inhabited by humans has been subject to inundation. The number of cultures with ancient stories of great floods is ample testament to that, as is the existence of Doggerland, territories walked by our mesolithic ancestors now beneath the waters of the North Sea. But it is probably (pace Utnapishtim, Noah and the rest) the first time that humankind has known what is coming, and it is certainly the first time since we invented nation states and have created laws of the sea.
This issue of sea level rise and inundation is about nation states, but it is at its heart a human tragedy at the scale of individuals. Which is why the case of Ioane Teitiota – called by some the ‘first climate change refugee’ – is so interesting. Teitiota is a national of Kiribati, one of the Pacific island nations seen to be most at risk from sea level rise. He sought refugee status in New Zealand on the basis of the risk to his life from climate change, and having failed through the New Zealand process he took the case to the UN Human Rights Committee for adjudication in 2015. The Committee issued its ruling towards the end of 2019.
The Committee agreed that forcibly returning a person to a place where their life would be at risk due to the adverse effects of climate change has the capacity to violate their right to life, one of the core human rights that most states have agreed to uphold. However, the majority opinion of the committee found, with two dissenters, that though there were indeed significant threats these were not imminent and they were not personal to Teitiota (officially called the author of these proceedings), as the law requires.
As so often, it is among the dissenting opinions that the clearest and most powerful statements are found. Committee member Duncan Laki Muhumuza stated: “The author presents evidence, which is not disputed by either the State party or the rest of the Committee, that sea level rise in Kiribati has resulted in the scarcity of habitable space, causing life endangering violent land disputes, and in severe environmental degradation resulting in contamination of the water supply and the destruction of food crops”. He goes on to say: “In my view, the author faces a real, personal and reasonably foreseeable risk of a threat to his right to life as a result of the conditions in Kiribati. The considerable difficulty in accessing fresh water because of the environmental conditions should be enough to reach the threshold of risk, without needing to reach the point at which there is a complete lack of fresh water.”
This lack of fresh water, and its worrying health consequences, are among the most striking elements of the evidence in the case. In part they are striking because saltwater infiltration contaminating freshwater supplies is a fundamental concern, not only in terms of the health of individuals but because it is one of the indicators of an island no longer being inhabitable – something which in law (as a shorthand) renders them no longer islands at all, but rocks, with much reduced status under the UN Convention on the Law of the Sea. Any such reclassifications would be fundamental, for example: “if the island of Kapingamarangi, the southernmost island in the Federated States of Micronesia, located some 300 kilometres south of the nearest island were to be reclassified as a rock, the Federated States of Micronesia would lose more than 30,000 square nautical miles of its exclusive economic zone” according to a 2020 report from the International Law Commission.
And this issue of the extent of the exclusive economic zone really matters: fisheries exports account for 94.7% of total exports for Federated States of Micronesia, 81.9% for the Cook Islands, 73% for Palau, 61.5% for Samoa and 23.8% for Tonga. As the extent of their land above sea level shrinks, the nations’ maritime waters will increasingly become high seas and these revenue sources will disappear.
It’s for this reason, as well as tradition and sentiment, that these states are arguing that rather than maritime boundaries shifting as land gets inundated, they should remain fixed. And that they should be fixed even after the point that the entire nations are fully submerged, or otherwise become uninhabitable. That at least would secure a revenue source for the displaced nation as it tried to build lives elsewhere above the waves.
There aren’t really alternatives. Maintaining territorial waters by artificial maintenance of land above sea level is prohibitively expensive, so much so that it “raises considerations of equity and fairness”, the International Law Commission states. Even wealthy Singapore baulks at the cost. It feels the need to prepare for a local mean sea level rise of 1 metre by 2100, quite fundamental for an island where around 30% is less than 5 metres above mean sea level. A comprehensive approach to coastal defences “could cost S$100 billion or more over the next 50 to 100 years”.
Lawyers are preparing for a time when certain entire nations will no longer exist, and are proposing legal ways forward in respect of those extraordinary circumstances. The basic conclusion that they are reaching, in support of these island nations, is that though the nations may cease to exist, their rights over the relevant pieces of ocean – their territorial waters – should persist. The hope being that this would provide at least some financial protection for their impoverished nationals.
I was initiated into these discussions by a remarkable series of webinars hosted by the great British Institute of International and Comparative Law, called Rising Sea Levels: Promoting Climate Justice through International Law. Despite the best efforts of some campaigning lawyers, the lack of justice, the lack of fairness, in what is happening is what came across to me most powerfully from these four thoughtful sessions.
So, on earth day, a blog on the sea, a blog on the impending loss of earth in perhaps the most fundamental of ways. A blog on the most unjust transition of all.
Today Deliveroo has delivered, for some at least. The tech business linking hungry people to immediate gratification literally on the backs of gig economy workers has listed at a slightly reduced, though still steep price for a heavily loss-making business that offers no prospect of a dividend “in the foreseeable future”. However, the share price tumbled steeply on its first day of trading on the London market and at the time of writing is down more than 20% from its 390p launch price (still valuing the company at approaching £6 billion).
Stuttering though its reception may be, Deliveroo could be a harbinger of more to come. For its dual class share structure, with a share class with 20 votes each retained by the founder and CEO, Will Shu (who will enjoy 57.5% of the voting rights and so retain entire control), looks like a model that many are urging as part of the brave new future for the London market.
For the debate about dual class shares has been revived. Following the Hill UK Listing Review, London is the latest market apparently determined to further undermine shareholder rights in order to encourage the listing of technology companies, on the strange understanding that this is the only way that tech founders can keep control and so be willing to bring their companies to the market.
The Review proposes allowing the inclusion of dual class share structures on the premium segment of the London market, as well as just the standard level – this would mean they would be included in the main market indices, which means many investors would invest in them automatically. Yet the whole point of public markets is surely that business leaders do cede some control, and invite other owners in to participate in business success, but also to have influence on it. If they are unwilling to cede control, founders do not need to sell – or they could retain a majority of the shares and not resort to a gerrymander of the votes.
Minority shareholder rights enable investors to protect their own interests and protect their investment; they represent the common law heritage of the rule of law and certainty of ownership. They have been built up over many years and provide a basis of certainty and security of ownership, enabling confidence that it is safe for investors to trust their money to the market. They are the very foundations of market confidence. There seems a risk that this is being forgotten in a rush of blood about a rush of money.
Investors call shares equities precisely because they are supposed to give their owners equal rights – their fungibility is fundamental to liquid markets. And yet dual class share structures deliberately create unfair shares: these are inequitable equities. Indeed, it seems that investment bankers and others appear to be arguing for foisting inequities on the investing public.
Instead, we need the regulators and markets to protect investors. Caveat emptor is a fine expectation. But it doesn’t work in a world that requires employees to save for their pensions, facing individual risk, and with charge caps that (rightly) oblige providers to invest mainly through passive index funds. Forced buyers cannot beware. While such retail investors simply cannot, institutional investors ought to be able to protect themselves, yet the way that most institutions currently invest means that they too would find themselves buying whatever is included in the index.
Forced buying of these inequitable equities seems to be the point. The proponents of including dual class stocks in the indices are keen to ensure that there is a ready market for the shares from index investors, that there is an automatic demand. This mindset is built into the Hill Review. Indeed, the Review rather oddly suggests that investors need to have a discussion about what standards they wish to require for inclusion in the index, rather than automatically linking index inclusion to premium listing on the London market. This ignores the fact that this exact linkage between index inclusion and the premium listing standards has been hard fought by investors on a number of occasions over the last several years: it is deliberate, not an accident, something that institutions have sought so that they continue to enjoy the minority shareholder rights that they so value. The Hill Review also argues for dismantling the terms ‘premium’ and ‘standard’ so that standard listing does not seem a lesser expectation – again in apparent ignorance that this quality distinction was entirely deliberate.
The debate on dual class shares specifically was last had back in 2017, in the US in particular, when the IPO of Snap forced it onto the agenda. Snap listed non-voting shares, with essentially all voting rights left in the hands of its two founders. In fact, because the company said that investors should not expect dividend payments, the instruments it issued were not really shares at all but in effect warrants – simply a right to hoped-for capital appreciation.
At that time, I and other investors tried to persuade the index providers to limit the inclusion of stocks in indices according to their voting power at the company, avoiding the inclusion of non-voters altogether and cutting the weighting of lower voting shares also. Some will argue that index investors would thereby have missed out on great recent performance by various tech businesses – but who knows what the share price performance of these companies would have been but for their full inclusion in indices and the heavy fund flows in recent years into passive investment strategies? I know that a number of investors are exploring ways to tailor indices so that they are not forced buyers of unfair shares, should the rules be changed as proposed.
Deliveroo’s, and Snap’s, lack of any prospect of a dividend is not unusual among the soaring technology stocks, even those with more mature businesses and huge market shares. There is no dividend from Alphabet (Google), Amazon or Facebook, let alone Spotify or Twitter. These companies’ cashflows, when not reinvested, are used for share buybacks – much of which is needed to neutralise the impact from the dilutive effect of heavy share issuance to employees as part of their compensation. That dilution, of course, does not affect the class of shares enjoyed by the founders, only that to which outside investors are exposed.
Investors in these companies thus do not expect dividends, and are not buying shares on the expectation of cash returns from the businesses themselves. Rather, they are willing to buy these shares on the expectation that someone else will later pay them more on the market. This is usually called the greater fool theory. Though sometimes it seems hard to believe, the world does always eventually run out of fools.
Perhaps the greatest risk with the dual share class approach is that the leaders of these businesses are insulated from influence. They literally do not need to listen to anyone, at least as long as they can fend off regulation. That ability to remain cloth-eared to outside influence is seen by them as a strength – they can ignore the supposed short-termism of the markets – but it can easily be a weakness, as shown by how slow many of the tech giants have been to respond to concerns about their role in fostering hatred and anger and in damaging democracy. Some tech leaders show an unwillingness even to consider rational questioning from within their own organisations.
The Deliveroo prospectus was blunt about this:
“The Founder’s ability, while he and any Permitted Transferees hold sufficient Class B Shares, to block any resolution to remove him as a Director will mean that his position on the Board, and his influence over the decision-making of the Company through decisions made by the Board, will effectively be entrenched for so long as the Founder wishes to remain on the Board.”
One hopes that Mr Shu at least will learn to listen. The remarkable efforts of the wonderful Bureau of Investigative Journalism show that, as is true of so many gig economy workers, Deliveroo riders can earn well below the minimum wage. Concerns about this, and about the unfair dual class share structure, were enough for some institutional investors to avoid investing in the company. For the time-being, they can choose, because the company is only making a standard listing; if the Hill Review proposals are adopted, not all of them will be able to avoid making an investment. At that point, absent regulatory intervention, Mr Shu alone will have been able to choose the ongoing business model for the company, and its treatment of its riders.
Let’s hope he chooses greater fairness in relation to the workforce than he has in relation to investors.
“Fairness matters. There is something powerful about the idea that we are all in this together – that until the lockdowns can be eased for everyone, they should be eased for nobody.”
Important words, and there is much to agree with (even though given the vastly differential outcomes from the pandemic, I personally have doubts over whether it is fair to say that we are actually all in this together).
Yet in his piece in last weekend’s FT Magazine, Tim Harford (the Financial Times’ and Radio 4’s counterintuitively rather public ‘Undercover Economist’), someone with whom it is generally hard to disagree, sets this sentence up as an assertion he should dispute for professional reasons. The words before the sentence quoted above are “But not even the Undercover Economist is just an economist.” Harford is suggesting – actually saying very clearly – that fairness is not good economics.
It’s odd. We do know that economists tend to rush to theory to simplify the excessive number of variables found among the complexities of the real world – it is not by chance that the old saw “That works very well in practice, but will it work in theory?” is most usually attributed to an economist. Yet why should economists live in denial of fairness, something that is so fundamental to who we are as human beings? Why set up such a false divide between feeling the sense of fairness and the analysis done by an economist?
It’s not just Harford, of course. I once challenged Branko Milanovic, the excellent economist of inequality, on precisely this point. His simple answer to the question of why favour analysing inequality over fairness was “Inequality is quantifiable, fairness is not”. But that is precisely the point: fairness matters viscerally to us, much more than mere inequality ever could do. Its unquantifiable nature is precisely its strength. Happily, I have noticed Milanovic using the term fairness more frequently of late.
And of course Harford is using the distinction in part to make his point. As he says repeatedly in the article, “we are not selfish”. Though it does have to be admitted that sometimes we do all forget not to be; as he says towards the end of the piece: “We just need the occasional reminder to look out for each other.” It’s a shame that economics works hard to find ways to exclude such thoughts from its calculations, rather than build in the sense of fairness that might lead us more systematically to such thoughtful and less selfish mindsets. After all, fairness matters.
“we advocate for research that centers the people who stand to be adversely affected by the resulting technology, with a broad view on the possible ways that technology can affect people”
It doesn’t sound like a particularly radical statement, certainly not radical enough to warrant being sacked. It says no more than the standard ethical question, just because we can, should we? Yet this seems a radical question to some, and indeed two of the authors of the paper from which this quote comes have been sacked by their employer from top roles in ethics, seemingly for reasons closely connected to the paper (though the exact circumstances and reasoning remain a matter of dispute).
Those former employees are Timnit Gebru and Margaret Mitchell (not really masked by being named as Shmargaret Shmitchell in the list of authors of the paper), and their former employer was Google, known to investors as Alphabet. There are also three further anonymous authors of the paper, prevented by their employer (it is specifically stated as a single employer) from being named; most assume that that employer is also Alphabet/Google.
The article at the heart of the dispute is On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?[it also includes a parrot emoji as part of its title]. Stochastic Parrots is a short article, with a substantial catalogue of references, which contains few surprises. The small ethics in AI discussion group that I am part of could see nothing at all that was truly controversial, but then we are perhaps a self-selecting crowd. Stochastic Parrots was prepared for the FAccT conference – the Association for Computing Machinery’s Fairness Accountability and Transparency conference due to start this coming week (it is to be discussed on March 10th, the last day of the conference) – and has been published by Emily Bender, a University of Washington academic who is another of the co-authors. The biggest surprise for most is that it is controversial enough to spark these departures.
The article points out that AI can use significant amounts of energy, and in a world of CO2 constraints, technology firms should not regard the use of endless amounts of data to train new algorithms as entirely cost-free. Again, it notes that the current trend for supposedly ‘natural’ language programs being trained on the entire Internet encodes the existing biases and unfairnesses built into current web activity, and further extends the domination of heavy users of tech while excluding those who are not part of the Internet community, those we might call ‘unwebbed’. It is like all the world were expected to adopt a West Coast US accent and verbal mannerisms – as if!
Both these concerns are clearly true and while they may be a little embarrassing for Big Tech they cannot be truly controversial.
The article goes on to criticise the large language programs. These, it argues, do not generate meaningful outputs but merely sewn-together repetitions of language they adopt from elsewhere, joined not by meaning but through statistical habit. They are, in short, the stochastic parrots of the title. Now this is a well-articulated criticism of a large area of activity for Google and its Big Tech peers, and the description (and emoji) makes it memorable, but it is hard to see it as a sackable offence for those charged with considering ethics to note that a work in progress has distinct weaknesses and doesn’t yet live up to the promises made for it.
So I suspect that it is the last aspect of Stochastic Parrots that is the genuinely controversial part, the raising of the fundamental ethical question of just because we can, should we? The fact that it was perceived as controversial reveals much about Google and Big Tech.
As well as the quote that heads this blog, Stochastic Parrots goes a little further: “we call on the field to recognize that applications that aim to believably mimic humans bring risk of extreme harms. Work on synthetic human behavior is a bright line in ethical AI development, where downstream effects need to be understood and modeled in order to block foreseeable harm to society and different social groups.”
So, it seems that Big Tech – or at least Google/Alphabet – is unwilling to countenance the very idea that some of its activities might have consequences that amount to extreme harms to people. And they seem to be unwilling to accept the idea that they should adopt a cautious approach of assessing what those harms might be before proceeding with their activities. The irony that these comments are very close to an articulation of the ‘don’t be evil’ mantra that framed Google at its creation is clear. It is also clear that ‘don’t be evil’ seems not to be applied by Alphabet now.
Such a denial of the possibility of risk and of the need to be cautious in the face of it seems a very dangerous position for such a powerful industry to take. And it flies in the face of the readily apparent harms from AI: perhaps it is no wonder that we are seeing such harms given the state of denial in the industry.
Ethics in AI remains a new field, worryingly. The overwhelming thinking does seem to be not to ask whether we should do it just because we can, but to do it simply because we can. A recent article in the New Yorker, Who should stop unethical AI?, sets out the trauma of beginning to consider ethics in AI. One quote, from an individual who was then an academic researcher but now works at Google, seems particularly telling: “Obviously, researchers are incentivized to pretend that there are no human subjects involved, because, otherwise, things like informed consent become issues, and that’s the last thing you want to do when you’re processing data with millions of data points.”
And there is a risk that ethics in AI may never get beyond its nascent phase. As Bender points out in an interview with MIT Technology Review, the experience over this paper may hinder further discussions of ethics in AI – it will have a “chilling effect” she is reported as saying. Having AI ethics specialists working for the Big Tech firms “has been beneficial in many ways. But we end up with an ecosystem that maybe has incentives that are not the very best ones for the progress of science for the world.”
Yet we all know that we need ethics in AI. We know that Big Tech has facilitated, indeed fuelled, a polarisation in politics, and society. The algorithm is the echo chamber. Big Tech has eroded privacy and increasingly exploits its knowledge of us to direct our consumer choices. As Stochastic Parrots worries, Big Tech risks reinforcing, not unwinding, existing unfairness in our society.
So we need ethics in AI. And part of that has to be more transparency. There is a huge irony in the fact that Big Tech is among the most secretive businesses in the world. It is ironic because largely it insists on no one else having secrecy, and thrives on undermining other intellectual property while keeping its own entirely hidden. The industry has at best a lax attitude to other’s copyright, whether those others are individuals or traditional media outlets (as has been seen most recently in the Facebook and Google spat with Australia). Big Tech has an avaricious attitude to every individual in the world’s personal data, repurposing it as the basis for a uniquely targeted advertising model.
Big Tech has even played a major role, according to Rana Foroohar in her searing Don’t be Evil: The Case Against Big Tech, in undermining the protection offered by the patent regime in the US. Since the passage of the America Invents Act in 2011, she reports, the US regime has swung from one of the easiest to gain a patent and to protect it to one where such protection is much harder.
Yet Big Tech also has an obsession with maintaining the privacy of its own organisations and of hiding the workings of its core business model, the algorithm, from the outside world.
Academics are used to testing their work in the outside world: in a sense, it does not exist unless it has been peer reviewed and published openly. Similarly, patents allow their owners unique benefit from their creativity for a period of years, on the basis of being made publicly available so that the knowledge can be freely available in due course and built upon by others. Open source software, which is publicly available and so can be tested and improved by developers around the world, is generally regarded as much more robust than secretive proprietary software: for example, it is not by chance that NASA chose Linux and other open source software to be the brains of Ingenuity, the drone currently flying itself in Mars’s scanty atmosphere.
Patent law developed because many inventions are easily replicable. New products can be reverse engineered and therefore copied, and society agreed that in order to foster creativity inventors need to be able to prevent such copying for enough time to generate a return from their invention. But the price of such exclusivity is publicity: the patent is made public and thus is freely available after the exclusive period ends. It can also be subject to testing and enquiry, and can be built on for future inventive leaps. A patent attorney friend notes that one of his first questions to a client or potential client is always do they actually want a patent at all. If reverse engineering isn’t possible so a secret cannot be readily discovered by the outside world, would it not be better to keep it as a secret and never release it to the public through the patent process? Coca Cola has proved over more than a century the value of a commercial secret that is kept as a secret. It is this model that IT giants – ironically given their obsession with freely revealing all the world’s information – are using.
I have previously lauded CEO of the British Academy Hetan Shah’s idea that there should be a data commons: that our data should not be owned by any business but rather available for common usage, perhaps after a period of commercial benefit (like the time-span of a patent) to enable the garnering of the data to be funded. But perhaps we should be brave enough to go further than this: all algorithms that affect more than a minimal number of people could be made subject to independent oversight by some properly funded regulator and accessible for academic research. This would enable others to test and challenge its quality, and to identify glitches and biases – in particular, to spot potential negative impacts of algorithms on society and individuals. It could prevent harms being hidden behind the veil of commercial interest.
Shah proposes that proprietary rights to data that is gathered should expire after 5 years, after which the data should be released to a charitable corporation and made available to all for research and study, “subject to scrutiny that ensure the data are used for the common good”. This argument is particularly strong, as Shah points out, for health data and other broad data sources that governments should never lazily allow to be the everlasting property of a single corporation.
Sunlight is the best disinfectant, we know. But little sunlight is currently allowed into the world of AI. These are black boxes from which light and attention are excluded.
Transparency is a protective model that has been attempted, to some extent at least, and in Europe at least. For the heart of some of the protections offered to consumers by the EU’s General Data Protection Regulation (universally known as GDPR) are the underlying principles of lawfulness, fairness and transparency. Under transparency, data processors are expected to be open about themselves and how they will use individuals’ data – this is often expressed as the individuals’ right to be informed.
Of particular relevance is Article 22 of GDPR, which protects individuals from abuse through solely automated decision-making. Such processing is only permitted in limited circumstances, including after having obtained explicit consent. Individuals are also given rights to challenge and insist on human intervention in the decision-making, and the processor is obliged to carry out regular checks to make sure that the automated decision-making is working as intended. All very sensible and fair.
This Article formed a key element of a recent British Institute of International and Comparative Law and Kings College London (KCL) Dickson Poon School of Law event on Contesting AI Explanations. Essentially, the key message of this fascinating session was that GDPR only goes so far, and that this is not far enough. The fact that Article 22 allows challenge only of solely automated decisions limits matters as few decisions as yet are solely automated. More significantly, though, the challenge can only be made where there is enough visibility of the AI activity to allow insight and challenge. It also happens that GDPR does not require disclosure of the detail of code, but of ‘meaningful’ information, a concept which is largely untested.
The comments of Robin Allen QC, of Cloisters chambers and co-founder of the AI Law Hub, were typical. He noted that transparency is helpful but often the use of AI is not visible so that it cannot be challenged. Take his example of a CV uploaded to the Internet; this may lead to interviews and job offers, but the individual will be unaware of all the occasions when an algorithm has screened them out for what might be arbitrary or wholly unacceptable reasons. Unless it is visible that this has happened, all current protections count for nothing.
The conclusion of most participants was therefore that GDPR does not in practice provide the protections that we might hope for. Rather, standard public law challenges and remedies are more often sought – such as judicial review. Some argue that this is beginning to amount to a presumption of disclosure for government and public bodies. However of course, such challenges are only possible where the AI is employed by public bodies, and do not exist in relation to the private sector.
We are left not just with holes in the application of the regulation, but with certain key conversations not even begun. The comments of Perry Keller, a reader in media and information law at KCL, were particularly striking. “We have not had enough of a debate about these harms,” he said. He noted that as well as the individual harm, there is a collective harm, a societal one, though GDPR considers only the individual aspects. “The excessive focus on the individual is not allowing us to focus on the societal one.” As Stochastic Parrots invites us to, we need to consider foreseeable harms, and explore whether their existence makes the new technology worthwhile.
We seem to have come a long way from don’t be evil. I can find no better ending than the first paragraph of Don’t be Evil: The Case Against Big Tech:
“’Don’t be evil’ is the famous first line of Google’s original Code of Conduct, what seems today like a quaint relic of the company’s early days, when the crayon colors of the Google logo still conveyed the cheerful, idealistic spirit of the enterprise. How long ago that feels. Of course, it would be unfair to accuse Google of being actively evil. But evil is as evil does, and some of the things that Google and other Big Tech firms have done in recent years have not been very nice.”
One of the main attractions of the concept of fairness is that it requires judgement, it is qualitative not quantitative. It is not reduceable to a number, and hence differs from inequality, the hard-edged term that tends to be favoured by most economists – because they mostly like to have a calculation and favour quantification. Measuring it doesn’t make inequality the better measure: the simple fact is, humankind can stand a good deal of inequality if it is fair; if it isn’t fair, equality is deeply unsettling.
Fairness is a clear sense that is innate in humans and crucial to our well-being. But does that make it an art?
It’s a question forced upon us by the title of an excellent new book The Art of Fairness: The Power of Decency in a World Turned Mean, by David Bodanis (author of bestseller e=mc2), published late last year. I am happy to recommend the book.
The Art of Fairness is alive with stories, finding fair and decent behaviour around the world and across history, as well as finding its opposites. I am now sorely tempted to do further research on the Bounty’s Captain Bligh, who appears to have swung between periods as a generous and inspirational leader and as a vicious monster. And I have to admit favouring any book that has a chapter on the experience of the second world war in India and Burma, and the leadership of the great and largely forgotten General Slim, under whom one of my grandfathers served (in a very junior capacity). It remains an oddity that the British obsession with that war almost entirely neglects the war in Asia.
From a business reader’s perspective, the two key chapters are one that contrasts Satya Nadella’s oversight of Microsoft with that of his predecessor Steve Ballmer, and another that compares the leadership that delivered the remarkably efficient construction of the Empire State Building with the venality that took apart an airline business, Eastern Airlines, apparently for personal greed. For me it felt that the Microsoft story held few surprises, as the downs and ups of that company in the years since Bill Gates’s departure have been played out in public, and the weaknesses of Ballmer seemed all too apparent to the outside world in his mostly mocked performances seeking to inspire staff, where he appeared unable to persuade even himself.
The Empire State Building and Eastern Airlines stories, while having some familiarities, are less known and so it felt they had more to teach. The Empire State story focuses on lead contractor Paul Starrett, who had what we might now call a strong stakeholder approach to business. In particular, he saw significant value in paying workers above the standard rate and offering them better treatment, not least making decent food available to them with subsidised restaurants created on a number of floors as the building rose. He was no soft touch: the iron fist within this velvet glove was a rigorous audit process, for people, tools and materials – so that no one was tempted to skim additional wages for non-existent workers or scam materials away from the building site. Both were problems typical of the construction business then as now. Starrett was rewarded by the construction being swifter and more efficient than rival sites even given the scale of the ambition that the building represented. It remained the tallest building in the world for decades.
The Eastern Airlines story is a contrast to this. Frank Lorenzo became CEO of the successful business in 1986. He took over a thriving and successful operation with good stakeholder relationships. And he proceeded to damage each of those relationships in order to make more money. He cut salaries among the workforce. He sold off assets, including the innovative reservations system that the business had carefully built. It was worse than just selling assets: often these were related party transactions that benefited him personally. He tried to divide and rule the workforce, trigger a strike among ground crew and mechanics and enter bankruptcy to offload liabilities. His plans failed and he was ousted, but not before having so damaged the business that it was split up among a number of acquirers.
One of the familiarities of this Eastern Airlines story is that it feels like a good deal of current business behaviour: too often it seems companies now are being overloaded with debt and, if they do not succeed quickly, deliberately put into bankruptcy to enable a phoenix business to emerge unburdened by former liabilities. Bodanis, in his generous discussion of further reading materials and explorations that provides a final two dozen pages of the book, makes an explicit link between Lorenzo’s behaviour and that of the worst activities of private equity. This unfortunately feels a fair comparison.
Each of these chapters in the first half of the book is used as evidence for what Bodanis calls koans, Zen Buddhism’s brief, sometimes apparently contradictory lessons. Examples are ‘Listen, without ego’ and ‘Give, but audit’. Collectively, these koans build through the book to form Bodanis’s understanding of what the art of fairness amounts to.
The second half of the book is given over to a contrast between the always flawed but ever-determined US President Franklin Delano Roosevelt, and the flawed and irresolute Nazi propagandist Joseph Goebbels. Though both feel like stories we have heard before, Bodanis manages to vest each man’s history with fresh insight – framed by the koans evidenced by earlier chapters – helping us to learn anew.
While Bodanis does not explicitly draw links between his discussion of Goebbels’ big lies and current political tendencies, those messages are clear between the lines (and between the lies). Reading the book over the weeks either side of this January’s insurrection in Washington DC, as I did, it felt disturbingly prescient. Further, the discussion of FDR feels particularly important given that President Biden, indeed all of us, seem to be some form of new New Dealers now, in our determination to Build Back Better – or better, to Build Back Fairer.
Actually, it seems to me that Bodanis’s book is less about fairness than it is about the key word in the subtitle, decency. When you read the book, it often seems that the words decency and decent appear more often than fair and fairness, and certainly when you hear Bodanis speak about the book it is decency that seems to keep coming up. It seems an old-fashioned term, but it is none the worse for that; Bodanis is perhaps challenging us to refashion it for our modern era. That would be no bad thing, and I think I would agree that decency at least is an art.
Fairness, actually, is all around it seems this year. I know that my RAS (reticular activating system) is particularly alert to the term, but it isn’t just that: it’s clear that the Covid-19 crisis has brought many to the conclusion that fairness matters, and that focusing on fairness is more worthwhile than on the narrower numerical concern of inequality.
A key example is the apparent rebranding of the Rethink series on Radio 4 – which started with the Pope’s comments on poverty – to be Rethink Fairness. As host Amol Rajan explained, fairness was a term that just kept on coming up in the Rethink commentaries. This led to a series of five special discussions over last week, considering Rethink Fairness on: wealth, regions, education, health and generations.
The most striking part of the series was that both the education and health discussions focused less on disparities in education and health outcomes in themselves, and much more on those disparities as symptoms of broader unfairness in society. In each case, the suggested responses seemed more to do with addressing poverty and a lack of opportunity, particularly in some regions of the country, than with issues in education and health specifically. Both the health and education systems, it was noted, are too often asked to backfill for other failings.
Typical was this exhortation from the excellent Sammy Wright, vice principal of Southmoor Academy in Sunderland, and a member of Social Mobility Commission:
“We have to support people outside school, we have to have an adequate welfare safety net, we have to have services and we have to have people who can provide that whole wrap-around culture for struggling families.”
Similarly, “Good health is not about the health service, the health service is sorting that out when it’s gone wrong,” said the wonderful Dame Julie Moore, former nurse and recently retired chief executive of University Hospitals Birmingham NHS Foundation Trust. “The health service ends up sometimes picking up the price for the basic inequality in income, in housing, in circumstances.” She went further: “Sometimes the health service is there to fulfil a need that should not have arisen; we can’t expect the health service to solve all these problems.”
“Most of the difference that we see between children can actually be explained by their family background, their family income, factors outside of their school,” agreed Anna Vignoles, professor of education at the University of Cambridge. “It’s not surprising when you think about the influence of families and the amount of resource that they have, that they are making more of a difference than schools.”
Professor Sir Michael Marmot identified four mechanisms that in his view link our poor health pre-pandemic with our poor health during the pandemic. These four mechanisms are explored in detail in Build Back Fairer, last month’s report he co-authored on lessons to be learned from Covid-19 and reflecting on his decade of recommendations to the government for reform. Again, these are issues that go much broader than health as such:
Quality of governance and political culture
Widening social and economic inequalities
Investment in public services
Health had stopped improving prior to the pandemic, which increased the risk of lethality during the pandemic
Vignoles put the call for action in similarly broad terms: “What we’re really facing pre-Covid, post-Covid, during Covid, is economic inequality in our society that also is mirrored in our education system and so if we’re serious about changing that we do have to work on the economic inequality that sits outside of schools first, or at least at the same time, because without tackling that we’re always going to face a problem in our education system.”
Furthermore though, it was clear that, just as many of the problems faced by the health and education systems arise from broader societal fairness issues, it is also true that addressing some of the health and education challenges – and investing more smartly in them – gives an opportunity to address future unfairness and create a nation in which we would all prefer to live. Getting these issues right can help with society as a whole, not just education and health in themselves. With so much focus on infrastructure spending, a key question is what infrastructure will actually matter for the future we want. There was some cynicism as to whether this should be physical transport links, and the suggestion that rather, it should be something smarter to suit the 21st century.
At its heart was a call not to see infrastructure just as concrete and steel. “View health as an investment in the nation’s infrastructure,” said Moore. “It’s a necessary part of our infrastructure, and it’s a really valuable investment on every front.”
Actually, the clearest articulation of the need for skills and investment in people and fairness rather than hard infrastructure came in the programme on Regions: “It’s not about faster trains between cities in the North of England, which would be used by richer people, it’s about trying to give people further down the ladder the skills that they need to get on in the world of work, and then also make that place a more attractive place to come and do business as well,” explained Paul Swinney, director of policy and research for the Centre for Cities.
This argument for a heavy investment in skills and training – and scope for retraining for those whose former jobs are no longer as valued – becomes more powerful when one considers the arguments of Harvard Professor Robert Putnam, of Bowling Alone fame. Putnam suggests that the rise of American public schooling in the progressive era (the years around 1900), which essentially provided a basic education for the entire population for the first time, readied the skills of its people and helped pump-prime the economy, providing the foundation for the economic success of the American century.
If we aim to have a successful next century, we need to prepare our young people to succeed in it.
There were six areas of recommendations in the original Marmot Review, Fair Society, Healthy Lives, which again go far beyond a narrow understanding of health. They are a good basic set of ambitions for a nation to set itself:
Give every child the best start in life.
Enable all children, young people and adults to maximise their capabilities and have control over their lives.
Create fair employment and good work for all.
Ensure a healthy standard of living for all.
Create and develop healthy and sustainable places and communities.
Strengthen the role and impact of ill health prevention.
It is Twelfth Night, the last of the Christmas festivities and by tradition when the decorations should be taken down. But Britain’s CEOs may be tempted to leave their decorations in place and risk the threat of the goblins that apparently invites, because today is also the day when top executive pay exceeds that of the average worker — named High Pay Day by my thoughtful friends at the High Pay Centre as a way of drawing attention to these differentials in pay between CEOs and the average full-time employee.
The bad news for FTSE 100 CEOs is that High Pay moment is a full hour later than last year as it now takes a CEO 34 hours of work to surpass average annual pay rather than the 33 hours it took last year. It is not that CEO pay has reduced; it has stayed relatively unchanged. Rather, average worker pay (perhaps counter-intuitively in these times of furloughing and Covid-driven unemployment) has risen a little. This pay ratio is around 120:1. High Pay moment this year is at around 5.30, about the time this blog is being posted.
Companies are now required to disclose their own internal pay ratios, that of CEO to the company’s average-paid worker in the UK (note ‘average-paid worker’, not ‘average worker’; no worker is average), and also a comparison of CEO pay to the lower and upper quartile worker in terms of pay. These ratios were analysed in another recent publication by the High Pay Centre, at whose launch late last month I had the privilege of being invited to speak.
The results are striking, though the headline averages are lower than the 120:1 marked today:
For the FTSE 100, the median CEO/median ratio is 73:1, and the median CEO/lower quartile ratio is 109:1
For the FTSE 350 as a whole, the median CEO/median employee pay ratio is 53:1, and the median CEO/lower quartile employee ratio is 71:1
These averages mask some remarkable variation, with the highest ratio, 2605:1, being that at Ocado, whose Tim Steiner enjoyed (and one assumes continues to enjoy) an extraordinary one-off level of pay of £58 million. Another seven companies saw CEO:median paid employee ratios over 200:1 (JD Sports, Tesco, Watches of Switzerland, GVC Holdings, Morrisons, CRH and WH Smith). The variation is shown a little in this chart:
It is clear from this that the biggest driver of the largest ratios is less the CEO’s individual pay (setting aside the unusual case of Mr Steiner) and more the pay of the workforce. That is still starker when the ratios in comparison with lower quartile workers by pay are considered. At Ocado, this ratio is 2820:1, at BP it is 543:1 and at Tesco it is 355:1. There are some sectors that are disproportionately represented among those with high ratios and notably low levels of pay — particularly retail and hospitality [Note that while AB Foods is classified as a ‘consumer goods’ company, its lower quartile paid workers are likely to be employed at its Primark subsidiary]:
In 11 cases, the High Pay Centre notes, the revealed lower quartile thresholds are below what would be earned from a 35-hour week paid at the £9.30 real living wage as calculated by the Living Wage Foundation (note, this is not the statutory minimum wage – no allegation of illegality is being made). We should note that in part these sectors being notably lower paid is an output of long-standing sexism in pay: retail and hospitality have traditionally been seen predominantly as women’s work and so have never been paid as well as other sectors. That is a clear unfairness.
And it is important to note that, as low as the pay revealed by these lower quartile figures is, fully a quarter of the workforces of each company is paid less.
When we focus on the lowest paid, it’s important to note as the High Pay Centre does that this data includes only those people that companies actually employ. HPC argues that outsourced workers should be included in pay ratios; I’m not convinced that this is practical, but I think it is realistic to ask for greater clarity of disclosure in business model reporting of all the additional workforce on which the company is dependent that are not necessarily staff members. More general enhancements to workforce discussions in business model reporting would also be helpful because these pay ratio disclosures include only UK workers. For some companies that will cover the whole business, while for others it is only a very small portion. Business model reporting can be made much more transparent and informative, and these sorts of disclosures would help towards a better understanding of how each company sits within its broader economic environment.
The High Pay Centre also calls for a number of ways in which the accountability of companies, and particularly remuneration committees, for pay decisions — including outcomes like pay ratios — should be enhanced. Personally, I have long said that there should be greater formal accountability of remuneration committees to staff members, in particular a formal presentation annually by the remuneration chair to employee representatives. I think that could prove salutary, and be a strong route for understanding staff perspectives that could then be fed into discussions in the boardroom. It might also help alleviate some of the ongoing suspicions that CEOs and other executives have too much influence on their own pay.
CEOs may be tempted to leave their decorations up. I suspect the rest of us will be moved to take ours down.
Technology will not of itself generate fairness. Merely asserting that the independence and lack of human intervention in a process removes bias isn’t true — indeed it may require human intervention to overturn failures to be fair. After all, fairness requires effort. This is the lesson that ought to be being learned in the world of algorithms, but unfortunately too often the creators and users of algorithms seem to prefer to state that they will be fair without actually carrying out the work effort that will deliver fairness in practice.
No parent of secondary school age children in England needs to be told that algorithms are not inherently fair. The debacle this summer of the supposedly standardised grades for students leaving school, who had lost the opportunity actually to sit their final exams because of Covid-19, brought the potential for technological unfairness fully into view.
The key political decision was that there should not be significant grade inflation arising from the use of teachers’ assessed grades. The way that was operationalised created the unfairness, particularly the view that results awarded should be in line with schools’ historic performance (startlingly, fully 40% of teacher assessed grades were downgraded). The unfairness arose despite regulator Ofqual’s explicit aim being to deliver fairness for students. For example, the regulator required less standardisation of smaller classes than larger ‘cohorts’, because adjustments in smaller classes might appear more arbitrary. But a key consequence of this was that fee-paying private schools (which market themselves in part based on offering more esoteric subjects and smaller class sizes) faced fewer downgrades than free state schools. That looked like unacceptable unfairness to many.
Furthermore, the tying of performance to historic grades anchored exceptional years, or improving schools, to their less successful heritage. Fundamentally in a system that has to be all about what the individual her- or him-self deserves, an automated process for determining performance is always going to create some degree of unfairness at a granular level — especially so when the algorithm directly limited the number of certain grades that could be awarded to a particular school, condemning some individuals to multiple downgrades from those their teachers predicted for them.
The scale and number of these apparent individual unfairnesses in the end led to a government decision to abandon the algorithm and simply revert to those teacher-assessed grades. No doubt there is some unfairness in that, but at least it appears less systematic unfairness.
So even where the aim explicitly is fairness, that is not necessarily what will be delivered if the design of the algorithm is faulty and it is subject to inadequate testing.
The scope for unfairness is much greater in cases where fairness is not the aim being sought. And where that application is to areas as sensitive as law enforcement and incarceration (as it increasingly is) then unfairness creates very fundamental problems.
Take facial recognition, a technology increasingly used particularly at border crossings to identify those who can be allowed into a country and those who should be excluded. In theory perhaps the use of technology should remove bias. In practice, however, the technology is racist.
A remarkable December 2019 study by the US’s National Institute of Standards and Technology of more than 100 leading facial recognition systems, including ones from the most famous names in technology, reveals that even the best facial recognition systems misidentify black people at rates 5-10 times higher than the misidentification of white people. This chart is but one sample of the striking data the analysis reveals:
As is obvious from this chart, not only are the facial recognition systems racist, they are also sexist. They are not as sexist as they are racist, but the algorithms are consistently worse at identifying women correctly. They also happen to be worse at identifying older people and children, getting worse and worse at the increasing extremes of age.
There is a consistent reason for these general errors: the development of the AI has been focused on one demographic, trained on a set of data. In most cases, that has predominantly been adult men from the locality of the developer. There is a strong home bias in the datasets used to develop the algorithms, meaning that there is a contrast between the data from most of the systems with the results from those developed in China: “with a number of algorithms developed in China this effect is reversed, with low false positive rates on East Asian faces”.
This reveals a general challenge about AI. There is a sense in which the creation of algorithms is based on the creators’ existing understanding of the world around them. Given that our world is riddled with unfairness, it is not surprising that unfairness is an outcome of their work — unless they deliver actions that actively work to remove the unfairness.
As Hetan Shah, then Executive Director of the Royal Statistical Society, now CEO of the British Academy (and also vice-chair of the Ada Lovelace Institute and chair of the Friends Provident Foundation), sets out in an excellent brief paper on Algorithmic Accountability:
“Algorithms for the most part are reflecting back the bias in our own world. A large part of ‘bias’ in algorithms comes from the data they are trained upon.”
It therefore follows that we need to lean actively against existing bias and prejudice in order to make algorithms more fair. Among the steps which would assist in this, Shah argues, are enhancing the diversity of the technology workforce, ethics training and a professional code of conduct for all data scientists, and new deliberative bodies to help set standards, and build ethics capacity.
The tech industry is alive to at least some of these issues — or sounds like it is. The language of fairness is used freely in its discussions of its own work. For example, in January 2018 Microsoft published its “ethical principles” for AI, which start with ‘fairness’. In May of that same year, Facebook announced its “commitment to the ethical development and deployment of AI” which includes a tool that it claims can systematically seek out and identify bias in algorithms, called ‘Fairness Flow’. That September saw IBM announce its ‘AI Fairness 360’, which is similarly designed to “check for unwanted bias”. The word appears prominently among the stated intents of Alphabet (Google) and Amazon.
The firms have also freely sponsored academic and other programmes supporting the development of fairness in AI, such as the Algorithmic Fairness and Opacity Group at Berkeley, that same University’s Center for Technology, Society & Policy (which has a project on Just Algorithms: Fairness, Transparency, and Justice), and the US National Science Foundation Program on Fairness in Artificial Intelligence.
Rodrigo Ochigame, who has worked within at least one of these organisations, is cynical about this: “Today’s champions of ‘algorithmic fairness’, sometimes sponsored by Silicon Valley firms, tend to frame discrimination and injustice as reducible to the stated distinction between the optimization of utility and other mathematical criteria”. His charming short article on the history of algorithmic fairness travels far — from 17th century English real property law, through 19th century US insurance contracts — and makes it clear that delivering fairness is not, and never has been, as simple as the champions of technology want it to seem.
In an earlier, excoriating article, Ochigame argues that the current efforts are little more than active lobbying for limited regulation. “Silicon Valley’s vigorous promotion of ‘ethical AI’ has constituted a strategic lobbying effort, one that has enrolled academia to legitimize itself,” he argues. This lobbying effort “was aligned strategically with a Silicon Valley effort seeking to avoid legally enforceable restrictions of controversial technologies”.
Certainly, despite all the language of fairness, the example of the facial recognition algorithms suggests that there is still a long way to go to deliver fairness in practice. Ochigame identifies a number of other uses of algorithms in law enforcement and the criminal justice system that appear in practice to be delivering clear unfairness.
The US’s Brookings Institute shares these concerns, highlighting a series of biased outcomes from algorithms, including gender bias in an Amazon recruitment algorithm, and bias in the online advertisements shown to online searchers, reinforcing existing prejudice and closing the door to difference. Just as with the facial recognition algorithms, the approach of building from what already exists or is near at hand leads to algorithms “replicating and even amplifying human biases, particularly those affecting protected groups”. This tendency, Brookings argues, is particularly concerning where technology is used in the criminal justice environment, to identify ‘potential’ criminals or sites of crime, or to help determine sentence lengths or the availability of bail. The solution, they believe, is not a technology one:
“companies and other operators of algorithms must be aware that there is no simple metric to measure fairness that a software engineer can apply, especially in the design of algorithms and the determination of the appropriate trade-offs between accuracy and fairness. Fairness is a human, not a mathematical, determination, grounded in shared ethical beliefs. Thus, algorithmic decisions that may have a serious consequence for people will require human involvement.”
Indeed, it seems as though algorithms risk becoming in some way a technological version of the sus law, the 1970s use of long-standing laws enabling the UK police to stop and question any individual suspected of intent to commit an arrestable offence. The sus law became a vehicle for blatant prejudice and harassment of ethnic minorities, and was repealed in 1981 following a series of race riots. The crudeness of social profiling by the police in the 1970s — even senior officers claiming that disproportionate targeting of black people was justified because they were over-represented among robbery and violence offenders — is reflected in much of the algorithmic modelling now being applied. Using technology does not make the work more fair or justifiable. It is just prejudice in another, perhaps less accountable, form.
The use of algorithms in the business arena has also been found to introduce unfairness. A recent study considers the German retail petrol (gasoline) market, which started to adopt algorithmic pricing in a significant way in 2017. Essentially, this study provides evidence that such pricing algorithms can drive anti-competitive behaviour: adoption of algorithms increases margins by 9% on average, even though non-monopoly markets show no margin enhancement at all. Looking just at duopoly markets, the researchers conclude: “we find that market-level margins do not change when only one of the two stations adopts, but increase by 28% in markets where both do.” The consequence is “a significant effect on competition”. It should probably come as no surprise that pricing algorithms push prices up; the question is whether this can be justified or whether it just amounts to a technological cover placed over the gouging of customers.
So fairness continues to be a victim in practice of the use of at least some algorithms, despite the apparent efforts by the technology giants to promote and assure the delivery of fairness. And rather than coming from the organisations they fund, in practice the best outlines of what is really necessary to deliver fairness in artificial intelligence — at least that I have been able to identify — come from other sources.
Of course the EU’s High-Level Expert Group on AI includes individuals from many of the leading technology firms, but the group is wider than that. In April 2019, it presented its Ethics Guidelines for Trustworthy Artificial Intelligence. Among the seven key requirements that AI systems need to meet to be deemed trustworthy, is the fifth, Diversity, non-discrimination and fairness. This states: “Unfair bias must be avoided, as it could could have multiple negative implications, from the marginalization of vulnerable groups, to the exacerbation of prejudice and discrimination. Fostering diversity, AI systems should be accessible to all, regardless of any disability, and involve relevant stakeholders throughout their entire life circle.”
In July 2020, the High-Level Expert Group took this work further and presented its final Assessment List for Trustworthy Artificial Intelligence. This provides questions for assessing whether AI delivers on the trustworthiness measure that the Group set. Among the questions under Diversity, non-discrimination and fairness are:
Is your definition of fairness commonly used and implemented in any phase of the process of setting up the AI system?
Did you consider other definitions of fairness before choosing this one?
Did you consult with the impacted communities about the correct definition of fairness, i.e. representatives of elderly persons or persons with disabilities?
Did you ensure a quantitative analysis or metrics to measure and test the applied definition of fairness?
Did you establish mechanisms to ensure fairness in your AI system?
This looks very similar to the simple model that Hetan Shah suggests for reducing and perhaps removing bias: (1) pilot to check for bias in multiple ways, with different datasets; (2) offertransparency to enable external scrutiny; (3) monitor outcomes for differential impacts; (4) provide a right to challenge and seek redress; and (5) enable enhancement through goodgovernance (e.g. through independent oversight). In the context he is considering, of public data being used in algorithms by private companies, he also suggests (6) use of negotiating strength by the public sector as monopoly owner of data which private sector rivals are competing for.
Brookings reaches similar conclusions, proposing the idea of ‘Algorithmic Hygiene’, “which identifies some specific causes of biases and employs best practices to identify and mitigate them”. It is unsurprising therefore that one of their key recommendations for enhancing fairness is to: “Increase human involvement in the design and monitoring of algorithms”.
As an aside, this danger that arises from assuming our starting point needs to be the world as it is and as we understand it is consistent with other concerns — as Brookings notes, the biases in search arise from similar errors. I worry about algorithms constraining our horizons and feeding confirmation biases. In particular, I worry about the role of algorithms in defining search and so much of our online lives, of cookies constraining what we see. This removes the joy of serendipity and happenstance. A friend complains that using any search engine other than his usual one of choice (you can guess which) means that he sees all sorts of untargeted, irrelevant material. Perhaps his approach is more efficient by fractions of seconds, but inefficiency occasionally has its benefits and I suspect we lose much by seeking to avoid it — or by handing the power of happenstance over to a machine which tends to want to confirm our certainties rather than enable us to happen upon challenge and difference. The algorithm is the echo chamber.
Algorithms, like any human technology, are neither fair nor unfair. They are not automatically fair, as their IT proponents would like us to believe; nor are they automatically unfair as many campaigners seem ready to argue. Like any human technology, they reflect the prejudices and understandings of their creators and the society in which they are created. We live in an unfair world and so most technology, if it does not actively lean against unfairness, will be unfair. There needs to be much more work to ensure that algorithms operate fairly — merely being technology does not deliver that, without much, much more. A technology industry that fails fully to engage with this challenge will not build the necessary trust and will see confidence in algorithmic technology erode. Current unfairnesses suggest that the industry has much more work to do.