Otago Law Review
|
|
Home
| Databases
| WorldLII
| Search
| Feedback
Otago Law Review |
|
Last Updated: 8 December 2019
Colin Gavaghan*
In June 2014, the attention of media – both traditional and social – was captured by images of ‘anti-homeless spikes’ outside a residential block in London.1 The images elicited an instant and powerful response. Welsh Assembly Member Jeremy Miles, for instance, Tweeted “If you want to see how callous society is becoming look at these anti-homeless spikes”, while Boris Johnson, at that time Mayor of London, described the spikes as “ugly, self-defeating & stupid.”
In May of this year, it transpired that New Zealand had its own version, when RNZ reported that some Auckland businesses had installed anti-homeless ‘sprinkler systems’ in their doorways.2 The initiative elicited a similar response, with Auckland City Councillor Cathy Casey describing it as “an inhumane and draconian response to a multi-faceted problem”.3
While the brutal-looking studs and sprinkler system were particularly unsubtle examples, the trend to which they belonged was already well established. The imposition of prohibitions and restrictions through design features, rather than more traditional orders and requests, is a familiar feature of urban landscapes, even if it is not invariably recognised as such. Elsewhere in London, for example, the so-called “Camden Bench”4 had been in use for some time. This innocuous-looking feature, while looking less overtly draconian, is thought to be every bit as effective at deterring rough sleepers. Furthermore, it has features specifically
* Colin Gavaghan is the first holder of the New Zealand Law Foundation
Chair in Law and Emerging Technologies at the University of Otago. He holds an LLB (Hons) and PhD from the University of Glasgow. He also lectures in medical and criminal law. This article is an expanded version of a keynote talk given at the Annual Conference of the Otago and Southland branches of New Zealand Law Society, November 2016. It has also been presented at the Faculty of Law, University of Ottawa, and the University of Newcastle La w School. I am grateful to all of those audiences, and to the students of LAWS428, for their insightful questions and suggestions which have considerably informed my thinking on this topic.
3 Ibid.
4 Frank Swain “Secret city design tricks manipulate your behaviour”
BBC (online ed, UK, 2 December 2013) <www.bbc.com/future/ story/20131202-dirty-tricks-of-city-design>.
designed – by the aptly-monickered Design Against Crime Research Centre – to prevent graffiti, skateboarding and littering.
The use of architecture to impose rules and restrictions is far from new. Yet there is reason to believe that the proportion of our lives subject to this form of regulation is increasing, and furthermore, that the kinds of values being imposed in this form is undergoing something of a transformation. Furthermore, as these examples show, it is certainly not the case that the state enjoys a monopoly on this sort of ‘regulation’.5 Although much of my focus here will be on the technological enforcement of the criminal law, several of the examples that I will discuss in this article are likely to be employed either by private actors, or in state-private collaboration of various forms.
A growing body of literature has addressed this phenomenon, with some going so far as to take seriously the suggestion that ‘techno‑regulation’ will displace traditional law as the predominant mode of rule enforcement.6 This tendency towards direct enforcement has given rise to a new vocabulary, including terms such as:
5 There may still be those who insist that ‘regulation’ applies only to actions
of the state, or its delegated bodies. For the purposes of this article, I employ a more expansive (if somewhat fuzzy) definition, of the sort made popular by Julia Black: “regulation is the sustained and focused attempt to alter the behaviour of others according to defined standards or purposes with the intention of producing a broadly identified outcome or outcomes, which may involve mechanisms of standard-setting, information-gathering and behaviour-modification” in Julia Black “Critical Reflections on Regulation” (2002) 27 Australian Journal of Legal Philosophy 1 at 26. See also Roger Brownsword and Morag Goodwin Law and the Technologies of the Twenty-first Century (Cambridge University Press, Cambridge, 2012) at 25.
In this article, I will use the term techno-regulatory mechanisms (TRMs), but whatever the preferred terminology, this trend has, to say the least, attracted considerable attention and concern – and frequent opposition – in academic and popular literature. Yet in many ways, its attraction can seem obvious. The lure of technologies of perfect enforcement can be appealing to the public, and irresistible to regulators.
In this article, it is my intention to investigate both the phenomenon of ‘techno-regulation’, and the concerns it has generated. In the first part, I will explore some of the areas where techno-regulation is already established, and some where it may be becoming more prevalent. As I will explain, it is not only in the built architecture of our cities that it is taking place. A great many other aspects of our lives are regulated in this way – not by rules and instructions that we are called upon to obey, but by structures and devices that render non-compliance practically or literally impossible. In addition to the geography of urban areas, I will focus on two other domains: road traffic and online interactions.
In the second part, I will examine some of the arguments and concerns that have been expressed about these techno-regulatory mechanisms (TRMs). Why should we care if society is so ordered that we lose the opportunity to act badly? If crime becomes impossible, is that really something to lament? Insofar as they generate feelings of unease, are these feelings rooted in anything normatively substantive, or are they really just manifestations of techno- or neo-phobia? I will suggest that, while some concerns may be somewhat overstated, there are legitimate grounds for caution and apprehension about some others.
In the third and concluding part, I will propose a checklist for good techno-regulation, a set of questions we might ask about particular initiatives that might help provide a guide as to whether and to what extent they should concern us. These questions are not intended to be wholly dispositive of the question of whether a particular initiative is acceptable. That will depend on a whole range of other issues, including cost-benefit analyses, and the values underlying the initiative. I do not deal with these concerns in this article, because they are common to all regulatory initiatives. Rather, my focus here is on the distinct or novel features of techno-regulation, and my task in this final section to determine whether those features are themselves a cause for concern.
11 Ian Kerr “Digital Locks and the Automation of Virtue” in Michael Geist
(ed) From “Radical Extremism” to “Balanced Copyright”: Canadian Copyright and the Digital Agenda (Irwin Law, Toronto, 2010) 247 at 249.
1 The rise of techno-regulation
For as long as there have been doors that lock, there have been attempts to impose rules directly rather than through the enforcement and deterrent functions of courts and the penal system. In the modern era, mundane and established examples abound: from speed bumps13 to ticket barriers at railway stations and sports stadiums, to home and car alarms, even wheel-locks on shopping trolleys.14
In that sense, the phenomenon I have discussed here is as far from new as can be. For a number of reasons, though, it may be that the role of techno-regulatory mechanisms, impossibility structures, ‘normative architecture’ and such like is growing. A number of reasons may be suggested for this shift.
A larger proportion of humanity now lives in cities than ever before – almost certainly more than half15 – and as such, almost constantly has to navigate a terrain built by other people. Not only is global car ownership also at record levels, but semi-autonomous car technology is increasingly a reality rather than a trope of science fiction. And an increasing proportion of our social and economic activity now takes place via the internet, in a manner that – as I discuss below – renders it particularly susceptible to direct control.
There has also been a well-documented shift in the objectives of law‑makers, and perhaps in the values and expectations of society; a shift towards risk aversion and characterized in part by demands that risks are prevented and complaints when they are not. This shift has, for example, been perceived in an increased concern with preventing rather than simply punishing crime. As David Garland expressed it: “Today, there is a new and urgent emphasis upon the need for security, the containment of danger, the identification and management of any kind of risk.”16
Much of this has been reflected in trends within the criminal justice system, where recent years have seen an increasing trend towards what has been labelled “preventive”17 justice or “pre-crime”18 initiatives. These initiatives – which include, for example, preventive detention and an increased criminalization of preparatory acts – is concerned less with what the offender has already done, than with what he is thought likely to do in future.
13 Some New Zealanders prefer the rather more exciting-sounding “judder
bars”.
14 Kerr, above n 11, at
278–279.
15 Mike Davis Planet of Slums (Verso,
London, 2006).
16 David Garland The Culture of Control: Crime
and Social Order in Contemporary
Society (Oxford University Press, Oxford, 2001) at 12.
I have written elsewhere about the use of detention as a ‘pre-crime’ measure.19 The sorts of measures I discuss in this article do not function in the same manner, but it may be that they are driven by the same sort of concerns.
In combination, I think these factors present a plausible case that ‘techno-regulation’ is on the rise. Even if it is not, it is clearly already a significant feature of our regulatory terrain, and one that merits some closer examination.
On the roads
TRMs are not confined to geographical features of city centres or devices to safeguard property. Where once rules against drunk driving or speeding were enforced by the post hoc apparatus of police, courts and sanctions, ‘interlocks’ or ‘speed limiters’ now seek pro-actively to render violations of those rules impossible.
The alcohol ignition interlock is a device that can be fitted to a motor vehicle with a view to preventing drink driving. In New Zealand, it has been available since 2012 as a sentencing option for more serious or repeat drink driving offences.20 The Land Transport Amendment Act 2017 introduces various changes to the Land Transport Act 1998, including mandatory interlock sentences for certain offences .
As explained in the Regulatory Impact Statement on the recent reforms, the interlock:21
is a breath-testing device that is hardwired into the ignition system of a vehicle. The driver must undergo a breath alcohol test before the vehicle can be started. The driver cannot start the vehicle if the analysed result is over the pre-set breath alcohol level. The interlock regime aims to reduce drink-driving by preventing people driving their vehicle if they have consumed any alcohol at all.
Use of the interlock is not limited to starting the vehicle. The driver is also required to provide a breath sample at random times when the vehicle is in use. Data relating to the vehicle use, alcohol readings and violations are recorded by the device.
In addition to alcohol interlocks, a variety of other in-car systems are available that variously prompt, nudge or compel drivers to behave in ways considered to be safer. Speed limiters, for example, have long
19 Colin Gavaghan, Jeanne Snelling and John McMillan. Better and Better
and Better? A Legal and Ethical Analysis of Preventive Detention in New
Zealand. (November 2014) Available at <www.otago.ac.nz/law/news/
otago083869.pdf>.
20 Land Transport Act 1998, s
65A.
21 John Edwards Ministry of Transport Regulatory Impact
Analysis – Reducing
Road Trauma and the Cost of Reoffending: Mandatory Alcohol Interlocks (18 April 2016) <www.transport.govt.nz/assets/Uploads/About/ Documents/RIS-Mandatory-Alcohol-Interlock.pdf>.
been mandatory for certain classes of vehicles in some jurisdictions,22 and optional speed limiters are becoming standard features in top end vehicles.23 Although speed limiters do not appear to be legally imposed on speeding drivers yet, it is not difficult to see the possible analogy with interlocks and drink drivers, a possibility to which I will return.
Furthermore, if predictions about the rise of ‘driverless cars’ come to fruition,24 the prospects for ‘impossibility structures’ around road traffic could increase exponentially.
The online domain
But it is perhaps in the online domain in which so much modern activity takes place that this form of regulation has mostly expanded. Cyber-law guru Lawrence Lessig has tracked the evolution of online culture, from the cyber-libertarianism of the internet’s early days, through an era of concern about online harms, to a new era in which he sees the internet as potentially ‘the most regulable space that man has ever known’:25
I believe that cyberspace creates a new threat to liberty ... We are coming to understand a newly powerful regulator in cyberspace. That regulator could be a significant threat to a wide range of liberties, and we don’t yet understand how best to control it.
As Lessig explains, while the physical world is constrained by physical laws, the online environment is controlled by computer code. And since code is written by human beings, there is potentially no limit to the extent that that environment can be controlled. Where real-life games may seek to impose rules, online games can make them part of the built terrain (the possibility of hacks and cheat codes notwithstanding). Insofar as the internet is not always a highly regulated environment, Lessig maintains,
See, for example, Directive 2002/85/EC, which makes the use of speed limiters mandatory within the European Community for heavy goods vehicles over a certain mass. <eur-lex.europa.eu/legal-content/EN/ TXT/?uri=celex%3A32002L0085>.
The Mercedes-Benz Vito series van, for example, comes with “Cruise control with SPEEDTRONIC speed limiter ... allowing you to effortlessly travel at a pre-set speed – and avoid inadvertently going over the speed limit. One less thing to think about, wherever you go.” <www.mercedes‑benz.co.uk/content/unitedkingdom/mpc/mpc—unitedkingdom— website/en/home—mpc/van/home/new—vans/models/vito/vito‑overview.html>.
This prediction in a recent article is quite typical: “Automakers have for years been introducing what the industry calls advanced driver assistance systems (ADAS), which can detect blind spots, provide emergency braking, and assist with parking, among other things. But most are now planning to market vehicles by the early 2020s that can drive without any human intervention in limited conditions, like on highways. Highly autonomous vehicles, which require even less human touch, are expected to arrive shortly thereafter.” Jonathan Masters “The Driverless Future: Autopia or Dystopia?” Foreign Affairs, August 17, 2017. <www. foreignaffairs.com/articles/2017-08-17/driverless-future>.
Lessig, above n 10, at 121.
this is only because those who own and control it have not yet chosen to make it so.
An example can be seen with internet filters, such as that operated by the New Zealand Government. Since 2007 the Department of Internal Affairs have made this available to internet service providers. As the DIA website explains, ‘The Digital Child Exploitation Filtering System has a very narrow purpose. It blocks access to known websites that contain child sexual abuse material.’26
The filter works by comparing the requested URL with a list of banned websites, compiled by the DIA’s Censorship Compliance Unit. If a match is found, ‘the system will present a landing page informing the user that the request has been stopped’. According to the DIA, while details of blocked searches are recorded, ‘no information enabling the identification of an individual will be stored’.27
It is not only online games, searches and social media platforms that could be subject to such techno-regulatory constraints. As Lessig’s Harvard colleague Jonathan Zittrain has explained, the growth in demand for virtual goods, such as e-books and digitised music, as well as the ‘tethered devices’28 on which they are read and played, has increased opportunities for manufacturers, retailers and intellectual property rights owners to impose conditions on buyers and users. Copyright agreements that previoulsy relied upon user compliance can now be enforced technologically. The notices that used to adorn university photo-copiers, advising users of allowable limits, are redundant in the age of online journals and e-books.
Moreover, as Zittrain points out, those conditions can, in principle, be modified or supplemented long after the product leaves the ‘shop’:29
If producers can alter their products long after the products have been bought and installed in homes and offices, it occasions a sea change in the regulability of those products and their users.
These sorts of possibility have led writers like Michael Rich to conclude that ‘the possibilities for computer-based impossibility structures are bounded only by the imagination of technologists and legislators’.30
26 Department of Internal Affairs <www.dia.govt.nz/Censorship-DCEFS‑
Common-Questions>.
29 At 107.
30 Rich, above n 8, at
797.
This ability was perfectly exemplified in July 2009, when Amazon recalled several Kindle e-books in response to a dispute about publication rights. Whereas sellers of hardcopy books would have been limited to removing the items from future sale, and perhaps offering refunds, Amazon found itself with a new ability; it was able to reach out via the internet, and remotely delete the books from their buyers’ Kindle readers.
Although Amazon subsequently offered apologies and reassurances31 that this should not have occurred, the incident alerted Amazon’s customers of the new reality. Whether or not we can rely on Amazon to choose not to act in the same way in future, the fact that they have the ability to do so is something of a game-changer, rendering more tentative our possession of our personal libraries.
In aggregate, these various technologies – including, but by no means limited to, semi- and wholly driverless cars, e-books and iTunes – create an environment in which the potential for ‘techno-regulation’ is growing rapidly. As Ian Kerr has put it:32
the potential for corporations, governments, and individuals to control behaviour by placing digital locks and related technological constraints on the devices we have so deeply come to rely upon in daily life is increasing in exponential fashion.
2 Reasons to be fearful?33
On a certain utilitarian analysis, this development is something to be welcomed. All else being equal, it is surely better to prevent criminal and anti-social conduct than to react after the fact. Allowing a harm to occur and then punishing the perpetrator results in the suffering of (at least) two people, harm that could be obviated by the prevention of the act in the first place. Even in the context of IP rights enforcement, there is surely no compelling moral claim to be able to infringe copyright or violate the terms of use of the products we choose to purchase or rent.
Yet this shift towards what has been variously described as a ‘techno‑regulatory’ or ‘pre-crime’ society has elicited considerable concern. As will become apparent, some of the concerns are practical or political, others more philosophical or jurisprudential. Some relate to particular manifestations of techno-regulation, others to the enterprise more generally. It is to these concerns that I now turn.
Inflexibility of rules
The first concern engages the utilitarian case for techno-regulation on its own terms. While ‘impossibility structures’ are aimed at preventing
As well as $150,000 in compensation to two high school students whose exam preparation had been significantly disrupted by the sudden disappearance from their Kindle readers of the texts they were studying. Kerr, above n 11, at 285.
This subtitle is borrowed from Charlie Brooker’s satirical television show Screenwipe.
harm, there may be situations where the impossibility of breaking the rules itself results in harm, perhaps even greater harm than it prevents.
In an article on speed-limiters in The Guardian newspaper, a spokesperson for the UK’s Automobile Association warned that the inability to exceed speed limits “could take away people’s ability to get themselves out of trouble with a quick burst of speed, such as in overtaking situations where the capacity to accelerate can avoid a head-on collision”.34
Similar arguments could be made with regard to interlocks and driving while intoxicated. Police v Coll35 concerned charges of drunk and dangerous driving. The defendant invoked the common law defence of necessity,36 claiming that he was driving only to get his injured and profusely bleeding friend to hospital. The Auckland District Court accepted this defence, taking the view that:37
In regard to the seriousness of the injuries sustained by the injured person, and the perceived necessity of obtaining urgent medical assistance from a hospital, the Court is of the opinion that the defendant can invoke the defence of necessity in excusing the manner of his driving at speed in Ascot Avenue on the night in question.
The Trespass Act 1980 contains a statutory defence of a similar nature. Section 3(2) provides a defence to criminal trespass:
if the defendant proves that it was necessary for him to remain in or on the place concerned for his own protection or the protection of some other person, or because of some emergency involving his property or the property of some other person.
Ian Kerr cites his first encounter with wheel locks on a supermarket trolley, by coincidence a short time after Hurricane Katrina devastated New Orleans. This coincidence gave him reason to wonder:38
what would have happened had there actually been “effective technological measures” on all shopping carts in New Orleans? What further devastation might have occurred for those thousands of unfortunate people using grocery retailers’ property out of necessity if it had been technologically disabled?
A concern with the impossibility structures of techno-regulation, then, is that they will eliminate the space in which ‘lesser evils’ judgments of this type can be made and acted upon in emergency situations.
34 “UK fights EU bid to introduce speed limit devices” The Guardian (online
ed, UK, 1 September 2013) <www.theguardian.com/world/2013/
sep/01/uk-fights-eu-speed-limit-devices>.
35 Police v Coll
[2007] DCR 700.
36 It may be, in light of a more recent Court
of Appeal decision, that the more
appropriate defence in these circumstances would be one of duress of
circumstances; Leason v Attorney-General [2013] NZCA 509. It is unlikely,
however, that the outcome would have been different.
37 Police
v Coll, above n 35, at [32].
38 Kerr, above n 11, at 280.
Whether these initiatives would pose greater risks than those they prevent is necessarily a speculative judgment that will vary as between specific cases. It is likely, however, to be an area in which cognitive biases abound. It is easier, perhaps, to imagine situations in which we will be imperilled by the inability to speed or drink drive than situations where we will create a hazard by availing ourselves of that same freedom. Nonetheless, the removal of the freedom to act in circumstances of necessity is a cost imposed at least by the more inflexible forms of TRMs, even if it is a cost that is outweighed by the reduction of other hazards.
Mission creep
A second concern is that techno-regulatory initiatives have the potential to expand beyond their original and limited remit. To take one example: at present, the interlock system is mandatory in New Zealand only for those convicted of drink driving offences. It is not difficult, though, to imagine a requirement that everybody who wants to drive a car has to blow into a breathalyzer tube before it starts.
Whether this would be a good or a bad thing, presumably, depends on a whole set of subsidiary questions. An expansion of the category of regulatory targets need not, in itself, be a problem. Perhaps the problem, rather, lies with the potentially surreptitious nature of the expansion. Imagine a scenario where, instead of the Government mandating that Interlocks be fitted in all cars, car manufacturers were somehow incentivized to fit them as standard. Their ubiquity would then look more like a commercial decision than a political one, and as such, may be spared the sort of scrutiny that any government initiative would attract.
Erosion of moral/political discourse
A related concern is that ‘techno-regulation’ can be insidious. What Roger Brownsword terms the ‘moral’ and ‘prudential’ registers (see next section) work by ordering us around, but at least they will almost always be visible. Almost by definition, the state cannot take away my freedoms via these routes without telling me that they are doing so.39
Brownsword’s third register – the ‘practical’ register – requires no such transparency. Instead of justifying new restrictions via some sort of democratic process, regulators may simply impose them directly by design decisions. One day, we find that the benches at our bus stop or railway station have been replaced, and the replacements are divided up by arm-rests, or are much narrower than before. If we notice at all, we give it little thought, and it never occurs to us that the change may have been made to prevent homeless people from sleeping on the benches.
This is not, of course, to deny the existence of concern about ‘stealth laws’ and ‘stealth taxes’. But these are, I suggest, stealthy in the weak sense of avoiding much media scrutiny. In democratic states, law changes cannot literally be state secrets.
Technology commentator Evgeny Morozov has expressed the concern like this:40
once laws and norms become cast in technology, they become harder to question and revise. They just fade into the background and feel entirely natural; indeed, they are often seen as an extension of the built environment rather than the outcome of deliberate planning by some wise social engineer.
Then again, we may note that the opposite may transpire. Techno‑regulation that is badly designed may in fact have the effect of stimulating civic engagement with ethical and political issues, such as the problems faced by homeless people. The London anti-homeless spikes probably drew more attention to the issue of homelessness, and our attitudes thereto, than the majority of awareness-raising measures. But what if the owners of the London flats had opted not for steel spikes, but for cobblestones? Or, say, a picturesque flowerbed?41 Would we, the majority of us with warm beds to go home to, even have noticed what had been taken away?
Morozov also worries that technologies of perfect enforceability will erode the space for protest and civil disobedience. He asks, rhetorically, how US history would have developed had Rosa Parks been physically prevented from entering the prohibited section of the famous Alabama bus.
In this respect, I suspect he underestimates how human ingenuity can respond in the face of such pragmatic obstacles. The London anti-homeless spikes were rapidly sabotaged.42 And, in a somewhat more surreal vein, Sarah Ross’s ingenious wearable art is designed to allow comfortable reclining on Los Angeles structures designed to deny them.43
40 Evgeny Morozov To Save Everything, Click Here: The Folly of Technological
Solutionism (Public Affairs, New York, 2013).
43 <www.insecurespaces.net/archisuits.html>.
Nonetheless, the potentially insidious nature of techno-regulation, and its ability to enforce politically and ethically contentious decisions without the need for scrutiny and deliberation, may be a legitimate area for concern.
‘Automating human virtue’
This is perhaps the most amorphous of the concerns about techno‑regulation. Roger Brownsword has sketched a taxonomy of regulatory strategies or ‘registers’.44 While the ‘moral’ and ‘prudential’ registers seek to engage with citizens at a rational level – either in the form of moral appeals or appeals to their enlightened self-interest – the register of possibility substitutes the normative language of ‘ought and ought not’ for the practical language of ‘can and cannot.’
For Brownsword, this is potentially troubling, as expressed in his rhetorical question: “if we are regulated so that we can only do the right thing, does it matter that we lose the opportunity to do the wrong thing?”45
Ian Kerr has sought to locate such concerns more explicitly within a context of virtue ethics:46
How could we possibly live well, let alone flourish, in environments that increasingly seek to control our behaviour with fine-tuned granularity, by the flick of a switch permitting or forbidding various courses of conduct not proscribed by law but by lock-makers? How are we to cultivate a moral compass, a sense of right and wrong, good and bad, if we are locked on a course that leads us only from here to there with no opportunity for moral journey, deliberation or error?
Bioethicist John Harris has also appealed to virtue as a basis for his concerns about at least one particularly contentious form of techno‑regulation:47
The space between knowing the good and doing the good is a region entirely inhabited by freedom. ... Without the freedom to fall, good cannot be a choice; and freedom disappears and along with it virtue. There is no virtue in doing what you must.
On the other hand, we may consider that there will remain ample scope to display virtuous or altruistic characteristics, even if we are denied the opportunity to act in actually criminal or antisocial ways.
As a more-or-less liberal lawyer, educated in old-style rules primarily reliant on the first of Brownsword’s two registers, I must confess to finding this shift more than slightly troubling too. But if I am to exclude
44 Roger Brownsword “Lost in Translation: Legality, Regulatory Margins,
and Technological Management” (2011) 26(3) Berkeley Technology Law
Journal 1321. Brownsword and Goodwin, above n 5, at
29.
45 Brownsword, above n 9, at 256.
46 Kerr, above
n 11, at 302.
47 John Harris “Moral Enhancement and
Freedom” (2011) 25(2) Bioethics
102.
the possibility that this is just knee-jerk conservatism or neo-phobia on my part, it is necessary to interrogate this intuition, with a view to establishing whether it really contains anything of ethical or political substance. Why should we care if society is so ordered that we lose the opportunity to act badly? If crime becomes impossible, is that really something to lament?
3 What we should be looking for in TRMs
After all, it surely cannot be the case that the practical register is always inappropriate. The fact is that we don’t tend to trust the moral or prudential registers when we leave our homes, but rather, place our faiths at least in part in the practical register, in the form of door locks and burglar alarms. We like to think that our bank details are kept safe not just by the consciences of our fellow citizens, nor their fear of apprehension and punishment, but by encryption and passwords and firewalls.
How, then, can we hope to get the best out of TRMs, while excluding the worst effects? In this section, I sketch out a number of questions that could be asked of any such initiative. My hope is that this may serve as a rough guide to whether, and to what extent, we should be concerned about the novel techno-regulatory character of such initiatives. I repeat, this says nothing about the more general questions we should ask about any new regulatory initiative: whether it is likely to be effective, what we think of the values it embodies, whether the likely benefit is worth the cost, and so forth.
Is it visible?
My proposed rule of thumb here is that techno-regulatory initiatives that are visible will, in general, pose less cause for concern than those which are not. Visible techno-regulation is open to scrutiny and challenge – at both micro and macro levels. That is to say, the existence of the regulation can be challenged, and so too can particular applications of it.
Visibility, in this context, can mean several different things. It could mean, for instance:
If we consider this through the example of New Zealand’s Internet Filter, the issue of visibility could relate to the degree of public awareness that it exists at all. Certainly, it is no secret that the Filter exists, though the extent to which the New Zealand public are aware that it exists is uncertain (if my law and technology students are any indication, its existence is not widely known).
Alternatively, visibility could relate to the question of timing: do we
know that our searches are being filtered at the time when it happens? A visible techno-regulation would be one that advertises itself as it is operating. As explained above, the DIA website explains that attempting to access anything that runs foul of the internet filter will divert to a DIA landing page, instantly alerting the individual to the fact that their search has come up against it. This is a high level of visibility.
It would be easy, though, to imagine an alternative filter, one that simply announced that a given website was unavailable, or even redacted certain results from an internet search without making clear this had happened. In such a case, the decision to block a particular website would be difficult, even impossible to challenge, since the user would typically have no reason to suspect it had happened.
Finally, visibility could relate to the criteria for blocking, to the algorithms that implement those decisions, or even to the specific sites that are blocked. A highly visible web filter would allow scrutiny of all of these.
The Department of Internal Affairs has refused to release the list of banned sites, citing section 6 (c) of the Official Information Act, which allows them to refuse on the grounds that the release would be “likely to prejudice the maintenance of the law, including the prevention, investigation, and detection of offences, and the right to a fair trial”.48 Given that releasing this information would have the effect of furnishing a list of child exploitation websites, it may be that the DIA’s decision is comprehensible and justifiable. (Although access to these URLs is blocked from New Zealand, there may be ways around this via virtual private networks.)
An additional safeguard is provided in the form of an Independent Reference Group, which meets regularly “to maintain oversight of the operation of the Digital Child Exploitation Filtering System to ensure it is operated with integrity and adheres to the principles set down in this Code of Practice”.49 All additions to and deletions from the list must be reported to the Group.
Whether these measures are entirely adequate to prevent the risk of mission creep is, no doubt, a moot point. InternetNZ, for example, have expressed concerns about this:50
we worry that once it is in place that scope creep will set in. That is, that once we’ve deployed a solution that others will look to use that solution for other means. We’ve seen big movie studios propose using the filter to block torrent sites; now we’re likely seeing the same thinking here.
48 Response from DIA to Matt Taylor, 12 March 2012, <techliberty.org.nz/
wp-content/uploads/DCEFS-DIA-Reply-Matt-Taylor-March-2012.pdf>.
49 <www.dia.govt.nz/Censorship-DCEFS-Code-of-Practice-July-2017#4>.
50 Andrew
Cushen “Worried about porn? Filtering the Internet is NOT the
answer” (25 January 2016) InternetNZ <internetnz.nz/blog/worried‑about-porn-filtering-internet-not-answer>.
The likelihood of such ‘scope creep’ is beyond the scope of this article. However, given the various checks discussed – and the attention of groups like TechLiberty and InternetNZ – it seems unlikely that it could be accomplished without being open to a degree of scrutiny.
Is it flexible?
Flexibility, too, could refer to a couple of different potential concerns. It might, for example, refer to flexibility on the part of regulators. As Zittrain has noted:51
The law as we have known it has had flexible borders. This flexibility derives from prosecutorial and police discretion and from the artifice of the outlaw. When code is law, however, execution is exquisite, and law can be self-enforcing. The flexibility recedes.
Cathy O’Neill’s Weapons of Math Destruction deals with a phenomenon that, if it is not properly a form of techno-regulation, is at least close kin to it: the increasing reliance on decisions by algorithms. Algorithms – or in her terminology, WMDs – now rule over many aspects of our lives, from our job prospects to our chances of getting a mortgage or an insurance policy, and increasingly, to decisions about bail, sentencing or parole.
O’Neill identifies a number of potential concerns about this phenomenon, one of which relates to the sort of flexibility with which I am presently concerned:52
you cannot appeal to a WMD. That’s part of their fearsome power. They do not listen. Nor do they bend. They’re deaf not only to charm, threats, and cajoling but also to logic – even when there is good reason to question the data that feeds their conclusions.
In addition to inflexibility on the part of the regulator or decision-maker, techno-regulation might trouble us insofar as it leaves no flexibility for the party who is regulated. Obviously, a regulation cannot be entirely flexible, as that would entirely defeat its purpose. A rule that is easy to break or avoid might barely qualify as a ‘rule’ at all, but rather a custom or a ‘nudge’.53
Nonetheless, as illustrated by the earlier examples of speed limiters, and interlocks, we may want them to be sufficiently flexible to admit rule-breaking in the case of genuine emergency situations.
Precisely how porous a barrier ought to be might depend on a number factors, including the importance of what it is trying to protect, and the likelihood and gravity of bad consequences resulting from inflexibility. It may also depend significantly on the perceived likelihood of rule-breakers taking advantage of such backdoors.
Zittrain, above n 27, at 107.
Cathy O’Neil Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Crown, New York, 2016) at 10.
Richard H Thaler and Cass R Sunstein Nudge: Improving Decisions About
Health, Wealth, and Happiness (Penguin Books, New York, 2008).
It may be that, instead of hard ‘impossibility structures’, something like ‘difficulty structures’ could be built into some devices. Perhaps interlocks and speed-limiters could be equipped with manual override mechanisms, that would allow rule-breaking in emergency situations, but which would automatically alert the police as to what was happening. This would allow rule-breakers to be apprehended or questioned, and the veracity and validity of their purported justification to be examined. Importantly, it would, we might assume, have the effect of deterring at least a proportion of unjustified rule-breakers. (Presumably not all; it is not entirely unknown for joy-riders to race around in stolen – or more accurately, converted54 – cars with the car alarms blaring.)
In other instances, it may be that sufficient flexibility could be permitted by requiring the possibility for human review. Perhaps pitch-side fences could be constructed so that they could be instantly dismantled by stewards or police, or emergency services could have an instant access pass for turnstiles. Algorithms of the type described by O’Neill could be, and sometimes are, reviewable by human supervisors, though it is probably unrealistic to expect all such decisions to be open to such review; if anyone receiving an unwelcome result could insist on such a review, it might be argued, many of the efficiencies of algorithmic decision-making would be lost. Nonetheless, for particularly important decisions, this may seem like an important safeguard.
A recent attempt to implement such a safeguard can be found in the EU’s General Data Protection Regulation,55 which contains a right “not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her”. It should be noted, though, that some skepticism has been expressed about the likely efficacy of this right.56
Details of the trade-off will depend on specifics of the various risks. Overall, though, an impossibility structure that renders non-compliance genuinely impossible will, on this criterion, merit closer scrutiny than one which allows for deviation in the case of genuine emergency.
Does it enforces rules already agreed upon by democratic means?
One concern about TRMs is that they can be used insidiously, to smuggle in rules that would not pass muster were they required to undergo democratic scrutiny. On the other hand, a TRM that merely enforces a rule already agreed upon and enforced by conventional law is less concerning.
Crimes Act 1961, s 226.
Regulation (EU) 2016/679, Art 22 (1): The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her. Note, however, the exceptions in paragraph (2).
Lilian Edwards and Michael Veale “Slave to the algorithm? Why a ‘right to an explanation’ is probably not the remedy you are looking for” Duke Law and Technology Review forthcoming.
Interlocks and speed-limiters plausibly pose little concern in this regard. Speeding and drink driving are, after all, already criminal offences. Similarly, the NZ Internet Filter, which is limited to blocking child sex abuse images, only enforces a prohibition that already exists. As he DIA says: ‘No one has the right to view illegal content that focuses on the sexual abuse of children; just as no one has a right to import illegal books and DVDs.’57
Other examples are more questionable. It is highly doubtful that the various anti-homeless measures discussed above enforce an objective as widely accepted as preventing drunk driving or sharing child pornography. Whether they involve enforcing rules that are already accepted depends on the specifics. Private land owners are certainly not obliged to leave their property unaltered in case homeless people wish to sleep there. The alteration of public spaces, like benches in public parks or railway stations, seems more contentious.
In the context of online interactions, concern has been raised that the compromise that previously underpinned the enforcement of IP rights has been obviated by technologies of direct enforcement. Previously, rights holders wishing to enforce those rights were largely reliant on the good will of end users and – where that failed – the enforcement apparatus of the state. This, in turn, allowed the state to require certain exceptions in the public interest – ‘fair use’ exceptions, and those for the purposes of reviews or teaching. In an era of direct enforcement, there is little need for the ‘middle man’, and hence, little necessity for rights holders to make allowance for these other interests.
It might be argued that there is a significant difference between the scenario where the state imposes new prohibitions without democratic deliberation, and the scenario where a private party builds certain conditions into the use of its products. There is doubtless much that could be said on that topic, but I will confine myself to the observation that, in an era where Facebook has an estimated 2 billion users, and Google accounts for around 90% of online searches, the variation of terms of use of private products and platforms can have at least as important an effect on the lives of most people as a great many actions by the state. If, for example, either of those platforms were to make it impossible to access certain content, it is far from implausible to surmise that this would have a greater effect than official attempts to ban.
Obviously, the scope for challenging private actors through, for example, judicial review or constitutional means is more limited, and this is likely to be even more true when the private actor is a website or company based in another jurisdiction. This article, however, is not concerned with the means to challenge TRMs, but merely with the question of when and to what extent they should concern us. With that in mind, decisions by major service providers that only uphold existing laws (such as those against threats of violence) are less concerning
<www.dia.govt.nz/Censorship-DCEFS-Common-Questions>.
than those which uphold widely agreed norms (such as those against ‘bullying’), and both are less troubling than those which enforce rules about which there is no consensus.
Does it employ more intrusive means of enforcement?
Even if a TRM only imposes rules that are already agreed to be acceptable, concern may be justified where it uses a means of enforcement that is more intrusive, degrading, discriminatory or inhumane.
The Mosquito MK4 Anti-Loitering Device, described by its distributor as an “ultrasonic teenage deterrent”, is:58
a low power device that makes a pulsing sound similar to an alarm clock. This sound is just out of the range of an adults hearing, which is why it only bothers people under the age of about 25. The sound is not loud or painful, just highly annoying after a short period of time.
Although deployed by private actors on private property, the Mosquito has received considerable criticism, and concern as to its legality. Some of this relates to the discriminatory attitudes to young people inherent in its application.59 This, however, seems to be an objection that is not specific to the Mosquito device. Rules against loitering or prohibiting unaccompanied children in certain shops and other premises are not unknown; insofar as the Mosquitos is discriminatory, it seems that this is equally true of signs denying admission.
Criticism has, however, also been levied at more novel aspects of the Mosquito, specifically, at the means used to enforce the rule. Mitchell Akiyama has described it as “a weapon that creates zones of discomfort intense enough to drive young people out of its range”.60 The Council of Europe Parliamentary Assembly, in its report in Mosquito technology, emphasised that:61
inflicting acoustic pain on young people and treating them as if they were unwanted birds or pests, are harmful, highly offensive and may thus result in a degrading treatment prohibited by Article 3 of the Convention. Under this provision, children and other vulnerable persons have the right to be protected from serious attacks against their physical and psychical integrity.
58 <www.compoundsecurity.co.uk/security-equipment-mosquito-mk4‑
anti-loitering-device>.
60 Akiyama, above n 57.
61 Council
of Europe Parliamentary Assembly “Prohibiting the marketing
and use of the ‘Mosquito’ youth dispersal device” 22 March 2010, at [11].
And UK human rights watchdog Liberty argued that “It exposes young people to extreme discomfort, and little is known about the long‑term effects on people’s hearing.”62 Leaving aside the justification for excluding people from certain areas on the basis of age, the means by which this is accomplished may itself be problematic.
4 The checklist applied
In this section, I will consider how the various measures discussed thus far in this article would measure up in terms of my proposed checklist. I emphasise again that this is not intended to answer the bigger question of whether these initiatives are desirable overall. That will depend on, inter alia, the seriousness of the mischief or harm that is intended to be prevented, the likelihood of successful prevention, and the various costs and harms caused by the initiative. The purpose here is only to evaluate whether the techno-regulatory aspects of the initiatives should provide cause for concern.
Interlock
The Interlock device is about as visible as a TRM could be. Would-be drivers must blow into the device before starting up their car. It is entirely obvious to them what is happening, and why.
In terms of flexibility, the interlock presents something of a mixed bag. At present, all interlock orders are at the discretion of sentencing judges, but the commencement of the relevant provisions of the Land Transport Amendment Act 2017 (in July 2018) will render them mandatory for certain offences. At the same time, however, the Bill introduces the option for a court to cancel an interlock order ‘if the person’s personal circumstances have changed significantly’, imposing a disqualification order instead.63 The imposition of the TRM, then, will be able to be challenged, both at sentencing (if it is believed that the criteria for its imposition are not in place) or subsequently (if the person’s circumstances change).
With regard to ‘emergency’ flexibility, the Interlock appears not to allow much or any. It is an offence to interfere with operation of the device,64 and while disabling the interlock in genuine emergencies might leave open a defence of duress of circumstances or necessity, the practical means to do so are far from straightforward.65 The most low-tech way around the ignition inhibiting function of the interlock is, of course, to have someone
62 <www.liberty-human-rights.org.uk/human-rights/fighting‑
discrimination/young-people/mosquito>.
63 Land Transport
Amendment Act 2017, s 34.
64 Land Transport Act 1998, s
55A.
65 A Google search reveals various websites purporting to
offer foolproof
ways to circumvent alcohol interlocks. See, for example, How to Bypass an Ignition interlock Device, <smighterofwrongs.blogspot.co.uk/2011/12/ for-information-and-entertainment.html>. For the purposes of this article, I have not attempted to investigate the reliability of any of them.
sober breathe into the device, thereby allowing the vehicle to be started. This, of course, relies upon the presence of a willing sober person.66 But provided one such could be found, the interlock could be circumvented.
As I explained earlier, the reason why this simple work-around does not render the device useless is that the interlock also requires the driver to be tested at regular intervals while driving. Even if a sober person could be found to start up the car, they would need to be in a position to breathe regularly into the device throughout the journey.
While this may be quite an effective barrier against drink driving in ordinary circumstances, it may not prevent the use of the car in emergencies. A failed test during driving will not disable the car – bringing the vehicle to a halt in transit could be dangerous – but will log the failure, where it will be discovered at the next mandatory check. It is also possible to install the device in such a way that a failure results in the car lights flashing or horn sounding, thereby drawing immediate attention to the infraction (though this is not typical in New Zealand). Furthermore, it seems like it should not be impossible to have the interlock communicate a signal to the police, together with GPS co‑ordinates, so that the drink driver could be intercepted at the time.
If it were considered, on the other hand, that the present situation was too restrictive, it would presumably be possible to adjust the interlock so as to permit the driver to start the car without passing the breath test, but with all or any of the attendant consequences (logging, lights flashing, horn sounding, police alerted). That might constitute an acceptable compromise, whereby casual drink driving is strongly deterred without entirely precluding the possibility of emergency driving. Alternatively, this might be thought to cause more danger than it prevents. Intoxication typically leads to impaired judgment, and drunk people may be poor judges when it comes to balancing risks, with the result that some may opt to utilise the manual override in circumstances that would do more harm than good. They would, of course, face criminal sanctions for this, but that largely negates the purpose of techno-regulation.
In all, then, the alcohol interlock may be seen as moderately flexible in emergency situations, and it is uncertain whether rendering it more flexible would be desirable.
In terms of the third and fourth items on the checklist: the interlock certainly imposes a rule already present in New Zealand law – that we cannot drive while over the limit – and insofar as it sets this limit at a ‘zero’ level, that is not unusual for those convicted of drink driving offences. Being required to give regular breath samples may be seen as mildly burdensome and possibly even slightly demeaning, but given that such samples can be required randomly even of non-convicted drivers, it seems unlikely to be seen as manifestly excessive.
Internet filter
New Zealand’s Internet Filter scores highly in terms of visibility. Its existence is a matter of public record. Users who encounter the filter are notified immediately that their search has been blocked. While a list of the specific blocked sites is not available, the criteria for blocking are. The contents of the list are reviewed regularly by an Independent Reference Group.
The Filter is flexible not in the sense that someone could violate it in an emergency (though it is hard to imagine what would one of those might look like!) but certainly in the sense that a decision to block a particular website can be challenged.
The criteria for blocking sites – assuming we can trust this – accord with already existing legal prohibitions; the DIA has regularly offered assurances that nothing banned by the filter is legally available otherwise.
Finally, the means of enforcement do not appear to be more intrusive or burdensome than the traditional law counterpart. Those who encounter the filter will be redirected to the DIA landing page, but the DIA insist that no record will be kept of their identity, and no legal or other consequences will follow for them.
As discussed earlier, concerns about various aspects of the Filter have been raised, in particular, the danger of ‘mission creep’ in terms of the content blocked. It is certainly the case that web filters are very open to expansion of the scope of what is blocked. The extent to which the Internet Filter should give cause for concern, then, depends in substantial part on the effectiveness of and trust in the various checks upon its use. If they function as they are supposed to, there seems little cause for concern.
Whether the Filter is an effective means of blocking access to child abuse images is altogether another question, but in terms of the techno‑regulation checklist, the Internet Filter is among the least concerning.
The Mosquito
Can the Mosquito be said to ‘visible’? There is no reason why its operation could not be accompanied by signs advertising its use, but it seems like these are not usually in place. Without such knowledge, those who would object to the fact or manner of its operation will not be able to do so. Additionally, the lack of ‘visibility’ in this case may lead to further problems. According to some reports, young people have found themselves subject to discomfort and anxiety with no explanation for the cause, while parents of young children have been left dealing with unaccountable distress.
In terms of ‘flexibility’, much will again depend on the details. In emergency situations, it is unlikely that the pain caused by the sound will be so great as to prevent young people coming onto or remaining upon the affected area to avoid other hazards, and any device that was likely to cause more than annoyance should be deemed problematic.
Does the Mosquito exceed the rules already agreed upon and imposed by conventional means? This will depend firstly on its reach. If its effects are limited to the area of private premises, that may be one thing; seeking to impose an effective ‘ban’ on congregating in public spaces, such us the areas outside of shops or schools, is another matter.
Where the Mosquito possibly does least well is in terms of the fourth criterion. A TRM that functions by imposing ‘extreme discomfort’ on a section of the population, regardless of whether they are breaking any law or disobeying any directive, seems like an obvious target for concern.
Sprinklers
In one sense, the Auckland sprinklers are highly visible, in that their presence and function is specifically advertised; they will, at certain points during the night, drench the steps and doorways where they have been fitted. In another sense, it may be that they are not particularly visible at all. Claims that they exist only to clean the areas in question, for the benefit of those arriving for early shifts, sit awkwardly with the claim that their real purpose is preventing rough sleeping. Much as with the flowerbed example I employed earlier, a TRM that effectively disguises its actual purpose is insulating itself from the sort of scrutiny and critique to which more traditional rules would be subject.67
The sprinklers are flexible in as much as they are deterence rather than impossibility structures. Like the Mosquito device, they do not make it impossible to enter the target area in cases of genuine emergency. Both the Mosquito and the sprinklers, however, lack the sort of flexibility possessed by the Internet Filter, in the form of a readily available point of contact in the event that the TRM seems to be malfunctioning.
More than with the Mosquito, it seems possible that the area of effect of the sprinkler systems could be confined to the private property of the owners. In as much as entry to private property can already be denied by walls and fences, there is arguably no extension of the scope of the existing rules.
However, the manner of their enforcement may be considered at least somewhat demeaning and inhumane. Drenching homeless people, especially in cold weather, could in fact by dangerous as well as unpleasant.
Spikes
And what, finally, of the infamous anti-homeless spikes with which I began this discussion?
They are highly visible, and their purpose was wholly transparent – considerably more so than with the Mosquito or the sprinklers. At least without some ingenuity, sleeping on them would not be an option, but they would not seem to prevent someone entering the area in question
in the event of an emergency. They protect private property and, unlike the Mosquito, have no potential to go beyond its borders. And again, unlike the Mosquito, the spikes function as a deterrent rather than a punishment; it is hard to imagine anyone sustaining injury by trying to sleep on the spikes, but easy to imagine young people hurt when unwittingly entering areas affected by the Mosquito.
The anti-homeless spikes are certainly aesthetically unattractive and highly unsubtle in terms of their message to and about the respective importance of shelter for homeless people and peaceful enjoyment of private property. But they are not, I argue, among the most troubling or pernicious applications of techno-regulation.
5 Conclusion
Techno-regulatory mechanisms or TRMs, as I have used the term, covers a wide range of measures that differ in various respects. Some are physical structures, part of the built architecture of towns or built into cars; others exist only as code in cyberspace or in electronic devices. Some are state initiatives, intended to enforce or supplement criminal law; others are utilised by private actors, perhaps to enforce legal rights, but sometimes perhaps to extend them. What all have in common is that they employ direct means to impose behavioural restrictions, rather than relying on threats or appeals.
In this article, I have made three claims about TRMs. First, that they are already widespread, and likely to become more so as our lives become more urbanized and technologized. Second, that while some concerns about TRMs may be overstated or lacking in normative substance, there are valid reasons to be concerned about at least some examples of this trend. And third, that there are a number of questions we should ask about any proposed TRM, the answers to which should inform our intuitions about its introduction.
Inform, however, does not mean the same as determine, and our overall response to those initiatives should quite rightly be shaped by a range of concerns, many of which are not unique to TRMs. But insofar as it is the ‘techno-’ aspect that presents concern, it is my hope that these sorts of questions should help sort out normatively substantive concerns from those that owe more to knee-jerk responses to the new or the newsworthy. Indeed, as I have argued, it may be the more apparently innocuous, insidious or invisible measures that should concern us most.
NZLII:
Copyright Policy
|
Disclaimers
|
Privacy Policy
|
Feedback
URL: http://www.nzlii.org/nz/journals/OtaLawRw/2017/7.html