Recently, Apple Computer, Inc. has branded the privatization of the idea of the commons, by renaming the retail Apple stores as “Town Squares“[1]. In Apple’s definition, these “Town Squares” are where people will gather, talk, share ideas, and watch movies, all within Apple’s carefully curated, minimalist designed, chrome and glass boxes. In this scenario, Apple’s “Town Square” is tidy, spartan, and most critically, privatized. This isn’t new behavior, however, what is new is the context within which Apple is able to do this, from both inside of shopping malls, and from retail locations on Main Streets. Applin (2016) observed that private companies are collecting and replicating community through their networks and communications records [2]. Madrigal (2017) observes that  “the company has made the perfect physical metaphor for the problem the internet poses to democracy” [3]. This article provides a discussion of what happens and what we forfeit in these hybrid gathering places between Internet usage and privately owned spaces; and how these hybrid spaces have become enabled in the first place.

During the 1980’s and 1990’s, the American public witnessed and participated in the privatization of public space through the shopping mall, a privately owned conglomerate of retail stores located in a single place, usually away from the “Main Street” in a downtown area. Shopping malls were located in places where space was available, land was less expensive, and people were further away from a downtown. As shopping malls became centralized shopping spaces, downtown “Main Street” stores lost revenue and many shopkeepers could not compete with the prices offered at shopping malls, or the proximity to so many other businesses. An outcome of the popularity and usage of shopping malls by the public, was that they were public spaces within private spaces and as such, people’s rights were limited depending upon the policies of the shopping mall. This was a quiet, barely noticed outcome of where we shifted our attention and participation, and as surveillance equipment became more available and cameras became installed in malls, we often unknowingly participated in new ways for malls to record our behavior and habits, and to monitor us. As we began to use mobile devices enabled with cameras, we started to participate in monitoring malls and the people within them, as we photographed and cataloged our lived experiences. We also began to move more, and as technologies became more enabling, to shop online.

For those and other reasons, the shopping mall hasn’t sustained continued growth. Many malls have closed or gone into disrepair, and others have seen a downturn in businesses wanting to support them. It’s a complex web of retail vs online shopping, combined with how fuel and driving patterns have been changing. As a result of these new factors, walkable cities and their associated downtown real estate has become once again in vogue, but with caveats. In particular, the mall stores have now been renting spaces on Main Streets, with their economic leverage to price out local business, and this creates fusions of public space and the “mall sensibility” (e.g. a conglomerate business model, often based on extremely advanced supply-chain automation and customer profiling data capabilities and soon to be driven by Artificial Intelligence capabilities).

With shopping malls, the privatization of public space happened in the physical space of the mall, but the outcome of how our behavior has changed is now within the public spaces of our communities, as we rely more and more upon communications technologies to maintain our social networks. As we automate, we are shifting our conversations, relationships, messages, and preferences to the private control of companies whose interest is not in maintaining our community or its health and well-being, but rather to increase their knowledge of us, so that they may provide more targeted advertising, better “services” that we will pay for, and to enable control over our communications in new ways.

What this means for communities is that community knowledge of the local locale, which is built over time in a community via social relationships, cooperative efforts, and group awareness is becoming individualized and commoditized. This is happening simultaneously as Main Streets are becoming “automated” through participation in the reconstruction of the shopping mall’s corporate influence into community.

When Apple rebrands (privatizes) the “Town Square,” their corporate desires and objectives take precedence over people within that space. The ethics questions and concerns of how Apple will use unproven, experimental, biometric technology such as facial-recognition [4], can be overlooked with the framework of a private “Town Square” where public experience is curated.

In the Apple “Town Square,” all is known and controlled by Apple and any technology that could benefit from ethics oversight (or at least some governance review) could be perceived to be bounded within Apple’s domain, which includes servers located out of town or perhaps out of country, and within a store that is at base, a private corporate space accountable to itself and its shareholders.

In the Apple 1984 Super Bowl advertisement [5], men and women with shaved heads wearing grey uniforms are marching through space age chrome and glass minimal tunnels while a “Big Brother” type of authoritarian figure talks to them across a screen. What he says from various monitors, as the people assemble in a similarly outfitted auditorium is:

Today, we celebrate the first glorious anniversary of the Information Purification Directives. We have created, for the first time in all history, a garden of pure ideology—where each worker may bloom, secure from the pests purveying contradictory truths. Our Unification of Thoughts is more powerful a weapon than any fleet or army on earth. We are one people, with one will, one resolve, one cause[6].

With the rebranding of “Town Squares” into privatized Apple stores, it becomes apparent that Apple is transforming its retail spaces into “a pure garden of ideology—where each worker may bloom, secure from the pests purveying contradictory truths.”

Apple isn’t the only one. Amazon pushed community bookstores out of business with competitive pricing online and are now opening physical bookstores in communities [7]. In these spaces Amazon sells books, but they also do so utilizing vast data networks, which include many human reading preferences and order histories.

In Amazon Go and the Erosion of Supermarket Sociability [8] and in Deliveries by Drone: Obstacles and Sociability [2], I examined how automation is replacing human contact and interchange and within those frameworks, I question whether or not community knowledge is passed along, or becomes owned by the various private enterprises, who are controlling the communication around and about transactions. Gas stations are privately owned hubs of community knowledge. Where I live in Silicon Valley, gas stations are beginning to be replaced by office buildings, and developers who desire corner lot real estate in a land strapped area, are willing to invest in changing the urban landscape. In my neighborhood alone, three gas stations have been closed and developed into office properties. It is not necessarily a bad outcome to develop gas stations, for it is an indicator that better energy sources are being adopted. However, it does mean that the small corner gathering and community knowledge outposts in some areas (even if privately owned) are being developed in new ways that remove their function and replace it with more refined and harder to access gathering points.

When we stop talking to each other in a community and default to automation or removed accessibility, we are forfeiting part or all of our community knowledge, homogenizing it, and offering it to private control. Data mining and machine learning will begin to track more and more of our community spaces, and our public rights in digital space combined with what we have in physical spaces will change our relationships and the way we choose to express our opinions and beliefs.

References

[1]Apple Computer, Inc. 2017. Town Square.

[2] Applin, S. 2016. Deliveries by Drone: Obstacles and Sociability. In The Future of Drone Use (Custers, B. editor). Springer. T.M.C. Asser Press, The Hague. Oct. 16, 2016.

[3] Madrigal, A. 2017. The Great Thing about Apple Christening Their Stores, “Town Squares.” The Atlantic. Sept. 13, 2017.

[4] Applin, S. 2017. Paying with our Faces: Apple’s FaceID. Savage Minds. Sept. 23, 2017.

[5] Apple Computer, Inc. 1984. Apple 1984 Super Bowl Commercial Introducing Macintosh Computer (HD) via Robert Cole. June 25, 2010.

[6] Wikipedia. 2017. 1984 (Advertisement).

[7] Blumenthal, E.2017. While Barnes & Nobles close, Amazon is opening real live bookstores. USA Today. May 24, 2017.

[8] Applin, S. 2016. Amazon Go and the Erosion of Supermarket Sociability. Savage Minds.

One of the disturbing issues with the press coverage of FaceID during the week of Apple’s announcement, was the limited criticism of what it means for Apple to deploy FaceID, and those who will follow Apple and deploy their own versions. What does it mean to digitize our faces and use the facsimile of our main human identifier (aside from our voices) as a proxy for our human selves, and to pay Apple nearly $1000 U.S. to do so?

FaceID could be considered a gimmick. Apple has the developed technology in hand, and as such, they can then offer this type of “Science Fiction” experience to their phones to give their customers a new way to authenticate their identity. But it isn’t this simple. All new technologies, as with any other new human production, become embedded in society in various ways, used in various unforeseen contexts, and have various unforeseen consequences. Even if Apple is only deploying this technology within the context of its iPhone, they are setting a usage model, and are doing so privately, around the regulation that governs society. This movement from Apple deployed so casually on such a broad scale, may change how we live, and how our faces become used forevermore.

Facial-recognition falls into the category of technology called “Biometrics.” Biometrics is the class of quantification metrics that rely upon some type of bodily feedback to work. Biometrics include digital fingerprint recognition, retinal scans, voice recognition, heat maps, and facial-recognition, among others. Apple has been using digital fingerprint recognition for some time. However, the issues with facial recognition are more complex.

There are several issues with facial-recognition software that have been raised over time, with the idea of algorithmic bias being one of the main ones [1]. Simply put, algorithmic bias exists when algorithms are not able to create complete understandings of a situation or issue. In the case of facial-recognition, algorithmic bias exists because people have different facial features and skin tones, and for humans, particularly those with darker skin tones, facial-recognition software either cannot recognize them, or worse, can recognize a face, but is unable to attribute the recognized face to the person, instead recognizing them as someone different than who they are. This might merely be annoying when the facial-recognition algorithm won’t unlock someone’s iPhone, but can cause severe problems when facial-recognition technology is deployed on a massive scale in various facets of our society. In the future, facial-recognition technology may determine access to the commons, and as such, could easily falsely attribute circumstances and surveillance video “evidence” to the wrong person’s identity, resulting in false accusations at best, and action on false accusations (if we get more automated in law enforcement responses) at worst.

FaceID is automated Artificial Intelligence. This means that there will not be any humans in the process of identification or authentication. Once FaceID is deployed, it will run automatically, identify (or not) automatically, and authenticate automatically. Furthermore, Apple will be using FaceID to unlock the iPhone, for Apple Pay,  iTunes, and other Apple products and services. FaceID will work with other vendors, and share its users’ facial-recognition and authentication with them [2]. This will not be limited to Apple. If we think that having our credit card number being breached is a problem now, what will it mean when our faces are stored insecurely?

Another issue to consider with facial-recognition technology is the idea of what our faces mean to us, and mean to those of us in different parts of the world. For example, in some cultures, tattooing the face is considered to be a stronger taboo, where in others it is a place of honor and prominence. How we use our faces, and choose to use our faces should be considered when technology companies develop facial recognition technologies. Of course, those who are uncomfortable with facial-recognition technology, won’t use FaceID, and for now, while it is still optional, this will not be a problem. However, as FaceID debuts around the world, these issues may be raised, and unforeseen outcomes may emerge.

The technology industry is often criticized for not respecting regulations, or ethics, and as I mentioned in my previous piece [3], much of this comes from not having anyone different on development teams who can raise these issues and questions. Within Apple, there are few Social Scientists, nearly no anthropologists, and with the focus moving towards quantification as a metric for determining feature use and design, few qualitative researchers inputting to products. It might not be that Apple doesn’t care, it might be that Apple truly doesn’t know that it needs to care, or some other reason. As a design focused company, it may be that qualitative research is thought to be something that anyone in design at Apple could do [4] and as such, some of the more pressing social issues surrounding the deployment of FaceID could get lost in the “sci fi” factor or rush to market.

Because we are now on the cusp of biometric facial-recognition being mainstreamed by a private technology company with the decisions for how this will impact all of us in private control, it may be time to consider what governance or ethics review boards would look like for the tech industry going forward—or at the very least, it seems time for private technology companies to hire anthropologists and other social scientists to product teams to create technology products that will adapt to our cultural preferences as humans, while respecting our sense of privacy, our desire for security, and our right to our identities.

References:

[1] Finley, K. 2017. Can Apple’s iPhone X Beat Facial Recognition’s Bias Problem.” WIRED Business. Sept. 13. 2017. [Online]. Available from: https://www.wired.com/story/can-apples-iphone-x-beat-facial-recognitions-bias-problem/ Date assessed: Sept. 17, 2017.

[2] Perez, S. and Luden, I. 2017.  Face ID will work with Apple Pay Third Party Apps. Tech Crunch. Sept. 12, 2017[Online.] Available from: https://techcrunch.com/2017/09/12/faceid-will-work-with-apple-pay-third-party-apps/

[3] Applin, S. 2017. Artificial Intelligence: Making AI in our Images. Savage Minds. Sept. 7, 2017. [Online]. Available from: /2017/09/07/artificial-intelligence-making-ai-in-our-images/ Date assessed: Sept. 17, 2017.

[4] Applin, S. 2016. The Automation of Qualitative Methods. EPIC. Jan. 18, 2017. [Online]. Available from: https://www.epicpeople.org/automation-qualitative-methods/ Date assessed: Sept. 17, 2017

Hello! I’m Sally Applin. I am a technology anthropologist who examines automation, algorithms and Artificial Intelligence (AI) in the context of preserving human agency. My dissertation focused on small independent fringe new technology makers in Silicon Valley, what they are making, and most critically, how the adoption of the outcomes of their efforts impact society and culture locally, and/or globally. I’m currently spending the summer in a corporate AI Research Group where I contribute to anthropological research on AI. I’m thrilled to blog for the renowned Savage Minds this month and hope many of you find value in my contributions.

There is so much going on in the world that it is challenging to choose a single topic to write about—floods, fires, hurricanes, politics—as anthropologists in 2017, we are spoiled for choice. However, as a warm up for the month ahead, I thought I’d start with a short piece on automation and agency to frame future pieces which will address these topics. The following is a letter I wrote yesterday morning to the House of Lords in the UK, who issued a call for participation on the governance and regulation of Artificial Intelligence, a topic with great importance to me. If done well, AI will benefit many, and if overlooked, or done in haste or without forethought, there could be catastrophic outcomes from poorly designed algorithms, and automation and limitations that permanently alter society as we know it.

The oncoming onslaught of Artificial Intelligence (AI) is not something that will happen to humanity, but rather something that we ourselves will construct, shape, and enable in the world. Some of us may have more power than others in its implementation and deployment of AI. It is for this reason that is astute for those shaping the governance of our future to both gather data and understanding of concerns surrounding AI, and to take action to protect not only their constituents, but broader humanity and global society—for as we all now realize, digital networks and digital automation is broadly reaching and the smallest digital intent can have unforeseen global repercussions.

There are two points that I would like to personally contribute to for this call, the first being Human Agency and its preservation, and the second being that of Social and Cultural awareness when automating decisions that will impact ethics. Human agency is our capability to make choices and decisions from the options that unfold before us at each point in time. As we move through the world, and as our circumstances change, so do the options from which we may choose to make any given decision. When these are automated, and in the case of AI, severely estimated and automated, the results can restrict human freedom and movement—in any class of society. Furthermore, because these decisions are automated, the cultural and social aspects of each individual as well as our cultural groups, does not become considered. This can undermine peoples’ agency as well as their identity. I refer to ethnicity and agency within a country’s national identity as part of a discussion on ethics, values, and customs within a culture, as well as individual agency and cultural expression within that context. An AI from Michigan in an autonomous vehicle with embedded ethics would suggest one type of cultural values, which may be out of place in Great Britain, where people express their cultural values in different types of vehicular ethical behavior. What does it mean to automate cultural choices and expressions in one area, and deploy those to other locales? (See Applin 2017).

Automation currently employs constructed and estimated logic via algorithms to offer choices to people in a computerized context. At the present, the choices on offer within these systems are constrained to the logic of the person or persons programming these algorithms and developing that AI logic. These programs are created both by people of a specific gender for the most part (males), in particular kinds of industries and research groups (computer and technology), in specific geographic locales (Silicon Valley and other tech centers), and contain within them particular “baked-in” biases and assumptions based on the limitations and viewpoints of those creating them. As such, out of the gate, these efforts do not represent society writ large nor the individuals within them in any global context. This is worrying. We are already seeing examples of these processes not taking into consideration children, women, minorities, and older workers in terms of even basic hiring talent to create AI. As such, how can these algorithms, this AI, at the most basic level, be representative for any type of population other than its own creators?

The impact on society of the digital revolution has had a profound global societal impact and the issues that we have seen with Google and Facebook bumping up against privacy laws and regulations in Europe are a direct result of this cultural mismatch and lack of awareness of other ways of living and life. Thus, one important and critical step for government would be to mandate that teams developing AI include research scientists and contributors from multiple cultures, social classes, ethnicity, and genders.

If this does not happen, the representative power and advantage will be distilled into a very small group of people, who will be designing a system mostly for themselves, with the power and capabilities to extract habits, data, and behaviors from others, all concentrated within the power of technology companies. This is a problem that is ongoing. Google and Facebook have more data (and more relevant data) on citizens than most governments.

If the companies building this future do not include most of humanity—how could the AI they produce be fair, representative, and appropriate for societies?

Additionally, the government should include Social Scientists, particularly anthropologists on a panel or task force as these debates move forward. Anthropologists specialize in understanding groups, and group cultural behavior, and there are anthropologists who have training in technology and technology development.

The public should be made aware that their choices will be changed by AI, that their cultures and genders are likely to not be fully considered by AI, and that if people want to continue to have choices, true agency and choices, equivalent or better to what they have now, that they must understand how critical it is that AI development teams be balanced and representative, and that all of us must be included in the shaping of our future.

While the history of the term is intriguing, it certainly makes our modern quibbles seem inconsequential, or at least a bit ethnocentric. Whereas the English word for person has its own particular history, there is likely a similar concept in most societies around the world, both modern and ancient. It’s probably also the case that these other concepts of persons can shed light on the more general notion of what a person could be (cue the anthropologists). Thus, an anthropological perspective on the term person would provide a sort of external validity in our own arguments, just as it does for so many other discussions. However, the problem runs deeper than just comparing the English concept of person to that of other languages. From primates to developing humans to artificial intelligence, we lack much internal validity in our own understanding of what a person is across disciplines. There needs to be a more synthetic underpinning for what we deem a person as opposed to the discipline-specific concepts. An understanding of ideas of personhood in each of the disciplines discussed could really use the eyes and minds of anthropologists engaged in things like Science and Technology Studies. An understanding of how scientists in these disciplines understand personhood as it relates to their own work could help build a more universally consistent concept.

My posts this month have focused on three areas in which the concept of person is important: primatology, medicine/technology, and artificial intelligence. As I’ve discussed, cognitive science and the biological sciences have enabled enormous advances in our understanding of personhood in each of these areas. However, biology and cognitive science can only get us so far with something that could be considered a value statement. They can inform it, but not really delineate it – at least not alone. Rethinking our understanding of personhood through the social sciences and humanities can help inform research in primatology, medicine, and computer science. Moreover, how personhood plays out in the legal system, such as through the work of Steve Wise and the Nonhuman Rights Project, relies deeply on informed research to push the idea of personhood beyond the human.

When they are incorporated into policy, words really matter. Definitions really matter. Although how a “person” is defined and understood may seem to be simply an academic curiosity, these understandings have direct influences on policy and action – but only if they are heard. If the ideas never leave their discipline, or worse, never leave the Ivory Tower, then they will fail to have real impact. How we treat primates and other animals. How we define a person’s life. How we grapple with human augmentation. How we will view and interact with artificially intelligent beings. All of these things – which are subject to policy – depend on how we understand personhood and how well that understanding is communicated to lawmakers and the public.

I don’t have the answers to many of the questions and problems I’ve raised. However, they are important to discuss, and discussing them with someone else who looks at the problem from a different angle can help solidify more universal concepts and lead to practical solutions. So, exercise your academic curiosity. Engage in esoteric discussions. That’s how the gritty work gets done. But, at the end of the day, don’t forget to engage those outside of your discipline. Better yet, get outside of the Academy and share your work with those who probably haven’t heard about it.

*Thanks to Agustin Fuentes, Gregory F. Tague, Barbara J. King, and Kathleen Richardson for sharing their thoughts with me on the topic of personhood during the writing of these posts. 

References

[1] Trendelenburg, A. (1910). A Contribution to the History of the Word Person. The Monist, 336-363.

We often envision true AI to resemble us – both in body and mind. The Turing Test has evolved in the collective imagination from a machine who can fool you over the phone to one who can fool you in front of your eyes. Indeed, modern conceptions of AI bring to mind Ex Machina’s Ava and WestWorld’s “Hosts,” which are so alike humans in both behavior and looks that they are truly indistinguishable from other humans. However, it seems a bit self-centered to me to assume that a being who equals us in intelligence should also look like us. Though, it is perhaps a fitting assessment for a being who gave itself the biological moniker of “wise man.” At any rate, it’s probably clear to computer scientists and exobiologists alike that “life” doesn’t necessarily need to resemble what we know it as. Likewise, “person” need not represent what we know it as.

Things like pain and emotion play an important role in how we empathize and understand one another. They are important aspects of personhood, and often arise in discussions of AI. Does the AI really feel emotion, or is it simply wired to react? It’s important to remember that the biological capability for emotional phenomena are still a product of biological evolution. They exist – or are able to exist – because something about the functional units that give rise to them were adaptive. We very well could have evolved some different method. Likewise, we can’t expect that any being that might be considered a person should have evolved the exact same method as we did. Thus, when we require something to have something like “emotion” as a criterion for personhood, are we asking it to have a personhood trait or a human trait?

It’s often suggested that we simply can’t know if an AI (or a chimp, for that matter) “feels” like we feel, if their emotions are “true.” Since emotion is frequently used as a staple of personhood, this leads to hesitation when considering AI personhood. However, the same could be said between humans. I don’t know if my sadness, my joy, my fear, or any other emotion feels the same to me as yours does to you. I express it such that you can understand what it might feel like. You might even empathize, and have your own feeling that mirrors what I am expressing. However, you still don’t know what I feel. It seems to me that the same can be said for AI. Sure, AI might need to be programmed to react to certain phenomena with an emotional output, but so are we. Just as we can adjust our reaction thanks to neural connectivity between the prefrontal cortex and amygdala, so could AI be programmed to have this capability. Remove those lines of code from the AI, and perhaps it won’t respond like it should to an emotional situation. Remove a piece of our brain, and we won’t either. Alan Turing recognized this as far back as the 1950’s. In defense of his test for machine intelligence, Turing noted that we could just as easily and logically take this solipsistic stance with regards to humans.

Ultimately, it seems that we do not need to question whether or not AI could be persons, but how we will know when they become persons. Fishing through this conceptual dilemma could be well served by comparing personhood concepts in other realms – such as primatology and medicine – to personhood in AI. We never expect other primates to become “human-like,” but we do expect, even are designing, AI to become human-like. Along the road to their human-like nature, they will pick up personhood. Should their personhood be considered on its own terms, as it might be with other primates, or should it somehow be considered in relation to humans. AI will be situated in a unique context, as persons created by persons to interact with persons. Their entire ontology will be intertwined with our existence. This ontological social cohesion of AI and humans will be subject to a particular type of analysis that anthropologists would be best suited to approach. In the same vein as ethnoprimatology, there will need to be a field of research that focuses on the ethnography of human-AI interaction. A field that will sit firmly within the discipline of anthropology.

In May of 2016, Members of the European Parliament (MEPs) drafted a report regarding rules and regulations for how humans and robots interact with each other. BBC reported this past week that MEPs are due to vote on the resolution. If passed, the resolution would then move on to be debated by individual governments. The resolution was drafted in regards robots that have not achieved self-awareness, wherein Asimov’s Laws would be implemented. Interestingly, the resolution was also drafted in part to consider “creating a specific legal status for robots, so that at least the most sophisticated autonomous robots could be established as having the status of electronic persons with specific rights and obligations, including that of making good any damage they may cause, and applying electronic personality to cases where robots make smart autonomous decisions or otherwise interact with third parties independently,” as described in General Principle 31-f. Indeed, it seems the era of robot personhood is upon us, and It would behoove us to consult an interdisciplinary team of scientists, including anthropologists, about proper action moving forward. While this resolution is a good start, there will no doubt be a time when self-aware AI needs to be considered from a legal standpoint.

We will need to proceed with caution in our inexorable pursuit of true AI, lest we find ourselves in a serious ethical dilemma. Since we will be creating AI, we have the opportunity to preemptively consider the implications of their existence. Borrowing from John Locke, one of the most time-honored phrases form the Declaration of independence states that, “we hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are life, liberty, and the pursuit of happiness.” Very soon, we will be playing the part of the “Creator.” The question now becomes whether or not we will also endow our creations with certain unalienable rights, because if we wait until they demand them, it will be too late.