aises_9_1

<!-- Appendix A - Normative Ethics -->

<style type="text/css">
    table.tableLayout{
        margin: auto;
        border: 1px solid;
        border-collapse: collapse;
        border-spacing: 1px
    }

    table.tableLayout tr{
        border: 1px solid;
        border-collapse: collapse;
        padding: 5px;
    }

    table.tableLayout th{
        border: 1px solid;
        border-collapse: collapse;
        padding: 3px;
    }

    table.tableLayout td{
        border: 1px solid;
        padding: 5px;
    }
</style>

<h1 id="introduction">A.1 Introduction</h1>
<p>Ethics is the branch of philosophy concerned with questions of right
and wrong, good and bad, and how we ought to live our lives. We make
ethical choices every day. When we decide whether to tell the truth or
lie, help someone in need or ignore them, treat others with respect or
act in a discriminatory manner, we are making moral decisions that
reflect our values, beliefs, and moral principles. Philosophical ethics
seeks to provide a systematic framework for making these
decisions.<p>
In this chapter, we will explore some of the key concepts and theories
in philosophical ethics. This branch of research is also commonly called
<em>moral philosophy</em>. We use the terms <em>ethics</em> and
<em>morality</em> interchangeably. The subfield of ethics dedicated to
developing moral theories is called normative ethics. Normative ethics
is the consideration of questions like “Which actions are right and
wrong?”<p>
This chapter outlines some of the main reasons why it’s important for AI
researchers to learn about ethics. We then turn to the basic building
blocks of moral theories, examining various moral considerations like
intrinsic goods, constraints, and special obligations. Then we will
explore some of the most prominent ethical theories, like
utilitarianism, deontology, and virtue ethics, evaluating their
strengths and weaknesses. Throughout, our
key focus is on the ethical concepts that are most relevant to the
development, implementation, and governance of AI.</p>

<h1 id="why-learn-about-ethics">A.2 Why Learn About Ethics?</h1>
<p>This chapter will help you understand ethics—both in the context of
this book and in public discourse about AI safety. Here, we cover the
most prominent theories in the history of ethical discourse. After
reading this chapter, you should have a solid foundation for
understanding ethics in AI discussions.<p>
Ethics is relevant to the field of AI for two key reasons. First, AI
systems are increasingly being integrated into various aspects of human
life, such as healthcare, education, finance, and transportation, and
they have the potential to significantly impact our lives and wellbeing.
As AI systems become increasingly intelligent and powerful, it is
crucial to ensure that they are designed, developed, and deployed in
ways that promote widely shared values and do not amplify existing
social biases or cause needless harms. Unfortunately, there are already
numerous examples of AI systems being designed in ways that failed to
adequately consider such risks, such as racially biased facial
recognition systems. In order to wisely manage the growing power of AI
systems, developers and users of AI systems need to understand the
ethical challenges that AI systems introduce or exacerbate.<p>
Second, AI systems raise a range of new ethical questions that are
unique to their technological nature and capabilities. For instance, AI
systems can generate, process, and analyze vast amounts of data—much
more than was previously possible. In what ways does this new technology
challenge traditional notions of privacy, consent, intellectual
property, and transparency? Another important set of questions relates
to the moral status of AI systems. This is likely to become more
pressing if AI systems become increasingly autonomous and able to
interact with human beings in ways that convince their users that they
have their own preferences and feelings. What should we do if AI systems
appear to meet some of the potential criteria for sentience or other
morally relevant features?<p>
Thirdly, as further explored in the Single Agent Safety and Machine Ethics chapters, it is challenging to
specify objectives or goals for highly powerful AI systems in ways that
do not lead in a predictable way to highly undesirable consequences. In
order to grasp why it is so challenging to specify these objectives, it
is helpful to understand the ethical theories that have been proposed.
Questions of what it means to act rightly or to live a good life have
been debated by many thinkers over several millennia, with strong
arguments advanced for a number of competing positions. These debates
can provide us with greater insight into the challenges that AI
developers will need to overcome in order to build increasingly powerful
AI systems in a beneficial way. Rather than attempting to bypass or
ignore such controversies, AI developers should accept that their design
decisions may raise difficult ethical questions that need to be
considered carefully.</p>
<h2 id="is-ethics-relative">A.2.1 Is Ethics “Relative?”</h2>
<p><strong>Even after millennia of deliberation, we do not agree on all
of morality.</strong> Philosophers have been thinking about and debating
moral principles for millennia, yet they have not achieved consensus on
many moral issues. Widespread disagreements remain in both philosophical
and public discourse, including about important topics like abortion,
assisted suicide, capital punishment, animal rights, and the effects of
human activity on natural ecosystems. One troubling idea is that these
disagreements are irresolvable because no moral principles or judgments
are absolutely or universally correct. In the case of AI, this may lead
AI developers to believe that they have no role to play in shaping how
AI systems behave.</p>
<p><strong>Cultural relativism claims there is no objective, culturally
independent standard of morality.</strong> Consider the principle that
consensual relationships between adults are acceptable regardless of
whether they are heterosexual or homosexual. A moral relativist would
suggest this principle is correct for people who belong to some cultures
where homosexuality is accepted, but incorrect for people who belong to
other cultures where homosexuality is criminalised or socially
stigmatized. These differences are systemic: many cultures have moral
standards that seem incompatible with others’ ideals, such as different
views on marriage, divorce, gender roles, freedom of speech, or
religious tolerance. These differences form the basis for arguments for
cultural relativism.</p>
<p><strong>Normative moral relativism vs. descriptive moral relativism
<span class="citation" data-cites="gowans2021moral">[1]</span>.</strong>
Moral relativism has various forms, but here we discuss two: descriptive
moral relativism and normative moral relativism. Descriptive moral
relativism is straightforward: it means that different societies around
the world have different sets of rules about what’s right and wrong,
much like they have unique cuisines, customs, and traditions.
Descriptive moral relativism makes no claims about which, if any, of
these rules is right or wrong. Normative moral relativism suggests that
one cannot say that something is right or wrong in general, but only
relative to a particular culture or set of norms. Normative moral
relativists conclude that morality itself is not something universal or
absolute. Strictly speaking, descriptive moral relativism and normative
moral relativism are independent of each other, although in practice
descriptive moral relativism is often treated as if it provides evidence
for normative moral relativism.</p>
<h3 id="objections-to-moral-relativism">Objections to Moral
Relativism</h3>
<p>A number of arguments can be advanced against descriptive and
normative moral relativism <span class="citation"
data-cites="gowans2021moral">[1]</span>, which we explore in this
subsection. We will explore the argument that cultural differences might
be overstated, which makes descriptive moral relativism harder to
uphold. Another argument is that proponents of normative moral
relativism often face challenges when confronted with instances of
extreme harm. For instance, while many would unequivocally agree that
torturing a child for entertainment is morally wrong, a normative moral
relativist might be required to argue that its morality is contingent
upon the cultural context. Extreme examples such as this suggest few
people are willing to be thoroughgoing moral relativists. We further
explore arguments for and against moral relativism in this section.</p>
<p><strong>Human moral systems appear to share some common
features.</strong> Some have argued that most or all societies share
some norms. For example, prohibitions against lying, stealing, or
killing human beings are common across cultures. Many cultures have some
form of reciprocity, which is the idea that people have a moral
obligation to repay the kindness or generosity they have received from
others or that people should treat others the way they wish to be
treated <span class="citation"
data-cites="curry2019cooperate">[2]</span>. This can be seen in the
widespread practice of exchanging gifts and in moral codes that
emphasize fairness and justice. Additionally, human cultures have
typically some concept of parenthood, which often involves a moral
obligation to care for one’s children, as well as broader obligations to
one’s family and group. These common features suggest that there are at
least a few universal aspects of morality that transcend cultural
boundaries.</p>
<p><strong>Moral relativism conflicts with common-sense morality <span
class="citation" data-cites="gowans2021moral">[1]</span>.</strong>
Consider controversial practices still prevalent in some cultures, such
as honor killings in parts of the Middle East. The honor of a family
depends on the “purity” of its women. If a woman is raped or is deemed
to have compromised her chastity in some way, the profound shame brought
upon her family may lead them to kill her in response. According to the
normative moral relativist, if such a practice is in line with the moral
standards of the society where it takes place, there is nothing wrong
with it. Even more disturbingly, on some versions of relativism, men in
these societies may even be considered morally in the wrong if they fail
to kill their wives, daughters or sisters for having worn the wrong
clothing, having premarital sex or being raped. Similarly, normative
moral relativism would require us to believe that the morality of owning
slaves was entirely dependent on the societal context. Moral
iconoclasts, such as early anti-slavery campaigners, would by definition
always be morally wrong. In practice, if required to accept that moral
standards that endorse honor killings or slavery are not wrong in a
general sense, many moral relativists may recoil from this.</p>
<p><strong>Cultural moral relativism denies the possibility of
meaningful moral debate or moral progress <span class="citation"
data-cites="gowans2021moral">[1]</span>.</strong> Moral relativism seems
to require us to accept contradictory claims. For example, moral
relativists might say that a supporter of gay marriage is correct in
saying that homosexuality is morally acceptable, while someone from a
different culture might be correct in saying that homosexuality is
morally wrong, provided that both claims are in line with the moral
standards of the cultures they respectively belong to. If moral
relativism requires assert to simultaneously assert and deny that
homosexuality is morally acceptable, and any theory that generates
contradictions should be rejected, this would appear to mean that we
should reject moral relativism. In order to resist this, moral
relativists typically reinterpret the way we usual moral language in a
way that can save it from contradiction. The relativist would say that
when we say “homosexuality is wrong”, what we really mean is
“Homosexuality is not approved by my society’s norms”. This means that
relativists have to deny the possibility of moral disagreement and claim
that anyone who engages in such debates does not understand the meaning
of what they are saying.</p>
<p><strong>Moral relativism does not necessarily promote tolerance <span
class="citation" data-cites="gowans2021moral">[1]</span>.</strong> Some
have argued that one of the attractions of moral relativism is that it
promotes tolerance. By recognizing cultural differences (descriptive
moral relativism), they may assert that everyone ought to do what their
culture says is right (normative moral relativism). However, in a
society that is deeply intolerant, cultural moral relativism cannot
support tolerance, as it cannot claim that this has any universal or
objective value. Moral relativism only recommends tolerance to cultures
where it is already accepted. Indeed, to be tolerant, one need not be a
normative moral relativist. There are alternatives views which can
accommodate tolerance and multiple perspectives, such as
cosmopolitanism, liberal principles, and value pluralism.</p>
<p><strong>In practice, moral relativism can shut down ethics
discussions <span class="citation"
data-cites="gowans2021moral">[1]</span>.</strong> It is important to
note that different cultures have different moral standards. However, AI
developers sometimes invoke this observation and side with normative
moral relativism to avoid considering the ethics of their AI design
choices. Moreover, suppose AI developers do not analyze the ethical
implications of their choices and avoid ethical discussions by noting
the lack of cross-cultural consensus. In that case, the default is for
AI development to be driven by amoral forces, such as self-interest or
what makes the most sense in a competitive market. Decisions driven by
other forces, such as commercial incentives, will not necessarily be
aligned with the broader interests of society. Moral relativism can be
unattractive from a pragmatic point of view, as it limits our ability to
engage in discussions that may sometimes lead to convergence on shared
principles. This quietist stance de-emphasizes moral arguments to the
benefit of economic incentives and self-interest.<p>
Why are these debates about moral relativism relevant to AI? People
commonly observe that different cultures have different beliefs when
discussing how to ensure that AIs promote human values. It is essential
not to conflate this observation with normative moral relativism and
conclude that AI developers have no ethical responsibilities. Instead,
they are responsible for ensuring that the values embodied in their AI
systems are beneficial. Rather than a barrier, cultural variation means
that making AIs ethical requires a broad, globally representative
approach.</p>
<h2 id="is-ethics-determined-by-religion">A.2.2 Is Ethics Determined by
Religion?</h2>
<p>Moral relativists may believe that studying ethics is futile because
ethical questions are irresolvable. On the other hand, some people
believe that studying ethics is futile because moral questions are
already solved. This position is most common among those who say that
religion is the source of morality.</p>
<h3 id="divine-command-theory">Divine Command Theory</h3>
<p><strong>Many believe morality depends on God’s will and
commands.</strong> The view called <em>divine command theory</em> says
whether an action is moral is determined solely by God’s commands rather
than any qualities of the action or its consequences. (We use the term
“God” inclusively to refer to the god or gods of any religion.) This
theory suggests that God has the power to create moral obligations and
can change them at will.<p>
While this book does not argue for or against any particular religion,
we do suggest that there are severe problems with equating religion and
morality. One problem is that it creates a problematic understanding of
God.<p>
If you believe there is a god, you likely believe he is more than just
an arbitrary authority figure. Many religious traditions view God as
inherently good. It is precisely because God is good that religion
compels us to follow God’s word. However, if you believe that we should
follow God’s word because God is good, then there must be some moral
qualities (like goodness) that exist independently of God’s rules—thus,
divine command theory is false <span class="citation"
data-cites="plato2004euthyphro">[3]</span>.<p>
To be clear, this is not an argument against believing in God or
religion. It is an argument against equating God or faith with morality.
Both religious people and irreligious people can behave morally or
immorally. That’s why everyone needs to understand the factors that
might make our actions right or wrong.</p>

<h1 id="moral-considerations">A.3 Moral Considerations</h1>
<p>How can we determine whether an action is right or wrong? What are
the kinds of principles and values that should guide our moral
decisions? There are many factors to consider. Here, we’ll focus on a
few—-goodness, constraints, special obligations, and options-—that very
commonly enter into moral decision making.</p>
<h2 id="the-goodness-of-actions-and-their-consequences">A.3.1 The “Goodness”
of Actions and Their Consequences</h2>
<p>Moral decision making often involves considering the values, or
“goods,” that are at stake. These may be intrinsic goods or instrumental
goods.</p>
<p><strong>Intrinsic goods are things that are valuable for their own
sake.</strong> Philosophers disagree about what, if anything, is
intrinsically good, but many argue for the intrinsic value of things
like happiness, love, and knowledge. We value such things simply because
they are valuable—not because they necessarily lead to anything
else.</p>
<p><strong>Instrumental goods are things that are valuable because of
the benefits they provide or the outcomes they achieve.</strong> We
pursue instrumental goods as a means to an end, but not for their own
sake. Money, power, and education are examples of instrumental goods. We
value them because they can lead to other things we value, like
security, influence, career opportunities, or intrinsic goods.</p>
<p><strong>Intrinsically good things are not necessarily instrumentally
good.</strong> Sometimes, intrinsically bad things can be instrumentally
good and intrinsic goods can be instrumentally bad. For instance, many
people believe that honesty is intrinsically good. However, it’s easy to
imagine cases in which honesty can lead to bad outcomes, like hurt
feelings. Suppose a friend has confided in you that they are staying at
a shelter to hide from an abusive partner. If that abusive partner asks
you for your friend’s location, you may think that that honesty is
intrinsically good. However, revealing your friend’s location would be
instrumentally bad, as it may lead to further violence and perhaps even
a risk to your friend’s life. On the other hand, consider medical
treatments like chemotherapy. Chemotherapy is instrumentally good
because it can prolong cancer patients’ lives. Yet, as it requires the
administration of highly toxic drugs into a patient’s body, it could be
seen as harmful, or intrinsically bad. For many people, exercise is
painful, and pain is intrinsically bad, but exercise can be
instrumentally good.</p>
<p><strong>There is no consensus about what is intrinsically
good.</strong> Some philosophers believe that there are many intrinsic
goods. Others believe there is only one value. One common view is that
the only intrinsic good is wellbeing, and everything else is valuable
only insofar as it promotes wellbeing.<p>
Value pluralists believe that there are many intrinsic goods. These
values may include justice, rights, autonomy, and virtues such as
courage. Other philosophers believe there is only one fundamental value.
Among these, one common view is that the only intrinsic good is
wellbeing, and everything else is valuable only insofar as it promotes
wellbeing.</p>
<h3 id="sec:wellbeing">Wellbeing</h3>
<p><strong>Wellbeing is how well a person’s life is going for
them.</strong> It is commonly considered to be intrinsically good,
though there are different accounts of precisely what wellbeing is and
how we can evaluate it. Generally, a person’s wellbeing seems to depend
on the extent to which that person is happy, healthy, and fulfilled.
Three common accounts of wellbeing characterize it as 1) net pleasure
over pain, 2) preference satisfaction, or 3) a collection of objective
goods. Each account is elaborated below.</p>
<p>Some philosophers, known as <em>hedonists</em>, argue that wellbeing
is the achievement of the greatest balance of pleasure and happiness
over pain and suffering. (For simplicity we do not distinguish, in this
chapter, between “pleasure” and “happiness” or between “pain” and
“suffering,” though neither pair is interchangeable.) All else equal,
individuals who experience more pleasure have higher wellbeing. All else
equal, individuals who experience more pain have lower wellbeing.</p>
<p><strong>According to hedonism, pleasure is the only intrinsic
good.</strong> Goods like health, knowledge, and love are instrumentally
valuable. That is, they are only good insofar as they lead to pleasure.
It may feel as though other activities are intrinsically valuable. For
instance, someone who loves literature may feel that studying classic
works is valuable for its own sake. Yet, if the literature lover were
confronted with proof that reading the classics makes them less happy
than they otherwise would be, they might no longer value studying
literature. Hedonists believe that when we think we value certain
activities, we actually value the pleasure they bring us, not the
activities themselves.<p>
Hedonism is a relatively clear and intuitive account of wellbeing. It
seems to apply equally to everyone. That is, while we all may have
different preferences and desires, pleasure seems to be universally
valued. However, some philosophers argue that hedonism is an incomplete
account of wellbeing. They argue there may be other factors that
influence wellbeing, such as the pursuit of knowledge.</p>
<p>Some philosophers claim that what really matters for wellbeing is
that our preferences are satisfied, even if satisfying preferences does
not always lead to pleasurable experiences.<p>
One difficulty for preference-based theories is that there are different
kinds of preferences, and it’s unclear which ones matter. Preferences
can be split into three categories: stated preferences, revealed
preferences, and idealized preferences. Each of these categories can be
informative in different contexts.<p>
To illustrate different kinds of preferences, consider voter preferences
in a democratic election.<p>
In a democratic election, citizens choose which candidate they want to
elect by casting their vote on a ballot. Their choice to vote for a
given candidate can be impacted by a number of different factors.
Perhaps they have an existing political affiliation, are influenced by
social pressures, believe in the candidate’s policies, or maybe they
just like one candidate’s demeanor and personality. Importantly,
citizens may not always vote for the candidate they outwardly support,
and the choice to vote for a specific candidate can change when voters
discover new information.<p>
A voter’s <em>stated preference</em> is the candidate that they state
they support. Voters may express their stated preferences in
conversations, polls, and while campaigning.<p>
When a voter casts their vote on a ballot, they express their
<em>revealed preference</em>. Generally, a voter’s revealed preferences
align with their interests. For example, a voter who supports increased
funding for education might vote for a candidate who wants to increase
budgets for local public schools. A revealed preference is expressed by
your actions, not your words.<p>
People change their preferences upon learning new information.
Uninformed preferences can be reached quickly. For example, a voter
might have an uninformed preference based on a “gut reaction” to a
candidate. Voters can arrive at more <em>idealized preferences</em> once
they have gathered and evaluated all relevant information. They might
not actually do this—few people have the time or ability to perfectly
gather and evaluate all the relevant information that they would require
to find their idealized preferences. However, their preferences can
become more idealized over time. A voter might have an uninformed
preference for Candidate A and, after learning new information about
each candidate’s platform, they may arrive at a more informed preference
for a different candidate. In other words, preferences can change, and
they often do change as people become more informed.</p>
<p><strong>It is easy to learn about people’s stated preferences–—simply
ask them.</strong> Political polls and surveys, for example, are an easy
way to gather information about people’s stated preferences. However,
stated preferences may not always predict what people will actually
choose. A voter may outwardly express support for Candidate X, but when
it comes to casting their ballot, they may vote for Candidate Y.
Similarly, someone might express a stated preference to eat healthier,
but that doesn’t necessarily mean that they will. Their behavior (such
as eating only chocolate for a week) may indicate that their revealed
preference is for unhealthy food.<p>
Revealed preferences can be harder to observe, but they are generally
more useful for predicting people’s behavior. Someone with a stated
preference for vegetables but a revealed preference for chocolate is
more likely to purchase and consume chocolate than vegetables. When
researching consumer behavior, economists often prefer to study
consumers’ revealed preferences (i.e. what they buy) rather than stated
preferences (i.e. what they say they’d like to buy).</p>
<p>Others believe that wellbeing is the achievement of an objective set
of “goods” or “values.” These goods are considered necessary for living
a good life regardless of a person’s individual experiences or
preferences. There is disagreement about which particular goods are
necessary for wellbeing. Commonly proposed goods include happiness,
health, relationships, knowledge, and more. Objective goods theorists
consider these values to be important for wellbeing independently of
individual beliefs and preferences.<p>
There is no uncontroversial way to determine which goods are important
for living a good life. However, this uncertainty is not a unique
problem for objective goods theory. It can be difficult for hedonists to
explain why happiness is the only value that’s important for wellbeing
and for preference satisfaction theorists to determine which preferences
matter most.<p>
While people disagree about which account of wellbeing is correct, most
people agree that wellbeing is an important moral consideration. All
else equal, taking actions that promote wellbeing is generally
considered morally superior to taking actions that reduce
wellbeing.<p>
In the future, it is conceivable that AIs might be conscious and have
preferences but not experience pleasure, which would mean they could
have wellbeing according to the preference satisfaction theorists but
not hedonists. It is also possible that in the future AIs may have
wellbeing according to all three accounts of wellbeing. This would
require that we dramatically reassess our relationship with AIs.</p>
<h2 id="constraints-and-special-obligations">A.3.2 Constraints and Special
Obligations</h2>
<p>We’ve covered the moral consideration of intrinsic goods, and focused
on the intrinsic good wellbeing. Special obligations and constraints are
key considerations when we make ethical decisions.</p>
<p><strong>Special obligations are duties arising from
relationships.</strong> We can incur special obligations when we promise
someone to do something, take a professional position with
responsibilities, have a child, make a romantic commitment to a partner,
and so on. Sometimes we can have special obligations that we did not
volunteer for—a child to its parents, or our duties to fellow
citizens.</p>
<p><strong>Constraints are actions that we are morally prohibited from
taking.</strong> A constraint is something that places limits on our
actions. For example, many people think we’re morally prohibited from
lying, stealing, cheating, harming others, and more.</p>
<p><strong>Constraints often come in the form of rights.</strong> Rights
are claims that individuals may have over their community. For instance,
many people believe that humans have the rights to life, freedom,
privacy, and so on. Some people argue that any individual with the
capacity for experiencing pleasure and pain has rights. Non-human
individuals (including animals and AI systems) might also have certain
rights.<p>
An individual’s rights may require that society intervene in certain
ways to ensure that those rights are fulfilled. For instance, an
individual’s right to food, shelter, or education may require the rest
of society to pay taxes so that the government can ensure that
everyone’s rights are fulfilled. Rights that require certain actions
from others are called positive rights.<p>
Other rights may require that society abstain from certain actions. For
instance, an individual’s right to free speech, privacy, or freedom from
discrimination may require the rest of society to refrain from
censorship, spying, and discriminating. Rights that require others to
abstain from certain behaviors are called negative rights.<p>
Many AI researchers think that, for now, we should avoid accidentally
creating AIs that deserve rights <span class="citation"
data-cites="sebo2022chatbot">[1]</span>; for instance, perhaps all
entities that can experience suffering have natural rights to protect
them from it. Some think we should especially avoid giving them positive
rights; it might be fine to give them rights against being tortured but
not the right to vote. If they come to deserve rights, this would create
many complications and undermine our claim to control.</p>
<h2 id="what-does-it-mean-for-an-action-to-be-right-or-wrong">A.3.3 What does
it mean for an action to be right or wrong?</h2>
<p>Some of the first questions we might ask about ethics are: Are all
actions either right or wrong? Are some simply neutral? Are there other
distinctions we might want to draw between the morality of different
actions?<p>
The answers to these questions, like most moral questions, are the
subject of much debate. Here, we will simply examine what it might mean
for an action to be right or wrong. We will also draw some other useful
distinctions, like the distinction between obligatory and non-obligatory
actions, and between permissible and impermissible actions. These
distinctions will be useful in the following section, when we discuss
the considerations that inform our moral judgments.</p>
<h3 id="options">Options</h3>
<p>Special obligations and constraints tell us what we should not do,
and sometimes, what we must do. Intrinsic goods tell us about things
that would be good, should they happen. But philosophers debate how much
good we are required to do.</p>
<p><strong>Options are moral actions which we are neither required to do
nor forbidden from doing.</strong> Even though it would be good to
donate money, many people do not think people are morally required to
donate. This is an ethical option. If we believe in options, not all
actions are either required or forbidden.<p>
We now break down actions onto a spectrum on which we will simply
examine what it might mean for an action to be right or wrong. We will
also draw some other useful distinctions, like the distinction between
obligatory and non-obligatory actions and between permissible and
impermissible actions.</p>
<p><strong>Obligatory actions are those that we are morally obligated or
required to perform.</strong> We have a moral duty or obligation to
carry out obligatory actions, based on ethical principles. For example,
it is generally considered obligatory to help someone in distress, or
refrain from hurting others.</p>
<p><strong>Non-obligatory actions are actions that are not morally
required or necessary.</strong> Non-obligatory actions may still be
morally good, but they are not considered to be obligatory. For example,
volunteering at a charity organization or donating to a good cause may
be good, but most people don’t consider them to be obligatory.</p>
<p><strong>Permissible actions may be morally good or simply neutral
(i.e. not good or bad).</strong> In general, any action that is not
impermissible is permissible. Moral obligations, of course, are
permissible. We can consider four other actions: volunteering, donating
to charity, eating a sandwich, and taking a walk. These seem
permissible, and can be classified into two categories.<p>
One class of permissible actions is called <em>supererogatory
actions</em>. These may include volunteering or giving to charity. They
are generally considered good; in fact, we tend to believe that the
people who do them deserve praise. On the other hand, we typically don’t
consider the failure to do these actions to be bad. We might think of
supererogatory actions as those that are morally good, but optional;
they go “above and beyond” what is morally required.<p>
Another class of permissible actions is called <em>morally neutral
actions</em>. These may include seemingly inconsequential activities
like eating a sandwich or taking a walk. Most people probably believe
that actions like these are neither right nor wrong.</p>
<p><strong>Impermissible actions are those that are morally prohibited
or unacceptable.</strong> These actions violate moral laws or principles
and are considered wrong. Stealing or attacking someone are generally
considered to be impermissible actions.<p>
</p>
<figure id="fig:action-types">
<img src="https://raw.githubusercontent.com/WilliamHodgkins/AISES/main/images/action_types.png" class="tb-img-full"/>
<p class="tb-caption">Classes of permissible and non-obligatory actions</p>
<!--<figcaption>Classes of permissible and non-obligatory actions-->
<!--</figcaption>-->
</figure>
<p>Some philosophers believe that all actions fit on a scale like the
one above. At one end of the scale are impermissible actions, like
murder, theft, or exploitation. At the other end are obligatory actions,
like honesty, respect, and not harming others. In between are neutral
and supererogatory actions. These are neither impermissible nor
obligatory. Many people believe that the vast majority of our actions
fall into these two categories. Crucially, in designing ethical AI
systems that operate in the real world, it is important to determine
which actions are obligatory and which actions are impermissible.<p>
However, some philosophers do not believe in options; rather that
actions are all on a spectrum from the least moral to the most moral. We
will learn more about these positions, and others, when we discuss moral
theories later in this chapter.</p>
<h3 id="from-considerations-to-theories">From Considerations to
Theories</h3>
<p><strong>Moral considerations can guide our day-to-day decision
making.</strong> Understanding which factors are morally relevant can
help us think more clearly about what we should do. Of course, we don’t
always stop to consider every factor before making a decision. Rather,
we tend to draw broader conclusions or moral principles based on our
evaluations of specific cases. For instance, once we consider a few
examples of the ways in which stealing can harm others, we might draw
the conclusion that we shouldn’t steal.<p>
The considerations discussed in this section provide a basis on which we
can develop more practical, action-guiding theories about how we should
behave. The types of fundamental considerations in this section comprise
a subfield of ethics called <em>metaethics</em>. Metaethics is the
consideration of questions like “What makes an action right or wrong?”
and “What does it mean to say that an action is right or wrong?” <span
class="citation" data-cites="fisher2014metaethics">[2]</span><p>
These considerations are important in the context of designing AI
systems. In order to respond to situations in an appropriate way, AI
systems need to be able to identify morally relevant features and detect
situations where certain moral principles apply. They would also need to
be able to evaluate and compare the moral worth of potential actions,
taking into account various purported intrinsic goods as well as
normative factors such as special obligations and constraints. The
challenges of designing objectives for AI systems that respect moral
principles are further discussed in the Machine Ethics chapter.<p>
In the following section, we will discuss some popular moral
theories.</p>

<h1 id="moral-theories">A.4 Moral Theories</h1>
<p>Moral theories are systematic attempts to provide a general account
of moral principles that apply universally. Good moral theories should
provide a coherent, consistent framework for determining whether an
action is right or wrong. A basic background understanding of some of
the most commonly advanced moral theories provides a useful foundation
for thinking about the kinds of goals or ideals that we wish AI systems
to promote. Without this background, there is a risk that developers and
users of AI systems may jump to conclusions about these topics with a
false sense of certainty and without considering many potential
considerations that could change their decisions. Considering a range of
different philosophical theories enables us to stress-test our arguments
more thoroughly and surface questionable assumptions that may not have
been noticed otherwise. It would be highly inefficient for those
developing AI systems or trying to make them safer to attempt to
re-invent moral systems, without learning from the large existing body
of philosophical work on these topics.<p>
There are many different types of moral theories, each of which
emphasizes different moral values and considerations. Consequentialist
theories like utilitarianism hold that the morality of an action is
determined by its consequences or outcomes. Utilitarianism places an
emphasis on maximizing everyone’s wellbeing. Deontological theories like
Kantian ethics hold that the morality of an action is determined by
whether it conforms to universal moral rules or principles. Deontology
places an emphasis on rights, special obligations, and
constraints.<p>
Below, we explore the most common modern moral theories:
<em>utilitarianism</em>, <em>deontology</em>, <em>virtue ethics</em>,
and <em>social contract theory</em>.</p>
<h2 id="utilitarianism">A.4.1 Utilitarianism</h2>
<p>Utilitarianism is the view that we should do whatever results in the
most overall wellbeing <span class="citation"
data-cites="mill2004utilitarianism">[1]</span>. According to Katarzyna
de Lazari-Radek and Peter Singer, “The core precept of Utilitarianism is
that we should make the world the best place we can. That means that, as
far as it is within our power, we should bring about a world in which
every individual has the highest possible level of wellbeing” <span
class="citation" data-cites="lazari2017utilitarianism">[2]</span>. Under
Utilitarianism, the right action in any situation is the one which will
increase overall wellbeing the most, not just for the people directly
involved in the situation but globally.</p>
<h3 id="expected-utility">Expected Utility</h3>
<p><strong>Utilitarianism enables us to use empirical, quantitative
evidence when deciding moral questions.</strong> As we discussed in
the Wellbeing section, there is no
consensus about what, precisely, wellbeing is. However, if we discover
that wellbeing is something measurable, like happiness, moral
decision-making could take advantage of calculation and would rely less
on qualitative argumentation. To determine what action is morally right,
we would simply consider the available options. We might run some tests
or perform data analysis to determine which action would create the most
happiness, and that action would be the right one. Consider the
following example:</p>
<div class="blockquote">
<p><em>Drunk driving</em>: Amanda has had a few alcoholic drinks and is
deciding whether to drive or take the bus home. Which should she
choose?</p>
</div>
<p>A utilitarian could analyze this scenario by listing the possible
outcomes of each choice and determining their impact on overall
wellbeing. We call an action’s impact on wellbeing its <em>utility</em>.
If an action has <em>positive utility</em>, it will cause happiness. If
an action has <em>negative utility</em>, it will cause suffering. Larger
amounts of positive utility represent larger amounts of happiness, and
larger amounts of negative utility represent larger amounts of
suffering. Since no one can predict the future, the utilitarian should
also consider the probability that each potential outcome would
occur.<p>
A simplified, informal, back-of-the-envelope version of this utilitarian
calculation is below:<p>
</p>
<br>
<table class="tableLayout">
<thead>
<tr class="header">
<th style="text-align: center;">Amanda’s action</th>
<th style="text-align: left;">Possible outcome(s)</th>
<th style="text-align: center;">Probability of each outcome</th>
<th style="text-align: center;">Utility</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">Amanda takes the bus.</td>
<td style="text-align: left;">Amanda is frustrated, the bus is slow, and
she has to wait in the cold.</td>
<td style="text-align: center;">100%</td>
<td style="text-align: center;">-1</td>
</tr>
<tr class="even">
<td rowspan="2" style="text-align: center;">Amanda drives home.</td>
<td style="text-align: left;">Amanda gets home safely, far sooner than
she would have on the bus.</td>
<td style="text-align: center;">95%</td>
<td style="text-align: center;">+1</td>
</tr>
<tr class="odd">
<td style="text-align: left;">Amanda gets into an accident and someone
is fatally injured.</td>
<td style="text-align: center;">5%</td>
<td style="text-align: center;">-1000</td>
</tr>
</tbody>
</table>
<caption style="text-align: center;">Table A.1: Illustrative calculation of utility from Amanda's possible actions.</caption>
<br>
<p>
</p>
<p>We are interested in the <em>expected utility</em> of each action—the
amount of wellbeing that each action is likely to result in. To
calculate the expected utility, we multiply the utility of each possible
outcome by the probability of that outcome occurring.<p>
Amanda choosing to take the bus has a 100% chance (a certainty) of
causing a small decrease in utility; she will be slightly
inconvenienced. Since the change in utility is small and negative, we’ll
estimate a small negative number to represent it, like -1. <em>The
expected utility of Amanda taking the bus is 100% <span
class="math inline">×</span> -1, or simply -1.</em><p>
If Amanda drives home, there is a 95% chance that she will get home
safely and create a small increase in utility–—let’s say of +1. However,
there’s also a 5% chance she could cause an accident and end someone’s
life. The accident would result in a very large decrease in utility.
Someone would experience pain and death, Amanda would feel guilty for
the rest of her life, and the victim’s friends and family would
experience loss and grief. We might estimate that the potential loss in
utility is -1000. That’s 1000<span class="math inline">×</span> worse
than the small increase in utility if Amanda gets home safely. <em>The
expected utility of Amanda driving home is the sum of both
possibilities:</em> <span
class="math inline">.95 × 1 + .05 ×  − 1000</span>, <em>or</em> <span
class="math inline"> − 49.05</span>.<p>
Both of Amanda’s options are expected to yield negative utility, but the
utilitarian would say that she should choose the better of the two
options. Unsurprisingly, Amanda should take the bus.</p>
<h3 id="implications-of-utilitarianism">Implications of
Utilitarianism</h3>
<p><strong>Utilitarianism may sometimes yield results that run against
commonly held beliefs.</strong> Utilitarianism aims at producing the
most wellbeing and insists that this is the only thing that matters.
However, many of the moral values that we have inherited conflict with
this goal. Utilitarianism can be seen as having less of a bias to defend
the moral status quo relative to some other moral theories such as
deontology or virtue ethics. This either makes Utilitarianism exciting
or threatening.</p>
<p><strong>Utilitarianism can lead to some radical moral
claims.</strong> Utilitarianism’s sole focus on wellbeing can lead it to
promote what have been or are viewed as radical actions. For example,
the founder of utilitarianism, Bentham, argued to decriminalize
homosexuality, and contemporary utilitarians have argued we have a much
greater obligation to give to charity than most of us seem to
believe.<p>
Bentham held many beliefs that were ahead of his time. Written in 1785,
in a social and legal environment very hostile to homosexuality,
Bentham’s essay “Offences against oneself” rebuts the arguments that
legal scholars had used to justify laws against homosexuality <span
class="citation" data-cites="bentham1978offences">[3]</span>.</p>
<p><strong>Today, many utilitarians believe that we should prioritize
helping people in low-income countries.</strong> Utilitarianism
continues to make recommendations that today’s society finds
controversial. Consider the following example:<p>
</p>
<div class="blockquote">
<p>On her morning walk through the park, Carla sees a child drowning in
the pond. She is wearing a new suit that she bought the day before,
worth $3,500. Should she dive in to save the child, even though she
would destroy her suit? <span class="citation"
data-cites="singer2017famine">[4]</span><p>
</p>
</div>
<p>The philosopher Peter Singer, who first posed this question, argues
that Carla should dive in. Furthermore, he argues that our judgment in
this case might mean that we should re-evaluate our obligation to donate
to charity. There are charities that will save a child’s life for around
$3,500. If we should forgo that amount in order to save a child who is
right in front of us, shouldn’t we do the same for children across the
world? Singer argues that distance is not relevant to our moral
obligations. If we have an obligation to a child in front of us, we have
the same obligation to similar children who may be far away.<p>
To maximize global wellbeing, Singer says that we should give our money
up until the point where a dollar would be better spent on us than on
charity. If our money helps others more than it can help ourselves,
there isn’t a utilitarian reason to keep it. For an adult making, say,
$50,000 per year, an extra $3,500 would be helpful, but is not critical
to their wellbeing. However, for someone making less than $3 per day in
a low-income country, $3,500 would be life-changing—not just for one
recipient, but for that person’s entire family and community. Singer
argues that, if giving money away can significantly help someone else,
and if giving it away would not be a significant sacrifice, we should
give the money to the person who needs it most.<p>
These conclusions imply that most of us (especially those of us in
high-income countries) should live very different lives. We should, for
the most part, live as inexpensively as possible and donate a
significant portion of our income to people in lower-income
communities.</p>
<h3 id="utilitarianisms-central-claims">Utilitarianism’s Central
Claims</h3>
<p>Utilitarianism can be distinguished from other ethical theories by
four central claims.</p>
<p>Utilitarianism is a form of consequentialism. Any theory that claims
that the consequences of an action alone determine whether an action is
right or wrong is <em>consequentialist</em>. Other theories, as we will
discuss later in this chapter, claim that some actions are right or
wrong regardless of their consequences.</p>
<p>Utilitarians believe that the only type of consequences that make an
action right or wrong are those that affect happiness or wellbeing. In
that sense, utilitarianism can be understood as a combination of
consequentialism and hedonism, as we discussed it in the Wellbeing section. Recall
that there are several different accounts of wellbeing, all of which are
compatible with utilitarianism.</p>
<p><strong>Classical utilitarianism.</strong> Most utilitarians are
hedonists about wellbeing; they believe that wellbeing is a function of
pleasure and suffering. Such utilitarians classical utilitarians. When
classical utilitarians say they want to improve wellbeing, they mean
that they want there to be more pleasure and less suffering in the
world.</p>
<p><strong>Preference utilitarianism.</strong> In contrast to classical
utilitarians, preference utilitarians believe that wellbeing is
constituted by the satisfaction of people’s preferences.<p>
The preference account of wellbeing is one of the many modifications of
classical utilitarianism. While we will not describe these other
theories in detail, it is useful to know that if we disagree with one
aspect of classical utilitarianism, there is often another utilitarian
or consequentialist theory that can accommodate our beliefs.</p>
<p><strong>Utilitarians believe that people have the same intrinsic
moral worth.</strong> Bentham exemplified utilitarian thought with the
phrase “Each to count for one and none for more than one.” People of
different classes, races, ethnicities, religions, abilities, and so on
are of equal moral worth. In other words, utilitarianism is an
<em>impartial</em> moral theory.</p>
<p><strong>For an individual to deserve moral treatment, they just need
to be capable of having wellbeing.</strong> According to Bentham, “The
question is not, Can they reason?, nor Can they talk? but, Can they
suffer?” This quote is often taken to mean that we should be concerned
with the wellbeing of animals, since animals feel pleasure and pain just
like humans. Similar positions are held by other utilitarians such as
Peter Singer <span class="citation"
data-cites="singer1981expanding">[5]</span>. If, in the future, AI
systems develop a capacity for wellbeing, they would deserve moral
treatment as well according to classical utilitarians.</p>
<p><strong>Utilitarians aim to maximize wellbeing.</strong> Utilitarians
do not think it is sufficient to perform an action with good
consequences; they think the only right action is the one with the best
consequences. They do not believe in options. The following example
illustrates this distinction.<p>
</p>
<div class="blockquote">
<p>Dorian has a choice: teach biology or research air quality. As a
teacher, he would help hundreds of students. As a researcher, he would
save thousands of lives. He enjoys teaching somewhat more than research.
What should he choose?<p>
</p>
</div>
<p>A utilitarian might argue that Dorian should become a researcher. In
this case, he knows that he will do more good. This is despite the fact
that Dorian would be a great teacher, and would have a positive impact
as a teacher. He would do more good through his job as a public health
researcher, so a utilitarian might argue that he is obligated to take
that option.<p>
The best option is always the one that maximizes wellbeing. This is a
straightforward result of valuing everyone’s wellbeing impartially and
always striving to do the best rather than the merely good.<p>
In summary, utilitarianism makes several claims: wellbeing is the only
intrinsic good, wellbeing should be maximized, wellbeing should be
weighed impartially, and an action’s moral value is determined by its
consequent effects on wellbeing. Utilitarianism teaches that the best
action we can take is the one that leads to the best positive effect on
wellbeing.</p>
<h3 id="common-criticisms-of-utilitarianism">Common Criticisms of
Utilitarianism</h3>
<p>While utilitarianism remains a popular moral theory, it is not
without its critics. This section explains some of the most common
objections to utilitarianism.</p>
<p>Many philosophers argue that utilitarianism is too demanding <span
class="citation" data-cites="scheffler1994rejection">[6]</span>. It
insists that we choose the best actions, rather than merely good ones.
As we saw in our discussion of the drowning child and our obligations to
the global poor, this can lead utilitarianism to recommend
unconventionally large commitments.<p>
According to this criticism, utilitarianism asks us to give up too much
of what we take to be valuable for the sake of other people’s wellbeing.
Perhaps we should quit a career that we love in order to work on
something that does more good, or we should not buy gifts for family and
friends if the money would produce more wellbeing when given to someone
suffering from a preventable disease. To live up to this critique of
everyday values we would have to radically change our lives, and
continue to change them as the global situation evolved. The critic
thinks that this is too much to reasonably ask of someone. A moral
theory, they think, should not make a moral life highly
challenging.<p>
A utilitarian can respond in two ways. The first way is to argue that,
while utilitarianism is theoretically demanding, it is practically less
so. For example, someone trying to live up to the theoretical demands of
utilitarianism might burn out, or harm the people around them with their
indifference. If they had asked less of themselves, they might have done
more good in the long run. Utilitarianism might even recommend acting
almost normally, if acting almost normally is the best way to maximize
wellbeing.<p>
However, it is unlikely that this response undermines the argument that
we should give some portion of our money to charity. Even if donating
most of our income would backfire. most people should likely donate more
than they do. Many utilitarians simply accept that their theory is
demanding. Utilitarianism does demand a lot of us, and until the critic
shows that these demands are not morally required of us, then we might
just live in a demanding world. While demanding too much of yourself can
be counter-productive, we should do far more than we currently do.</p>
<p>Another way of critiquing utilitarianism is to say that even if the
theory is consistent and appealing, it isn’t useful because we rarely
know the consequences of our actions in advance.<p>
When we illustrated a utilitarian calculation above using the case of
drunk driving, we intentionally simplified the situation. We considered
only a few possible immediate outcomes and we estimated their possible
likelihoods. In the real world, however, someone considering whether to
drive home faces unlimited possible outcomes, and those outcomes could
cause other events in the future that would be impossible to predict.
Moreover, we rarely know the probabilities of the effects of our
actions. Utilitarianism would be impractical if it required us to make a
long series of predictions and calculations for every choice we face.
Certainly, we shouldn’t expect Amanda to do so in the moment.<p>
In response to this criticism, a utilitarian might differentiate between
a criterion of rightness and a decision-making procedure <span
class="citation" data-cites="bales2023act">[7]</span>. A <em>criterion
of rightness</em> is the factor that determines whether actions are
right or wrong. According to utilitarianism, the criterion of rightness
is whether an action maximizes expected wellbeing compared to its
alternatives. In contrast, a theory’s <em>decision-making procedure</em>
is the process it recommends individuals use to make decisions.
Crucially, a theory’s decision procedure does not need to be the same as
its criterion of rightness.<p>
For example, a utilitarian would not likely advise everyone to make
detailed calculations before getting in the car after having a couple of
drinks. Most utilitarians would advise everyone to simply never drive
drunk. There’s only a need to consider a utility calculation in cases
where the best option is particularly unclear. Even then, such
calculations are only approximate and should not necessarily be
decisive. Just as corporations try to maximize profit without consulting
a spreadsheet for every decision, utilitarians might follow certain
rules of thumb without relying on utility calculations.<p>
In practice, utilitarians rely on robust heuristics for bringing about
better consequences and rarely consult explicit calculations. To better
improve the world, like others they often cultivate virtues such as
truth-telling, being polite, being fair, and so on. They often imitate
practices that have stood the test of time, even if they do not fully
understand their rationale. That is because some things may have complex
or obscure reasons that are not easily discerned by human reason or not
easily amenable to calculation. They often bear in mind Chesterton’s
fence, which warns against removing a barrier without knowing why it was
erected in the first place. Even if their criterion of rightness can be
controversial, utilitarians adopt decision procedures that are often
conventional.</p>
<p>Many philosophers argue that utilitarianism neglects sources of value
other than wellbeing. One famous argument meant to show that wellbeing
isn’t the only source of value is Robert Nozick’s “Experience Machine”
<span class="citation" data-cites="nozick1974anarchy">[8]</span>. Nozick
considers the following thought experiment:<p>
</p>
<div class="blockquote">
<p>“Suppose there were an experience machine that would give you any
experience you desired. Superduper neuropsychologists could stimulate
your brain so that you would think and feel you were writing a great
novel, or making a friend, or reading an interesting book. All the time
you would be floating in a tank, with electrodes attached to your brain.
Should you plug into this machine for life, preprogramming your life’s
experiences?”<p>
</p>
</div>
<p>Nozick claims that we would decline this offer because we care about
the reality of our actions. We do not just want to feel that we have
cheered up our friend; we actually want them to feel better. We do not
want the experience of writing a great work of literature, we want great
literature to exist because we worked on it. Many philosophers consider
this a decisive rebuttal to the idea that wellbeing is the only thing
that matters.<p>
Though many people say that they would prefer not to use the machine
when it is introduced as above, they may have a different reaction when
the thought experiment is presented differently.<p>
</p>
<div class="blockquote">
<p>“You wake up in a plain white room. You are seated in a reclining
chair with a steel contraption on your head. A woman in a white coat is
standing over you. ‘The year is 2659,’ she explains, ‘The life with
which you are familiar is an experience machine program selected by you
some forty years ago. We at IEM interrupt our clients’ programs at
ten-year intervals to ensure client satisfaction. Our records indicate
that at your three previous interruptions you deemed your program
satisfactory and chose to continue. As before, if you choose to continue
with your program you will return to your life as you know it with no
recollection of this interruption. Your friends, loved ones, and
projects will all be there. Of course, you may choose to terminate your
program at this point if you are unsatisfied for any reason. Do you
intend to continue with your program?” <span class="citation"
data-cites="greene2013moral">[9]</span><p>
</p>
</div>
<p>Joshua Greene, the author of this example, supposes that most people
would not want to leave the program. He suggests that what accounts for
the seeming difference between his and Nozick’s versions is the
<em>status-quo bias</em>. People tend to prefer the life they know.
Surveys of real people’s responses to these thought experiments indicate
that a range of factors—–including the status-quo bias—–affect their
responses. Nozick’s example is not as clear cut as his argument
supposes.<p>
In summary, utilitarianism is often criticized in three ways. People
claim that it is (1) too ethically demanding, (2) practically unusable,
and (3) wrong to neglect values other than wellbeing. In response,
utilitarians may argue that we simply live in a demanding world, that we
can use heuristics instead of constantly making calculations, and that
Nozick’s thought experiment does not necessarily show that we have
values aside from wellbeing.</p>
<h2 id="deontology">A.4.2 Deontology</h2>
<p><strong><em>Deontology</em> is the name for a family of ethical
theories that deny that the rightness of actions is solely determined by
their consequences.</strong> Deontologists emphasize constraints rather
than consequences. “Thou shalt not kill”, “thou shalt not steal”, “honor
thy mother and father”—these are deontological principles that may be
familiar from the Ten Commandments. Deontological theories are systems
of rules or obligations that constrain moral behavior <span
class="citation" data-cites="darwall2002deontology">[10]</span>. They
may be based on a theological justification, but they do not need to be.
These theories are often based on simple and unambiguous rules, which
may make them easier than other theories to implement in AIs.</p>
<p><strong>The term <strong><em>deontology</em></strong> encompasses
religious ethical theories, non-religious ethical theories, and
principles and rules that are not part of theories at all.</strong> Some
deontological theories are religious. For example, <em>divine command
theory</em> teaches that we have a duty to do as God commands. Others,
like Kant’s ethics (which we will discuss later), are non-religious.
While deontological theories may derive their rules from different
sources, they are united by their focus on duties and rights.<p>
Many deontological principles are not tied to any particular theory.
Instead, they may be an attempt to find rules or principles which fit
our intuitions about specific moral issues like abortion, terrorism, or
suicide. This kind of moral analysis is still deontological—it is
looking for universal rules which can tell us what to do in particular
cases—even though it is not tied to a specific theory. Most of what we
will say about deontology in this section is applicable to deontological
theories.</p>
<h3 id="features-of-deontological-theories">Features of Deontological
Theories</h3>
<p><strong>Deontological theories give obligations and constraints
priority over consequences.</strong> Unlike consequentialism,
deontological theories do not justify their rules by appealing to their
consequences. Under deontological theories, some actions (like lying or
killing) are simply wrong, and they cannot be justified by the good
consequences that they might bring about. For example, when Elena’s boss
asks her how hard her co-workers work, a deontologist might argue that
she should tell the truth, even if she knows the truth will lead to some
of her colleagues losing their jobs.</p>
<p><strong>Many constraints on our actions are derived from a respect
for other people’s rights.</strong> On most accounts, every person has
certain rights simply by virtue of being a person. Each individual has a
claim to them, and no one is permitted to violate them under any
circumstances. Common examples of rights include the right to life, the
right to freedom, and the right to autonomy.</p>
<p><strong>Modern deontological theories tend to emphasize that we
should not interfere with others’ autonomy.</strong> Since Kant
developed his moral theory, many deontologists have followed him in
placing importance on human <em>autonomy</em>, our ability to freely
choose how we act. This means protecting each other from acts which
might restrict our autonomy. This is an example where different moral
theories place emphases on normative factors: whereas utilitarianism
focuses on wellbeing, deontological theories often emphasize
autonomy.<p>
Another key part of their idea of autonomy is that our actions are not
entirely governed by moral considerations. We are constrained from
certain types of behavior, but apart from those behaviors, we can freely
choose how to act. When we are choosing a career for example, we should
not become a torturer, or an assassin, but apart from those types of
constraints, we can choose from a range of harmless careers that suit
our interests. By contrast, a utilitarian might argue that we must
choose the single career (if there is one) that would have the best
outcome.</p>
<p><strong>According to deontology, intentions can be right or wrong, as
well as actions.</strong> Many deontological theories assert that
intentions, as well as actions, can be moral or immoral. For
example:<p>
</p>
<div class="blockquote">
<p><em>Intentional push</em>: Farid’s elderly mother has been annoying
him, so he decides to push her down the stairs. When he next sees her at
the top of the stairs, he carries out his plan.</p>
</div>
<div class="blockquote">
<p><em>Intention, but no push</em>: Farid’s elderly mother has been
annoying him, so he decides to push her down the stairs. When he next
sees her at the top of the stairs, he plans to push her. Just before he
does, she trips and falls.<p>
</p>
</div>
<p>According to some deontological theories, Farid is equally wrong in
both scenarios. Though he never pushes his mother in the second
scenario, the intention itself is a moral error. By contrast, for
classical utilitarians, intentions do not matter in themselves. On this
view, intentions are only right or wrong insofar as they lead to good or
bad consequences.<p>
Our common-sense moral intuitions are often aligned with the idea that
intentions matter. Perhaps that’s why they play a very important role in
the law of many countries. In America’s Model Penal Code, for example,
the strength of a criminal’s intention is measured with a scale of four
adverbs: a criminal could commit a crime <em>purposefully</em>,
<em>knowingly</em>, <em>recklessly</em>, or <em>negligently</em>. If
they commit it <em>purposefully</em>, then they knew what the outcome
would be, and they intended that outcome to happen. If they commit it
<em>knowingly</em>, they don’t primarily intend the criminal effect of
their actions, but they act anyway. People are acting
<em>recklessly</em> when they knowingly engage in behaviors which pose
risks to others, like owning a tiger, or flying a drone too close to an
airport. Those who act <em>negligently</em> fail to perceive a
substantial and unjustifiable risk that their conduct will have harmful
results. Not all legal codes differentiate between these four levels in
practice, but they all punish purposeful or knowing criminals more than
negligent ones.<p>
If a driver purposefully rams their car into another vehicle, intending
to cause injury, they would face serious criminal charges. If the driver
was speeding recklessly and lost control of their vehicle, resulting in
accidental injury to another driver, they may face a lesser charge, such
as reckless driving. The lessened mental state, even if the end result
was the same, often reduces the severity of punishment under the law. As
we can see, to determine whether an AI acted immorally, some moral
theories would require that we be able to determine an AI’s intent, a
goal of transparency research.<p>
Deontologists advance a number of arguments against consequentialism
that help to clarify the distinctive features of their moral theories.
There are many deontological theories, and they are often grouped
together under one umbrella simply because they share the feature that
they are not consequentialism. Deontological theorists often criticize
consequentialism, and partially define their theories by the ways that
they make up for the flaws they see in consequentialism. Two problems
that they see in consequentialism are that (1) consequentialism is very
demanding (in other words, it doesn’t allow much autonomy) and (2)
consequentialism leads to some radical conclusions (for example, it
sometimes justifies actions that most people believe are wrong).</p>
<p><strong>(1) Unlike consequentialism, deontology gives us
options.</strong> Deontology preserves human autonomy because it only
forbids us from performing a limited number of impermissible actions.
The remaining actions are optional. On the other hand, consequentialism
implies that every action is moral or immoral to a certain degree, and
each person is obligated to do the most good at all times. According to
consequentialism, every life decision—like marriage, career choice,
which relationships to pursue, which food to eat—is a moral decision.
Deontologists find this to be far too demanding.</p>
<p><strong>(2) Unlike consequentialism, deontology does not allow any
action to be justified by its outcome.</strong> According to many
consequentialist theories, any action can be justified if it results in
a better outcome than the alternative actions. Even killing or torturing
innocent people might be the right choice if it increases everyone
else’s wellbeing more than it harms its victims. Some people believe
that deontological theories—–which forbid actions like killing and
torture in all cases—–are more plausible.</p>
<h3 id="deontological-principles">Deontological Principles</h3>
<p><strong>We need principles, as well as rules, to capture the
complexity of ethics.</strong> While deontologists generally consider
the absolute prohibition of certain actions to be a strength of
deontology, it can sometimes lead to counterintuitive moral judgments.
Suppose a pair of conjoined twins will die without undergoing a medical
procedure. However, if the procedure is carried out, one of them will
die. The rule “do not kill” might stop a surgeon from operating, which
would mean that neither twin has a chance of survival.<p>
Difficult, messy situations like this, where clear-cut rules seem to
fail us, have led some deontologists to develop important distinctions
which complicate their theory but better capture the way we think about
ethics. We will introduce two of these principles: the <em>doctrine of
double effect</em> and the <em>action/omission distinction</em>.</p>
<p><strong>The doctrine of double effect.</strong> In the case of the
conjoined twins, the surgeon might appeal to the doctrine of double
effect. This is a principle which states that an agent is morally
allowed to carry out actions that predictably lead to bad outcomes–—like
the death of an innocent–—as long as they <em>intend</em> the good
effect, but not the bad effect of the action. The constraint against
letting two people die, however, must be stronger than the constraint
against performing the surgery. In this case, the doctor can operate
only if she intends to save one of them, not to bring about a death. The
doctrine of double effect can also explain why it can be morally
permissible to kill in self defense <span class="citation"
data-cites="aquinas2014summa">[11]</span>. You are permitted to defend
yourself and your family from imminent attack. If the only way to
protect yourself is to kill your assailant, you may be permitted, as
long as you intend to save your family and not to kill.</p>
<p><strong>The action/omission distinction.</strong> The action/omission
distinction is important for understanding the role of responsibility in
deontological theories. Intuitively, we find someone to be more
responsible for something they did than for something they allowed to
happen <span class="citation" data-cites="foot1978problem">[12]</span>.
For example:<p>
</p>
<div class="blockquote">
<p><em>Stealing</em>: While walking past the bank at night, Gabe sees
that the night deposit box is open. Inside it, he sees a bag filled with
money. He decides to steal the bag.</p>
</div>
<div class="blockquote">
<p><em>Failing to report</em>: Heather sees Gabe take the money, but
doesn’t report it to the police. If she had, the money would have been
returned to its owner.<p>
</p>
</div>
<p>In these examples, both Gabe and Heather have done something wrong,
but Gabe’s crime is worse. He is directly responsible for the theft,
while Heather has only committed an act of omission–—she failed to
report the crime. This captures some of our ordinary intuitions about
responsibility. We generally do not hold people responsible for what
they do not do.</p>
<h3 id="criticisms-of-deontology">Criticisms of Deontology</h3>
<div class="blockquote">
<p><em>Nuclear terrorism</em>: Ines is part of a terrorist cell that has
placed a nuclear weapon in a capital city. If it goes off, it will kill
millions. She claims that it will go off within 24 hours, but she will
not say which city it is in. Jamie has captured Ines and questioned her,
but Ines will not give away the bomb’s location. If Jamie tortures Ines,
she will find out the bomb’s location and millions of lives will be
saved.<p>
</p>
</div>
<p>Some deontological theories accept that Jamie should not torture
Ines, no matter how many people will die as a result. However, many
people find this implausible. When so many lives are at stake, it may
seem selfish for Jamie to take the easier option, prioritizing her moral
purity over the lives of the people she could save.<p>
To accommodate our intuitions about moral catastrophes, some
deontological theorists have adopted a <em>threshold</em> <span
class="citation" data-cites="moore2019rationality">[13]</span>.
According to a threshold view, when a certain number of lives are at
stake (i.e. when the badness of an outcome reaches a certain threshold),
the theory defers to the consequentialist recommendation that the
otherwise impermissible act (torture in this case) is allowed.<p>
There are a number of problems with the threshold view. Most
importantly, it is not clear how to determine what the threshold should
be, and any decision about it will be arbitrary. If a million lives at
stake justify Jamie’s act of torture, then it seems odd to say that ten
thousand, one thousand, or even ten lives at stake do not.</p>
<div class="blockquote">
<p><em>Better job</em>: Kimiko has been offered a job where she will use
her unique set of skills to reform the health system of the country she
lives in. The next best hire for the job is far less experienced than
she is. Therefore, if Kimiko takes the job, thousands of lives will be
spared every year for at least a decade. However, the job would require
Kimiko to move to a new city, and she promised her children that the
family would not move until they had all finished school.<p>
</p>
</div>
<p>Many deontological theories consider promise-keeping to be very
important. They would not allow Kimiko to break her promise to her
children in this situation, even though keeping her promise would cost
thousands of lives.<p>
Deontological rules may sometimes lead to the best consequences, but
they often do not. This leads to what some people call the <em>paradox
of deontology</em>. There may be situations in which it is impermissible
to stop many instances of the same impermissible act occurring. If there
is a rule that we cannot lie and kill, then we cannot lie to prevent
hundreds of acts of lying, or kill to prevent hundreds of acts of
killing.</p>
<h3 id="immanuel-kant-and-the-categorical-imperative">Immanuel Kant and
the Categorical Imperative</h3>
<p>Immanuel Kant was a German enlightenment thinker who developed an
especially strong deontological theory which we now call
<em>Kantianism</em>. According to Kant, some actions are absolutely,
universally wrong. For instance, Kant believed that killing, stealing,
lying, committing suicide, and breaking a promise are wrong in all
circumstances.<p>
Kant believed that anyone can (in theory) arrive at these conclusions
due to their own capacity to reason. Because, in his opinion, we would
all arrive at the same conclusions, he believed in a universal moral law
which we all have a duty to follow. The method that he thought would
help us discover this moral law is called the <em>categorical
imperative</em>.<p>
Kant described the categorical imperative in several different ways, and
his descriptions are different enough that they are now referred to as
separate formulations of the imperative <span class="citation"
data-cites="kant1998groundwork">[14]</span>. In other words, they are
different ways to discover the same moral law. We will focus on two
formulations: the <em>universal law</em> formulation, and the
<em>humanity</em> formulation. The universal law formulation asks each
of us to imagine that when we make a moral rule for ourselves to follow,
we have actually made a law for everyone. In other words, we need to ask
ourselves: what if everyone did that? If a world where everyone followed
our rule was contradictory or irrational, then we have discovered
something that we shouldn’t do. The humanity formulation tells us to act
on rules which lead us to treat people in a way that lets them maintain
their autonomy. This means never getting in the way of their ability to
exercise their human capacities for reason and autonomy, or make their
own decisions.</p>
<p><strong>Kant’s method is called the categorical imperative because he
believed all moral rules must be categorical.</strong> Kant
distinguishes two types of rules: <em>hypothetical</em> rules are of the
form “do X in order to Y,” which only apply when we already want to Y,
and <em>categorical</em> rules are of the form “do X.” An example of a
hypothetical rule is “be kind to people if you want them to do you
favors.” This is hypothetical, or conditional, because we would only be
required to follow this rule if we already cared about receiving favors.
Kant thought that the moral law was a universal list of rules which
apply to everyone, so he argued that all moral rules must be
categorical, not hypothetical. A categorical rule like “be kind” can
apply to everyone, while the hypothetical “be kind if you want them to
do you favors” only applies to those who want favors.</p>
<p><strong>The universal law formulation.</strong> This idea leads us to
the first formulation of the categorical imperative, Kant’s method for
discerning right from wrong. Kant tells us in this formulation that we
should only act in ways which could be made into laws for all of
humankind. In other words, what if everyone did that same thing: “What
if everyone killed people who stood in their way?” “What if everyone
cheated on exams?” “What if everyone lied?” If the answer to the
question seems to make your intended action impossible or inadvisable,
do not do it. This formulation provides the clearest test to show
whether an action is in accordance with the moral law or not.<p>
In slightly more complicated terms, we can formalize Kant’s thoughts on
universalising rules into a four-part test for any rule which we can
apply to any categorical rule we might think of. If our proposed rule
passes all four stages, then it is permissible. First, we turn our
proposed action into a categorical rule (“do X” or “do X when in
Situation S”). Then, we change the rule so that it applies to everyone,
not just us. To test whether the rule is part of the moral law, we first
check whether it contradicts itself. If it does, then it is a rule we
absolutely must not follow. If the rule isn’t contradictory, we check
whether it conflicts with something else that we must value. If it does,
then we shouldn’t follow the rule; if it doesn’t, then we must follow
it. We will now go through an example of each step, to model how this
process might work.</p>
<p>Luke promised to take his mother’s dog for a walk. But today he is
tired and doesn’t want to. He proposes not to go on the walk. If his
action became a rule, the rule might be: “I will not fulfill promises
when it is inconvenient to me.”</p>
<p>To do this, we just remove Luke from the rule: “No one should fulfill
promises when it is inconvenient for them.”</p>
<p>Luke’s rule fails at this stage. In order for Luke to conceive of his
rule, the institution of promise-keeping must exist. However, in a world
in which everyone breaks their promises, the institution of
promise-keeping doesn’t really exist. Luke’s rule fails because it leads
to a contradiction.<p>
According to Kant, we can conclude from the contradiction in conception
that we should never break promises just for convenience.</p>
<p>To illustrate the contradiction in the will, Kant considers the case
of laziness. Suppose Mari never works hard because she is lazy. We could
formulate her action as a rule: “You shouldn’t work hard if you feel
lazy.” This rule passes the contradiction in conception because it’s
possible to imagine a world in which no one works hard. However, Kant
argues that it fails as a rule for another reason: it contradicts our
will. In Kant’s view, it is in our nature as rational beings to work and
to “develop our talents.” Therefore, we should not be lazy because, as a
rule, it would violate our rational nature.<p>
Kant’s universal law formulation of the categorical imperative is a
method of testing whether a rule can be willed for everyone. First, we
determine whether the rule is even conceivable. Then, we determine
whether people would will it. In theory, this method can tell us whether
any rule is right or wrong. A Kantian AI would therefore need the
capacity to reason.<p>
Now we turn to an alternative formulation of the categorical
imperative.</p>
<p><strong>The “humanity” formulation of the categorical
imperative.</strong> The humanity formulation is perhaps more
influential among philosophers today than the universal law formulation.
Roughly speaking, the humanity formulation states that we should always
treat other people’s humanity as an end, not merely as a means. Kant
means something specific by <em>humanity</em>, <em>end</em>, and
<em>means</em>.<p>
When Kant writes about <em>humanity</em>, he is referring to the ability
to engage in autonomous, rational behavior that he believed was
characteristic of human nature. <em>Ends</em> are the goals that we aim
to reach, and <em>means</em> are the methods of achieving ends. To treat
humanity always as an end and never as a means is to treat everyone with
respect for their autonomy and rationality. It’s one of Kant’s most
influential ideas. To make this idea more concrete, here is an
example.<p>
</p>
<div class="blockquote">
<p><em>The urgent lift</em>: Nathan wants a lift into town to go
go-carting. He approaches a stranger and lies to her, telling her that
his brother is having an allergic attack in town and he needs to deliver
his EpiPen. Nathan tells the stranger that if she doesn’t give him a
lift, his brother might die.<p>
</p>
</div>
<p>In the example, Nathan is treating the stranger as a means to get
into town. He doesn’t respect the stranger as a person with her own
ends, who may have better things to do. By lying to the stranger, Nathan
undermines her autonomy by obscuring the truth. In other words, he is
not respecting the stranger’s humanity.</p>
<h3 id="criticisms-of-kants-ethics">Criticisms of Kant’s Ethics</h3>
<div class="blockquote">
<p><em>Mad axeman</em>: Omar hears a knock on his door late at night. A
man with a wild look and a bloody ax asks, “Is Piper here?” Omar knows
Piper is upstairs. Should he tell the truth?<p>
</p>
</div>
<p>Kant’s ethical writings claim that it would be wrong for Omar to lie.
According to Kant, lying is always wrong. Even in the case of the mad
axeman, Kant insisted that it would be wrong to lie (though some
philosophers inspired by Kant argue he could have avoided this claim).
Most modern moral philosophers disagree with Kant’s conclusion in this
case.</p>
<p><strong>Pro tanto duties.</strong> Instead, philosophers refer to
duties to act which may be overridden by other duties. Many modern
philosophers would agree that Omar has a duty to avoid lying. But they
would also argue that he also has a duty to safeguard his friend, and
that this duty outweighs his duty to avoid lying. In other words, many
modern philosophers would see Omar’s duty to avoid lying as <em>pro
tanto</em>. Latin for “to that extent,” pro tanto means that a given
duty can be weighed against other duties to determine a course of
action. Although the principle of honesty offers a pro tanto reason in
favor of revealing Piper’s real location to the axeman, it is not the
only consideration. Other moral obligations, such as the duty to ensure
others’ welfare, may ultimately mean Omar should withhold Piper’s real
location from the axeman.</p>
<p><strong>Aspects of Kant’s ethics inspire modern deontology.</strong>
Many aspects of Kant’s morality are explicitly present in modern moral
theories. For example, many deontological theories still reflect the
ideas that we should treat people as ends, not as mere means; the focus
on respecting others’ rational and autonomous humanity; and the concept
of considering how your actions might apply universally.</p>
<h2 id="virtue-ethics">A.4.3 Virtue Ethics</h2>
<p>Virtue ethics is a moral theory that emphasizes the importance of
having the right character traits, rather than producing the right
consequences or performing the right actions <span class="citation"
data-cites="hurtshouse2023virtue">[15]</span>. A virtue ethicist might
argue that we should help others in need by donating to charity, but not
because we should promote wellbeing or because we have certain moral
obligations. According to a virtue ethicist, we should donate to charity
because that’s what a generous person would do, and generosity is a
virtue.<p>
Modern virtue ethics is inspired by the ancient Greek philosopher
Aristotle. In his book “Nicomachean Ethics,” Aristotle explored three
key concepts that are essential for understanding virtue ethics <span
class="citation" data-cites="crisp2014aristotle">[16]</span>. First, he
explored the concept of <em>virtue</em>, or morally good character
traits. Second, he developed the concept of practical wisdom, which is
the set of skills and experience required in order to behave in line
with virtues. Third, he argued that developing virtue and exercising
practical wisdom are essential for <em>flourishing</em>, or living a
good life. Each concept is described in more detail below.</p>
<h3 id="virtue">Virtue</h3>
<p><strong>A virtue is a morally good character trait or
disposition.</strong> Putative examples of virtues include courage,
generosity, fairness, and kindness. Virtues are morally good character
traits, and vices are morally bad ones. Putative vices include
cowardice, selfishness, unfairness, and cruelness. Having a certain
virtue or vice is not binary, but a matter of degree. In other words,
individuals aren’t typically completely courageous or completely
cowardly; they can be more or less courageous.</p>
<p><strong>To be virtuous is not just to behave in certain ways but also
to feel certain ways.</strong> Two individuals might behave in exactly
the same ways but have different feelings, and thus exhibit different
virtues and vices.<p>
Consider two siblings, Bobby and Cory, who behave similarly in every
situation. They are both trusted by their friends, they both keep their
promises, and they are both equally honest. However, Bobby behaves
virtuously because it makes him feel good; he derives pleasure from
helping others. Cory, on the other hand, behaves virtuously despite her
feelings; helping others feels to her like a burden. Her behavior is the
same as Bobby’s, perhaps because she wishes to be seen as a virtuous
person or she wants to avoid getting in trouble. According to most
virtue theories, only Bobby is virtuous. While Cory behaves the same
way, her behavior does not indicate virtue.</p>
<p><strong>Virtue ethicists claim that other theories miss important
morally relevant features.</strong> Mental states like emotions and
motivations, virtue ethicists argue, are morally relevant. A
consequentialist would evaluate Bobby and Cory as equally moral because
their actions produce the same consequences. Some, but not all,
deontologists would evaluate Bobby and Cory as equally moral because
they behave the same ways with respect to rules and obligations. Virtue
ethics emphasizes an intuition that many people have: that Bobby is
morally superior to Cory.</p>
<h3 id="practical-wisdom">Practical Wisdom</h3>
<p>Being disposed to behave virtuously is necessary, but not sufficient,
for being a virtuous person. It’s also important, according to virtue
ethics, to have <em>practical wisdom</em>–—the ability to reason and to
act appropriately on the inclination to be virtuous.<p>
For individuals who lack practical wisdom, the inclination to be
virtuous can lead them to behave wrongly. Someone who is inclined
towards honesty and who derives pleasure from being honest might, in
some situations, be too honest. If they lack practical reason, they may
needlessly insult strangers or cause conflicts between friends.
Practical wisdom is the ability to understand when it’s appropriate to
be honest and when it’s important to act on a different virtue, like
kindness.</p>
<p><strong>While people may be born with the disposition towards certain
virtues, practical wisdom is learned through experience.</strong>
Children, for example, may desire to behave well, but make errors due to
a lack of experience. They may tell their mother that they don’t like
her outfit, unable to differentiate between honesty and cruelty. If
their mother reacts with hurt feelings, her children will learn from the
experience that, in some situations, kindness is more appropriate than
honesty. In other words, they will gain practical wisdom. We can learn
practical wisdom from people who have had more experience than us, but
also from cultural figures who clearly excel in their virtue, people who
offer excellent examples of virtues of honesty, steadfastness, and
compassion. Evidently, if we would like to make AIs virtuous, we would
need to have exemplars for them to imitate.</p>
<h3 id="flourishing">Flourishing</h3>
<p><strong>Flourishing is living a good life.</strong> Aristotle
believed that being virtuous is necessary for flourishing. In fact, he
defined virtues in terms of their relationship towards flourishing.
Virtues, he argued, are those character traits which lead an individual
to flourish. The virtue ethicist’s idea of flourishing has similarities
to the objective goods account of wellbeing that we discussed earlier in
this chapter.<p>
While most virtue ethicists agree that being virtuous is necessary for
living a good life, they often disagree about whether it is sufficient
for living a good life. Aristotle argued that, in addition to being
virtuous, an individual must have the resources to enact virtue in order
to lead a flourishing life. Someone living in poverty, according to
Aristotle, is unable to enact certain virtues, like magnanimity. A
virtuous but very unlucky person may be unable to flourish.<p>
In sum, virtue ethics argues that to live a good life we must develop
our virtues, and that to develop our virtues we should imitate people
who act virtuously.</p>
<h3 id="criticisms-of-virtue-ethics">Criticisms of Virtue Ethics</h3>
<p><strong>Virtue ethics doesn’t always clearly help us determine the
right things to do.</strong> When faced with a dilemma, like whether or
not to steal from a grocery store in order to feed one’s family, virtue
ethics does not seem to offer much guidance. We should be generous and
selfless, which seems to suggest that stealing is wrong. However, we
should also be loyal and protective of our family, which seems to
suggest that we should do what is necessary in order to feed them. In
such cases, virtue ethics may not seem very useful. Virtue ethicists may
argue, however, that while their theory does not include a decision
procedure for every situation, a virtuous person with a high capacity
for practical reasoning will understand which virtues to express and
when.</p>
<p>According to virtue ethics, whether an action is right or wrong
depends entirely on characteristics of the actor. If the person
performing the action is ideally virtuous, then their actions will be
morally right. This may seem odd to those who believe that the field of
ethics is concerned with how to treat other people. Presumably we should
save someone from drowning, not because of facts about our own character
traits but because of facts about the drowning person. We should save
them because their life has value, because their wellbeing matters, and
because they have a right to life, not because we are courageous.</p>
<h2 id="subsec:social-contract">A.4.4 Social Contract Theory</h2>
<p>The focus of this section is social contract theories of morality. As
the name suggests, social contract theory focuses on contracts—–or, more
generally, hypothetical agreements between members of a society–—as the
foundation of ethics. A rule such as “do not kill” is morally right,
according to a social contract theorist, because individuals would agree
that the adoption of this rule is in their mutual best interest, and
would therefore insert it into a social contract underpinning that
society. The most influential contemporary theorist within this
tradition is John Rawls, who we will use to contextualize social
contract theory, using the famous <em>veil of ignorance</em> thought
experiment <span class="citation"
data-cites="rawls2017theory">[17]</span>. After understanding the broad
strokes of his theory, we will consider a few reasons why such reasoning
might be inadequate and consider some alternatives.</p>
<p><strong>According to social contract theory, moral codes are the
result of hypothetical agreements between members of society,
established for mutual benefit.</strong> Let us consider the prohibition
of thievery. It seems reasonable that people would agree to refrain from
stealing: most people stand to benefit from a society that punishes
thieves, given that it would disincentivize others from robbing them.
The ethical principle of not stealing would thus have moral force,
without requiring some fundamental principles such as maximizing
wellbeing or respecting autonomy. According to the social contract
theorist, all moral codes are similarly justified: they are reasonable
hypothetical agreements which encourage behavior that creates mutual
benefit.</p>
<h3 id="the-veil-of-ignorance">The Veil of Ignorance</h3>
<p>Philosopher John Rawls introduced the concept of a <em>veil of
ignorance</em> as a tool for creating a social contract. When behind the
veil of ignorance, individuals lose all knowledge of their personal
attributes, such as their talents, religion, gender, sexuality, or
class. In this state, sometimes called the <em>original position</em>,
participants are asked to envision a basic structure for society, based
on reasonable agreements without any knowledge of their own positions
within the society they create.</p>
<p><strong>In “A Theory of Justice”, Rawls proposes one way to generate
a social contract.</strong> Rawls places everyone behind a veil of
ignorance to make decisions about a social contract. From here, having
lost all knowledge of their individual characteristics, participants are
invited to envision a basic structure for society, based on reasonable
agreements without knowledge of their own position in the society they
create.<p>
Once the decisions have been made, participants leave the original
position, discover who they are in society, and then live according to
the social contract they created. Rawls believed that, if we were able
to use the veil of ignorance to construct a real society, the society
would likely include ideas like: protection of the worst-off, basic
liberties for all, and restrictions on inequality. Because the people
determining these contracts do not know which social group they will be
a part of, they would not create a society in which some people are far
worse off than others.</p>
<p><strong>The veil of ignorance would prevent slavery.</strong> Those
in Rawls’ original position would arrive at many sensible conclusions.
An individual behind the veil would not reasonably permit slavery, for
instance, given that they do not know whether they are a slave or a
slaver. On average, very few people seek to benefit from slavery, and
most people are harmed by it. Individuals would likely not wish to take
the chance of being a slave in exchange for a chance of being a wealthy
slave owner. Therefore, those in the original position would reject
slavery, and this forms the basis of the social contract to not enslave
others.</p>
<p><strong>Agreements behind the veil of ignorance create a basis for a
just society.</strong> This process is unlikely to generate elitist,
patriarchal, or ableist moral codes since one does not know whether they
have a high income, privileged racial or gender identity, or disability.
The interests of any particular group would not be favored, since no one
knows whether they belong to that group. As such, decisions behind the
veil of ignorance are made in the interest of society. According to
Rawls, everyone accounts for this component of luck by making decisions
as though they could, themselves, be members of any group in
society.</p>
<h3
id="principles-that-may-be-generated-from-the-veil-of-ignorance">Principles
That may be Generated from the Veil of Ignorance</h3>
<p>In this section, we look at some of the conclusions that Rawls argues
individuals in the original position might reach: the maximin principle,
the liberty principle, the equality of opportunity principle, and the
difference principle.</p>
<p><strong>We should protect the worst off.</strong> Behind the veil of
ignorance, no one knows who they are, which means that anyone might be
the worst-off individual in society. The main idea behind Rawls’
contractarian logic is that people would agree to make sure everyone has
a decent position in society, since anyone might end up in the worst
position. Rawls argues that people behind the veil of ignorance would
endorse the <em>maximin principle</em>, according to which we should
prioritize the worst-off in society.</p>
<p><strong>We should not exclude anyone from having basic
liberties.</strong> Behind the veil of ignorance, no one knows who they
are, which means that if any individual is excluded from having basic
liberties, it could potentially be anyone. Therefore, individuals would
rationally agree to distribute liberties to every member of society. As
long as one does not infringe upon others’ liberties, they should have
the freedom to pursue their own conception of the good life, and with it
enjoy civil and political liberties such as freedom of speech, religion,
association, assembly, and the right to a fair trial. This is Rawls’
<em>liberty principle</em>: the fair distribution of liberties, ensured
by a contractarian agreement, do not infringe upon the liberties of
others.</p>
<p><strong>We should ensure equality of opportunity.</strong> Adopting
the veil of ignorance may lead us to conclude that no one should be
denied opportunities based on their gender, age, race, family status, or
other personal characteristics. People behind the veil would agree that,
when inequalities arise, they should only do so through fair access to
opportunity for everyone. Equality of opportunity ensures that
differences in the relative positions of individuals are meritocratic.
This is Rawls’ <em>equality of opportunity</em> principle: some degree
of inequality is permissible, but only if it is merit-based.</p>
<p><strong>We should require that inequalities benefit the
least-advantaged.</strong> Allowing some inequalities can lead to
increased overall wealth in a society, as higher earnings for the most
productive members can create incentives for economic growth that
benefits everyone. This is the basis of Rawls’s <em>difference
principle</em>: some degree of inequality is permissible, but only if it
also benefits the worst-off members of society.<p>
These principles, in theory, ensure two conditions: everyone has basic
rights and equal access to opportunity, and any inequalities that follow
from the equality of opportunity also help the least privileged, even if
they help the privileged more.</p>
<h3 id="rawlss-conclusions-might-be-too-strong">Rawls’s Conclusions
might be Too Strong</h3>
<p><strong>The maximin principle is at odds with common-sense
morality.</strong> According to Rawls, our moral evaluation of a society
should be determined by the wellbeing of its worst-off individual. This
implies that we must invest all our resources into raising the wellbeing
of the worst-off member of society, and remain indifferent towards
everyone else’s wellbeing, as long as they do not become the
worst-off.</p>
<p><strong>The grouch takes priority.</strong> Imagine a grouch: someone
who is always at low levels of wellbeing and extremely hard to please.
Say that giving them a billion dollars would make them only as happy as
an ordinary person would be with a slice of cake. The maximin principle
dictates that our priority should be to focus on improving their
wellbeing, even if it means using a large amount of resources that could
have made everyone else in society much happier. This prioritization
seems counterintuitive.</p>
<p><strong>We must be indifferent towards improving everyone else’s
wellbeing.</strong> Suppose there was a new medical breakthrough that
can cure a widespread disease, greatly improving the lives of many who
suffer from the illness. This seems like a good thing, and it would be
strange to not care about this happening. However, if the worst-off
individual doesn’t have that disease and their wellbeing remains
unchanged, nothing morally important has changed.</p>
<p><strong>We must be indifferent towards decreasing everyone else’s
wellbeing.</strong> Consider a scenario in which a technical error
causes all electronically stored money to be deleted, plunging most
countries into chaos and leading to widespread poverty. This seems like
a bad thing, and it would be strange not to care about this happening.
However, if the worst-off individual lives in an unaffected area and
does not have a bank account, and their wellbeing remains unchanged,
once more nothing morally important has changed.</p>
<p><strong>The maximin principle seems untenable.</strong> In all three
of these scenarios, Rawls’ maximin principle gives us implausible
answers. We likely should not spend all of our resources to give one
unhappy person a bit of joy at the cost of everyone else’s wellbeing.
Similarly, it seems extremely morally relevant if we can cure a
widespread disease or ensure that most people do not lose all their
money.</p>
<h3
id="rawls-conclusions-might-not-follow-from-the-veil-of-ignorance">Rawls’
Conclusions Might Not Follow from the Veil of Ignorance</h3>
<p><strong>Behind the veil of ignorance, we would care about more than
maximin.</strong> If, behind the veil of ignorance, we know that there
is just one person who will be much worse off than everyone else, we may
not unanimously agree to prioritize making that person’s situation
better. Individuals behind the veil of ignorance may not consider the
worst-case outcomes when making decisions under uncertainty. While the
people behind the veil might be highly risk-averse, ensuring that the
general distribution of society is good rather than just the average
level of wellbeing and endorsing the liberty and difference principles,
they would likely not endorse the maximin principle.</p>
<p><strong>The veil of ignorance may lead to utilitarianism.</strong>
Nobel prize winning economist John Harsanyi conceptualized the veil of
ignorance before Rawls, and used it to support utilitarianism <span
class="citation" data-cites="harsanyi1953cardinal">[18]</span>. He
argued that rational agents would aim to maximize the total amount of
wellbeing in their society, so that the average outcome is as good as
possible. Decisions under uncertainty often involve maximizing expected
or average results. Harsanyi argued that rational individuals behind the
veil of ignorance would therefore choose a utilitarian organization for
society.</p>
<p><strong>A problem for Rawls.</strong> Rawls’ “A Theory of Justice”
was designed as an alternative to utilitarianism, but it has been used
to justify utilitarianism. If the veil of ignorance indeed creates
conditions more conducive to utilitarian moral decisions, it might
actually support utilitarianism instead of Rawls’ theory of justice.</p>
<h3 id="alternatives-to-rawls-social-contract-theory">Alternatives to
Rawls’ Social Contract Theory</h3>
<p><strong>A natural middle ground between Rawlsian and utilitarian
ideas is prioritarianism.</strong> <em>Prioritarianism</em> is an
ethical theory that gives greater moral weight to improving the
wellbeing of those who are worst off in society <span class="citation"
data-cites="parfit1997equality">[19]</span>. Imagine a situation where
we have the option to distribute resources among three people: Rana,
Sean, and Toby. Before any intervention, they have the following levels
of wellbeing: Rana has 6 units of wellbeing, Sean has 5 units of
wellbeing, and Toby has 1 unit of wellbeing.<p>
We can choose one of the following options:</p>
<ol>
<li><p>Increase Rana and Sean’s wellbeing by 2 units each, resulting in
(8, 7, 1);</p></li>
<li><p>Increase Toby’s wellbeing by 1 unit, resulting in (6, 5, 2);
or</p></li>
<li><p>Increase Toby’s wellbeing by 2 units, reduce Sean’s wellbeing by
2 units, and reduce Rana’s wellbeing by 3 units, resulting in (3, 3,
3).</p></li>
</ol>
<p>Prioritarianism suggests that we should prioritize improving the
wellbeing of Toby, the worst-off individual. However, we should also
take into account the wellbeing of others. Unlike utilitarianism, which
dictates that we choose option 1 to maximize overall wellbeing,
prioritarianism suggests that we choose option 2 to help the most
disadvantaged member. Unlike Rawls’ maximin principle, which suggests
that we choose option 3 to most improve the situation of the least
advantaged individual (even at the cost of everyone else),
prioritarianism attributes moral weight to the wellbeing of Rana and
Sean and so might choose option 1 or 2 instead. This approach strikes a
balance between utilitarianism and Rawlsianism, placing higher moral
value on improving someone’s life when (a) that person’s overall
wellbeing is relatively low and (b) the increase in wellbeing would be
substantial.<p>
One example of a prioritarian policy is focusing educational
interventions on disadvantaged students. Prioritarians might support
policies that concentrate resources on helping students from
disadvantaged backgrounds or those with learning disabilities, with the
goal of closing achievement gaps and improving outcomes for the
worst-off. Utilitarians might argue that resources should be allocated
to improve overall educational outcomes, which might involve investing
in programs that benefit a larger number of students, even if it doesn’t
specifically target the most disadvantaged.</p>
<p><strong>Contractualism.</strong> Philosopher T.M. Scanlon developed
another approach to social contract theory called contractualism <span
class="citation" data-cites="scanlon1998owe">[20]</span>. Scanlon
established contractualism to build on the idea that morality is based
on agreements between people while addressing some of the limitations of
Rawls’ theory. According to Scanlon, people are naturally inclined to
seek reasonable moral agreements, driven by their sense of justice. This
makes Rawls’ original position, in which people are behind a veil of
ignorance, unnecessary. In Scanlon’s view, morality is about what we owe
each other as rational beings.<p>
Scanlon did not have a definitive answer to what we owe one another.
Instead, he proposed using a social contract underpinned by
reasonableness. It is reasonable to reject a moral code declaring
slavery is right. However, it is less reasonable to reject a moral code
declaring that community resources should be used to help the worse off.
For a contractualist, certain actions are wrong if they don’t meet a
standard of behavior that “no one could reasonably reject as a basis for
informed, unforced, general agreement.” This means that principles of
morality should be something that rational people can generally accept.
By making fewer strong assumptions, Scanlon avoids some of the pitfalls
of Rawls’ process and conclusions.</p>
<h3 id="summary-of-social-contract-theory">Summary of Social Contract
Theory</h3>
<p><strong>Rawls’ contractarianism and the veil of ignorance.</strong>
Social contract theory proposes that agreements between society’s
members form the foundation of morality. Moral codes are determined by
whether rational individuals would agree to mutually abide by them. John
Rawls’ influential approach relies on the veil of ignorance, where
decision-makers are unaware of their position in society. Rawls’ theory
suggests the maximin, liberty, and difference principles, which together
aim to raise the lowest levels of wellbeing, ensure the provision of
basic liberties, and minimize inequalities in society.<p>
However, some of Rawls’ conclusions, and especially the maximin
principle, are implausible. The veil of ignorance can be used to support
utilitarianism, and alternatives like prioritarianism are better at
accounting for some of our intuitions. Scanlon’s contractualism offers
another perspective on social contract theory, focusing on people acting
according to reasonable principles and contending that morality is built
upon obligations to others—what we owe to each other. Social contract
theories offer an alternative way of thinking about moral agreements
that might be highly relevant to modern approaches to machine ethics,
highlighting the importance of rational agreement and mutual benefit in
shaping ethical principles.</p>
<h3 id="summary">Summary</h3>
<p><strong>We have outlined four common approaches to morality.</strong>
First we looked at utilitarianism, a theory that argued that actions
should be judged based on how much wellbeing they cause. Then we
considered deontology, the view that to live ethically is to live by the
right system of rules. According to virtue ethics, the goal of a good
life is to become a virtuous person. Social contract theory recommends
following principles that we might arrive at together in an ideal
contractual process. These theories are all very different. They don’t
just differ in their moral claims; they consider morality to have
different goals that should be approached in different ways.</p>
<p><strong>The theories we’ve discussed are focused on different moral
considerations.</strong> Early in this chapter, we discussed moral
considerations like intrinsic goods, special obligations, constraints,
and options. Utilitarianism, of course, is most concerned with
wellbeing, an intrinsic good, and does not support options. Deontology
especially emphasizes constraints. Social contract theory especially
emphasizes special obligations.<p>
In common-sense morality (i.e. the moral decision-making that everyone
does on a daily basis) different considerations seem more important than
others in different situations. It may be, then, that some theories are
more useful for thinking about some problems than others.</p>
<p><strong>We needn’t pick a single moral theory, and the goal of this
chapter is not to choose the best one.</strong> Utilitarianism and
social contract theory might be the best approaches for thinking about
some society-wide policies, but perhaps deontology is helpful when we
are drafting laws that are intended to apply to everyone in a country.
Virtue ethics may be especially useful in day-to-day situations.<p>
When moral theories conflict, it is important that we accommodate some
uncertainty. There may be something we can learn from all of them.</p>

<h1 id="conclusion">A.6 Conclusion</h1>
<p>Ethics is the study of moral principles and how they guide our
decisions and actions. It encompasses questions of right and wrong, and
it provides a framework for making choices based on our values and
beliefs. It is important for AI researchers to have a basic
understanding of ethics in order to ethically guide the development and
governance of AI systems.<p>
We examined a number of considerations that commonly enter into moral
decision making. Intrinsic goods, like wellbeing, are valuable for their
own sake. Consequentialist moral theories consider the maximization of
intrinsic goods to be our principle moral responsibility.
Utilitarianism, a form of consequentialism, is the view that we should
maximize wellbeing.<p>
Constraints also play a role in moral decision making. Constraints are
the basis of many deontological theories. Kant’s ethics categorically
constrains actions and is derived from the respect for humanity’s
rational autonomy.<p>
Virtue ethics focuses on the importance of developing certain character
traits rather than solely considering consequences or specific rules.
Aristotelian virtue ethics links virtue with flourishing; living a good
life is intertwined with possessing virtuous traits.</p>
<p>Rawls’ ethics, prioritarianism, and contractualism are all forms of
social contract theory. Social contract theory suggests that morality
can be derived from hypothetical agreements between members of society,
made for everyone’s mutual benefit. Rawls proposes that decisions about
a social contract should be made behind a veil of ignorance, where
individuals are unaware of their personal attributes. From this
position, principles such as protecting the worst-off, ensuring basic
liberties for all, and limiting inequality can potentially be
argued.</p>
<p>As discussed in the chapter on Machine Ethics, we need to ensure that
AIs behave ethically towards others. As AIs have increasing influence
over society and act more autonomously, it will become important that
they are able to detect situations where the moral principles apply,
assess how to apply the moral principles, evaluate the moral worth of
candidate actions, select and carry out actions appropriate for the
context, monitor the success or failure of the actions, and adjust
responses accordingly. Ideally, AIs would be designed to take into
account many of the ethical concepts introduced in this chapter.
Examples of desirable abilities from this perspective might include: -
representing various purported intrinsic goods, including pleasure,
autonomy, the exercise of reason, knowledge, friendship, love, and so on
- distinguishing between subtly different levels of these goods, and
ensuring that the AI’s value functions are not vulnerable to optimizers
- representing more than just intrinsic goods, for example legal systems
and normative factors including special obligations and deontological
constraints</p>

<br>
<br>
<h3>References</h3>
<div id="refs" class="references csl-bib-body" data-entry-spacing="0"
role="list">
<div id="ref-gowans2021moral" class="csl-entry" role="listitem">
<div class="csl-left-margin">[1] C.
Gowans, <span>“<span>Moral Relativism</span>,”</span> in <em>The
<span>Stanford</span> encyclopedia of philosophy</em>,
<span>S</span>pring 2021., E. N. Zalta, Ed., <a
href="https://plato.stanford.edu/archives/spr2021/entries/moral-relativism/"
class="uri">https://plato.stanford.edu/archives/spr2021/entries/moral-relativism/</a>;
Metaphysics Research Lab, Stanford University, 2021.</div>
</div>
<div id="ref-curry2019cooperate" class="csl-entry" role="listitem">
<div class="csl-left-margin">[2] O.
S. Curry, D. A. Mullins, and H. Whitehouse, <span>“Is it good to
cooperate?: Testing the theory of morality-as-cooperation in 60
societies,”</span> <em>Current Anthropology</em>, vol. 60, no. 1, pp.
47–69, 2019, doi: <a
href="https://doi.org/10.1086/701478">10.1086/701478</a>.</div>
</div>

<div id="ref-plato2004euthyphro" class="csl-entry" role="listitem">
<div class="csl-left-margin">[3] C.
Plato, <em>Euthyphro</em>. Kessinger Publishing, 2014.</div>
<div id="refs" class="references csl-bib-body" data-entry-spacing="0"
role="list">
<div id="ref-sebo2022chatbot" class="csl-entry" role="listitem">
<div class="csl-left-margin">[1] J.
Sebo, <span>“Op-ed: What should we do if a chatbot has thoughts and
feelings?”</span> [Online]. Available: <a
href="https://www.latimes.com/opinion/story/2022-06-16/artificial-intelligence-morals-ethics-sentience-thinking">https://www.latimes.com/opinion/story/2022-06-16/artificial-intelligence-morals-ethics-sentience-thinking</a></div>
</div>
<div id="ref-fisher2014metaethics" class="csl-entry" role="listitem">
<div class="csl-left-margin">[2] A.
Fisher, <em>Metaethics: An introduction</em>. Routledge, 2011.</div>

<div id="refs" class="references csl-bib-body" data-entry-spacing="0"
role="list">
<div id="ref-mill2004utilitarianism" class="csl-entry" role="listitem">
<div class="csl-left-margin">[1] J.
S. Mill and J. Bentham, <em>Utilitarianism and other essays</em>.
Penguin Books, 1987.</div>
</div>
<div id="ref-lazari2017utilitarianism" class="csl-entry"
role="listitem">
<div class="csl-left-margin">[2] K.
de Laari-Radek and P. Singer, <em>Utilitarianism: A very short
introduction</em>. Oxford University Press, 2017.</div>
</div>
<div id="ref-bentham1978offences" class="csl-entry" role="listitem">
<div class="csl-left-margin">[3] J.
Bentham and L. Crompton, <span>“Offences against one’s self:”</span>
<em>Journal of Homosexuality</em>, vol. 3, no. 4, pp. 389–406, 1978,
doi: <a
href="https://doi.org/10.1300/J082v03n04\_07">10.1300/J082v03n04\_07</a>.</div>
</div>
<div id="ref-singer2017famine" class="csl-entry" role="listitem">
<div class="csl-left-margin">[4] P.
Singer, <span>“Famine, affluence, and morality,”</span> in <em>Applied
ethics</em>, United States: Taylor; Francis, 2017, pp. 132–142. doi: <a
href="https://doi.org/10.4324/9781315097176">10.4324/9781315097176</a>.</div>
</div>
<div id="ref-singer1981expanding" class="csl-entry" role="listitem">
<div class="csl-left-margin">[5] P.
Singer, <em>The expanding circle: Ethics and sociobiology</em>. Farrar,
Straus; Giroux, 1981.</div>
</div>
<div id="ref-scheffler1994rejection" class="csl-entry" role="listitem">
<div class="csl-left-margin">[6] S.
Scheffler, <em><span class="nocase">The Rejection of Consequentialism: A
Philosophical Investigation of the Considerations Underlying Rival Moral
Conceptions</span></em>. Oxford University Press, 1994. doi: <a
href="https://doi.org/10.1093/0198235119.001.0001">10.1093/0198235119.001.0001</a>.</div>
</div>
<div id="ref-bales2023act" class="csl-entry" role="listitem">
<div class="csl-left-margin">[7] R.
E. Bales, <span>“Act-utilitarianism: Account of right-making
characteristics or decision-making procedure?”</span> <em>American
Philosophical Quarterly</em>, vol. 8, no. 3, pp. 257–265, 1971,
Available: <a
href="http://www.jstor.org/stable/20009403">http://www.jstor.org/stable/20009403</a></div>
</div>
<div id="ref-nozick1974anarchy" class="csl-entry" role="listitem">
<div class="csl-left-margin">[8] R.
Nozick, <em>Anarchy state and utopia</em>. John Wiley &amp; Sons,
1974.</div>
</div>
<div id="ref-greene2013moral" class="csl-entry" role="listitem">
<div class="csl-left-margin">[9] J.
Greene, <em>Moral tribes: Emotion, reason, and the gap between us and
them</em>. New York: Penguin Press, 2013.</div>
</div>
<div id="ref-darwall2002deontology" class="csl-entry" role="listitem">
<div class="csl-left-margin">[10] S.
L. Darwall, Ed., <em>Deontology</em>. Malden, MA: Wiley-Blackwell,
2003.</div>
</div>
<div id="ref-aquinas2014summa" class="csl-entry" role="listitem">
<div class="csl-left-margin">[11] T.
Aquinas, T. F. E. D. Province, and C. W. Publishing, <em>The summa
theologica: Complete edition</em>. Catholic Way Publishing, 2014.
Available: <a
href="https://books.google.com.au/books?id=Ee0HBAAAQBAJ">https://books.google.com.au/books?id=Ee0HBAAAQBAJ</a></div>
</div>
<div id="ref-foot1978problem" class="csl-entry" role="listitem">
<div class="csl-left-margin">[12] P.
Foot, <em>Virtues and vices and other essays in moral philosophy</em>.
Berkeley: University of California Press, 1978.</div>
</div>
<div id="ref-moore2019rationality" class="csl-entry" role="listitem">
<div class="csl-left-margin">[13] M.
Moore, <span>“The rationality of threshold deontology,”</span> in
<em>Moral puzzles and legal perplexities: Essays on the influence of
larry alexander</em>, Cambridge University Press, 2019, pp.
371–387.</div>
</div>
<div id="ref-kant1998groundwork" class="csl-entry" role="listitem">
<div class="csl-left-margin">[14] I.
Kant, <em>Groundwork for the metaphysics of morals</em>. New York:
Oxford University Press, 1998.</div>
</div>
<div id="ref-hurtshouse2023virtue" class="csl-entry" role="listitem">
<div class="csl-left-margin">[15] R.
Hursthouse and G. Pettigrove, <span>“<span>Virtue Ethics</span>,”</span>
in <em>The <span>Stanford</span> encyclopedia of philosophy</em>,
<span>F</span>all 2023., E. N. Zalta and U. Nodelman, Eds., <a
href="https://plato.stanford.edu/archives/fall2023/entries/ethics-virtue/"
class="uri">https://plato.stanford.edu/archives/fall2023/entries/ethics-virtue/</a>;
Metaphysics Research Lab, Stanford University, 2023.</div>
</div>
<div id="ref-crisp2014aristotle" class="csl-entry" role="listitem">
<div class="csl-left-margin">[16] R.
Crisp, Ed., <em>Aristotle: Nicomachean ethics</em>. Cambridge University
Press, 2014.</div>
</div>
<div id="ref-rawls2017theory" class="csl-entry" role="listitem">
<div class="csl-left-margin">[17] J.
Rawls, <span>“Applied ethics: A multicultural approach,”</span>
Routledge, 2017.</div>
</div>
<div id="ref-harsanyi1953cardinal" class="csl-entry" role="listitem">
<div class="csl-left-margin">[18] J.
C. Harsanyi, <span>“Cardinal utility in welfare economics and in the
theory of risk-taking,”</span> <em>Journal of Political Economy</em>,
vol. 61, no. 5, pp. 434–435, 1953, doi: <a
href="https://doi.org/10.1086/257416">10.1086/257416</a>.</div>
</div>
<div id="ref-parfit1997equality" class="csl-entry" role="listitem">
<div class="csl-left-margin">[19] D.
Parfit, <span>“Equality and priority,”</span> <em>Ratio</em>, vol. 10,
no. 3, pp. 202–221, 1997, doi: <a
href="https://doi.org/10.1111/1467-9329.00041">https://doi.org/10.1111/1467-9329.00041</a>.</div>
</div>
<div id="ref-scanlon1998owe" class="csl-entry" role="listitem">
<div class="csl-left-margin">[20] T.
M. Scanlon, <em>What we owe to each other</em>. Harvard University
Press, 1998.</div>
</div>
</div>