text:
\documentclass[a4paper]{article}

\input{temp}

\usetikzlibrary{positioning}

\setcounter{section}{-1}

\begin{document}

\title{Topics in Set Theory Sheet 4}

\author{Xiaojia Rao}

\maketitle

\newpage

\section*{34.}
Fix $A \in M[G]$, a formula $\varphi(x,y,x_1,...,x_n)$ which specifies the function that we want to use for replacement, and fix parameters $a_1,...,a_n \in M[G]$. We want a name for
\[
B:=\{y: M[G] \vDash \exists x \in A \varphi(x,y,a_1,...,a_n)\}
\]
Take a name $\sigma$ for $A$ and some names $\tau_1,...,\tau_n$ for $a_1,...,a_n$. Define a formula
\[
\psi(x,y,x_1,...,x_n,A) := x \in A \wedge \varphi(x,y,x_1,...,x_n)
\]
and now consider the name
\[
\rho := \{(\pi,p): p \Vdash^* \exists x \psi(x,\pi,\tau_1,...,\tau_n,\sigma)\}
\]
We claim that $val(\rho,G) = B$.\\
$\subseteq$: suppose $y \in val(\rho,G)$. So there is $(\pi,p) \in \rho$ with $p \in G$ and $val(\pi,G) = y$. So $p \Vdash^* \pi \in \exists x \psi(x,\pi,\tau_1,...,\tau_n,\sigma)$. By definition, this means that the set
\[
D:=\{r:\exists \mu \in M^\P (r \Vdash^* \psi(\mu,\pi,\tau_1,...,\tau_n,\sigma))\}
\]
is dense below $p$. \underline{Also $p \in G$, so $G \cap D \neq \phi$},\footnote{On lecture 19 this was just stated as a fact without proof, but I can't see why it's trivial. My proof of this claim: suppose otherwise, that $G \cap D = \phi$. Add all elements $q$ s.t. there's no $r \in D$ s.t. $r \leq q$ to $D$ to form a new set $D'$. Then $D'$ is dense, so $G \cap D' \neq \phi$, so pick $r \in G \cap D'$. Now if $r \not\in D$, then since $G$ is a filter, pick $q \in G$ s.t. $q \leq p,r$. Since $D$ is dense below $p$ and $q \leq p$, we can pick $s \in D$ s.t. $s \leq q$. But then $s \leq q \leq r$, and $s \in D$, $r \in D'\setminus D$, contradicting our definition of $D'$.} so take some $q \in G \cap D$. $q \in D$ means there exists a name $\mu$ s.t.
\[
q\Vdash^* \psi(\mu,\pi,\tau_1,...,\tau_n,\sigma)
\]
But $q \in G$ as well, so by FT,
\[
M[G] \vDash \psi(\mu,\pi,\tau_1,...,\tau_n,\sigma)
\]
which translates to: there exists $x$ ($:=val(\mu,G)$) that
\[
&M[G] \vDash \psi(x,y,a_1,...,a_n,A)\\
\iff &M[G] \vDash x \in A \wedge \varphi(x,y,x_1,...,x_n)
\]
we can rewrite this as
\[
M[G] \vDash \exists x (x \in A \wedge \varphi(x,y,x_1,...,x_n))
\]
i.e. $y \in B$.\\
$\supseteq$: suppose $y \in B$. So $M[G] \vDash \exists x \psi(x,y,a_1,....a_n,A)$. Now take a name $\pi$ for $y$; so by FT, there exists $p \in G$ s.t. $p \Vdash^* \exists x \psi(x,\pi,\tau_1,...,\tau_n,\sigma)$. But this is exactly the requirement for $(\pi,p) \in \rho$. So $y \in val(\rho,G)$.

\section*{36.}
I'm assuming every instance of '$\P$-generic over $M$' actually means '$\P$-generic filter over $M$' (and similarly for later questions).

Also, in this proof we omit all suffixes on $\leq_\P,\leq_\Q$ because in all places it's quite obvious which $\leq$ we're referring to; we also just say \emph{$p$ and $p'$ are compatible} instead of saying \emph{$p$ and $p'$ are compatible w.r.t. $\P$} for the same reason.

$G$ is a $\P$-generic filter:\\
$\bullet$ Let $p \in G$, and $p \leq p'$. Then $i(p) \leq i(p')$. But $H$ is a filter, and $i(p) \in H$, so $i(p') \in H$. So $p' \in G$.

$\bullet$ Let $p,p' \in G$; we want to prove that $p,p'$ have a witness of compatibility in $G$. Now $i(p),i(p') \in H$, so they have some witnesses of compatibility in $H$. By (b), $p$ and $p'$ are also compatible, but we don't know yet that there's a witness in $G$.\\
Let $q \in H$ s.t. $q \leq i(p),i(p')$. We first claim that the set
\[
B:=\{i(p_c): p_c \leq p,p', p_c \in \P\}
\]
is $\Q$-bar below $q$ (obvious meaning): suppose $q' \leq q$. By (c), take some $p_{q'} \in \P$ s.t. it satisfies the condition in (c). Then $i(p_{q'
})$ and $q'$ are compatible. But $q' \leq q$, so $q' \leq i(p),i(p')$. So $i(p_{q'})$ and $i(p),i(p')$ are both compatible. By (b), $p_{q'}$ and $p,p'$ are both compatible. So take some $p_1 \leq p_{q'},p$, $p_1 \in \P$; but now $p_1 \leq p_{q'}$, so by (c), $i(p_1)$ and $q'$ are still compatible, i.e. $i(p_1)$ and $i(p')$ are still compatible. So $p_1$ and $p'$ are compatible. So take come $p_c \leq p_1,p'$ in $\P$. Now $p_c \leq p,p',p_{q'}$, so $i(p_c) \in B$; and it is compatible with $q'$ by (c).

Now we prove that $B\cap H \neq \phi$ (\emph{again, this might already be trivial, but I can't see why -- see footnote on the previous page}): we know $H$ is a $\Q$-generic filter, so it's a $\Q$-bar generic filter. Suppose $B \cap H = \phi$, so $B$ is not a $\Q$-bar. Add all elements $s \in \Q$ that witnesses this fact to form a new set $B'$, then $B'$ is a $\Q$-bar, so pick $r \in H \cap B'$. Now if $r \not\in B'$, then since $H$ is a filter, pick $s \in H$ s.t. $s \leq r,q$. Since $B$ is $\Q$-bar below $q$ and $s \leq q$, we can pick $t \in B$ s.t. $t$ and $s$ are compatible. But $s \leq r$, so $t$ and $r$ are compatible, contradicting our definition of $B'$.

So $G$ is a filter.

$\bullet$ Now we prove that $G$ is $\P$-generic over $M$. In fact we prove that it's $\P$-antichain generic over $M$. Let $A \subset \P$ be a maximal antichain, we want to prove that $\exists a \in A (i(a) \in H)$.\\
Suppose otherwise. Write $i(A)$ for the set containing images of element of $A$ under $i$. Then $i(A)$ cannot be an maximal antichain (as $H$ is $\Q$-antichain generic, so every maximal antichain has non-empty intersection with $H$). Obviously by (b), $i(A)$ is still an antichain. So (as $M[H] \vDash AC$) we extend it to a maximal antichain $A'$. Now $A' \cap H \neq \phi$, so take $q \in A' \cap H$. By assumption, $q \not\in i(A)$. Now apply (c), we get a $p \in \P$ s.t. $p' \leq p \implies i(p')$ and $q$ are compatible. But $A$ is a maximal antichain in $\P$, so $p$ has to be compatible with some element of $A$, say $p_a$. So let $p_c \in \P$ witness that $p$ and $p_a$ are compatible; in particular, $p_c \leq p$. By (c), $i(p_c)$ and $q$ are compatible. But $p_c \leq p_a$ and hence $i(p_c) \leq i(p_a)$, so $i(p_a)$ and $q$ are compatible; but both $i(p_a)$ and $q$ are in the antichain $A'$ of $\Q$. Contradiction (see diagram below).

\begin{tikzpicture}[node distance = 2cm]
\node(pa)			{$p_a$};
\node(p)	  [above right=0.5cm and 1.5cm of pa] {$p$};
\node(pc)	  [below right of= pa] {$p_c$};

\node(ipa)	[right=4cm of pa] {$i(p_a)$};
\node(q) [right of =ipa] {$q$};
\node(ipc) [below right of =ipa] {$i(p_c)$};

\draw(pa) -- (pc);
\draw(p)--(pc);
\draw(ipa) -- (ipc) node [midway, fill=white] {$?$};
\draw(q)--(ipc) node [midway, fill=white] {$?$};
\end{tikzpicture}

So $G$ is a $\P$-generic filter over $M$.

Now suppose $x \in M[G]$. So pick a name $\sigma$ for it. For each $(\pi,p) \in \sigma$, we \emph{transform} $p$ to $i(p)$, and we do so recursively, i.e. not only for the elements $(\pi,p)$ of $\sigma$, but also for all $(\tau,p') \in \pi$ where $(\pi,p) \in \sigma$, and so on. After this recursive transformation we get a name in $M^\Q$, let's call it $\sigma_q$.

We claim that $val(\sigma_q,H)=x(=val(\sigma,G))$. Prove this by induction on complexity of name $\sigma$: the case $\sigma=\phi$ is trivial. Now suppose $\sigma \neq \phi$. Note that because of our definition of $\sigma_q$, for each $(\pi_q,q) \in \sigma_q$ we can find a corresponding 'preimage' $(\pi,p) \in \sigma$ where $(\pi_q,q)$ is the result of our transformation from $(\pi,p)$. We know two things about them:\\
$\bullet$ $i(p) = q$;\\
$\bullet$ $val(\pi,G) = val(\pi_q,H)$ by induction hypothesis.

So we only need to show that $p \in G$ iff $i(p) \in H$, but fortunately that's exactly how we defined $G$. So done.

\section*{37.}
36(a) is trivial. For 36(c), suppose $q \in \Q$. By 37(c) we have some $p \in \P$ s.t. $i(p) \leq q$. Now if $p' \leq p$ then $i(p') \leq i(p) \leq q$, so in particular $i(p')$ and $q$ are compatible.

Now 37(b) gives that if $i(p)$ and $i(p')$ are compatible then $p$ and $p'$ are compatible (needs care here because \emph{compatible} and $\perp$ are opposite things). So the direction we need to prove in 36(b) is that if $p$ and $p'$ are compatible then so are $i(p)$ and $i(p')$. But this is trivial because $i$ preserves order, so if $p_c$ witnesses the compatibility of $p$ and $p'$ then $i(p_c)$ witnesses the compatibility of $i(p)$ and $i(p')$.

\section*{38.}
Note that in $\T$, $t$ and $t'$ being compatible necessarily means that $t\leq t'$ or $t' \leq t$, as otherwise $t$ and $t'$ must have already differed at some place in the sequence and there's no way to repair it (unlike the case of subsets of $\N$). So in graph theorical sense, if we only count edges between sequences of length that differ by 1, then $\T$ looks like a tree where the root is the empty sequence, and each node has $\N$ many offsprings.

Enumerate all elements in $\Q$ as $(q_i)$ (we can write this since $\Q$ is countable). Now define the required map $i:\T \to \Q$ recursively:\\
$\bullet$ the empty sequence is mapped to $1_\Q$;\\
$\bullet$ suppose a sequence $(s_i)$ of length $n \geq 0$ is mapped to some $q \in \Q$. Now, temporarily, we restrict our enumeration of $(q_i)$ to only those elements that are $\leq q$, but preserving the original order of enumeration; i.e. we get a subsequence of the original enumeration. We call that subsequence $q_i$ temporarily for convenience.\\
Now define recursively the image of $(s_1,...,s_n,j)$: for each $j$, let $k$ be the smallest number s.t. $q_k$ is incompatible with any of the images of $(s_1,...,s_n,i)$ where $i<j$ (we'll prove that this exists later). Now if $q_k$ is incompatible with some other elements that are not yet compatible with any of the images of $(s_1,...,s_n,i)$ where $i<j$ (let's call this condition(*)), then we let the image of $(s_1,...,s_n,j)$ be $q_k$; otherwise, since $\Q$ is splitting, take some $q_{k_1},q_{k_2} \leq q_k$ that are incompatible, then $q_{k_1}$ will have the above property that we desire to make it the image of $(s_1,...,s_n,j)$, and we do so. We see from the construction that our claim of existence above is justified, since when we choose the image of $(s_1,...,s_n,j)$, we always leave some elements that are not yet compatible with any of the previous images out.

It's obvious that this construction will satisfy 37(a) and (b), so we'll just need to prove 37(c). Take any $q \in \Q$. It must be $q_i$ in the enumeration for some $i$. There are three cases:\\
$\bullet$ we picked $q_i$ as the image of some sequence $(s_i)$ in the above construction. In this we already get an image of $\T$ below $q_i$ (which is itself);

$\bullet$ $q_i$ is not chosen in the above construction of picking images for sequences, because doing so would violate our condition (*) in the construction above. But in that case, we have taken some $q_{i_1},q_{i_2} \leq q_i$ that are incompatible, and chosen one of them as an image of some sequence. Therefore we also have an image of some sequence below $q_i$;

$\bullet$ $q_i$ is skipped in the process of picking images for sequence, because it is already compatible to some previous $q_j$ ($j<i$) that has been picked as an image of some sequence. Suppose $(s_n)$ is mapped to that $q_j$. Then when we recursively construct image of extensions of $(s_n)$ and restrict our vision to only those elements that are $\leq q_j$, $q_i$ will now have a lower index. For each time this case happens, the index (after restricted vision) of $q_i$ decreases by at least 1, so this case cannot take place infinitely many times, and we must get one of the above two cases after finitely many steps. So done.

\section*{39.}
We first prove that the composition of two dense embeddings is still a dense embedding. 37(a) and (b) are quite trivial. For 37(c), suppose $i:\P \to \Q$, $j:\Q \to \R$ are dense embeddings. Let $r \in \R$. Then there's some $q \in \Q$ s.t. $j(q) \leq r$. But then there's some $p \in \P$ s.t. $i(p) \leq j(q)$. So $j\circ i(p) \leq r$. So the image of $\P$ under $j \circ i$ is dense in $\R$.

Now consider $\T_3$, the partial order of finite 0-1-2 sequences. We claim that there's no dense embedding from $\T_{bin} \to \T_3$: suppose $i$ is such a dense embedding, and WLOG let $i(\phi_{bin}) = \phi_3$ (meaning of notations should be clear). Now consider what the images of $(0)_{bin}$ and $(1)_{bin}$ under $i$ are. In particular, the first term of those two images can only take two values among $\{0,1,2\}$, so at least one of $(0)_3,(1)_3,(2)_3$ is not compatible with either of the two images; suppose $(2)_3$ is so. But we've argued in q38 that being dense in $\T$ (and similarly, $\T_{bin}$ and $\T_3$) is equivalent to one is $\leq$ than the other. That means there's no element in $\T_{bin}$ that has image below $(2)_3$, so the image of $\T_{bin}$ under $i$ is not dense in $\T_3$. Contradiction.

However, $\T_3$ is still a countable splitting partial order, so by q38, there is a dense embedding $\T \to \T_3$. Combining our previous two results we see that there cannot be a dense embedding from $\T_{bin} \to \T$.

\section*{41.}
Following the same argument in lecture, we could get a similar version of lemma that gets us an approximation $F \in M$ for any function $f$ in $M[G]$ s.t. $|F(x)| < \kappa$ for any $x$. Now suppose $\P$ does not preserve some $\lambda \geq \kappa$. So $M[G] \vDash \exists \gamma < \lambda$ s.t. there's a bijection $f:\gamma \to \lambda$. Apply the lemma to get an $F$, and by surjectivity of $f$ we know $\lambda = \bigcup_{\alpha < \gamma} F(\alpha)$; then we get a similar contradiction that $M \vDash |\lambda| < |\gamma \times \kappa| \leq \lambda$ if $\lambda \geq \kappa$.

\section*{42.}
Consider instead $\P := Fn(\omega,2)$, and consider $f_G:=\bigcup G: \N \to 2$. We interpret this as characteristic function of a subset. Now let
\[
S_G :&= \{n \in \N: f_G(n) = 1\}\\
&=\{n \in \N, \exists p \in G(p(n)=1)\}
\]
Is $S_G$ splitting over $M$?\\
Consider
\[
D_{A,m}:=\{p: \text{ there are } a_1,...,a_m \in A s.t. p(a_1) = p(a_2)=...=p(a_m) = 1\}
\]
and let $\bar{D}_{A,m}$ be the same thing but with 1 replaced by 0. These are both dense (in $M$!).

For $Fn(\omega,\omega)$, one way is to consider the mod 2 function, and use $\bar{f}(n) := f(n) \pmod 2$.

\section*{43.}
Note that $A$ has to play a role here, otherwise this forcing is just the one in the previous question; if $\Q = Fn(\omega,\omega)$ and $f_G$ $\Q$-generic, we say $f$ dominates $M$ if $\forall g \in M$ $\exists N \forall n \geq N g(n) < f(n)$. Obviously if $g$ dominates $M$ then $g$ bounds $M$; we claim that $f_G$ does not dominate $M$.\\
Why is that? Fix $g \in M$. Then consider the set
\[
D_{g,n}:= \{p: \text{ there are } m_1 < ... < m_n (p(m_i) \leq g(m_i))\}
\]

Let's talk about some nomenclature. The partial order $Fn(\omega,\omega)$ is usually called Cohen forcing, which doesn't actually do anything. The one we actually used was $Fn(\kappa \times \omega,\omega)$ was the forcing to add $\kappa$ many Cohen reals (generic object for Cohen forcing). We've also looked that $Fn(\omega,\kappa)$ which is called the collapse forcing.

Most of the forcings (except one) are named by the person who named them. The forcing in this question is called the Mathias forcing.

In this case, in a pair $(s,A)$, $s$ is the part that matters (defines what function we have), and $A$ is sometimes called the \emph{reservoir set}; it is the rule of how to extend the function when you want to get stronger (specifying the range).

If $G$ is $\P$-generic over $M$, $f_G:\N \to \N$: $f_G(n) = k$ if there is $(s,A) \in G$ s.t. $s(n) = k$.

So following the same convention, we call this Mathias reals (doesn't really matter).

Claim that $f_G$ bounds $M$: consider the dense set
\[
D_{g,n}:=\{(s,A) : \text{ there are } \geq n \text{ points } s.t. s > g\}
\]
To prove this is dense, suppose given $s$. We copy $s_0,...,s_k$, then extend $a_1>g(k+1),...,a_n>g(k+n)$ and call this $s^*$. Then $(s^*,A) \leq (s,A)$ (note $A$ is infinite, so it contains arbitrarily large numbers).

45 is actually hard. Let's get back to this at the end.

\end{document}

ext: tex


text:
\chapter{To verify truth table for OR gate}
%\ref{sec:background}.

%\section{Aim}
%\label{sec:objectives}
%	To verify the truth table for AND Gate.

\section{Apparatus}
%\label{sec:objectives}
	\begin{itemize}
		\tightlist
		\item Kit for realization of gates
		\item Connecting Leads
	\end{itemize}

\section{Theory}
	The OR gate is an electronic circuit that gives a high output (1) if one or more of its inputs are high. A plus (+) is used to show the OR operation.
	\begin{figure}[h]
		\centering
		\includegraphics{img/exp2/1}
		\caption{Symbol for OR gate}
		\label{fig:2:1}
	\end{figure}
	\begin{figure}[h]
		\centering
		\includegraphics{img/exp2/2}
		\caption{Truth Table for OR gate}
		\label{fig:2:2}
	\end{figure}
	OR gate can be realized by DRL (Diode-Resistance-Logic) or by TTL (Transistor-Transistor-Logic). Presently, we will learn how to implement the OR gate using DRL (Diode-Resistance-Logic). To realise OR gate, we will use a diode at every input of the OR gate. The anode part of diode is connected with input while the cathode part is joined together and a resistor, connected with the cathode is grounded. In this case, we have taken two inputs which can be seen in the circuit below.

	When both the inputs are at logic 0 or low state then the diodes D1 and D2 become reverse biased. Since the anode terminal of diode is at lower voltage level than the cathode terminal, so diode will act as open circuit so there is no voltage across resistor and hence output voltage is same as ground. When either of the diodes is at logic 1 or high state then the diode corresponding to that input is forward bias. Since this time anode is at high voltage than cathode therefore current will flow through forward biased diode and this current then appears on resistor causing high voltage at output terminal also. Hence at output we get high or logic 1 or +5V. So, if any or both inputs are high, the output will be high or “1”.

	\begin{figure}[h]
		\centering
		\includegraphics[width=0.3\linewidth]{img/exp2/3}
		\caption{Circut for making OR gate}
		\label{fig:2:3}
	\end{figure}

\section{Procedure}
	\subsubsection{Simulator 1}
	\begin{itemize}
		\tightlist
		\item Connect the supply(+5V) to the circuit.
		\item Press the switches for inputs "A" and "B".
		\item The bulb glows if any one or both the switches are ON else it won't glow.
		\item Repeat step-2 and step-3 for all state of inputs.
	\end{itemize}

	\subsubsection{Simulator 2}
	\begin{itemize}
		\tightlist
		\item Enter the Boolean input "A" and "B".
		\item Enter the Boolean output for your corresponding inputs.
		\item Click on "Check" Button to verify your output.
		\item Click "Print" if you want to get print out of Truth Table.
	\end{itemize}

\section{Observations}
	\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp2/4}
		\caption{}
		\label{fig:2:4}
	\end{figure}
		\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp2/5}
		\caption{}
		\label{fig:2:5}
	\end{figure}
		\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp2/6}
		\caption{}
		\label{fig:2:6}
	\end{figure}
		\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp2/7}
		\caption{}
		\label{fig:2:7}
	\end{figure}

\section{Conclusion}
Thus we can see that when both inputs are low then only the output is low otherwise it is high when even one of the inputs is high.

\section{Precautions}
	\begin{enumerate}
		\tightlist
		\item Make the connections when power supply is OFF.
		\item Ensure that the connections are tight.
		\item Change the status of inputs only when power supply is OFF.
	\end{enumerate}


text:
\hypertarget{classification}{%
\chapter{Classification}\label{classification}}

This case study is related to ``Machine Bias'', an article published by
ProPublica in 2016 (see
\url{https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing}).
The article explores the use of predictive algorithms in the criminal
justice system. The arguments presented in the article are statistical;
specifically, they are related to binary classification.

The goal of this case study is to replicate the analysis reported in the
article and to review ways to evaluate binary classification algorithms.

\begin{itemize}
\item
  In this chapter, we start with the confusion matrix and the metrics
  that are derived from it, including accuracy, sensitivity,
  specificity, predictive value, false positive rate, and false negative
  rate.
\item
  In the next chapter, I review the arguments made in a response article
  published in the Washington Post. I compute and interpret calibration
  curves, and explore the trade-offs between predictive value and error
  rates.
\end{itemize}

\hypertarget{machine-bias}{%
\section{Machine Bias}\label{machine-bias}}

We will start by replicating the analysis reported in
``\protect\hyperlink{machine-bias}{Machine Bias}'', by Julia Angwin,
Jeff Larson, Surya Mattu and Lauren Kirchner, and published by
ProPublica in May 2016.

This article is about a statistical tool called COMPAS which is used in
the criminal justice systems of some states to inform decisions about
which defendants should be released on bail before trial, how long
convicted defendants should be imprisoned, and whether prisoners should
be released on probation (see
\url{https://en.wikipedia.org/wiki/COMPAS_(software)}).

COMPAS uses information about defendants to generate a ``risk score''
which is intended to quantify the risk that the defendant would commit
another crime if released.

The authors of the ProPublica article used public data to assess the
accuracy of those risk scores. They explain:

\begin{quote}
We obtained the risk scores assigned to more than 7,000 people arrested
in Broward County, Florida, in 2013 and 2014 and checked to see how many
were charged with new crimes over the next two years, the same benchmark
used by the creators of the algorithm.
\end{quote}

In their notebook (see
\url{https://github.com/propublica/compas-analysis/blob/master/Compas\%20Analysis.ipynb}),
they explain in more detail:

\begin{quote}
We filtered the underlying data from Broward county to include only
those rows representing people who had either recidivated in two years,
or had at least two years outside of a correctional facility.

{[}\ldots{]}

Next, we sought to determine if a person had been charged with a new
crime subsequent to the crime for which they were COMPAS screened. We
did not count traffic tickets and some municipal ordinance violations as
recidivism. We did not count as recidivists people who were arrested for
failing to appear at their court hearings, or people who were later
charged with a crime that occurred prior to their COMPAS screening.
\end{quote}

If you are not familiar with the word ``recidivism'', it means ``a
tendency to relapse into a previous condition or mode of behavior,
especially relapse into criminal behavior
(\url{https://www.merriam-webster.com/dictionary/recidivism})''.

In this context, a person is a ``recidivist'' if they are \emph{charged}
with another crime within two years of release. However, note that there
is a big difference between \emph{committing} another crime and being
\emph{charged} with another crime. We will come back to this issue.

The authors of the ProPublica article use this data to evaluate how well
COMPAS predicts the risk that a defendant will be charged with another
crime within two years of their release.

Among their findings, they report:

\begin{quote}
\ldots{} the algorithm was somewhat more accurate than a coin flip. Of
those deemed likely to re-offend, 61 percent were arrested for any
subsequent crimes within two years.

\ldots{} In forecasting who would re-offend, the algorithm made mistakes
with black and white defendants at roughly the same rate but in very
different ways.

\begin{itemize}
\item
  The formula was particularly likely to falsely flag black defendants
  as future criminals, wrongly labeling them this way at almost twice
  the rate as white defendants.
\item
  White defendants were mislabeled as low risk more often than black
  defendants.
\end{itemize}
\end{quote}

This discrepancy suggests that the use of COMPAS in the criminal justice
system is racially biased.

Since the ProPublica article was published, it has been widely discussed
in the media, and a number of researchers have published responses. In
order to evaluate the original article and the responses it provoked,
there are some technical issues you need to understand. My goal for
these notebooks is to help you analyze the arguments and interpret the
statistics they are based on.

Note: The ProPublica article writes ``white'' and ``black'' starting
with lower-case letters, which was the most common style in 2016. But
things are changing. As I revise this document in 2021, many style
guides now suggest that ``Black'' should be capitalized, and some
suggest that ``White'' should, too. For now, I will follow the
convention of the original article, but I will revisit this decision in
future editions.

\hypertarget{data}{%
\section{Data}\label{data}}

The authors of ``Machine Bias'' published their data and analysis at
\url{https://github.com/propublica/compas-analysis}.

We can use Pandas to read the data file and make a
\passthrough{\lstinline!DataFrame!}.

\begin{lstlisting}[language=Python,style=source]
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

cp = pd.read_csv('compas-scores-two-years.csv')
cp.shape
\end{lstlisting}

\begin{lstlisting}[style=output]
(7214, 53)
\end{lstlisting}

The dataset includes 7214 rows, one for each defendant, and 53 columns.

\hypertarget{the-confusion-matrix}{%
\section{The Confusion Matrix}\label{the-confusion-matrix}}

The authors of ``Machine Bias'' describe their analysis in a
supplemental article at
\url{https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm}.
It includes this table, which summarizes many of the results they
report:

\includegraphics{figs/machine_bias_table.png}

The table summarizes results for all defendants and two subgroups:
defendants classified as white (``Caucasian'' in the original dataset)
and black (``African-American'').

For each group, the summary includes a confusion matrix (see
\url{https://en.wikipedia.org/wiki/Confusion_matrix}) and a number of
metrics, including

\begin{itemize}
\tightlist
\item
  FP rate: false positive rate
\item
  FN rate: false negative rate
\item
  PPV: positive predictive value
\item
  NPV: negative predictive value
\item
  LR+: positive likelihood ratio
\item
  LR-: negative likelihood ratio
\end{itemize}

I will explain what these metrics mean and how to compute them, and
we'll replicate the results in this table.

But first let's examine the data and compute some recoded variables. The
following function displays the values in a Series and the number of
times each value appears.

\begin{lstlisting}[language=Python,style=source]
def values(series):
    """Count the values and sort.

    series: pd.Series

    returns: series mapping from values to frequencies
    """
    return series.value_counts(dropna=False).sort_index()
\end{lstlisting}

Here are the values of \passthrough{\lstinline!decile\_score!}, which is
the output of the COMPAS algorithm. \passthrough{\lstinline!1!} is the
lowest risk category; \passthrough{\lstinline!10!} is the highest.

\begin{lstlisting}[language=Python,style=source]
values(cp['decile_score'])
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  decile\_score \\
\midrule
1  &          1440 \\
2  &           941 \\
3  &           747 \\
4  &           769 \\
5  &           681 \\
6  &           641 \\
7  &           592 \\
8  &           512 \\
9  &           508 \\
10 &           383 \\
\bottomrule
\end{tabular}

It's important to note that COMPAS is not a binary classifier; that is,
it does not predict that a defendant will or will not recidivate.
Rather, it gives each defendant a score that is intended to reflect the
risk that they will recidivate.

In order to evaluate the performance of COMPAS, the authors of the
ProPublica article chose a threshold, \passthrough{\lstinline!4!}, and
define decile scores at or below the threshold to be ``low risk'', and
scores above the threshold to be ``high risk''. Their choice of the
threshold is arbitrary. Later, we will see what happens with other
choices, but we'll start by replicating the original analysis.

I'll create a Boolean \passthrough{\lstinline!Series!} called
\passthrough{\lstinline!high\_risk!} that's
\passthrough{\lstinline!True!} for respondents with a decile score
greater than 4.

\begin{lstlisting}[language=Python,style=source]
high_risk = (cp['decile_score'] > 4)
high_risk.name = 'HighRisk'
values(high_risk)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  HighRisk \\
\midrule
False &      3897 \\
True  &      3317 \\
\bottomrule
\end{tabular}

The column \passthrough{\lstinline!two\_year\_recid!} indicates whether
a defendant was charged with another crime during a two year period,
after the original charge, when they were not in a correctional
facility.

\begin{lstlisting}[language=Python,style=source]
values(cp['two_year_recid'])
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  two\_year\_recid \\
\midrule
0 &            3963 \\
1 &            3251 \\
\bottomrule
\end{tabular}

I'll create another \passthrough{\lstinline!Series!}, called
\passthrough{\lstinline!new\_charge\_2!}, that is
\passthrough{\lstinline!True!} for defendants who were charged with
another crime within two years.

\begin{lstlisting}[language=Python,style=source]
new_charge_2 = (cp['two_year_recid'] == 1)
new_charge_2.name = 'NewCharge2'
values(new_charge_2)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  NewCharge2 \\
\midrule
False &        3963 \\
True  &        3251 \\
\bottomrule
\end{tabular}

If we make a cross-tabulation of
\passthrough{\lstinline!new\_charge\_2!} and
\passthrough{\lstinline!high\_risk!}, the result is a DataFrame that
indicates how many defendants are in each of four groups:

\begin{lstlisting}[language=Python,style=source]
pd.crosstab(new_charge_2, high_risk)
\end{lstlisting}

\begin{tabular}{lrr}
\toprule
HighRisk &  False &  True  \\
NewCharge2 &        &        \\
\midrule
False      &   2681 &   1282 \\
True       &   1216 &   2035 \\
\bottomrule
\end{tabular}

This table is called a \textbf{confusion matrix} or error matrix.
Reading from left to right and top to bottom, the elements of the matrix
show the number of respondents who were:

\begin{itemize}
\item
  Classified as low risk and not charged with a new crime: there were
  2681 \textbf{true negatives}; that is, cases where the test was
  negative (not high risk) and the prediction turned out to be correct
  (no new charge).
\item
  High risk and not charged: there were 1282 \textbf{false positives},
  cases where the test was positive and the prediction was incorrect.
\item
  Low risk and charged: there were 1216 \textbf{false negatives}, cases
  where the test was negative (low risk) and the prediction was
  incorrect (the defendant was charged with a new crime).
\item
  High risk and charged: there were 2035 \textbf{true positives}, cases
  where the test was positive and the prediction was correct.
\end{itemize}

The values in this matrix are consistent with the values in the
ProPublica article, so we can confirm that we are reading the data and
replicating their analysis correctly.

I'll also check the confusion matrices for white and black defendants.
Here are the values of \passthrough{\lstinline!race!}:

\begin{lstlisting}[language=Python,style=source]
values(cp['race'])
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  race \\
\midrule
African-American &  3696 \\
Asian            &    32 \\
Caucasian        &  2454 \\
Hispanic         &   637 \\
Native American  &    18 \\
Other            &   377 \\
\bottomrule
\end{tabular}

Here's a Boolean Series that's true for white defendants.

\begin{lstlisting}[language=Python,style=source]
white = (cp['race'] == 'Caucasian')
white.name = 'white'
values(white)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  white \\
\midrule
False &   4760 \\
True  &   2454 \\
\bottomrule
\end{tabular}

And here's the confusion matrix for white defendants.

\begin{lstlisting}[language=Python,style=source]
pd.crosstab(new_charge_2[white], high_risk[white])
\end{lstlisting}

\begin{tabular}{lrr}
\toprule
HighRisk &  False &  True  \\
NewCharge2 &        &        \\
\midrule
False      &   1139 &    349 \\
True       &    461 &    505 \\
\bottomrule
\end{tabular}

\passthrough{\lstinline!black!} is a Boolean Series that is
\passthrough{\lstinline!True!} for black defendants.

\begin{lstlisting}[language=Python,style=source]
black = (cp['race'] == 'African-American')
black.name = 'black'
values(black)
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &  black \\
\midrule
False &   3518 \\
True  &   3696 \\
\bottomrule
\end{tabular}

And here is the confusion matrix for black defendants.

\begin{lstlisting}[language=Python,style=source]
pd.crosstab(new_charge_2[black], high_risk[black])
\end{lstlisting}

\begin{tabular}{lrr}
\toprule
HighRisk &  False &  True  \\
NewCharge2 &        &        \\
\midrule
False      &    990 &    805 \\
True       &    532 &   1369 \\
\bottomrule
\end{tabular}

All of these results are consistent with the ProPublica article.
However, before we go on, I want to address an important issue with this
dataset: data bias.

\hypertarget{data-bias}{%
\section{Data Bias}\label{data-bias}}

Systems like COMPAS are trying to predict whether a defendant will
\emph{commit} another crime if released. But the dataset reports whether
a defendant was \emph{charged} with another crime.

Not everyone who commits a crime gets charged (not even close). The
probability of getting charged for a particular crime depends on the
type of crime and location; the presence of witnesses and their
willingness to work with police; the decisions of police about where to
patrol, what crimes to investigate, and who to arrest; and decisions of
prosecutors about who to charge.

It is likely that every one of these factors depends on the race of the
defendant. In this dataset, the prevalence of \emph{new charges} is
higher for black defendants, but that doesn't necessarily mean that the
prevalence of \emph{new crimes} is higher.

If the dataset is affected by racial bias in the probability of being
charged, prediction algorithms like COMPAS will be biased, too. In
discussions of whether and how these systems should be used in the
criminal justice system, this is an important issue.

However, I am going to put it aside \emph{for now} in order to focus on
understanding the arguments posed in the ProPublica article and the
metrics they are based on. For the rest of this notebook I will take the
``recidivism rates'' in the dataset at face value; but I will try to be
clear about that they mean (and don't mean).

\hypertarget{arranging-the-confusion-matrix}{%
\section{Arranging the confusion
matrix}\label{arranging-the-confusion-matrix}}

In the previous section I arranged the confusion matrix to be consistent
with the ProPublica article, to make it easy to check for consistency.

In general, there are many possible ways to arrange a confusion matrix:
you can put the predictions along the columns and the actual conditions
along the rows, or the other way around, and you can sort the rows and
columns in either order.

There is no universal standard arrangement, but the one that seems to be
most common is the one on the
\href{https://en.wikipedia.org/wiki/Confusion_matrix}{confusion matrix
Wikipedia page}, which looks like this:

\includegraphics{figs/confusion_matrix1.png}

In this arrangement:

\begin{itemize}
\item
  The predictions are down the rows.
\item
  The actual results are across the columns.
\item
  The rows and columns are sorted so true positives are in the upper
  left and true negatives are in the lower right.
\end{itemize}

In the context of the ProPublica article:

\begin{itemize}
\item
  ``Predicted condition positive'' means the defendant is classified as
  high risk.
\item
  ``Predicted condition negative'' means low risk.
\item
  ``Condition positive'' means the defendant was charged with a new
  crime (recidivated).
\item
  ``Condition negative'' means they were not charged (survived).
\end{itemize}

\hypertarget{accuracy}{%
\section{Accuracy}\label{accuracy}}

Based on these results, how accurate is COMPAS as a binary classifier?
Well, it turns out that there are a \emph{lot} of ways to answer that
question. One of the simplest is overall \textbf{accuracy}, which is the
fraction (or percentage) of correct predictions. To compute accuracy, it
is convenient to extract from the confusion matrix the number of true
positives, false positives, false negatives, and true negatives.

\begin{lstlisting}[language=Python,style=source]
tp, fp, fn, tn = matrix_all.to_numpy().flatten()
\end{lstlisting}

The number of true predictions is \passthrough{\lstinline!tp + tn!}. The
number of false predictions is \passthrough{\lstinline!fp + fn!}. So we
can compute the fraction of true predictions like this:

\begin{lstlisting}[language=Python,style=source]
def percent(x, y):
    """Compute the percentage `x/(x+y)*100`."""
    return x / (x+y) * 100
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
accuracy = percent(tp + tn, fp + fn)
accuracy
\end{lstlisting}

\begin{lstlisting}[style=output]
65.37288605489326
\end{lstlisting}

As a way to evaluate a binary classifier, accuracy does not distinguish
between true positives and true negatives, or false positives and false
negatives.

But it is often important to make these distinctions, because the
benefits of true predictions and true negatives might be different, and
the costs of false positives and false negatives might be different.

\hypertarget{predictive-value}{%
\section{Predictive Value}\label{predictive-value}}

One way to make these distinctions is to compute the ``predictive
value'' of positive and negative tests.

\begin{itemize}
\item
  \textbf{Positive predictive value (PPV)} is the fraction of positive
  tests that are correct.
\item
  \textbf{Negative predictive value (NPV)} is the fraction of negative
  tests that are correct.
\end{itemize}

In this example, PPV is the fraction of high risk defendants who were
charged with a new crime. NPV is the fraction of low risk defendants who
``survived'' a two year period without being charged.

The following function takes a confusion matrix and computes these
metrics.

\begin{lstlisting}[language=Python,style=source]
def predictive_value(m):
    """Compute positive and negative predictive value.

    m: confusion matrix
    """
    tp, fp, fn, tn = m.to_numpy().flatten()
    ppv = percent(tp, fp)
    npv = percent(tn, fn)
    return ppv, npv
\end{lstlisting}

Here are the predictive values for all defendants.

\begin{lstlisting}[language=Python,style=source]
ppv, npv = predictive_value(matrix_all)
ppv, npv
\end{lstlisting}

\begin{lstlisting}[style=output]
(61.350618028338864, 68.79651013600206)
\end{lstlisting}

Among all defendants, a positive test is correct about 61\% of the time;
a negative test result is correct about 69\% of the time.

\hypertarget{sensitivity-and-specificity}{%
\section{Sensitivity and
Specificity}\label{sensitivity-and-specificity}}

Another way to characterize the accuracy of a test is to compute

\begin{itemize}
\item
  \textbf{Sensitivity}, which is the probability of predicting correctly
  when the condition is present, and
\item
  \textbf{Specificity}, which is the probability of predicting correctly
  when the condition is absent.
\end{itemize}

A test is ``sensitive'' if it detects the positive condition. In this
example, sensitivity is the fraction of recidivists who were correctly
classified as high risk.

A test is ``specific'' if it identifies the negative condition. In this
example, specificity is the fraction of non-recidivists who were
correctly classified as low risk.

The following function takes a confusion matrix and computes sensitivity
and specificity.

\begin{lstlisting}[language=Python,style=source]
def sens_spec(m):
    """Compute sensitivity and specificity.

    m: confusion matrix
    """
    tp, fp, fn, tn = m.to_numpy().flatten()
    sens = percent(tp, fn)
    spec = percent(tn, fp)
    return sens, spec
\end{lstlisting}

Here are sensitivity and specificity for all defendants.

\begin{lstlisting}[language=Python,style=source]
sens, spec = sens_spec(matrix_all)
sens, spec
\end{lstlisting}

\begin{lstlisting}[style=output]
(62.59612426945556, 67.65076961897553)
\end{lstlisting}

If we evaluate COMPAS as a binary classifier, it is a little more
specific than sensitive:

\begin{itemize}
\item
  About 63\% of the recidivists were classified as high risk.
\item
  About 68\% of the non-recidivists were classified as low risk.
\end{itemize}

It can be hard to keep all of these metrics straight, especially when
you are learning about them for the first time. The following diagram
might help (from \url{https://en.wikipedia.org/wiki/Confusion_matrix}):

\includegraphics{figs/confusion_matrix2.png}

PPV and sensitivity are similar in the sense that they both have true
positives in the numerator. The difference is the denominator:

\begin{itemize}
\item
  PPV is the ratio of true positives to all positive tests. So it
  answers the question, ``Of all positive tests, how many are correct?''
\item
  Sensitivity is the ratio of true positives to all positive conditions.
  So it answers the question ``Of all positive conditions, how many are
  detected?''
\end{itemize}

Similarly, NPV and sensitivity both have true negatives in the
numerator, but:

\begin{itemize}
\item
  NPV is the ratio of true negatives to all negative tests. It answers,
  ``Of all negative tests, how many are correct?''
\item
  Specificity is the ratio of true negatives to all negative conditions.
  It answers, ``Of all negative conditions, how many are classified
  correctly?''
\end{itemize}

\hypertarget{false-positive-and-negative-rates}{%
\section{False Positive and Negative
Rates}\label{false-positive-and-negative-rates}}

The ProPublica article reports PPV and NPV, but instead of sensitivity
and specificity, it reports their complements:

\begin{itemize}
\item
  \textbf{False positive rate}, which is the ratio of false positives to
  all negative conditions. It answers, ``Of all negative conditions, how
  many are misclassified?''
\item
  \textbf{False negative rate}, which is the ratio of false negatives to
  all positive conditions. It answers, ``Of all positive conditions, how
  many are misclassified?''
\end{itemize}

In this example:

\begin{itemize}
\item
  The false positive rate is the fraction of non-recidivists who were
  classified as high risk.
\item
  The false negative rate is the fraction of recidivists who were
  classified as low risk.
\end{itemize}

The following function takes a confusion matrix and computes false
positive and false negative rates.

\begin{lstlisting}[language=Python,style=source]
def error_rates(m):
    """Compute false positive and false negative rate.

    m: confusion matrix
    """
    tp, fp, fn, tn = m.to_numpy().flatten()
    fpr = percent(fp, tn)
    fnr = percent(fn, tp)
    return fpr, fnr
\end{lstlisting}

Here are the error rates for all defendants.

\begin{lstlisting}[language=Python,style=source]
fpr, fnr = error_rates(matrix_all)
fpr, fnr
\end{lstlisting}

\begin{lstlisting}[style=output]
(32.349230381024476, 37.40387573054445)
\end{lstlisting}

FPR is the complement of specificity, which means they have to add up to
100\%

\begin{lstlisting}[language=Python,style=source]
fpr + spec
\end{lstlisting}

\begin{lstlisting}[style=output]
100.0
\end{lstlisting}

And FNR is the complement of sensitivity.

\begin{lstlisting}[language=Python,style=source]
fnr + sens
\end{lstlisting}

\begin{lstlisting}[style=output]
100.0
\end{lstlisting}

So FPR and FNR are just another way of reporting sensitivity and
specificity. In general, I prefer sensitivity and specificity over FPN
and FNR because

\begin{itemize}
\item
  I think the positive framing is easier to interpret than the negative
  framing, and
\item
  I find it easier to remember what ``sensitivity'' and ``specificity''
  mean.
\end{itemize}

I think ``false positive rate'' and ``false negative rate'' are more
problematic. For example, ``false positive rate'' could just as easily
mean either

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  The fraction of positive tests that are incorrect, or
\item
  The fraction of negative conditions that are misclassified.
\end{enumerate}

It is only a convention that the first is called the ``false discovery
rate'' and the second is called ``false positive rate''. I suspect I am
not the only one that gets them confused.

So, here's my recommendation: if you have the choice, generally use PPV,
NPV, sensitivity and specificity, and avoid the other metrics. However,
since the ProPublica article uses FPR and FNR, I will too. They also
report LR+ and LR-, but those are combinations of other metrics and not
relevant to the current discussion, so I will ignore them.

However, there is one other metric that I think is relevant and not
included in the ProPublica tables: prevalence, which is the fraction of
all cases where the condition is positive. In the example, it's the
fraction of defendants who recidivate. The following function computes
\textbf{prevalence}:

\begin{lstlisting}[language=Python,style=source]
def prevalence(df):
    """Compute prevalence.

    m: confusion matrix
    """
    tp, fp, fn, tn = df.to_numpy().flatten()
    prevalence = percent(tp+fn, tn+fp)
    return prevalence
\end{lstlisting}

Here's the prevalence for all defendants:

\begin{lstlisting}[language=Python,style=source]
prev = prevalence(matrix_all)
prev
\end{lstlisting}

\begin{lstlisting}[style=output]
45.06515109509287
\end{lstlisting}

About 45\% of the defendants in this dataset were charged with another
crime within two years of their release.

\hypertarget{all-metrics}{%
\section{All Metrics}\label{all-metrics}}

The following function takes a confusion matrix, computes all the
metrics, and puts them in a DataFrame.

\begin{lstlisting}[language=Python,style=source]
def compute_metrics(m, name=''):
    """Compute all metrics.

    m: confusion matrix

    returns: DataFrame
    """
    fpr, fnr = error_rates(m)
    ppv, npv = predictive_value(m)
    prev = prevalence(m)

    index = ['FP rate', 'FN rate', 'PPV', 'NPV', 'Prevalence']
    df = pd.DataFrame(index=index, columns=['Percent'])
    df.Percent = fpr, fnr, ppv, npv, prev
    df.index.name = name
    return df
\end{lstlisting}

Here are the metrics for all defendants.

\begin{lstlisting}[language=Python,style=source]
compute_metrics(matrix_all, 'All defendants')
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &    Percent \\
All defendants &            \\
\midrule
FP rate        &  32.349230 \\
FN rate        &  37.403876 \\
PPV            &  61.350618 \\
NPV            &  68.796510 \\
Prevalence     &  45.065151 \\
\bottomrule
\end{tabular}

Comparing these results to the table from ProPublica, it looks like our
analysis agrees with theirs.

Here are the same metrics for black defendants.

\begin{lstlisting}[language=Python,style=source]
compute_metrics(matrix_black, 'Black defendants')
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &    Percent \\
Black defendants &            \\
\midrule
FP rate          &  44.846797 \\
FN rate          &  27.985271 \\
PPV              &  62.971481 \\
NPV              &  65.045992 \\
Prevalence       &  51.433983 \\
\bottomrule
\end{tabular}

And for white defendants.

\begin{lstlisting}[language=Python,style=source]
compute_metrics(matrix_white, 'White defendants')
\end{lstlisting}

\begin{tabular}{lr}
\toprule
{} &    Percent \\
White defendants &            \\
\midrule
FP rate          &  23.454301 \\
FN rate          &  47.722567 \\
PPV              &  59.133489 \\
NPV              &  71.187500 \\
Prevalence       &  39.364303 \\
\bottomrule
\end{tabular}

All of these results are consistent with those reported in the article,
including the headline results:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  The false positive rate for black defendants is substantially higher
  than for white defendants (45\%, compared to 23\%).
\item
  The false negative rate for black defendants is substantially lower
  (28\%, compared to 48\%).
\end{enumerate}

\begin{lstlisting}[language=Python,style=source]
error_rates(matrix_black)
\end{lstlisting}

\begin{lstlisting}[style=output]
(44.84679665738162, 27.985270910047344)
\end{lstlisting}

\begin{lstlisting}[language=Python,style=source]
error_rates(matrix_white)
\end{lstlisting}

\begin{lstlisting}[style=output]
(23.45430107526882, 47.72256728778468)
\end{lstlisting}

In other words:

\begin{itemize}
\item
  Of all people who \emph{will not} recidivate, black defendants are
  more likely to be be classified as high risk and (if these
  classifications influence sentencing decisions) more likely to be sent
  to prison.
\item
  Of all people who \emph{will} recidivate, white defendants are more
  likely to be classified as low risk, and more likely to be released.
\end{itemize}

This seems clearly unfair.

However, it turns out that ``fair'' is complicated. After the ProPublica
article, the Washington Post published a response by Sam Corbett-Davies,
Emma Pierson, Avi Feller and Sharad Goel: ``A computer program used for
bail and sentencing decisions was labeled biased against blacks. It's
actually not that clear,'' at
\url{https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas}.

I encourage you to read that article, and then read the next chapter,
where I will unpack their arguments and replicate their analysis.

max_stars_repo_name: AllenDowney/ElementsOfDataScienceBook

max_issues_repo_name: AllenDowney/ElementsOfDataScienceBook

max_forks_repo_name: AllenDowney/ElementsOfDataScienceBook