\section{Early Life}

{\bf Rosenberger:} Tell us a little about your early life. Where did
you grow up and what did your parents do?

{\bf Flournoy:} I was born in Long Beach, CA, and grew up in Los
Angeles County in a lemon orchard surrounded by oil wells and a flood
plain. There was a dairy farm nearby and we had a donkey. My father was
a plumbing contractor who plumbed Los Angeles: restaurants,
dormitories, cemeteries. He had 11 trucks go out every day. My Mom was
always unhappy about not finishing college, so she enrolled in college
when I went to college and then directed a preschool for many years. I
have three brothers and one sister. I was sent to Polytechnic School in
Pasadena as a sophomore in high school. On the entrance exam I had the
second highest score in math in history, but I flunked the English exam
because I didn't know the words in the instructions. (So even then I
had a one-sided brain!)

\begin{figure}

\includegraphics{495f01.eps}

\caption{Nancy at home in a part of Los Angeles County that was then
called Potero Heights, 1949.}\label{baby}
\end{figure}
\begin{figure}

\includegraphics{495f02.eps}

\caption{Nancy at her graduation from Polytechnic School, Pasadena, 1965.}\label{hsgraduation}
\end{figure}

{\bf Rosenberger:} As a young person, were you interested in
mathematics, statistics, data? What made you excited about statistics?

{\bf Flournoy:} High school algebra really made me happy; I would lay
on the floor and work problems for hours. I had a new female instructor
whose husband had gotten a professorship across the street at Cal Tech
while she just landed a high school job; her anger came through and I
got the message that mathematics is worth being passionate about.

My love of statistics came as a junior at UCLA, when I took a course
taught by Don Ylvisaker. I just assumed that Don was a great teacher
for all time, but he later told me that he never had another class like
it. Four or five students from that class went on to get doctorates in
statistics.

\begin{figure*}[b]

\includegraphics{495f03.eps}

\caption{Nancy with her parents, Elizabeth Blincoe and
Carr Irvine Flournoy, at her graduation from the University
of Washington in 1982.}\label{uwgraduation}
\end{figure*}

{\bf Rosenberger:} You were fortunate to be at UCLA at a time when
there were some of the great names in biostatistics: Abdelmonen Afifi,
Frank Massey, Wil Dixon, Olive Dunn, Virginia Clark. What professors
excited you at UCLA?

{\bf Flournoy:} Afifi was the young dynamic professor and taught out
of Scheff\'e; all the students loved Afifi. Dixon had a bimodal
distribution among the students; you either loved him or hated him. He
put out a thousand ideas a minute; if you paid close attention, you
would find they were pearls. It was a challenge to get what he was
saying as he didn't change the tone of his voice when he switched from
one topic to another. He taught the power of data analysis as a tool
for learning and a thousand little ways to make the data sing. I had a
class with Frank Massey; I learned a lot, but he was quiet and not dynamic.

{\bf Rosenberger:} Did you have any connection to the Department of
Statistics? You mentioned Ylvisaker. What about Paul Hoel?

{\bf Flournoy:} A separate statistics department did not exist at that
time; it was a math department with a few statisticians. I used the
Hoel, Port and Stone probability book when it was just a set of notes.
I don't think Hoel was the instructor though.

{\bf Rosenberger:} What interested you in biostatistics?

{\bf Flournoy:} Most of the statistics courses that were offered at
UCLA were in the Department of Biostatistics. Prior to taking
statistics, I had loved biochemistry and was a nutrition major, leading
to my major in the School of Public Health (SPA). When I recognized
that with a degree in nutrition, I would probably only be able to run a
cafeteria in a hospital, I decided to get my degree in mathematics
instead. I applied repeatedly to change from SPA to the College of Arts
and Sciences (CAS), but my application would get turned down. In tears,
I didn't know what to do. Then the SPA Dean asked why I was flunking
out, which didn't make sense since I always had gotten As and Bs. They
had lost all my records because I had changed names when I was
previously married, and nothing followed me. So that's why they didn't
accept me at CAS. By the time this got settled, I had enough credits to
get a degree in biostatistics.

\section{Graduate School}

{\bf Rosenberger:} What did you do after you graduated? How did you
get to graduate school?

{\bf Flournoy:} When I got a job at Regional Medical Programs as a
Statistician I, one old man would come around and ask me to add
numbers; I told him he could hire a statistical clerk for half my
salary. I was told that, as a young woman, my presentations were not
credible. So they hired a male DrPh to present my reports in his name.
As a mild way of protesting, I put my hair in a bun, dyed it white, and
got fired. They said I was an ``uppity woman.'' At that time, Virginia
Clark was department chair. She said, ``We have a fellowship, why don't
you come to grad school?'' I have some happy memories of my master's
program at UCLA: Olive Dunn supervised my master's thesis; Mary Ann
Hill was a great teaching assistant for Dixon's class; Carol Newton
taught a mean FORTRAN programming course; and Ray and Jean Mickey were
influential in my career decisions.

{\bf Rosenberger:} When you won the David Award, you talked about
meeting F. N. David. Tell us about that.

{\bf Flournoy:} I was in the Los Angeles chapter of the ASA; around
1972, a group of us carpooled out to UC Riverside where David was
giving a talk. She had a strong presence, standing with one leg up on a
stairstep and smoking a cigar while she talked. It was a roomful of
people, and she exuded such confidence. So I immediately started
smoking cigars. I had been used to seeing female statisticians such as
Clark and Dunn behind a desk and not commanding an audience.

{\bf Rosenberger:} How did you get to University of Washington (UW)?

{\bf Flournoy:} After the M.S., I thought I knew everything about
statistics. I got a job at Southwest Education and Laboratory for
Research, where there were a lot of education psychologists who were
into experimental design. On my second day, they presented me with
computer output that had more than one error term; I had the good sense
to keep my mouth shut. I immediately called Wil Dixon and asked what
they were talking about. He replied, ``Oh well, we can't teach you
everything.'' He suggested I get a book by Walt Federer. The book was
out of print, but Walt got a preprint from India and sent it to me; so
I spent my nights reading Federer's book.

Later, I was trying to read the {\em Journal of the American
Statistical Association} to implement some of the stuff I wanted to do,
and I found I couldn't read the literature. I also wanted to escape the
smog of Los Angeles. So I applied to the UW, my only application. Dick
Kronmal said there was a research assistant position with the bone
marrow transplant team, which was then located in the Old Public
Hospital (recently Amazon) in Seattle.

At that time, there was no statistics department at UW. The
mathematical statistics courses were taught in the Department of
Mathematics. I took the mathematical statistics sequence from Galen
Shorack. I had courses from Ron Pyke and Fritz Schultz in
nonparametrics. Shortly after Fritz left for Boeing, the remaining
statistics faculty formed the Department of Statistics. In the
Department of Biostatistics, there were some female faculty: Paula
Diehr and Pat Wahl. I took the first categorical data analysis class
taught at UW from Norman Breslow. He gave quizzes at the end of class,
so I never paid so much attention in a course before. I took survival
from Ross Prentice early in the days of the Cox proportional hazards model.

{\bf Rosenberger:} What was it like working with your dissertation
advisor, Lloyd Fisher?

{\bf Flournoy:} It worked out well because we have similar work
styles. Both of us had busy consulting lives; we would schedule
meetings and get our business done.

\section{The Seattle Bone Marrow Transplantation Team}

{\bf Rosenberger:} Today every street corner seems to have a contract
research organization for data coordinating centers on large clinical
trials. But when you went to the Fred Hutchinson Cancer Research
Center, information technology was primitive, such places did not
exist. You had to create that environment on your own. What was it
like? What were the challenges?

\begin{figure*}[b]

\includegraphics{495f04.eps}

\caption{Yash Mittal, first female director of the probability program,
and Nancy, first female director of the statistics program, at NSF.}\label{yash}
\end{figure*}

{\bf Flournoy:} That's an interesting story. Dick Kronmal had invested
a lot of effort in creating a database management system without
requiring a rectangular data structure. Updates required physically
sorting the cards (remember all data records had to fit into the 80
digits of a Hollerith punch card---so I tend to use the words ``card''
and ``record'' interchangeably). There was a transplant data set in
place with seven different kinds of cards. Kronmal had E. Donnal ``Don''
Thomas (Director of the Clinical Research Division at the Cancer
Center) buy a computer. The computer weighed 50 pounds (I could toss it
in my van and take it home; the cost was about \$50,000), and data
storage was on Phillips cassette tapes. Records could be transmitted
across the phone wires and then integrated into the database at UW.
Initially, there was not much data (only 10 patients) and the first
update took my whole computing budget for the year! What I did then for
some period of time was, when it was time to do an update, punch cards
of the whole database and the new dataset; I would physically sort and
merge the cards by hand and load them into SPSS. That was my ``dirty
laundry'' story because the laundromat had big long tables and I sorted
cards while doing laundry. Kronmal told me that, if I had any trouble
with my new computer, I should call Leonard Hearne. Index sequential
files were brand new at that time, and Leonard used them to create an
early database management system before the word was in the literature
(see Flournoy and Hearne, \citeyear{FloHea,FloHea90N1,FloHea90N2}). We used it for several
years until a commercial system came on the market. At site visits,
someone would ask a question and I would pass a note down to a
programmer, who would extract the answer in 15 minutes or so. We set
the bar for oncology programs.

Ross Prentice came from the University of Waterloo with a box of cards
on the Cox proportional hazards model; we were really early using that.
The doctors were smart enough to understand the limitations in using
discriminant analysis and they were thrilled to be able to incorporate
censored survival data in their regression models. My work documenting
graft vs. leukemia in humans was very important (see Weiden et al.,
\citeyear{Weietal79,Weietal81N1,Weietal,Weietal81N2}).
One hypothesis motivating the development of bone marrow
transplantation was that the marrow graft would attack residual
leukemia also. Immunological activity of the graft was apparent when
the graft instigated an immunological attack on the patient. I modeled
the impact of this attack on the relapse rate. The protection of the
graft attack against relapse greatly complicated post-transplant
treatment strategies. But our findings have withstood the test of time.
It was, perhaps, the first major application of the proportional
hazards model with time-dependent covariates.

{\bf Rosenberger:} When you think of the success of the bone marrow
transplant program (Don Thomas won the Nobel prize in 1990 for
developing bone marrow transplantation as a treatment for leukemia),
how much did statistics and data management play a role in that? Do you
think statistics and data management will ever get its due?

{\bf Flournoy:} We had this rudimentary set of records that could be
added onto infinitely. It started out that bacteriology wanted to add a
card, then virology, then specific studies would add a card with their
data. Before you knew it, we had an interdisciplinary shared database
with assigned patient numbers so all the integrated data was available.
I was able to say ``do you know what they're doing in virology that's
related'' because I knew everybody's data. It wasn't until many years
later that people started talking about having integrated shared
databases. Most were established for billing purposes, not for research
purposes. They are different constructs. Hospitals would archive data
after the bill was paid but we wanted to keep it around forever.

When the program started, there was one of everybody (one statistician,
one virologist, etc.), and we would sit around the table and share
results. It was important to be influential and to catch problems in
data collection and quality control before they got big. When working
with new doctors, there were humps you had to get over because they
would claim that there were no quality control issues: their lab people
never made a mistake. A lot of negotiation had to go on before we could
agree. Yes, we had a huge influence. Even randomization and blinding
was controversial. If it was in the middle of the night the cards might
get shuffled; there was too much room for bias. We introduced them to a
very careful randomization regimen for treatment assignments, with a 24
hour on-call person.

It will be hard for statistics and data management to ever get its full
due because the doctors are so enamored of themselves (laughs). It's
really a strange system where the people with the least science
background usually run the science. Also, the data management budget
was always the first to be cut; yet it is very expensive to do a
quality job.

{\bf Rosenberger:} What has your role been in fostering
interdisciplinary research?

{\bf Flournoy:} Having conducted interdisciplinary research for more
than a decade at the Cancer Center, I knew the power that teams of
interdisciplinary researchers could bring to bear on important
scientific questions. Coincidentally, when I went to the National
Science Foundation (NSF) in 1986, the Division of Mathematical Sciences
(DMS) had funded the Institute of Mathematical Statistics (IMS) to
write a report on cross-disciplinary research. I watched the growth in
their thinking as they interacted with each other. At the time, the
discipline did not appreciate the role of applications in academic
settings. I think I was able to influence the IMS cross disciplinary
committee on the valuable nature of interdisciplinary work. The report
of the committee had a dramatic effect on the discipline. The report
proposed establishing the National Institute of Statistical Science.
Since I was at NSF, I was able to promote the idea of establishing a
broad institute that would work on problems of national importance.

At the same time, I would receive proposals from statisticians
motivated by applications. Because our budget was small, I took such
proposals around to the relevant disciplines that were involved, and
was able to get some joint funding. This resulted in my getting an
award in 1988 for facilitating the funding of these interdisciplinary
projects. This also led to specific DMS requests for proposals for
interdisciplinary research projects, which are now common throughout NSF.

\section{Adaptive Designs}

{\bf Rosenberger:} How did you get interested in adaptive designs?

{\bf Flournoy:} While at the Cancer Center, the major program project
grant had five-year reviews. When we prepared for the third one of
these, we spent a year reviewing what we had done and how we would go
forward. In the course of that retrospective, I developed some feelings
about the two arm clinical trial. The standard ideas about the two arm
clinical trials came from the Peto paper in the mid 70s (Peto et al.,
\citeyear{Petetal76,Petetal77}). But, in my experience, a treatment is a point in a high
dimensional space: involving drugs, radiation, including how much, how
often; and one learned little about this high dimensional space using
the traditional two arm clinical trial. For instance, we spent five
years comparing A to B; but then we go back to the high dimensional
space and pick out point C, and have another five years of
experimentation and compare A to C. Then we compare C to D, and after
15 years we have knowledge of four points in a high dimensional space.
I believed it would be more efficient and informative to know which
direction we should head in the high dimensional space. So that led me
to think about adaptive designs. I recommended several to the group and
the physicians liked the ideas, but thought they may be too radical to
get funded.

Another thing that promoted my interest was looking at pilot studies to
decide what to take forward to larger studies. Bob Tsutukawa was
visiting the Cancer Center from the University of Missouri at the time.
I thought his Bayesian ideas were appealing and I used expert opinion
for prior elicitation (see Flournoy, \citeyear{Flo93}). The prior was way off, so
we wound up with a lot of toxicities. You just can't trust the best
expert opinion of the best experts, and so there needed to be some way
to use interim data faster to adapt and put much less weight on the
prior. My later work showed how random walk rules could be constructed
to do this (see Durham and Flournoy, \citeyear{DurFlo94}; Durham, Flournoy and
Rosenberger, \citeyear{DurFloRos97}; Flournoy and Oron, \citeyear{FloOro}).

\begin{figure*}

\includegraphics{495f05.eps}

\caption{1994 IMS Workshop on Sequential Analysis at University of North Carolina, Chapel Hill.  Included are Nancy Flournoy (seated second from left), Lynne Billard (immediately behind Nancy), Janis Hardwick (to the right of Lynne),
Bill Rosenberger (third row from back, middle), and Steve Durham
(directly in front of right window).}\label{seqanal94}
\end{figure*}

{\bf Rosenberger:} The first time I heard the name Nancy Flournoy was
in the context of the 1989 session on adaptive designs at Joint
Statistical Meetings (JSM) in Washington. It turned out to be one of
the most controversial sessions in the history of JSM. Talk about that.

{\bf Flournoy:} My experience at NSF was that you don't make progress
without community. One person alone doesn't get much done. So I had the
idea that a JSM session on adaptive design would bring together people
who are interested in adaptive designs. I didn't know anyone
personally. I invited based on my impressions of their interests. I
invited Don Berry, Richard Simon and Janis Hardwick. I gave a
straightforward technical talk on the topic. The remainder of the
session focused primarily on criticism of the extracorporeal membrane
oxygenation (ECMO) trials (e.g., Barlett et al., \citeyear{Baretal85};
O'Rourke et al.,
\citeyear{ORoetal89}; Ware, \citeyear{War89}).
(The ECMO trial was an implementation of the
randomized play-the-winner rule of Wei and Durham, \citeyear{WeiDur78}, in which 11
babies were assigned to an experimental arm, and all survived, while
one baby assigned to the conventional arm, died. The historical death
rate on the conventional arm was 80 percent.) To my dismay, all the
negative focus of the session was directed toward the adaptive design
aspect of the clinical trial, rather than on the sample size and what
kind of sample size would be needed for the trial to be convincing. The
press that was generated by this session set adaptive designs back a
long time.

{\bf Rosenberger:} How much do you think the failed ECMO trial
inhibited the development of adaptive designs?

{\bf Flournoy:} What would have been a reasonable approach? The
original trial was unconvincing due to having few patients, in spite of
the fact that a probabilistically reasonable stopping rule was applied.
The controversy over the subsequent two arm trial in clinical research
set back adaptive designs wrongly. The adaptive trial was so successful
that only one baby died; is that bad?

{\bf Rosenberger:} In your 1992 AMS/IMS/SIAM conference on adaptive
designs (Flournoy and Rosenberger, \citeyear{FloRos1995}), you brought together some of
the  groundbreakers of adaptive designs along with a number of younger
faculty who are now at the forefront of the discipline. At the opening
session, you started by talking about the need to streamline the
process of clinical trials, the end to phases and the incorporation of
dynamic interim decisions. You said that will revolutionize the way we
do clinical trials, and that this conference would be an ambitious
beginning to that revolution. Now, over two decades later, there are
70-some sessions on adaptive designs at the Joint Statistical Meetings,
``big-pharma'' working groups, Food and Drug Administration white papers
and guidelines, companies like ADDPLAN, and CYTEL devoted to adaptive
design software and everyone wants to do adaptive designs. What took so long?

{\bf Flournoy:} ECMO made a steep hole to climb. We also had to
develop theory. It was one thing to say ``this is a good idea,'' and
another to adequately support it. Some ideas were NOT good. This
includes a class of procedures that derive from stochastic
approximation, that Val Fedorov coined ``best intention'' designs. In
these designs, a target dose is estimated (such as the dose having a
particular percent toxicity or one that maximizes some utility
function); then that estimate is the dose given to the next subject.
Some, including Lai and Robbins (\citeyear{LaiRob82}), understood early on that using
this procedure without safeguards may result in treatment sequences
that converge to the wrong dose. But others, including myself (Li,
Durham and Flournoy, \citeyear{LiDurFlo95}), were enamored of this idea and ignorant of
earlier warnings. This approach remains popular today even as recent
publications are exposing just how misleading it can be (e.g., Azriel,
\citeyear{Azr12}, Oron and Hoff, \citeyear{OroHof13}).

In the 1980s, John Whitehead spent a year visiting the Cancer Center
from the University of Reading, and promoted the idea of using
sequential stopping rules taking censoring into account. Its value was
so obvious that I expected that by 1990 every clinical trial would be
using these techniques. So I focused instead on adaptive allocation. At
American University (AU), I worked on theoretical problems in these
areas. When I pulled my head out and looked around I was shocked to see
that stopping rules incorporating censoring were not being used, except
a bit in cancer. So things that seem obvious to some can take a long
time to enter the medical arena. Take the ``$3+3$'' dose escalation design
as an example. It has been soundly discredited (Reiner, Paoletti and
O'Quigley, \citeyear{ReiPaoOQu99}; Lin and Shih, \citeyear{LinShi01}),
and yet remains a standard
practice in oncology phase I trials.

Adaptive allocation is still in its infancy compared to sequential
monitoring and stopping. Now there has developed a new belief that
simulation is adequate for assessing an adaptive design. But relying
solely on simulation muddies the water because there is no global view
of what is driving the design. In addition, there are many papers in
the literature that report only averages over simulations without
measures of variability. When you consider measures of variability, a
completely different picture emerges (Oron and Hoff, \citeyear{OroHof13}). I fervently
believe in developing the theory underlying classes of designs.
Fortunately, many people are interested in working on the theoretical
challenges, and there are a lot of interesting open questions.

{\bf Rosenberger:} Many times when I hear talks on adaptive designs I
want to scream out ``Nancy Flournoy thought of that in the 1980s.'' How
do you feel about some of your early ideas being ignored?

{\bf Flournoy:} Well, I'm hardly alone in this. For instance, Chris
Jennison invented many clever techniques for sequential and adaptive
clinical trials very early that are sometimes ``rediscovered'' without
reference (e.g., Jennison, Johnstone and Turnbull, \citeyear{JenJohTur82};
Kulkarni and
Jennison, \citeyear{KulJen86}; Jennison, \citeyear{Jen87}).
In my case, it amazes me that there
are a large number of people who will reference a paper from the 1980s
and ignore 30 years of my research. For example, the early up and down
paper of Storer (\citeyear{Sto89}) is often cited without reference to my later
papers that have much more sophisticated control of the adaptive
process. This early paper is used as a whipping post to declare up and
down procedures inferior. An up and down design is a random walk that
can end anywhere. The last state (dose) visited should not be used as
an estimator. But this is done when the up and down design is compared
to other procedures that derive from stochastic approximation (e.g.,
Zacks, \citeyear{Zac09}). That bothers me a lot.

{\bf Rosenberger:} How did you meet Steve Durham? This began one of
the great collaborations in statistics. Tell us about that.

{\bf Flournoy:} One of the few positive consequences of the 1989 JSM
session was meeting Steve Durham from the University of South Carolina.
When I walked out the door after the session, Steve introduced himself
and was very excited because we were basically working on the same
mathematical problems, his from an engineering motivation, and mine
from a medical motivation. We began working together right away. He
would come to Washington, DC, to meet me, and I went to South Carolina.
After a stint as Chair at AU, I was on a sabbatical at the University
of North Carolina Chapel Hill; Leonard and I bought a house close to
campus so that we could host visitors. In particular, Steve Durham and
I worked together quite a lot in that house and at the Department of
Statistics. Several other collaborators came down for extended periods,
including you (W.F.R.) and two of my doctoral students: Eloi Kpamagen
(now at Novavax) and Misrak Gezmu (now at National Institutes of Health).

{\bf Rosenberger:} The introduction of the random walk rules coincided
with the introduction of the continual reassessment method (CRM;
O'Quigley, Pepe, and Fisher, \citeyear{OQuPepFis90}) in the Bayesian context. In
particular, you and Steve worked out the entire exact distribution
theory of a class of designs, while others were relying on simulation.
How does this rank in terms of your contributions to statistics?

{\bf Flournoy:} The random walk rules are extremely practical and
mathematically elegant, so it was a lot of fun to develop the theory.
They are the standard in many areas of science (e.g., American Society
for Testing and Materials, \citeyear{ASTM1991}; Treutwein, \citeyear{Tre95};
National Institute of
Environmental Health Sciences, \citeyear{NIEH2001}).
The key property that we
discovered was how to control the allocation coverage by introducing an
appropriate bias (Durham and Flournoy, \citeyear{DurFlo94}). Steve was always
thinking in terms of engineering applications; I was always thinking in
the dose-response context. We did ``reverse engineering,'' in that we
had a target allocation in mind, and we found design parameters to
facilitate this. The designs are nonparametric in that allocation does
not depend on estimates of model parameters. They are extraordinarily
simple to illustrate and have exact distribution theory that is
unavailable for other, more complicated designs.

{\bf Rosenberger:} Some have lumped random walk rules in the context
of generic dose escalation designs, such as the $3+3$ design, that has no
optimal properties. At the same time, Bayesian approaches, such as the
CRM were becoming increasingly well-known. Talk about the historic
interplay among these approaches.

{\bf Flournoy:} Lloyd Fisher and John O'Quigley (from the University
of Leeds) were hired at the Cancer Center to replace me when I left for
the NSF. Lloyd and I laughed that it is not often one's dissertation
advisor replaces his student! John was initially responsible for
implementing a random walk rule that I had designed in a pilot study
for a bone marrow clinical trial. He let them get away with a simple
dose escalation procedure, but he and Lloyd got introduced to the
subject at that time. They immediately thought of doing a Bayesian
alternative, and it was published in 1990 in {\em Biometrics}
(O'Quigley, Pepe and Fisher, \citeyear{OQuPepFis90}); the major random
walk paper
appeared in {\em Biometrics} in 1997 (Durham, Flournoy and
Rosenberger, \citeyear{DurFloRos97}). Most of the Bayesian literature
 was, by necessity,
simulation based, whereas Steve and I were busy obtaining a complete
workable probabilistic theory of the random walk procedures.

There are a number of philosophical differences among the approaches.
Fedorov would call the CRM a ``best intention'' approach, because it
involves predicting a target dose and treating the next patient at that
dose, sequentially. Our approach is estimation-motivated. The idea is
to get allocations into a region of interest that allows us to
efficiently estimate the dose-response curve in that region.

There is also a short-memory and long-memory distinction: allocation
probabilities for the random walk rule converge exponentially fast to
their asymptotic limits. Alternatively with best intention designs
(which to date are long-memory designs), nonrepresentative early
allocations can cause the design to converge to the wrong dose (see,
e.g., Azriel, Mandel and Rinott, \citeyear{AzrManRin11};
Oron, Azriel and Hoff, \citeyear{OroAzrHof11};
Azriel, \citeyear{Azr12}). Such phenomena were observed early on in the context of
stochastic approximation designs (e.g., Lai and Robbins, \citeyear{LaiRob82};
Bozin
and Zarrop, \citeyear{BozZar91}).

Adaptive optimal designs are promising long memory designs, but they
depend on parameter estimates to get started. Random walk procedures
that target optimal design points provide good start-up information
with small sample sizes. Alternatively, one can regularize the
information matrix, a ``fix'' that is often called ``Bayesian designs''
even though no posterior distribution is obtained. True Bayesian
estimator updates coupled with dose allocations made in some stable
optimal way, rather than in a ``best intention'' way are also promising.

{\bf Rosenberger:} What is the future of adaptive designs? Do you
think all clinical trials will eventually be adaptive?

{\bf Flournoy:} I think there is a great future for adaptive designs.
I think experimentation will always involve a series of trials; the
question is how well one utilizes information from one to the next.
There is a lot of value in relatively small but sequential trials (see
Flournoy, \citeyear{Flo}), because these trials involve many design features,
including the grid size and range on which you are operating. The best
use of one experiment may be to tell you how you could have better
selected design characteristics; then you can refine the estimate of
the target of interest.

\begin{figure*}[b]

\includegraphics{495f06.eps}

\caption{Nancy, husband Leonard, and Lynne Billard at their home
in Chapel Hill, NC, 1994.}\label{lenandlynne}
\end{figure*}

Some of my work has been on inference and estimation following adaptive
designs (e.g., Rosenberger, Flournoy and Durham, \citeyear{RosFloDur97};
Ivanova and
Flournoy, \citeyear{IvaFlo}; May and Flournoy, \citeyear{MayFlo09};
Lane, Yao and Flournoy, \citeyear{LYF2014}).
One has to be careful doing everything sequentially because some of the
interim changes may cause final estimates to lack normality. For
example, in best-intention designs, the estimate of a slope parameter
can march off to infinity for some common models. Also, even if an
adaptive dose-finding procedure has a fixed total sample size, the
sample sizes at each dose are random variables. In up-and-down
procedures, the proportion of subjects allocated to each dose tends to
a constant and standard asymptotic normality results. But in many other
adaptive designs, proportions of subjects allocated to each dose tend
to a random variable. This causes the conditional information matrix to
be random, even in the limit, in which case standard conditions for
asymptotic normality fail. These are many interesting questions to be
explored about adaptive designs.

\section{Women in Statistics}

{\bf Rosenberger:} Talk about the creation of Pathways to the Future,
its successes, and its legacy.

\begin{figure*}

\includegraphics{495f07.eps}

\caption{Nancy, Ingram Olkin and Elizabeth Margosches
(formerly with the Environmental Protection Agency) at the
Campanile at University of California, Berkeley, 2003.}\label{ingram}
\end{figure*}

{\bf Flournoy:} I went to the 1984 Annual IMS Meeting in Lake Tahoe.
At that meeting, there were five women out of about 200 attendees. It
became quite clear to me that this was an important place for academic
statisticians to meet and focus on academic interests. In anticipation
of the 1988 Fort Collins IMS meeting, which was separate from the Joint
Statistical Meetings, I decided it would be great to see more women
there. So I bounced ideas off Mary Ellen Bock (Purdue University) and
Lynne Billard (University of Georgia). Lynne agreed to take the lead in
organizing a workshop for women at the upcoming IMS meeting. Lynne had
the brilliant idea of having Elizabeth Scott (University of California,
Berkeley) give the keynote lecture. At that time, we were debating
whether there was any gender inequity in academia, and we weren't sure.
I had never experienced problems at UCLA or UW. However, when I went to
the NSF, Yash Mittal (the first female director of the probability
program) and I saw that there were almost no female grantees, and very
few were even applying for grants.

The evening presentation by Scott really hit us very hard: she had tons
of data and randomized studies on gender inequity. Any questions about
inequities in how women were recruited, judged and valued were thrown
out the window. Scott's way of handling this lecture was wonderful
because she went through all this horribly depressing data, but she
then turned around and finished the lecture by telling us what we could
do to protect ourselves. She ended with two positive notes: that
outcomes are not predetermined, and one can take one's career in one's
own hands. Lynne ran the workshop for the next two decades, and she
presented Scott's lecture with updated data every year. That lecture
was the last lecture Scott gave before she passed away. I remember well
that there was a palpable sigh of relief from Scott---that she could
turn over her cause to the next generation.

{\bf Rosenberger:} How did you become NSF program director? What was
your experience with gender issues there?

{\bf Flournoy:} Ingram Olkin has long been a great friend and mentor.
He is the one who recommended me to the NSF for the program director
position. I was the first female director in the statistics program the
same year that Yash Mittal was the first female probability director.
Some people had indicated to the division director their fear I was
going to give all the grant money to biostatistics. I convinced him
that I could represent the entire statistics field.

One day I remember answering the phone and a professor on the line
yelled ``I said I wanted to speak to the director,'' thinking a woman
on the phone must be a secretary.

We had a good travel budget and I went to as many young women's
lectures as I could. I would go up at the end of their talk and ask if
they would be interested in applying for a grant. By the time I left
NSF, the proportion of grant proposals from women was proportional to
their presence in the field. A suggestion is such a small thing, and
yet clearly important messages weren't being transmitted to female faculty.

{\bf Rosenberger:} Was discrimination subtle or not so subtle when
your career was developing?

{\bf Flournoy:} Well, there was always sexist behavior and many things
that were said and done are considered inappropriate or even sexual
misconduct today. When I went on the job market for a fully academic
position I found that many men were incredulous. Some would make
outrageous comments directly to me as if I were invisible (or a man).
Men in my own age category were often dismissive or oblivious to my
presence. Some of the older generation was very helpful and supportive
(I think of Shanti Gupta, Purdue University; Norman Johnson, University
of North Carolina at Chapel Hill; Lucien LeCam, University of
California, Berkeley; Ingram Olkin, Stanford University; and Manny
Parzen, Texas A\&M University). The younger generation just thought of
me as another senior person, so they were fine.

{\bf Rosenberger:} What is your feeling about the role of women in
statistics today? I can say, from my perspective on 20 years of search
committees, that from a hiring perspective, we are thrilled to have
qualified women candidates and compete hard to get them. And certainly
policies on tenure to allow maternity leave have vastly improved over
the years, as have the composition of committees and senior
administrators. Is there any work left to be done?

{\bf Flournoy:} You can see improvement, but there are still troubling
facts: just try to find a woman in the 2013 JSM awards brochure, for
instance. Women are getting hired at proportional rates now, but
awards, tenure and advancement are areas where there much is left to be
done. See Lynne Billard's new update of Scott's old data on the subject
(Billard and Kafadar, \citeyear{BilKaf14}). That will depress you.

\section{Conclusion}

{\bf Rosenberger:} You talked a little about your transition into a
fully academic position. The latter part of your career was spent at AU
and University of Missouri (MU), and considerable time as department
chair, and a mentor to many diverse
students. Talk about this.

{\bf Flournoy:} AU was a great place for me when I went there in 1988.
I had left the Cancer Center with a staff of 23, a budget of \$700,000
and responsibilities that had become a burden when I became convinced
of the need for more nimble learning strategies in dose-finding
clinical trials. I had eight doctoral students at AU, and all but two
of them developed mechanisms to control random walks and urn models,
and to provide mathematical descriptions of their controlled behavior.
One worked on issues of inference following an adaptive design and one
worked on a problem in economics. I am proud that four of these
students are black and two are women.

\begin{figure}

\includegraphics{495f08.eps}

\caption{Nancy near Aasgard Pass in the Enchantment Lakes Wilderness
Area, Washington, where she was hiking with her husband Leonard and her
colleague Lori L. Thombs (University of Missouri) following the 2006
Joint Statistical Meetings in Seattle.}\label{hiking}
\end{figure}

Unfortunately, a very destructive president came to AU, and by 2000 it
was clear that STEM graduate programs were going to be dismantled. AU
had one of the oldest statistics doctoral programs in the country and
it was sad to see it threatened by ignorance and arrogance. To remain
in a department with a doctoral program, I needed to move and this led
me to accept the chair at Missouri in 2002. When I stepped down as
chair in 2011, I had doubled the number of tenure-track faculty and
added five teaching faculty positions. I increased the presence of the
department across campus through joint appointments and a targeted
increase in service courses, and I increased the prestige of the
department nationally, personally promoting our faculty and enabling
their participation in national and international activities. More
details can be found in a Chapter I recently wrote on the history of
statistics at MU (Flournoy and Galen, \citeyear{FloGal}).

I have graduated seven doctoral students from MU. We worked on adaptive
and optimal designs; we developed new models for specific, challenging
dose-response problems and we have illuminated the effect of having
dose allocations depend on the history of prior allocations and
responses. My students continue to bring me a great deal of pleasure.

{\bf Rosenberger:} What are your hobbies and interests?

{\bf Flournoy:} I love hiking. I am not happy with a trip that takes
less than four days. A four-day trip has two days out and two days
back---so one is never very far from a road. After hiking for more than
two days, one must rely on one's self much more completely. It is so
peaceful. I gave up trying to hike in the East and the Midwest United
States. One just can't get far enough away from roads; and the
mountains aren't high enough. I like trekking around timberline for a
week or more where the views are spectacular. I keep going back to
Yosemite, Kings Canyon and Sequoia National Forests. Nepal was great,
too. I try to get in one long hike each year. In the meantime, I~dance.
I~resumed ballet classes while at AU; it is great mind-to-body exercise
and wonderful for strength and balance. Leonard and I enjoy English
country dance together. Throw in Pilates and yoga and I am happy.

To survive a severe health challenge that had the doctors stumped, I
gained considerable knowledge of alternative methods and became
accomplished in some. But that is another story.

{\bf Rosenberger:} What's next for Nancy Flournoy?

{\bf Flournoy:} Well I have a lot of ideas. I'm really interested in
questions of inference following adaptive designs. We have some
examples in two stage designs that maximum likelihood estimators are
mixtures of normals; some designs lead to estimators that are normal
with random variances. I think our preliminary results are
generalizable, but this remains to be shown. I'm optimistic that
tractable solutions to seemingly intractable problems are at hand.


\section{Introduction}
\label{sec: Introduction}
We discuss an intuitive alternative to classical Principal Components analysis that selects the (singular) affine transformation of the original scatter, visualized as N points embedded within a p-dimensional Euclidean space, that reduces its dimensionality to q $<$ p while attempting to reproduce all pair-wise, squared Mahalanobis inter-point distances.  We show that this criterion is not coordinate-free; it places extra emphasis upon reproducing the squared Euclidean distances of all N points from the origin.  Our origin-centric perspective on affine reduction of dimensionality encourages interpretation of the recovered scatter using either polar or Cartesian coordinates.

In traditional principal component and coordinate approaches to reduction of dimensionality, the origin is conventionally placed at the geometric centroid of the overall scatter by subtracting the observed arithmetic mean value from each given coordinate.  This convention can, optionally, be retained in the origin-centric approach discussed here.  On the other hand, the impact of details such as how the given coordinates were originally scaled and intercorrelated are minimized in this new, alternative approach.  This is accomplished by placing the given data scatter in its affine invariant canonical form, Obenchain(1971, 1972a).  Interestingly, this substitution makes reduction of dimensionality using conventional principal components analysis impossible; all one-dimensional orthogonal projections of the revised data have the same variance, and any two mutually orthogonal projections are uncorrelated.  In particular, this means that the ultimate impact of researcher initiatives, Glaeser(2006), in choice of included variables and of their initial scaling upon potential recovered configurations of low dimensionality is thereby greatly reduced.

The format of the paper is as follows. In the next three Sections, we introduce basic notation and define terminology related to Euclidean distance calculations, matrix norms in linear algebra, and origin-centric coordinate concepts.  Sections \ref{sec: example1} and \ref{sec: example2}, then display two relatively simple numerical examples.  We end this overview paper with some general discussion points in Section \ref{sec: Discussion}. The final four Sections, \ref{sec: affine invarcomm} to \ref{sec: plarge}, are somewhat more technical Appendices covering historical background information and some relatively advanced observations and conjectures.

\section{Origin-Centric Coordinates and Euclidean Inter-point Distances}
\label{sec: origin-centric}
When a N $\times$ p matrix of finite real numbers, $\vec X$, is used as a Cartesian coordinate representation of N points in Euclidean space relative to p = 2 or more mutually orthogonal axes, the conventional view is that the location of the origin is not important.  After all, the Pythagorean theorem dictates that the squared distance between the i-th and j-th points in the scatter is given by
\begin{align}
\label{eqn: sqdist}
\vec d^2_{ij} &= (\vec x'_{i} - \vec x'_{j})(\vec x_{i} - \vec x_{j})\nonumber\\
    &= \vec x'_{i}\vec x_{i} + \vec x'_{j}\vec x_{j} - 2\vec x'_{i}\vec x_{j},
\end{align}
where $\vec x'_{i}$ denotes the i-th row of $\vec X$.  In the first line of the above expression, note that only differences in coordinates between points are involved.  It follows that  $\vec d^2_{ij}$ is, by definition, invariant under shifts in the location of the origin resulting from simple (additive) translations of coordinates.

The second line of equation (\ref{eqn: sqdist}) shows an equivalent expression containing origin-centric terms.  The first two terms are the squared Euclidean distances of the i-th and j-th points from their current origin, $\vec 0$.

Similarly, the full N $\times$ N matrix of inter-point squared distances can be written as
\begin{align}
\label{eqn: sqdistmtx}
\vec D^{(2)} &= ((d^2_{ij})) \mbox{  ...for all i and j from 1 to N} \nonumber\\
    &= (\vec d^2_0)\vec 1' + \vec 1(\vec d^2_0)' - 2\vec {XX'},
\end{align}
where $\vec d^2_0$ is the N $\times$ 1 column vector that consists of the ordered elements of $Diag(\vec {XX'})$.  In other words, the i-th element of $\vec d^2_0$ is the $\vec x'_{i}\vec x_{i}$ term of equation (\ref{eqn: sqdist}).  Thus $\vec d^2_0$ is the vector of squared Euclidean distances of all N points from their current origin, $\vec 0$.  The $\vec D^{(2)}$ matrix is symmetric, and its N diagonal elements are null.

Suppose now that a given $\vec X$ matrix is to be optimally approximated by a N $\times$ q matrix of finite real numbers, $\vec Z$, where q $<$ p but q is at least 1.  Specifically, suppose that the objective is to minimize the squared Frobenius norm of the difference between the $\vec D^{(2)}$ matrices computed from the $\vec X$ and $\vec Z$ coordinates:
\begin{align}
\label{eqn: sqdifnorm}
\vec {Norm}^2 &= \sum_{i=1}^N \sum_{j=1}^N (d^2_{ij}(\vec X) - d^2_{ij}(\vec Z))^2 \nonumber\\
    &= || {\rho}\vec 1' + \vec 1 {\rho}' - 2(\vec {XX'} - \vec {ZZ'})||^2 \nonumber\\
    &= 4||\vec {XX'} - \vec {ZZ'}||^2 + 2({\rho}'\vec 1)^2 + 2N{\rho}'\rho,
\end{align}
where $\rho = d^2_0 (\vec X) - d^2_0 (\vec Z)$ is the column vector of discrepancies in squared difference of points from the origin in $\vec X$ and $\vec Z$ coordinates.

In the special case of primary interest here, the $\vec {XX'}$ outer-products matrix ($N \times N$) is what is know as an Association matrix, $\vec A$, for ``individuals'' (points), Gower(1966), while the $\vec {X'X}$ inner-products matrix ($p \times p$) contains the corresponding Adjusted Sums-of-Squares and Cross-Products for ``variables.''  The objective in Principal Component Analysis (PCA) is to minimize $||\vec A - \vec {ZZ'}||^2$ where the $\vec X$ and $\vec Z$ matrices have each been ``centered'' as outlined below in Section \ref{sec: affinermd}.  This particular point is more fully discussed in an Appendix; see Section \ref{sec: center}.

In any case, minimizing $\vec {Norm}^2$ of equation (\ref{eqn: sqdifnorm}) is potentially quite different from ordinary PCA.  Minimizing $\vec {Norm}^2$ places emphasis upon balancing two separate criteria: [1] reducing the size of the $\rho$ vector, which contains the discrepancies in {\em squared distance from the origin} for all N points within the original $\vec X$ and derived $\vec Z$ configurations, and [2] reducing the PCA norm of the difference in outer-product matrices.  Tension between these competing objectives make choice of $\vec Z$ so as to minimize $\vec {Norm}^2$ of equation (\ref{eqn: sqdifnorm}) an origin-centric approach.

\section{Affine Reduction of Multivariate Dimensionality}
\label{sec: affinermd}

The {\em Singular Value Decomposition} of a N $\times$ p matrix consisting of the ``centered,'' real valued $\vec X$ coordinates of N points in a Euclidean space of dimension (linear rank) r can be written as:
\begin{equation}
( \vec I - \vec {1\gamma'} ) \vec X = \vec H \vec \Lambda^{1/2} \vec {G'}, \label{eqn: svdcenter}
\end{equation}
where the 1 $\times$ N row vector $\vec {\gamma'}$ is any generalized inverse of the N $\times$ 1 column vector of all ones, $\vec 1$, the semi-orthogonal matrix $\vec H$ is the N $\times$ r matrix of standardized {\em Principal Coordinates}, $\vec \Lambda^{1/2}$ is a r $\times$ r diagonal matrix of strictly positive singular values, and $\vec {G'}$ is a r $\times$ p semi-orthogonal matrix of component {\em Direction Cosines}.  When r = p, the $\vec G$ matrix becomes orthogonal (and invertible); $\vec {GG'} = \vec I$ of order p.

Every choice for $\vec \gamma$ is such that $\vec {\gamma'1} = 1$; it follows from (\ref{eqn: svdcenter}) that the semi-orthogonal matrix $\vec H$ is orthogonal to $\vec \gamma$.  This $\vec {\gamma'H} = 0$ restriction assures that r $\leq min(p, N-1)$.  More complete information about this critical choice of initial translation is given in the Appendix of Section \ref{sec: center}; choice of $\vec \gamma$ determines the location of the origin, $\vec 0$, of Cartesian coordinates.

Affine reductions in dimensionality are those in which the recovered $\vec Z$ configuration is of the restricted form:
\begin{equation}
\vec Z = \vec H \vec B, \label{eqn: rqaffrmd}
\end{equation}
where $\vec H$ is the N $\times$ r matrix of (\ref{eqn: svdcenter}) and $\vec B$ is a r $\times$ q matrix of $rank(\vec B) = q < r$ consisting of real scalars.  In particular, any row of $\vec H$ that is null assures that the corresponding row of $\vec Z$ will also be null ...corresponding to a point coincident with the origin, $\vec 0$.

One of two forms of preliminary data ``standardization'' are typically applied prior to performing traditional PCA, but neither is nearly as ``drastic'' as replacing one's initial $\vec X$ matrix by its Mahalanobis principal coordinates matrix, $\vec H$, as in (\ref{eqn: rqaffrmd}).  Traditional ``mean centering'' uses the $\vec {1^+}$ vector, with each of its N elements equal to 1/N, as choice of $\vec {\gamma'}$ in (\ref{eqn: svdcenter}).  Traditional standardization to ``correlation form'' performs an addition step in which each column of the ``mean centered'' $\vec X$ matrix is then divided by its sample standard deviation.  This is essentially equivalent to the standardization in which the N elements in each column of $\vec X$ are made to not only sum to zero but also to have sum-of-squares set equal to 1; on the other hand, this standardization still allows the different columns of $\vec X$ to be (linearly) correlated or confounded.

In sharp contrast, the semi-orthogonal $\vec H$ matrix of (\ref{eqn: svdcenter}) is essentially r-dimensional from the perspective of traditional PCA.  Each column of this $\vec H$ matrix is orthogonal to the current choice of $\vec \gamma$ (defining a mean or multivariate median measure of location), has sum-of-squares set equal to 1, and is exactly uncorrelated with the derived coordinates along any direction strictly orthogonal to it.

For example, consider the 1-dimensional case, q = 1 in (\ref{eqn: rqaffrmd}), where the $\vec Z$ and $\vec B$ matrices become single column vectors, $\vec z$ and $\vec b$.  It follows that $\vec {\gamma'z} = 0$ while the corresponding sum-of-squares is $\vec {z'z} = \vec {b'H'Hb} = \vec {b'b}$, which is determined solely by the squared length of the $\vec b$ vector.  Similarly, any two different choices for this $\vec b$ vector yield uncorrelated $\vec z$ representations if and only if these two choices are mutually orthogonal vectors, $\vec {b_1'b_2} = 0$.

In other words, since PCA cannot be used to reduce the dimensionality of $\vec H$, one's primary hope may well be to adopt an origin-centric perspective based upon minimizing the squared norm of equation (\ref{eqn: sqdifnorm}).

\section{Adding a Point at the Origin}
\label{sec: added origin}

If none of the rows of the $\vec H$ matrix of (\ref{eqn: svdcenter}) is null, then the given scatter contains no points coincident with the origin, as determined by the current choice of $\vec \gamma$ vector.  On the other hand, there will always be at least one point at the current origin whenever exactly one element of the current $\vec \gamma$ vector is a 1 and all other elements are null.

Any null row of the $\vec H$ matrix of (\ref{eqn: svdcenter}) can be relocated to its first row simply by renumbering the N points in the initial scatter.  The resulting $d^2_0 (\vec X)$ vector of (\ref{eqn: sqdifnorm}) that contains the squared Euclidean distances of all N point from their current origin will then naturally have a zero in its first row.  In fact, this $d^2_0 (\vec X)$ vector will also then be the first column of the $\vec D^{(2)}$ matrix of (\ref{eqn: sqdistmtx}), while its transpose will be the first row of this same matrix.

When a $\vec H$ matrix from (\ref{eqn: svdcenter}) contains no null row, a null row can be added as, say, row N+1. This augmented $\vec H$ matrix is still semi-orthogonal, and $\vec {H'H}$ is still the r $\times$ r identify matrix.  The corresponding augmented $\vec \gamma$ column vector ($(N+1) \times 1$) can be taken to have a zero as its final element, and the augmented $\vec D^{(2)}$ matrix of (\ref{eqn: sqdistmtx}) will have the original $d^2_0 (\vec X)$ vector as the above the diagonal elements in its last column ...and its transpose in the bottom row.

In summary, the $d^2_0 (\vec X)$ vector that contains the squared Euclidean distances of all N original points from their current origin is either already contained within the original N $\times$ N squared distances matrix, $\vec D^{(2)}$ of (\ref{eqn: sqdistmtx}), or else it will appear within the corresponding (N+1) $\times$ (N+1) augmented version of this matrix when a new point coincident with the origin is included.

In other words, the $d^2_0 (\vec X)$ vector is either an explicit or implicit part of every N $\times$ N matrix of squared distances, $\vec D^{(2)}$ of (\ref{eqn: sqdistmtx}).  In sharp contrast with PCA, efforts to minimize $\vec {Norm}^2$ of (\ref{eqn: sqdifnorm}) place special emphasis upon accurate reproduction of $d^2_0 (\vec X)$ values within the recovered Z-configuration.

\section{First Numerical Example: Six Points on a Plane}
\label{sec: example1}

Perhaps the most simple sort of ``interesting'' example for comparing PCA with our origin-centric approach is depicted as H coordinates (p=2 and N=6) in Figure 1.  When reducing this configuration to a single dimension (q=1), a singular affine transformation can produce either ``a pair of triplet points" or else a ``triplet of pairs.''  In fact, these particular configurations happen to correspond to two of the three local minima of $\vec {Norm}^2$ of equation (\ref{eqn: sqdifnorm}); see Figure 2.

\begin{figure}[htb]
\center{\includegraphics[width=\textwidth]{H_6in2.jpg}}
\caption{\label{fig:H6in2} Mahalanobis H Configuration for 6 Points in 2 Dimensions.  Numerical searches for the best q=1 representation can use, as their starting points, orthogonal projections of the 6 points onto the line through the origin making angle $\phi$ with the horizontal axis.  Initial values for the $\vec B$ matrix of (\ref{eqn: rqaffrmd}) are then simply 2-element column vectors of the form $(sin(\phi), cos(\phi))'$, and the objective of the search is to find the best length for $\vec B$ in each given direction.}
\end{figure}

This example is also interesting in the sense that, while the initial H-configuration contains no point(s) at the origin, the optimal q=1 dimensional recovered configuration [case (c) in Figure 2] contains 2 points coincident with the origin.

\begin{figure}[htb]
\center{\includegraphics[width=\textwidth]{z1_6in2.jpg}}
\caption{\label{fig:z6in2} Depiction of three Local Minima of $\vec {Norm}^2$ for Dimension q=1.  While a pair of local minima occur in case (a) at 3.50 and in case (b) at 4.80, the global minimum of 3.11 corresponds to case (c).}
\end{figure}

\section{Second Numerical Example: The Longley Data in q=2 Dimensions}
\label{sec: example2}

Here we use the infamous Longley(1972) dataset (p=6, N=16) to illustrate some key distinctions between the PCA and affine, origin-centric methods of (\ref{eqn: rqaffrmd}) for reduction of multivariate dimensionality. While Longley used seven variables in his ill-conditioned regression model, here we use YEAR = 1947 to 1962 only as labels for points, rather than as an exact ``linear trend" variable. The six US economic variables used here are $X_1 =$ GNP deflator, $X_2 =$ GNP, $X_3 =$ number Unemployed, $X_4 =$ size of Armed Forces, $X_5 =$ total Population and $X_6 =$ total Employed.

\begin{figure}[htb]
\center{\includegraphics[width=\textwidth]{longPCAcorr.jpg}}
\caption{\label{fig:longPCA} The PCA solution of Dimension q=2 for the Longley data when standardized to ``correlation'' form.  It is clear from the labels on the points as well as from the direction cosines for Axes 1, 2, 5 and 6 that this recovered configuration is best interpreted as representing ``linear growth" over time.}
\end{figure}

\begin{figure}[htb]
\center{\includegraphics[width=\textwidth]{longAROD.jpg}}
\caption{\label{fig:longAROD} This is the optimal origin-centric representation in q=2 dimensions for the Longley data when standardized to Mahalanobis H form.  The ``year'' labels on points suggest that time variation is being depicted here as clockwise or counter-clockwise rotations.  In particular, the direction cosines for Axes 3 and 4 suggest that these two measures are the true, primary determinants of year-to-year variation in q=2 dimensions.}
\end{figure}

This example is rather typical of cases I have explored, especially when N greatly exceeds p.  Specifically, the optimal affine reduction in Mahalanobis dimensionality depicted in Figure 4 has a ``Dyson swarm'' interpretation, featuring a characteristic ``hole" surrounding the explicit origin.

\section{Discussion}
\label{sec: Discussion}
In the terminology of Glaeser(2006), ``Researcher Initiatives'' can deliberately bias findings towards the perspective of one particular subset of stakeholders.  For example, it is well know that including several different quantitative X-variables that are surrogate measures of a single construct in a PCA tends to orient the first principal axis to represent that construct.  The corresponding phenomena in an origin-centric analysis would be to include several points that are actually coincident, thereby placing greater emphasis upon recovering the distance of this composite from all other points within the scatter.

Origin-centric methods allow the scatter to be visualized much like a ``Dyson Swarm'' for a finite universe of N or fewer points. For example, a q=3 dimensional recovered scatter might be primarily viewed as a 2-dimensional small-scale conformal mapping where radial distance information is disregarded. Alternatively, the size of each of the N plotting symbols displayed on this sort of ``sky view" map could represent some monotone function of its recovered radius.

Nonnegative Matrix Factorization (NMF) techniques rely upon locating the origin so that all N initial points have p strictly nonnegative coordinates.  In other words, all initial points must lie on or within the boundaries of the ``first'' Cartesian $2^p$-ant of p-dimensional Euclidean space. Similarly, the q dimensions to be recovered using NMF are to be represented as vectors emanating from the origin that have strictly nonnegative direction cosines because they point exclusively into this same first Cartesian $2^p$-ant.

\section{Appendix: Affine Invariance and Commutativity}
\label{sec: affine invarcomm}

Linear algebra is typically used to manipulate data structures, consisting of vectors and matrices of real numbers, in multivariate statistical analyses.  Insights into the analytical geometry of such manipulations is provided by visualizing the data as consisting of a scatter of N points within a p-dimensional Euclidean space.

A fundamental property of Euclidean space is that distance is computed from coordinates relative to orthogonal axes using the Pythagorean theorem.

A N $\times$ p matrix of real numbers, $\vec Y$, is said to represent a nonsingular affine transformation of an initial N $\times$ p matrix of real numbers, $\vec X$, if and only if there exists a 1 $\times$ p ``translation" vector $\vec {a'}$ and a nonsingular p $\times$ p matrix $\vec B$ of real values such that
\begin{equation}
\vec Y = \vec {1a'} + \vec {XB}. \label{eqn: afftran}
\end{equation}
In particular, if any row of $\vec X$ consists of all zeros, the corresponding row of $\vec Y$ will be $\vec {a'}$. The notation of equation (\ref{eqn: afftran}) differs slightly from that of Obenchain(1971) simply because the $\vec X$ and $\vec Y$ matrices have been transposed here.

A statistic $S(Y)$, consisting of a scalar, vector or matrix of real numbers, will be said to be affine invariant if and only if its numerical value when computed after any nonsingular affine transformation has been applied to a dataset is identical to its value before such transformation:
\begin{equation}
S(\vec Y) = S(\vec X) \mbox{  ...for every translation $\vec a$ and nonsingular $\vec B$} \label{eqn: affinvar}
\end{equation}

The $\vec H$ matrix of (\ref{eqn: svdcenter}) is not uniquely determined. For example, any of the r columns of $\vec H$ (each N $\times$ 1) can always be multiplied by $-1$ if the corresponding row of $\vec {G'}$ (1 $\times$ p) is also multiplied by $-1$.  On the other hand, $\vec {HH'}$ is an idempotent and symmetric (N $\times$ N) matrix that is uniquely determined and corresponds to the orthogonal projection matrix (linear operator) that characterizes the column space of the $( \vec I - \vec {1\gamma'} ) \vec X$ matrix; see Rao(1973), pages 46-48.  In fact, Obenchain(1971) showed that $\vec {HH'}$ and $\vec D^{(2)}$ are equivalent ``maximal affine invariant'' statistics which can be viewed as a standardized, canonical form for a given (N $\times$ p) data matrix of ``centered'' scalar values subject to nonsingular linear $\vec B$ transformations, (\ref{eqn: afftran}).

The $\vec {HH'}$ and $\vec D^{(2)}$ matrices are unique but distinct, and each can be computed from the other.  PCA focuses upon reproducing $( \vec I - \vec {1\gamma'} ) \vec {XX'} ( \vec I - \vec {\gamma 1'} )$ by decomposing it into ordered and additive components. PCA fails to reduce dimensionality when applied to $\vec {HH'}$ because all of its components are equally good or bad and, thus, cannot be uniquely ordered.

By focusing upon reproducing $\vec D^{(2)}$ instead of $\vec {HH'}$, origin-centric methods can reduce the dimensionality of configurations that PCA cannot.  There appear to be at least two separate parts to the price-one-has-to-pay to achieve affine invariance properties that ignore vagaries caused by analyst choice of variable scalings and/or confounding between variables.  First of all, recovered configurations will not be additive; the optimal solution of dimension (q+1) usually does not exactly ``contain'' the optimal solution of dimension q.  Secondly, minimizing $\vec {Norm}^2$ of equation (\ref{eqn: sqdifnorm}) is a problem of constrained optimization subject to multiple local minima.  Fortunately, modern numerical search software, such as the ``optimx'' R-package of Nash and Varadhan(2011), is now available to researchers.

\section{Appendix: Detail on Data Centering and Affine Median Vectors}
\label{sec: center}

A N $\times$ p matrix of real numbers, $\vec Y$, will be said to be a ``centered'' version of a N $\times$ p matrix of real numbers, $\vec X$, if and only if
\begin{equation}
\vec Y = ( \vec I - \vec 1\gamma' ) \vec X, \label{eqn: center}
\end{equation}
where the 1 $\times$ N row vector $\vec \gamma'$ is any generalized inverse of the N $\times$ 1 column vector of all ones, $\vec 1$.  In other words, any 1 $\times$ N row vector of  real values that sum to 1, $\vec \gamma'1 = 1$, is a valid choice for $\vec \gamma'$ in (\ref{eqn: center}).

The ``centering'' operation of equation (\ref{eqn: center}) frequently yields $\vec Y$ coordinates that differ from those of the initial $\vec X$ matrix.  In these cases, the location of the implicit origin, $\vec 0$, has been shifted via a simple ``translation'' of $\vec Y$ coordinates.

The unique Moore-Penrose inverse of $\vec 1$, usually denoted by $\vec 1^+$, is the row vector with each of its N entries equal to 1/N.  This choice for $\vec \gamma'$ corresponds to centering the $\vec X$ matrix at its overall mean vector, i.e. at its traditional centroid.

When $\vec \gamma_1$ and $\vec \gamma_2$ are two possibly different choices for the $\gamma$ vector of (\ref{eqn: center}), the following equality is easily verified by direct multiplication and algebraic simplification:
\begin{equation}
( \vec I - \vec 1\gamma'_2 )( \vec I - \vec 1\gamma'_1 ) = \vec I - \vec 1\gamma'_2 .
\label{eqn: idempot}
\end{equation}
When $\vec \gamma_1 = \vec \gamma_2$, equation (\ref{eqn: idempot}) shows that all valid choices for the $\gamma$ vector of (\ref{eqn: center}) yield a centering matrix, $\vec I - \vec 1\gamma'$ , that is idempotent and, thus, corresponds to a geometrical projection in N space that generally is oblique.  In fact, the choice $\vec \gamma'_1 = \vec \gamma'_2 = \vec 1^+$ is the only choice that makes the $\vec I - \vec {11^+}$ matrix symmetric as well as idempotent, thus corresponding to a strictly orthogonal projection in N space.

Perhaps, an even more interesting result follows from equation (\ref{eqn: idempot}) when $\vec \gamma_1$ and $\vec \gamma_2$ are distinct generalized inverses.  In these cases, note that any valid, final choice of centering, as in equation (\ref{eqn: center}), will simply ``wipe-out'' and ``replace'' any and all previously applied choices of centering of form (\ref{eqn: center}).  The corresponding implication for (\ref{eqn: svdcenter}) is that changing one's choice of $\vec \gamma$ vector typically changes the $\vec H$ matrix so that it becomes orthogonal to the new $\vec \gamma$.

To center an $\vec X$ matrix at the point corresponding to its i-th row, the i-th element of $\gamma$ would then be a 1 and all other elements would be null.

A multivariate median vector can be defined, as in Obenchain(1972a), using ``convex hull'' concepts to assure that the $\vec \gamma$ vector is an invariant function under all strictly nonsingular affine transformations of a given $\vec X$ scatter, i.e. where no systematic reductions in dimensionality are being enforced.  In direct analogy with the concept of a univariate median, successive ``exterior'' convex hulls (which are preserved under nonsingular affine transformations) of the scatter initially spanning p-dimensional Euclidean space would be determined and successively ``peeled away.''  Whenever two or more points on an exterior hull are coincident, only one such point is set aside (assigned a $\gamma$ weight of zero) at each stage of such peeling.  The final convex hull remaining at the end of this sequence will contain neither any coincident points nor any strictly interior points.  The affine invariant $\vec \gamma$ vector, which defines the corresponding affine commutative median vector, $\vec {\gamma'X} = \vec {-a'}$ of equation (\ref{eqn: afftran}), gives equal, positive weight to each of the unique points on the innermost hull and zero weight to all other points.

\section{Appendix: Restrictions to Distinct Points, possibly Weighted.}
\label{sec: coincident}

In this penultimate appendix, we comment on the role of mutually coincident points in determining the dimensionality of a given scatter.

Since coincident points clearly cannot increase dimensionality and always remain coincident in the affine reduction formulation of (\ref{eqn: rqaffrmd}), it makes good sense computationally to limit attention to scatters of only distinct, non-coincident points.  Since no reasonable method will fail to exactly reproduce the zero diagonal elements of the $\vec D^{(2)}$ matrix of (\ref{eqn: sqdistmtx}), minimizing the squared norm of equation (\ref{eqn: sqdifnorm}) rightly focuses on minimizing the sum of the $N(N-1)/2$ terms with $1 \leq i < j \leq N$, where N now denotes the total number of distinct points.  On the other hand, individual terms in this summation could certainly be differently ``weighted'' to account for the presence or absence of additional points coincident with either the i-th or the j-th distinct point.

Limiting attention to unique points and affine reduction methods also assures, when many variables are analyzed for a fixed number of points, that the maximum possible value of r will indeed equal $(N-1) \leq p$ in equation (\ref{eqn: svdcenter}).  For an arbitrary configuration of N unique points, one then knows in advance that $\vec {H'H}$ will ultimately become the $(N-1) \times (N-1)$ identity matrix as the number of columns of $\vec X$ is arbitrarily increased.  Furthermore, in this same limit, the corresponding $\vec {HH'}$ matrix will also approach the $\vec {I - \gamma\gamma^+}$ matrix, which is the N $\times$ N orthogonal projection matrix for the vector space orthogonal to $\gamma$.

\section{Appendix: Affine Invariance and the Restriction to p less than or equal to $N-1$.}
\label{sec: plarge}

When $\vec {\gamma'} = \vec {1^+}$, the large-p asymptotic H-configuration is that of N points equally spaced in $N-1$ dimensions, which corresponds to a sparse, high-dimensional Dyson sphere with no point coincident with the origin.  Specifically, the squared distance between any 2 of these N points will be 2, and the squared distance from each to the origin will be $d^2_0$ = $(N-1)/N$.

When all of the elements of $\vec \gamma$ except one are zeros, the asymptotic configuration will depict that single point as being coincident with the origin while the remaining $N-1$ points are again equally spaced.  While the squared distance between any two of these $N-1$ points will still be 2, the squared distance from each of them to the origin point will now be $d^2_0$ = 1. The first $N-1$ rows of the asymptotic $\vec H$ matrix may then be any $(N-1) \times (N-1)$ orthogonal matrix (such as some permutation of the rows and columns of the $\vec I$ matrix), while the final row is null.  This configuration again spans $N-1$ dimensions and depicts another sparse, high-dimensional Dyson sphere as well as its origin point.

\section{Acknowledgments}
\label{sec: acknow}

The author received extensive feedback from reviewers of his 1972 submission on this topic to {\em Biometrika} as well as his 1973 submission to {\em Journal of Multivariate Analysis}.  The current manuscript is brief and completely revised in organization and presention style.  The basic concepts outlined here are unquestionably sound, and our objective of incorporating affine invariance and commutivity properties into algorithms for reduction of multivariable dimensionality is not ``impossible.''  Thanks to modern statistical computing systems for graphical visualization and numerical search, our innovative origin-centric approach can be of much more ``practical interest'' today than it was 40+ years ago.

\section{References}
\label{sec: refs}

Dyson, FJ. Search for Artificial Stellar Sources of Infra-Red Radiation. {\em Science}, 1960, 131, 1667-1668.

Gabriel, KR. The Biplot Graphic Display of Matrices with Application to Principal
Component Analysis. {\em Biometrika}, 1971, 58(3), 453-467.

Gabriel, KR. Analysis of Meteorological Data by Means of Canonical Decomposition
and Biplots. {\em Journal of Applied Meteorology}, 1972, 11, 1071-1077.

Glaeser, EL. Researcher incentives and empirical methods. {\em Harvard Institute of Economic Research, discussion paper \#2122}. http://scholar.harvard.edu/glaeser 2006.

Gower, JC. Some distance properties of latent root and vector methods used in multivariate analysis. {\em Biometrika}, 1966, 53, 325-338.

Gower, JC. Adding a point to vector diagrams in multivariate analysis. {\em Biometrika}, 1968, 55, 582-585.

Gower, JC. Euclidean distance geometry, {\em Math. Sci.}, 1982, 1, 1-14.

Gower, JC. Properties of Euclidean and Non-Euclidean Distance Matrices {\em Linear Algebra and its Applications}, 1985, 67, 81-97.

Lee, DD and Seung, HS. Learning the parts of objects by non-negative matrix factorization. {\em Nature}, 1999, 401(6755), 788-791.

Longley, JW.  An appraisal of least squares programs for the electronic computer from the point of view of the user.  {\em Journal of the American Statistical Association}, 1967,  62, 819-841.

Nash, JC and Varadhan, R. Unifying Optimization Algorithms to Aid Software
System Users: optimx for R.  {\em Journal of Statistical Software}, 2011, 43(9), 1-14.

Mahalanobis, PC. On the generalized distance in statistics. {\em Proc. Nat. Inst. Sci. India, Part A.} 1935, 12, 49-55.

Obenchain, RL. Multivariate procedures invariant under linear transformations. {\em Annals of Mathematical Statistics}, 1971, 42, 1569-1578.

Obenchain, RL. Correction Note: Multivariate procedures invariant under linear transformations. {\em Annals of Mathematical Statistics}, 1972a, 43, 1742-1743.

Obenchain, RL. Regression optimality of principal components.  {\em Annals of Mathematical Statistics}, 1972b, 43, 1317-1319.

Obenchain, RL. Affine Multidimensional Scaling. Presentation at {\em Third International Symposium on Multivariate Analysis}, Dayton, OH, 1972c.

Obenchain, RL. The geometry of linear reparameterizations in regression. {\em Contributions to Statistics: Essays in Honor of Norman L. Johnson}, ed. P. K. Sen, Amsterdam: North-Holland 1983, 347-365.

Obenchain, RL. ICE Preference Maps: Nonlinear Generalizations of Net Benefit and Acceptability.  {\em Health Services and Outcomes Research Methodology}, SpringerLink (open access), 2008, 8, 31-56. (two-dimensional, origin-centric maps of economic preferences.)

Okamoto, M. Optimality of principal components. {\em Multivariate Analysis, 2} P. R. Krishnaiah, ed. New York: Academic Press, 1969, 673-685.

Rao, CR.  Linear Statistical Inference and Its Applications, 2nd Edition.  New York: John Wiley and Sons, 1973.

Schoenberg, IJ. Remarks to Maurice Freehet's article ``Sur la definition axiomatique d'une classe d'espaces vectoriels distancies applicables vectoriellement sur l'espace de Hilbert'', {\em Ann. Math.} 1935, 36, 724-732.

Torgerson, WS. {\em Theory and methods of scaling.} New York: Wiley, 1958.

Young, G. and Householder, AS. Discussion of a set of points in terms of their
mutual distances. {\em Psychometrika}, 1938, 3, 19-22.

\end{document}