IEEE Technology and Society Magazine - September 2019 - 64

outer boundary for meaningful analysis of mature systems. The data could have shown increasing numbers
of RXD events as one got closer to the edge of the
20-year time horizon, but instead the data all showed a
strong decline.
A thought experiment furthers demonstrates how the
TRACES report could have been accurate in the 1960s,
but not accurate today if new basic science has not continued to play a foundational role in engineering.
Assume that besides the time horizon, the broader
methodological differences between TRACES and Hindsight did not contribute to their conclusions. Then suppose that, as TRACES proposes, Hindsight had used a
longer time horizon, like 50 years. This would capture

TRACES was the U.S. National Science
Foundation response to the Hindsight
report, commissioned to trace the
sources of innovation in science research.
much of TRACES' period of study and would likely, for
several of the weapon systems in Hindsight, lead to the
conclusion that basic science plays a significant role in
engineering developments. Indeed, a 50-year time horizon beginning in 1967 would take into account some of
the earliest advances in engines and aircraft design.
The next question is whether the trend of basic science
contributions has continued since the 1960s. In many
ways, modern science and engineering began with WWII
and the dawn of major government investment in R&D
[2]. Given that and the ensuing proliferation of engineering developments, we posit that a 2014 study (50 years
hence from the beginning of Hindsight) would replicate
the results that Project Hindsight found with a 20-year
time horizon.
This is to say, for engineering-intensive systems,
basic science plays only a small role after the initial system is developed.9 The above-noted decline in RXD

9
An astute reviewer asked whether we ourselves were embracing a form
of the linear model of innovation, which we discussed in footnote 2. Of
the two reports, it seems likely that TRACES seems motivated by a linear
model of innovation, in that its cases seem almost chosen to represent
cases where science linearly led to innovation. Hindsight does push for
the primacy of engineering in driving key learning events, but there does
not seem to be a direct linear model of innovation present. To be clear, we
do not endorse the linear model of innovation, but support more detailed
study of the mechanisms by which innovation occurs and by which engineers can better pursue innovation. Getting such data (such as might
confirm the hypothesis noted above) can help engineers make greater
progress at achieving public values.

events makes such a relative decline seem plausible if
not likely. TR ACES' conclusion that basic science
research contributes to technological developments, if
to be found in ensuing studies, would then require longer and longer time horizons.
Scope differences on civilian versus military
technology: TRACES' [10], [11] arguments have a different scope than did Hindsight as it examined many
civilian technologies. DOD was and is engaged in many
areas of development that build on highly advanced
technology, such as missiles and aircraft. The TRACES
study, performed in 1967, examined a number of technologies that were relatively recent, including birth
control pills and video recorders. It is very likely that
video recorders, being a wholly new technology, would
have a nearer origin to basic science. If this is the
case, then continued development in already established military technologies may be most influenced
by "engineering" and "technology" RXD events. The
findings of TRACES then cannot be said to invalidate
Hindsight because they study different subjects.

Assumptions of Both HINDSIGHT and TRACEs
In many ways, Hindsight and TRACES represent wholly
different conceptions of technological innovation. Hindsight manifests the historic engineering tradition concerned with fielding new systems in an efficient manner.
This tradition has long been at odds with a Vannevar
Bush-like view of science as the fount of all knowledge,
a tradition manifest in TRACES [13]-[15]. Given the different methodologies, we should not take the reports to
necessarily disagree, especially because they do not
necessarily have conflicting definitions of research, as
shown in Table 5. Instead, Hindsight and TRACES offer
complementary findings on innovation.

Other Criticisms of the Hindsight Methodology
Despite their uniqueness, neither the final Hindsight nor
TRACES report seem to have been seriously assessed in
the science policy literature. At the time of the writing of
this paper, the final 1969 Hindsight report had been
cited only 45 times on Google Scholar. Almost all references to the report and its earlier versions are simply
references to the existence of Project Hindsight and do
not engage with the data presented in it. Even the substantive criticisms mentioned below do not reference
the contributions of temporal data such as those in Figure 1. This could be due to the timing of when the DOD
declassified the Hindsight final report. It is unclear the
extent to which critics had access to lower level Hindsight data. Meanwhile, TRACES has only been cited a
few times as well.
A philosopher, Kreilkamp, who worked at the National
Science Foundation wrote a scathing critique [16],

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

SEPTEMBER 2019

IEEE Technology and Society Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2019