Signal Processing - May 2017 - 33

Number of Samples (Future)

from those in typical SP applications. Related SP problems and
have always been skeptical about models and statistical results
solutions for such big data and modeling applications need to
summarized from historical data, especially so after the 2008
be carefully formulated.
financial crisis. Some people have even gone so far as to say
that the past cannot inform about the future, and therefore
models and statistics coming from historical data are not useful
Other related works in SP
or trustworthy.
We have mainly focused on the literature in economics and
In the following, we show some interesting empiribusiness research and provided the SP understanding on the
cal results as a small step forward to answer this question.
literature. Note that there have been three special issues on SP
The stock daily returns of Russell 3000
for finance [1]-[3]: one in IEEE Signal
component stocks from January 1995 to
Processing Magazine in 2011, and the
All theories and models
September 2014 are examined. The Rusother two in IEEE Journal of Selected
are based on historical
sell 3000 is used because those stocks are
Topics in Signal Processing in 2012 and
data with the assumption
relatively liquid and the price data are a
2016. Readers can find more SP examples
that the past can
good reflection of real market transacthere, especially on portfolio and risk analrepresent the future.
tions. Note that the component stocks of
ysis, HFT, and algorithmic trading. When
the Russell 3000 are changing over time.
going through SP technologies, readers can
The test period is set to about four years from January 2011
focus more on the economic problem formulation and evaluato September 2014. Starting from the first trading day of
tions of solutions with the concepts discussed in this tutorial.
2011, the return distribution of the historical data of the past
Also, an overview of business analytics related to SP can be
16 years is used to predict the return distribution of the next
found in [65] and [66], which provide perspectives on system
day. We want to examine overall how accurate it is to use
modeling of a business.
the historical stock return distribution as a representation
of the future distribution. The following procedure is used
Conclusions and thoughts
to examine the differences between historical distributions
In this tutorial, we present an introduction to some fundamenand future distributions.
tal economic theories that govern financial markets and peoAt time t, we use all historical daily returns of the preceding 16
ple's economic decisions, including expected utility, RP,
years to create a distribution f. We then define K equal probabilportfolio theory and asset-pricing models, EMH, prospect theity quantile bins Q k, k = 1, f, K with bin (stock return) boundory, and behavioral economics. We also introduce basic
econometric tools and theories from an SP perspective.
aries q 1, f q K -1 . Apparently, P (R x ! Q k) = ^1/K h, 6x # t. If
We emphasize that when analyzing data and building ecothe future distribution is the same as the past distribution, we
nomic models, researchers should keep in mind existing ecocan expect that at time t, the returns of the 3000 stocks will
nomic theories and hypotheses. The burden of proof is high
fall into each bin uniformly, i.e., P (R t ! Q k) = ^1/K h . The
when findings contradict or are inconsistent with existing
chi-square test can be conducted to test this hypothesis. We
theories. For example, any price anomaly found in financial
aggregate all bin counts and plot two scenarios in Figure 13
markets needs to be carefully examined because it may conwith 50 bins, i.e., K = 50. Figure 13(a) shows the performance
tradict existing models and the EMH and could be a chance
of the past unconditional distribution. Figure 13(b) shows the
result. It is always difficult to predict prices because these
performance of the past distribution conditional on the precedare usually determined by market equilibrium no matter how
ing ten-day patterns.
The bar charts should be flat if the past distribution and
the future distribution are the same. Given the large number
of samples, by chi-square tests, we can statistically reject
Conditional
Unconditional
the null hypothesis that the future distribution is the same
× 104
× 104
6
7
as the past distribution for the testing periods. Note that
6
5
both distributions underestimate tail risks. The statistical
5
test results justify people's concern that the past data do
4
4
not represent the future. However, as Figure 13 shows the
3
3
past distribution does contain some information about the
2
2
future, e.g., the mean values. In addition, the past distri1
1
butions conditional on the preceding ten-day price pattern
0
0
contain more information about the future than the uncon10 20 30 40 50
10 20 30 40 50
ditional past distributions.
Equal Probability
Equal Probability
These findings from the data are encouraging for SP
Bins (Past)
Bins (Past)
because they indicate that sophisticated structures contain(a)
(b)
ing information in data need to be identified and understood.
Meanwhile, these findings also pose challenges because
FIGURE 13. The bar charts should be flat if the past distribution and the
hypotheses and systems in these economic data are different
future distribution are the same.
IEEE Signal Processing Magazine

May 2017

Table of Contents for the Digital Edition of Signal Processing - May 2017