Signal Processing - January 2016 - 52

User Behavior Modeling [45]-[50]

Learning-Based Equilibrium
Finding [51]-[53], [56], and [57]

Game-Theoretic Learning
Algorithms [54], [55], [58],
[59], and [66]-[68]
Legacy Multiarm Bandit [74]-[76]
Strategic Multiarm Bandit for
Website Ad Auction [77]-[79]
Active Learning [40]-[44]
Incentive-Compatible
Crowdsourcing [69]-[73]

Machine Learning

Strategic
Decision Making

[FIG18] a summary of related works.

was playing the strategy of avoiding difficult tasks according to
our exit survey. When a much lower sampling probability of 0.3
is used, it becomes profitable to increase the number of submissions by submitting lower-quality solutions, as most errors will
simply not be detected. This explains why the majority of participants had very low accuracies, as shown in Figure 17(b). Notably, a few workers, five of 41, still exhibited very high accuracies
in this case. Our exit survey suggests that their behaviors are
influenced by a sense of work ethics, which prevents them from
playing strategically to exploit the mechanism vulnerability.
With the incentive mechanism M t, as the introduction of training tasks makes it more costly to submit wrong solutions, participants need to reevaluate their strategies to achieve a good
tradeoff between accuracy and the number of submitted tasks.
From Figure 17(c), we can see that the accuracy of the participants with the incentive mechanism M t has a very similar distribution to that of the group using the reward accuracy
mechanism M a with the highest sampling probability. Therefore, through the use of quality-aware worker training, the
incentive mechanism M t can greatly improve the effectiveness
of the basic reward accuracy mechanism M a with a low sampling probability to a level that is comparable to the one that
has the highest sampling probability.
reLateD works
Although not referred to specifically as decision learning, there
has been a growing body of literature in recent years on the
intersection of learning and strategic decision making, as

summarized in Figure 18. One class of related works is learning
to understand how human beings make strategic decisions from
real data. For example, classical machine-learning techniques
are used in [46] to predict how people make and respond to
offers during negotiations and how they reveal information and
their response to potential revelation actions by others. Their
results showed that the strategies derived from machine-learning algorithms, even when not optimal, can beat real human
beings [46]. The year-long study of empirical data shows that an
experienced human being in a repeated game will be more
cooperative but turn the tables more definitely when he or she
is betrayed by the opponent [47]. Additionally, the study in [48]
shows that human beings have very limited memory space and
computation capability, which limits the optimality of their
decisions. It has also been shown in [49] that a dynamic belief
model, by ignoring the older signals in constructing the belief,
works best in predicting human decisions. Through empirically analyzing the purchase history on Taobao, a large-scale
online shopping social network, Guo et al. revealed that a real
human values purchase experiences shared by his or her
friend and would be willing to pay a higher price for trustworthy vendors [50]. Nevertheless, in such a complicated system,
it is still difficult to predict the purchase decisions with more
than 50% accuracy using traditional machine-learning algorithms [50]. In [51], how users make decisions on social computing systems is learned from real data and used to guide the
design of mechanisms for the systems. The aforementioned
works establish a solid foundation of decision learning. These

IEEE SIGNAL PROCESSING MAGAZINE [52] jANuARy 2016
Table of Contents for the Digital Edition of Signal Processing - January 2016
