IEEE Consumer Electronics Magazine - January/February 2024 - 67

TABLE 2. Types of deepfakes on human visual input(s).
Deepfake
Type
Description
Deepfakes that manipulate pose,
gaze, mouth, body, and
Reenactment
expressions in It such that they
are driven by Is. Mouth
enactments can also lead to
realistic voice-dubbing.
Potential Attack Model
Examples
Deepfake manipulations of CCTV
footage can be used as false
Attacker impersonating an
identity and controlling
speech and actions in a
visual.
pieces of evidence in a trial, and
can also enable an attacker to
impersonate t and gain the trust
of friends, family members, or
service providers to facilitate
access to money, assets, or any
other service.
Replacement
Deepfakes in which the content
of It is replaced with Is) keeping
the identity of Is as it is.
An attacker can swap a
victim's face on a body of
a look-alike to facilitate
malicious intentions.
Revenge porn, where the face of
the victim is superimposed on
the body of a look-alike actor/
actress for defamation. They can
also potentially be used for
spreading false political opinions
of politicians to damage their
reputation or election campaigns
Editing
Deepfakes in which attributes of
It such as facial hair, clothes,
ethnicity, age, and weight, etc.,
are manipulated.
Synthesis
Deepfakes in which Of is
created without any target It.
They can help create
characteristics for games or
movies.
The attacker (s) can
potentially use this
technique to create a
misleading, false persona.
An attacker can create
false online personas for
various forms of
deception and predation.
A child predator can change his/
her online profile by editing age
and gender. This technique also
allows attackers to remove the
victim's clothes for defamation
or entertainment purposes.
Deepfakes can lead to the
creation of completely synthetic
identities.
identity-proofing process. This is true even if a
combination of operator and automated systems
is deployed to detect and prevent such manipulations,
as shown in Figure 1.
Categories of deepfakes identified in the literature
are summarized in Table 2. Note that, Is
and It in Table 2 represent visuals of source (s)
and target (t), respectively. The resultant deepfake
produced is represented by Of.
The Ease of Generating Deepfakes
The generic process involved in creating deepfakes
is illustrated in Figure 3,with most deepfakes
used for replacement and reenactment
employing a slightly modified version of this process.
In these cases, the input image I undergoes
i) face detection and cropping, ii) intermediate
representation extraction, iii) generation of a new
face based on another face, and iv) blending the
generated face onto the target image.
January/February 2024
Of the numerous deepfakes listed in Table 2,
reenactment and replacement deepfakes are the
most prevalent. They can facilitate identity impersonation,
thereby posing a significant threat to
remote identity proofing in cases, where the
device used to stream the video is not under the
service provider's control (i.e., IAL2). Tools that
allow for the creation of deepfakes are widely
available online,3,7,8 making it increasingly likely
to encounter them. In the following section, we
examine current state-of-the-art methods for
detecting and identifying such imagery.
Retrospectively, numerous mechanisms for
reenactment and replacement deepfakes have
been proposed, and those relevant to RiDP are
summarized in Nanda et al. (Appendix 2a, 2b).3
Face replacement, also known as face-swapping,
was historically achieved manually through editing
software like Photoshop. However, advancements
in neural networks have enabled fully
automated andmore realistic face-swapping.
67
IEEE Consumer Electronics Magazine - January/February 2024

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2024
