IEEE Computational Intelligence Magazine - February 2020 - 41

A hidden Markov chain model can effectively recover the
history of the entire decision-making process based on the service appearances [38]. We only need to know the status S of
network nodes, as well as the corresponding action A. This policy r, on the other hand, can use the mentioned IPP model as
an initial value. The optimal policy is calculated by setting up
an evaluation parameter Q (including the transmission overhead, the switching overhead, the reward for demand satisfaction, etc.). Afterwards, the learning formula
Q(s, a) = Q(s, a) + a(r (s, a) + c max alQ (sl , al ) - Q(s, a))

(5)

can be used to influence the power control policy. Ultimately, a
policy r corresponding to the different values of the state
matrix S will be used as the solution for the power control system to maximize energy savings.
When the cloud collection of control data is large enough,
we can even use deep reinforcement learning to train in the
cloud; in the memory replay approach, a random selection of
some historical control data is added to the recent reinforcement learning. The data are mixed into the input sample (as a
mini-batch), thus balancing the impact of recent events on the
overall network; the network clone technique involves setting
up a parallel space for the predictive network Ql. Compared to
learning after each round of data updating, learning and testing
are completed only after the collection of M rounds of data,
thus maximizing stability.
Using the trend prediction, the BSs can adjust the sleeping
strategy over time, even if facing sudden changes. In general, the
optimization of throughput or power only concerns the size of
each message, and the value of content has not been thoroughly
investigated. Because of the diversity of IoT nodes, the existence
of various types of messages makes the optimization task more
difficult. However, if we could effectively use the inner
connections between these messages' content, our network
management would be even more accurate.

(a)

V. Experiment and Comparisons
A. Implementation

First, we perform the field test of our LoRa and ZigBee ad hoc
network. Installing a LoRa gateway on top of a building allows
all the nodes embedding LoRa modules to access the core network through that gateway. As shown in Fig. 7, the installation
includes the integration of Raspberry Pi with a gateway module, connection with a database and registration with an online
service. For an individual IoT node, the collected data can be
transmitted to a remote antenna through either the LoRa
module or the ZigBee module. At present, the installation has
been completed, and the resulting network successfully delivers
packets within ten kilometers.
B. Network Flow Control

Traffic flow control of the network is divided into two different
dimensions. In practice, the power control of communication
modules in the network is mainly divided into two steps: the
establishment of an active node connection diagram and the
network traffic flow control. When a new time frame starts,
each center node and the gateway are aware of the switch conditions for each edge node. In other words, the future control
scheme of the previous power control system is also stored at
these intermediate nodes that can form a network similar to
that shown in Fig. 8(a). After establishing the temporary active
node connection diagram, we can select a traffic control policy
for data flows. According to the existing literature, the use of
relay priorities that maximize hop number first and destination
closeness first can optimize the overall network response delay
and the total throughput, respectively [39].
Next, we continue and finish the optimization of the network settings. To understand the range setting clearly, we use
the deduction process mentioned before, and the calculated
results are shown in Fig. 8(b). The size of a star topology network varies from 100 to 400, and the maximum throughput

(b)

FIGURE 7 Deployment of LoRa and ZigBee transmission module. (a) Integration of Raspberry Pi with a gateway module. (b) User-side transmission module connecting with an IoT device.

FEBRUARY 2020 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - February 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2020