LSR: Energy-Efficient Multi-Modulation Communication for Inhomogeneous Wireless IoT Networks

An important change occurs when one or multiple nodes are switched on (or reset) and start participating in the protocol. The goal of this first phase of the protocol, the Bootstrapping phase, is to let nodes synchronize their local time with the time of the host, register a data stream, and apply optimizations (discussed in Section 4). An example of the bootstrapping process is depicted in Figure 6.

We assume that a node starts in the beginning without any knowledge about the network except for the set of used modulations. As a first step, each node listens with modulation Mod0 for schedule floods initiated by the host and potentially forwarded by other nodes. Since the nodes start with Mod0, all nodes which are able to communicate with at least one modulation in the set of used modulations will be able to receive and relay a schedule packet. Once a schedule packet is received, the node can synchronize its time. The node will then start to listen for the second schedule floods sent with modulation Mod1. Because the modulations are ordered according to decreasing link budget, already in the first round, each node can determine a first estimate of the best modulation, based on the received/not received schedule floods. Nodes that successfully received at least the first schedule flood and which have a traffic demand now use the contention slot corresponding to the best modulation to request a data stream. To support floods initiated by other nodes, the nodes keep participating in floods for all modulations with smaller or equal modulation indices as the active modulation. A random exponential backoff mechanism is applied if a stream is not confirmed by the host in the following schedule message. Once a stream request is confirmed and the node is registered, the node starts sending data messages in the assigned data slots and starts the optimizations (discussed in Section 4). The example in Figure 6 shows the process of registering nodes; however, no optimizations are applied. An example which applies optimizations is discussed in the evaluation in Section 5.2.3.

In the following, we describe the most important changes and the corresponding reactions according to the LSR protocol.

Nodes continuously update the best modulation based on received schedule messages. In case the best modulation is determined to be different from the active modulation, the node sends a stream request to the host to request a change of the modulation. If a switch of modulation is possible according to the host, the host acknowledges the updated stream, and the active modulation is updated on the host and the node. If the modulation is changed to a modulation with a smaller link budget, switching modulations is seamless and does not cause data packets to be lost, assuming ideal RF conditions. In the other direction, the need for switching the modulation is based on missed schedule floods and therefore reduces the data transmission throughput but no energy is wasted as no data floods are initiated.

In case a node is disconnected (i.e., it has no assigned data slot but data to send), it needs to register a data stream at the host again. Thanks to the flooding approach, only one round is required for a node to register a stream, when assuming no packet loss, which is a realistic assumption if no other node is trying to reconnect simultaneously. If a node is freshly connected, also the corresponding optimization state (see Section 4) is reset. Consequently, reconnecting a node is fast, but the optimizations related to the reconnected node take a few rounds to converge.

On the host, there is a ring buffer of size \(B_h\) for every node to store the \(B_h\) most recent hop distance measurements of floods without delayed retransmissions. When the host needs the hop distance to a node for a decision, it calculates an estimate of the hop distance based on the measurements stored in the ring buffer. Whenever there is such a ring buffer that is not completely filled with measurements, such as in the beginning or when a node just joined the LSR network, the host instructs all nodes to not apply any delay to the retransmissions to measure the unaffected hop distance. If the ring buffer of every registered node is completely filled and fresh connectivity information is required (graph contains unconnected nodes or topology change occurred, see Section 3.4.2), the host initiates a probing sequence with delayed retransmissions as described in Section 4.4 and measures the potentially affected hop distances.

To evaluate the performance, we use the following metrics:

We do not include a metric for latency for transferring data from a node to the host in our evaluation because by design this property does not significantly differ for the considered round-based protocols. The upper bound of the latency is determined by the round period (when assuming successful flood receptions). The aspect of unsuccessful data transfers is covered in the evaluation by the reliability metric.

As discussed in Section 1, straightforward approaches such as star-based protocols or protocols that only use short-range transmissions do not satisfy the requirement of covering all considered scenarios. As this difference in coverage makes a fair comparison impossible, we do not include such protocols in the evaluation. Instead, we compare the following two protocol implementations, which have the potential to cover all scenarios:

We implement the mentioned protocols in a simulation as well as on real hardware:

The basic configuration values which are used for the simulation are listed in Table 2. The transmit power has been set to the same level as in the evaluation on the testbed (see Section 5.3). For the simulation, it has no influence on the connectivity, as the used path loss matrix is constructed by taking into account the available link budget. However, it has an influence on the absolute power consumption values (i.e., the energy per round metric). For measurements regarding packet loss, hop distance without delay, and hop distance with delay, a history depth of three measurements (i.e., \(B_m=B_h=B_d=3\)) is used, unless otherwise noted. The number of hops setting for the Gloria flooding primitive limits the maximum supported network diameter. The number of available nodes in the network is used as an upper bound for the number of skipped rounds due to the random exponential backoff mechanism.

For simplicity, we only use the two modulations also used in the description of the LSR protocol in Section 3.1 for the evaluation. For a quantitative comparison of different modulations, we provide the most important characteristics of the modulations used in the evaluation plus one additional modulation in Table 3. For the calculations, we used the configuration used in the simulations (Table 2). The range values are based on the Friis free-space loss equation [17] and are merely indicative, as the achievable range in a realistic environment depends on many additional factors (e.g., see [20, 31]).

First, we verify that the protocols work reliably in all scenarios and compare the energy consumption. For this, we use the simulation to execute the LWB and the LSR protocols in all scenarios. For the LSR protocol, we use two modulations and full thinning. For each combination of protocol and scenario, the simulation is executed for 500 rounds, whereas the first 30 rounds are omitted in the presented data to remove the influence of the Bootstrapping phase. Visualizations of the collected reliability and energy per round metrics are depicted in Figures 11(a) and 11(b), respectively. In the case of reliability, one sample corresponds to the reliability of a single node. The data of all nodes form one distribution (i.e., one box in the boxplot). For the energy per round, one sample corresponds to the consumed energy of a single node in a single round. The data of all nodes for all rounds form one distribution (i.e., one box in the boxplot).

The results show that the LWB and the LSR protocols provide sufficiently high reliability (i.e., they work for all scenarios). The reliability of LSR is comparable to the reliability of LWB. There are cases in which the reliability of LSR is lower. This can be explained by the fact that LSR applies thinning optimizations which reduce the redundancy. However, in other cases, the reliability of LWB is lower. Such cases can be explained by the reduced potential of colliding packets within a flood as the number of simultaneously transmitting nodes is reduced by the thinning as well. The energy consumption of LSR is significantly reduced compared to LWB. There are two reasons for this. The main reason is the thinning optimizations which disable a subset of nodes during data floods. The second contributing factor is the use of two modulations that allows certain nodes to transmit data efficiently via short-range communication. Together with energy load balancing, this allows for reducing the maximum energy consumption of all nodes. More detailed results are provided in Section 5.2.3.

The reduction of energy per round is most significant for scenarios with nodes connected to the host via a short-range path, such as scenarios SR and CR. Scenarios that require most or even all nodes to send data using the long-range modulation, such as scenarios LM, LI, and FO, show a higher energy consumption. As all transmissions within a flood are required to use the same modulation, this also applies to scenario RC. The LI scenario is the most extreme one, as indicated by the wide range of energy per round. For this scenario, all nodes are required to send data with the long-range modulation. The nodes close to the host represent a bottleneck and the opportunities to apply thinning optimizations are limited.

In summary, the LSR protocol allows for reducing power consumption without losing the property that every node can always be connected via the long-range modulation if required by the environment, which ensures that it is able to support all scenarios.

As the aggregated results only provide a high-level view, we have a closer look at the CR scenario. For this, we simulate the CR scenario for 1,000 rounds. To see the influence of the number of modulations when using the LSR protocol, we add the LSR protocol variant that uses only a single modulation, the long-range modulation. We apply full thinning for both LSR variants. In Figure 12, we plot the distribution of the energy per round for each node separately. Again we omit the first 30 rounds to remove the effect of the Bootstrapping phase (i.e., each bar represents the data from rounds 30 to round 999).

As expected, the energy consumption of the LWB protocol is similar for all nodes. With this protocol, all nodes always use long-range modulation and all nodes participate in each data flood. Using LSR with only one modulation already reduces energy consumption significantly. The thinning reduces the energy consumption and the information from the connectivity graph allows to evenly distribute the energy load for relaying messages to different nodes. Concretely, the forwarding of messages from nodes 2 and 4 is distributed to nodes 3, 5, and 6 compared to an assignment in which node 3 does all the work. However, this scheme is still inefficient in this case as the relaying nodes send their own data as well as data from remote nodes with the more expensive long-range modulation. If we use LSR with two modulations, the relaying nodes (nodes 3, 5, and 6) can send their own data via the short-range modulation. This allows to further reduce the maximum energy consumption across all nodes and therefore would increase the lifetime of a battery-powered system.

In the following, we further inspect the LSR protocol using two modulations. To investigate the temporal behavior, we plot the energy per round metric over time in Figure 13(a). In this case, we do not omit any rounds. From the energy consumption, we see that the Bootstrapping phase starts in round 0 and lasts around 20 rounds. The Bootstrapping phase consists of two parts: (1) the registration of the nodes at the host and (2) the collection of measurements for the thinning. The time required for the registration is mainly determined by the number of participating nodes. Each node determines the best modulation and sends a stream request. Per round, the host can receive at most one stream request for each modulation successfully. In the example, 8 rounds are required for all nodes to get registered. Once a node has a stream assigned (i.e., the active modulation has been determined), the thinning procedure is started. In the example in Figure 13(a), the thinning lasts until round 18. Afterward, the energy consumption is significantly reduced and remains stable for all nodes for a longer period of time. In rare cases, the network is adapted to a detected change even in a static scenario. For illustration purposes, we specifically picked a case where this happens within the first 100 rounds, concretely in round 33. This behavior can be explained with the simulation which is configured to use probabilistic transmissions (i.e., individual packets can be lost and the arrival of floods can be delayed). This can lead to situations where a used link in the connectivity graph is considered to be no longer available or a node is considered to have a different distance to the host. In these cases, the thinning is partially reset and the connections between nodes are measured again. As a consequence, the energy consumption is increased for a short amount of time. By configuring the history depth, it is possible to adjust the trade-off between fast adaptation to changes versus stability. Overall, LSR is able to continuously collect information about link characteristics without increasing the energy consumption significantly.

To examine the contributions of different radio operations, we plot the individual components in Figure 13(b). Here, we again omit the first 30 rounds. The results show that transmitting with long-range modulation accounts for the largest share of energy consumption. In addition, the nodes that are connected to the host via a short-range path and do not have to forward data (nodes 7 and 8) have a non-negligible long-range part. This is due to the forwarding of the schedule packets and the participation in contention slots. This represents the cost of supporting continuous dynamic adaptation to a changing environment. In certain cases, this is not necessary and further optimizations, as discussed in Section 6.4.5, could be applied.

To demonstrate how the LSR protocol adapts itself to a changing environment, we artificially change the link characteristics at runtime. For this, we use the RC scenario and simulate 100 rounds. At round 50, we change the link between the host (node 1) and node 5 from supporting only long-range communication to supporting long- and short-range communication.

In Figure 14, the active modulation as registered on the host node is depicted over time. In the first 10 rounds, the bootstrapping process is taking place and all nodes successfully register a stream for the long-range modulation (Mod0). After the path loss of the link between node 1 and node 5 is artificially decreased, all nodes could potentially switch to using the short-range modulation (Mod1). The transition happens with a delay due to the history of measurements. In this example of this section, the depth of the history is configured to eight measurements. In Figure 15, the shortest connectivity graphs for Mod0 and Mod1 as observed by the host node before and after the change are depicted. The two time instants are marked with arrows in Figure 14. When comparing the shortest connectivity graphs in Figure 15 to the RC scenario in Figure 10, we find that the graphs match the original graph well. This example shows that LSR is capable of continuously measuring the link characteristics and switching the modulation during normal data transfer rounds. In many cases, this even works without interrupting the data stream (i.e., without losing data packets). This is also the case in this example (Figure 16). For the other direction (switch from Mod1 to Mod0), such an uninterrupted switching is not supported by LSR as the change of the topology is detected by observing missed packets.

In this section, we investigate the convergence behavior and the accuracy of the graph-based thinning method of LSR with full thinning. For this, we introduce an additional metric, the graph distance. The graph distance is a measure that describes how much the graph obtained by LSR differs compared to the original graph defined by the scenario (ground truth). To compute the graph distance, for both graphs we first determine the shortest connectivity graph and extract the set of all edges. Then, we compute the symmetric difference between the two sets. The number of elements in the resulting set corresponds to all nodes that are either missing or superfluous compared to the ground truth. Ideally, the optimization of LSR is able to determine the original graph (i.e., the graph distance reaches zero). To prevent any influence from the registration procedure, we pre-register all nodes to Mod0 for the evaluation in this section.

First, we use the RC scenario to investigate the influence of the history depth. In Figure 17, we plot the graph distance for Mod0 over the first 40 rounds for four different history depths and 10 different simulation runs (i.e., different initialization of the random number generator) each. In addition, we indicate the best-case convergence time calculated based on the used scenario variation and used configuration parameters as vertical lines. In this case, all nodes are pre-registered and thus no rounds are required for registering streams at the host. However, a number of rounds are needed to fill the history buffer for hop distance measurements and collect thinning information. The best case for the first part is directly given by the history depth configuration. The best-case duration of the second part is given by Equation (1) discussed in Section 4.4.

The results show that the thinning with a history depth of one element often does not converge. However, for more suitable history depths of three or more elements, the thinning converges in many cases within 25 rounds. In rare cases, the obtained connectivity graph differs from the original graph by one or two edges; however, in most cases, LSR determines the correct graph.

To further investigate the influence of the number of hops and the number of nodes in the scenario, we additionally introduce the Convergence (CO) scenario depicted in Figure 18. This scenario defines several variations that differ by the number of hops and columns.

In Figure 19, we plot the graph distance for Mod0 over the first 60 rounds for the different variations of the CO scenario. For each scenario variation, we plot 10 runs. In addition, we use a fixed history depth of five elements. Again, we depict the best-case convergence time corresponding to each scenario variation as a vertical line. The results show that the best-case convergence times can indeed be achieved. However, there are also runs that converge more slowly. The results suggest that convergence becomes more challenging with scenarios involving more nodes. In addition, the results indicate that a larger number of hops and nodes increases the remaining graph distance.

The scenarios used for the comparison between the simulation and the testbed are based on the FlockLab 2 testbed [46] installation. FlockLab 2 supports tests with up to 30 nodes. The FlockLab 2 testbed provides different actuation and tracing capabilities including tracing the serial output, logic actuation and tracing, power profiling, and interactions and tracing using debug probes. The accurate timestamping required for the actuation and tracing services is achieved via time synchronization based on the Global Navigation Satellite System (GNSS) and Precision Time Protocol (PTP).

In the experiments in this section, we use two scenarios that both use the same set of 13 nodes depicted in Figure 20. Nine nodes are located inside an office building, and four additional nodes are located farther apart at distances of up to 2 km on rooftops of campus and city buildings. The first scenario (CR_FL2) is similar to the CR scenario of Section 5.2. We use node 24 as the host node. It is part of a cluster with short link distances. In addition, the scenario includes four rooftop nodes with large path loss links to the cluster. The second scenario (RC_FL2) is similar to the RC scenario of Section 5.2. In this scenario, we use node 15 as the host node and the remaining nodes form a cluster that includes links with high and low path losses.

A main reason for the evaluation in this section is the verification that the simulation is meaningful compared to real-world deployments. We therefore tried to match the real scenarios as well as possible in the simulation. We obtain the corresponding link characteristics information by performing link tests on the FlockLab 2 testbed which we then feed into the simulation in the form of a path loss matrix.

The implementation used for the testbed is based on the DPP2 LoRa target architecture [46] which consists of the DPP2 LoRa communication board [5] and a simple target adapter that enables the connection to the FlockLab 2 target interface. The DPP2 LoRa communication board consists of an STMicroelectronics STM32L433CC microcontroller and a Semtech SX1262 low-power long-range radio transceiver [40]. The microcontroller features an ARM Cortex-M4 core that supports clock frequencies of up to 80 MHz. The radio transceiver provides up to \(+\)22 dBm of transmit power, operates in the 868-MHz band, and supports the long-range LoRa and the short-range Frequency Shift Keying (FSK) modulation schemes using the same radio front end and antenna.

The software used to run on the nodes is available as open source as part of the Flora software² [44]. An overview of the message structures of the implementation is depicted in Figure 21. The header of each message contains a network ID and the message type identifier. The Slots field of the Schedule message contains a list of node IDs that defines the structure of the round for the corresponding modulation. In this simple implementation, fixed-size bit fields are used for the data acknowledgment, the delay mask, and the participation updates. As discussed in Sections 4.1 and 4.4, this representation could be replaced by alternative representations to allow for better scaling of the protocol to a larger number of nodes. The implementation on the DPP2 LoRa communication board supports the full LSR scheme for the sensor nodes and the basic scheme (see Section 3.3) for the host node. A fully featured scheduler that implements all optimizations is implemented in Python and runs on a separate device that interfaces with the host node via a serial connection. This partitioning of the functionality has the advantage that any ordinary node with a serial connection to a device that supports the execution of Python scripts can serve as a host node. Consequently, in the evaluation with the testbed, any node can be configured to be the host.

The configuration values that are used for both the simulation and the tests on the FlockLab 2 testbed are listed in Table 4. The transmit power is chosen such that the four remote rooftop nodes are not directly reachable from the remaining nodes using the short-range modulation (Mod1) but are reachable using the long-range modulation (Mod0). A history depth of three measurements (i.e., \(B_m=B_h=B_d=3\)) is used for the simulation and the testbed.

For the simulation as well as the testbed and for each scenario, we run the LSR protocol with full thinning and using two modulations for 180 rounds. We omit the first 30 rounds to remove the influence of the Bootstrapping phase. The resulting reliability and the energy per round values for the CR_FL2 and the RC_FL2 scenarios are plotted in Figure 22. For the testbed, the reliability values are determined by analyzing serial logs, whereas the energy is measured using the power profiling service of FlockLab 2.

We observe that in both scenarios, the reliability values for both implementations are in a similar range. For the CR_FL2 scenario, the reliability obtained from the testbed is less than 1 % higher compared to the results from the simulation. For the RC_FL2 scenario, the simulation has a tendency of being slightly too optimistic as the reliability determined by the simulation is 7% higher compared to the value measured with the testbed.

For both scenarios, the energy per round values exhibit the same order of magnitude and a similar pattern for both implementations. In the RC_FL2 scenario, some nodes display more significant deviations, but the tendency is comparable. The energy per round measurements for the CR_FL2 and RC_FL2 scenarios obtained from the testbed are on average 12 % and 18 % higher when compared to the corresponding values determined in the simulation, respectively.

The results from the testbed demonstrate that the LSR protocol including its optimizations can be implemented on real IoT hardware and works as expected. Various factors may contribute to differences between the simulation and the testbed implementation. On the one hand, the wireless channel inherently contains various probabilistic elements which are changing over time. On the other hand, the simulation includes a number of simplifications and abstractions to keep the computations tractable. For example, the probabilistic model of the simulation is based on fixed path loss values, whereas in a real system the path loss values can significantly vary over time. Additionally, overheads such as the power consumption of the microcontroller as well as frequency synthesis and amplifier ramp-up time of the radio which are not considered in the simulation contribute to differences. Furthermore, it is difficult to accurately model real-world effects such as the capture effect [49]. Considering the described complexity stemming from the properties of real systems and the probabilistic elements of the LSR protocol, we do not necessarily expect the metrics to agree on a per-node level, but that the network-wide trends align with our simulations. Thus, we find that the results of the simulation match the ones of the testbed sufficiently well. As a consequence, we consider the results of the evaluation in Section 5.2 to be representative.

So far, we have assumed that the set of available modulations is given and does not change. For simplicity, we use only two modulations in this work. However, the protocol could easily be extended to use more than two modulations. The mechanism to determine the best modulation would be unchanged and still relies on collecting statistics on the received schedule floods. But the overhead would potentially increase as each node needs to listen to additional schedule floods. The optimal set of modulations (number of modulations and which modulations are available) could be determined offline or online based on a model of the network or using sample measurements.

Up to now, we have limited the number of streams for each node to a single stream. Furthermore, the gateway is always the sender or receiver of the data. With extensions of the basic scheme, more flexibility for data streams could be added. By introducing stream IDs and adaptation of the scheduler on the host, multiple streams per node could be supported. In addition, streams between two nodes could be integrated by including the destination node ID into stream requests and adapting the scheduler. In this case, thinning could be applied but full thinning to a single shortest path would not be possible in every case as the shortest connectivity graph, obtained as described in Section 4.4, does not contain all edges between nodes that have the same hop distance from the host. The scheme for measuring the connectivity graph could be extended as well but additional hop distance measurements need to be collected and the overall complexity would be increased.

Due to the uncontrollable and fluctuating RF environment, flooding messages with blind retransmissions, as it is implemented in Gloria, does usually achieve a packet delivery ratio below \(100\%\). To improve this, an additional protocol layer on top of the basic scheme could be added. This additional layer would resend data that has not been delivered successfully. With the aggregated acknowledgment of data transmissions in schedule messages, the basic LSR scheme provides a mechanism to implement such a layer on top.

We described the LSR protocol applied to a system with a single gateway. One straightforward solution to support multiple gateways is to assign each of the gateways the role of an LSR host and to statically assign each node to exactly one gateway. More sophisticated schemes where nodes for example are being passed between gateways at runtime or schemes where nodes can be registered to multiple gateways at the same time are possible but outside the scope of this article.

If it is known from the scenario that topology changes do not occur or are extremely rare, the protocol could be further optimized by thinning non-data transmissions such as schedule and contention floods. However, this is more critical with respect to recovery from a failure than thinning data floods since this could impact the flow of control information. To ensure recovery from a failure, a mechanism is required to reset the thinning of non-data floods. A simple example of such a mechanism would be that all nodes are required to participate in schedule floods in every \(j\)th round to receive control information.

We propose to use a simple approach for thinning based on the shortest connectivity graph information, namely selecting a single path out of the available shortest paths. The selection of participating nodes could potentially be improved or tailored to the needs of the application by improving the load balancing and allowing for a defined level of redundancy.