Deterministic distributed resource discovery (brief announcement)

The resource discovery problem was introduced by Harchol-Balter, Leighton and Lewin in [HLL99], as a part of their work on web caching. They developed a randomized algorithm for the problem in the weakly connected directed graph model, that was implemented within LCS at MIT, and then licensed to Akamai Technologies.

The directed graph is a logical one (as opposed to the underlying communication graph). It represents the nodes' “knowledge” about the topology of the underlying communication network. In [HLL99] the notion of topology “knowledge” is simplified, by modeling it as a “knowledge” of an ID of another node. In the general case such a “knowledge” may a include whole route, as well as any additional information needed in order to establish a connection (e.g. a cryptographic public key).

It is assumed (here, and in [HLL99]) that the logical graph G is weakly connected. This can result from topology changes: e.g., a loss of a connection to a name server, or a gain of new knowledge that is uneven between different nodes, since it is obtained by distributed algorithms, and in non-homogeneous network.

Note that weak connectivity is a necessary condition for the solvability of the problem. Dealing efficiently with the weakly connected graph was, in fact, the main contribution in [HLL99]. The alternative of transforming the graph into an undirected one, and then solving the problem on the resulting undirected graph, is possible. If E₀ = O(n) it even leads to efficient solutions: A time optimal algorithm (for undirected graphs) appears in [SV82], It was designed for CRCW PRAM, but can be translated into the model of [HLL99] using O(|E₀| log n) messages. The assumption that |E₀| = O(n) may be suitable if topology changes are assumed to be limited. Many practical distributed systems, however, attempt to deal with the case that the network may be partitioned. Thus, E₀ should be as big as possible, to enable the disconnected components to regain connectivity.

The current paper proposes a deterministic algorithm for the problem in the same model as that of [HLL99]. Our algorithm is near optimal in all the measures: time, message, and communication complexities. Each previous algorithm had a complexity that was higher at least in one of the measures. Specifically, previous deterministic solutions required either time linear in the diameter of the initial network, or communication complexity O(n³) (with message complexity O(n²)), or message complexity O(|E₀| log n) (where E₀ is the edge set of the initial graph). Compared to the main randomized algorithm of the Harchol-Balter, Leighton, and Lewin, the time complexity is reduced from O(log²n) to O(log n log^* n), the message complexity from O(n log ²n) to O(n log n log ^* n), and the communication complexity from O(n² log³ n) to O(|E₀|log² n + n² log n).