CN113037872B - Caching and prefetching method based on Internet of things multi-level edge nodes - Google Patents
Caching and prefetching method based on Internet of things multi-level edge nodes Download PDFInfo
- Publication number
- CN113037872B CN113037872B CN202110553109.8A CN202110553109A CN113037872B CN 113037872 B CN113037872 B CN 113037872B CN 202110553109 A CN202110553109 A CN 202110553109A CN 113037872 B CN113037872 B CN 113037872B
- Authority
- CN
- China
- Prior art keywords
- data
- edge node
- request data
- level edge
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5681—Pre-fetching or pre-delivering data based on network characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Transfer Between Computers (AREA)
- Computer And Data Communications (AREA)
Abstract
A caching and prefetching method based on Internet of things multi-level edge nodes belongs to the technical field of digital information transmission and comprises the following steps: step S1, establishing an Internet of things multi-level edge node architecture; the multi-level edge node architecture of the Internet of things comprises an Internet of things terminal device, a second edge layer, a first edge layer and a cloud end; step S2, caching edges of the sensing data; in step S3, edge prefetching of the second-level edge node. Aiming at the characteristics that the number of interaction times between the terminal equipment of the Internet of things and the edge node is large and the data of single interaction is small, an edge layer is added between an original edge layer and the terminal equipment of the Internet of things, and the first edge layer and the second edge layer are set to be a lightweight data processing center, so that the data flow is reduced, and the access delay is reduced.
Description
Technical Field
The invention belongs to the technical field of digital information transmission, and particularly relates to a caching and prefetching method based on an Internet of things multi-level edge node.
Background
The edge node is a logic abstraction of basic common capability of a plurality of product forms of the edge side such as an edge gateway, an edge controller and an edge server, and the product forms have common capability of real-time data analysis, local data storage, real-time network connection and the like of the edge side. Currently, edge nodes are in their entirety at the early stages of layout and development. The edge node position is between the user and the cloud center, and compared with the traditional cloud center, the edge node is closer to the user (data source), and has the characteristics of miniaturization and distribution.
The Internet of things (IoT), i.e., "Internet of things," is an extended and expanded network based on the Internet, and combines various information sensing devices with the Internet to form a huge network, thereby realizing the intercommunication of people, machines and things at any time and any place.
In the years, with the introduction and practice of concepts such as intelligent communities and the like, a large number of internet of things terminal devices are accessed into a network, and information interaction is carried out, so that mass data are generated. Common terminal devices of the internet of things include cameras, computers, mobile communication tools, sensors and the like. If the source data of the terminal equipment of the internet of things are directly uploaded to the cloud computing center to be processed, on one hand, a large amount of bandwidth is occupied, and on the other hand, huge burden is brought to the processing of the cloud computing center.
In the prior art, there are cases of performing data processing on video by using edge calculation, for example, chinese patent No. CN202011414580.0 discloses a method and an apparatus for performing structural analysis on video based on edge calculation, the entire system presents a flat distributed architecture, a central end only needs to complete scheduling work of edge end nodes, and receives calculation results transmitted by the edge end nodes, and the work calculation amount of the central end is much smaller than that of the edge end nodes, so that the overall reliability and fault tolerance of the system are significantly improved, normal function use condition of the entire system is not affected due to single node failure, and the system reliability is improved; meanwhile, by utilizing the distributed architecture with the organically combined center and edge, the problems of insufficient computing power and weak data processing capability of the center or edge end node can be solved, the computing power bottleneck of a single device is not limited any more, the computing and analyzing capability is greatly improved, and the data processing speed is accelerated.
However, the above scheme is directed to processing of video data, on one hand, a large amount of calculation is required in the processes of video structuring processing, data encryption and the like, and on the other hand, the amount of source data is large, and a large amount of storage space of an edge node is occupied.
Compared with video data, the number of the terminal devices of the internet of things is large, the number of times of interaction between the terminal devices of the internet of things and the edge nodes is large, and single interaction data is small. The probability that the terminals of the internet of things with similar positions send the same request data in the same time period is higher. Thus, the network has duplicate data processing and highly redundant network traffic. For example, a sensor (for example, a temperature and humidity sensor) for monitoring weather is installed in the smart community, Zhang III and Li IV in the community all send requests for accessing weather data to edge nodes through mobile terminals in the same time period, if the data are only stored in a cloud, all the requests need to access the cloud, and the cloud repeatedly sends the same weather data.
Due to frequent interaction times between the terminal equipment of the internet of things and the edge nodes and the complex high-access judgment algorithm, the processing burden of the edge nodes is increased, and network delay is caused.
Therefore, aiming at the problems that the number of interaction times between the terminal equipment of the internet of things and the edge node is large and the data of single interaction is small, the edge node needs to be arranged near the terminal equipment, high-demand data does not need to be uploaded to the cloud for processing, the data is processed at the network edge side, a simple and efficient high-access-capacity judgment mode is adopted, the cache hit rate is ensured, the request response time is shortened, and redundant cache is eliminated.
Disclosure of Invention
In view of the shortcomings of the prior art, the invention aims to provide a caching and prefetching method based on the multi-level edge node of the internet of things.
In order to achieve the above object, the present invention adopts the following technical solutions.
A caching and prefetching method based on Internet of things multi-level edge nodes comprises the following steps:
step S1, establishing an Internet of things multi-level edge node architecture; the multi-level edge node architecture of the Internet of things comprises an Internet of things terminal device, a second edge layer, a first edge layer and a cloud end;
the terminal equipment of the Internet of things comprises user equipment and a sensor;
the second edge layer is provided with a second-level edge node with a cache; each Internet of things terminal device is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wireless network, a wired network or a cellular network;
the first edge layer is provided with a first-level edge node with a cache; each second-level edge node is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wired network or a cellular network;
the cloud is provided with a database for storing all past data; all the first-stage edge nodes are in communication connection with the cloud end, and the communication connection adopts a wired network or a cellular network;
step S2, caching edges of the sensing data; the sensing data with high freshness and high access rate is preferentially reserved by adopting a data replacement mode;
step S3, performing edge prefetching of the second-level edge node; the second level edge node stores all associated sensed data for cache misses in its cache.
Step S2 further includes the steps of:
step S2a, sensing new sensing data by the terminal equipment of the Internet of things at intervals, and uploading the new sensing data to a second-level edge node of a second edge layer;
step S2b, the second-level edge node allocates a frequency value for each sensing data in the cache, the initial value of the frequency value is 0, and the frequency value is more than or equal to 0; every fixed time period, if the sensing data is not accessed, the frequency value of the sensing data is reduced by 1 until the frequency value is reduced to 0; otherwise, increasing the frequency value of the sensing data by 1; the sensing data and the frequency values are in one-to-one correspondence;
step S2c, when the terminal device of the internet of things uploads new sensing data to the second level edge node, the second level edge node first determines whether the cache of the edge node is full: if the cache is not full, the second-level edge node stores the new data in the cache of the second-level edge node, and assigns a frequency value for the new sensing data, wherein the initial value of the frequency value is 0; if the cache is full, deleting the historical sensing data of the terminal equipment of the Internet of things with the minimum frequency value from the cache; if a plurality of sensed data have the same frequency value, the oldest sensed data will be deleted.
Step S2d, the second-level edge node uploads new sensing data of the terminal equipment of the Internet of things to the first-level edge node, and the first-level edge node stores all the sensing data uploaded by the second-level edge node in communication connection with the first-level edge node; the first-level edge nodes upload new sensing data of the terminal equipment of the Internet of things to the cloud, and the cloud stores all the sensing data uploaded by the first-level edge nodes in communication connection with the cloud.
Step S3 further includes the steps of:
step S3a, the second-level edge node establishes a type relation set; the second-level edge node collects the request data of all the terminal equipment of the Internet of things obtained within a period of time into 1 request data sequence, and the request data at least comprises 4 fields, namely the type of a sensor requested by a user, the position of the sensing data requested by the user, the time requested by the user and the time for generating the sensor data; extracting the request data from each request data sequence according to each association rule and forming a typed request data sequence;
step S3b, when the second level edge node receives a new data request, it will search the sensing data corresponding to the data request in the cache; if the sensing data is not in the cache, the condition of cache missing data occurs, and edge prefetching is triggered:
if the proportion of the same type of request data of the new data request is larger than or equal to the proportion threshold value, finding other same type of request data associated with the same type of request data according to the type relation set, and then adding the same type of request data sequence of the new data request and the associated other same type of request data sequences into a prefetching list;
otherwise, adding the same type request data sequence where the new data request is located into the prefetching list;
step S3c, the second level edge node finds out the request data of cache loss according to all the request data in the prefetch list, and then sends the request data of cache loss in the prefetch list to the first level edge node;
the first-level edge node receives the request data of the second-level edge node, acquires the sensing data lost by the cache from the cloud if the request data lost by the cache exists, stores the sensing data in the current-level cache, and then sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node; the first-level edge node receives the request data of the second-level edge node, and if the request data of the cache loss does not exist, the first-level edge node sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node;
and the second-level edge node stores all the sensing data corresponding to the request data in the pre-fetching list in the current-level cache, and sends the sensing data corresponding to the new data request to the terminal equipment of the Internet of things.
In step S3a, the association rule is as follows:
1, classifying all request data in each request data sequence, counting the number of the request data of the same type, and dividing the number of the request data of the same type by the total number of the request data in the request data sequence to obtain the proportion of the request data of the same type; when the proportion of the request data of the same type is larger than or equal to the proportion threshold value, grouping all the request data of the type into 1 request data sequence of the same type; there may be a plurality of request data sequences of the same type extracted from each request data sequence; sequentially processing all the request data sequences to obtain k request data sequences of the same type (k is more than or equal to 2 and less than or equal to the total number of the types of the request data);
namely: type (A) = number of request data of the same type/total number of request data in the request data sequence, and type (A) is more than or equal to a proportion threshold value; wherein A is a classification type of request data, and type (A) represents the proportion of A type request data;
calculating the association degree of each Type to other types, wherein the association degree (A → B) = Type (A ^ B)/Type (A), and A and B are both classification types of the request data;
the degree of association (a → B) indicates the possibility that the type B request data appears after the type a request data appears;
and presetting a confidence coefficient, and when the association degree (A → B) is higher than the confidence coefficient, pairing the two request data sequences of the same type A and B and including the two request data sequences of the same type A and B into a type relation set.
The scheme has the following advantages:
1. aiming at the characteristics that the number of interaction times between the terminal equipment of the internet of things and the edge node is large and the data of single interaction is small, an edge layer (namely, a second edge layer) is added between an original edge layer (equivalent to a first edge layer in the application) and the terminal equipment of the internet of things, and the first edge layer and the second edge layer are set to be a lightweight data processing center, so that the data flow is reduced, and the access delay is reduced.
The edge node of the second edge layer is a lightweight edge node with limited computing capacity and limited cache space, and the deployment cost of the whole framework is saved under the condition of meeting the requirements of edge cache and edge prefetching. In a traditional edge architecture, all internet of things terminal devices are directly connected to edge nodes, and network traffic bottleneck can be caused when traffic load is high. In the edge architecture, the second edge layer shares the terminal access burden of the first edge layer.
And 2, edge caching, wherein a data replacement mode is adopted in consideration of the freshness and the communication cost of the data of the Internet of things, and the sensing data with high freshness and high access rate are preferentially reserved, so that the traffic load of part of the network and the access delay of users are reduced.
And 3, edge prefetching. In the past, edge prefetching generally researches the request trend of a single user, and the prefetching algorithm is very complex and needs to be configured with a large amount of computing resources. The edge prefetching of the scheme does not take the request trend of a single user as a research object, but predicts the access to a group of users. And the second-level edge node stores all associated sensing data into the cache thereof through edge prefetching and sends the sensing data with cache loss to the user. Thus, if other users also request such data, the local second level edge node can satisfy the services without having to obtain the sensed data from the upper layers.
Drawings
Fig. 1 is a schematic diagram of a multi-level edge node architecture of the internet of things.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
Compared with video data, the terminal equipment of the Internet of things has the following characteristics:
the method comprises the following steps that 1, the number of the terminal devices of the Internet of things is large, the number of times of interaction between the terminal devices of the Internet of things and edge nodes is large, and single interaction data is small. Therefore, the traditional complex high-access judgment algorithm is no longer applicable.
And 2, the data of the terminal equipment of the Internet of things comprises the current working parameter value of the terminal equipment reporting the data of the Internet of things and the data value collected by the terminal equipment. The current operating parameter value typically takes up more memory than the data value. Therefore, the storage space of the edge node is mostly used for storing the current operating parameter values of the terminal device, and much of this information is duplicated and redundant.
Internet of things data is sensitive to freshness, as people are most interested in the latest data. Unlike video data, data values of the terminal devices of the internet of things vary with time and location, such as data values of a temperature and humidity sensor, a PM2.5 sensor and an atmospheric pressure sensor.
Aiming at the characteristics of the terminal equipment of the Internet of things, the scheme adopts the architecture of the multilevel edge nodes of the Internet of things. The internet of things terminal equipment uploads data to a closer edge node instead of the cloud computing center, so that data flow is reduced, and access delay is reduced.
Aiming at the characteristics that the interaction times between the terminal equipment of the Internet of things and the edge node are large and the data of single interaction is small, the scheme adopts a data caching and data prefetching technology. The data caching and data prefetching have the effects of reducing network load, reducing network energy consumption and reducing access delay. The data caching of the edge calculation means that data is stored in the edge cache and is directly used by a terminal user, and the situation that the user and an edge node call the data from a cloud is avoided. The data prefetching of the edge calculation refers to reading the required data from the upper layer in advance and storing the data to a cache of the edge node before the user needs to access the data. The key to data caching and data prefetching is how to judge high-access data.
A caching and prefetching method based on Internet of things multi-level edge nodes comprises the following steps:
step S1, establishing an Internet of things multi-level edge node architecture; the multistage edge node architecture of the Internet of things comprises terminal equipment of the Internet of things, a second edge layer, a first edge layer and a cloud end.
The terminal equipment of the Internet of things comprises user equipment and a sensor. The user equipment can be a mobile phone, a computer, an intelligent bracelet, an intelligent watch and the like. The sensor can be a temperature sensor, a humidity sensor, an air pressure sensor, a water pressure sensor and the like. The user device may generate request data and sensory data. The sensor generates sensory data.
And the second edge layer is provided with a second-level edge node with a cache. The second-level edge node adopts lightweight equipment to reduce the deployment cost of the edge node, for example, the second-level edge node may adopt Raspberry pi (Raspberry Pis) providing a network-based file sharing service for clients. Each terminal device of the internet of things is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wireless network, a wired network or a cellular network.
Because thing networking terminal equipment is in large quantity, and many thing networking terminal equipment possess this wireless transmission module, consequently, second level edge node disposes wireless transmission module, and wireless transmission module includes but not restricted to Wi-Fi module, bluetooth module and zigBee module.
And the first edge layer is provided with a first-level edge node with a cache. The first-level edge node adopts traditional edge node equipment and has larger computing capacity and storage capacity, such as a server or a base station. Each second-level edge node is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wired network or a cellular network, so that the stability of mass data transmission between the first edge layer and the second edge layer is guaranteed.
The cloud is a data center with a large amount of computing capacity and storage space, and can be a public cloud, a private cloud or a hybrid cloud. The cloud is configured with a database that stores all past data. All the first-level edge nodes are in communication connection with the cloud end, and the communication connection adopts a wired network or a cellular network, which is a reliable communication connection mode. And the first-level edge node uploads all data to the cloud for storage.
Aiming at the characteristics that the number of interaction times between the terminal equipment of the Internet of things and the edge node is large and the data of single interaction is small, the framework adopts lightweight equipment as a second-level edge node. The second-level edge node is responsible for receiving the sensing data from the terminal equipment of the Internet of things, storing the sensing data in a local cache, and uploading the sensing data to the first-level edge node; in addition, the second-level edge node receives the access request of the terminal equipment of the Internet of things, and can collect and cache sensing data and process part of the request of the user, so that the burden of the first-level edge node is reduced, and the data is cached from the body or captured from the upper-level edge node according to the request and the request is replied.
The first level edge nodes may cache larger data (e.g., video), may perform complex tasks, and provide greater computing power. The first edge layer and the second edge layer are set to be a lightweight data processing center, the terminal equipment of the internet of things can upload data to the edge nodes, the edge nodes collect, cache and preprocess the data, and users can access the data from the edge nodes instead of acquiring the data from the cloud, so that data traffic is reduced and user access delay is reduced. On the contrary, if the data processing centers are all arranged at the cloud end, the long-distance information transmission between the terminal device of the internet of things and the cloud end causes access delay.
Step S2, caching edges of the sensing data;
step S2a, the terminal device of the internet of things (e.g., the temperature sensor) senses new sensing data at intervals (e.g., every five minutes), and uploads the new sensing data to the second-level edge node of the second edge layer.
The sensing data comprises the current working state parameter value of the terminal equipment of the Internet of things and the data value collected by the terminal equipment of the Internet of things. In the scheme, the sensing data is divided into 4 fields, namely, the device type, the position of the generated value, the time of the generated value and the sensing value. The device type, the position of the generated value and the time of the generated value belong to the current working state parameter value, and the sensing value belongs to the data value acquired by the terminal device of the Internet of things.
In the four fields, the sensor type and the position of the generated value are always the same, and the time for generating the value changes periodically, so that the information in the 3 fields is redundant. If the data uploaded by the terminal equipment of the internet of things each time are independently stored in the cache of the edge node, the data occupy a lot of space. Therefore, the sensing data are stored in the table data set of the sensing data after being summarized, so that the data of the specific sensor with the same sensor type and the same position of the generated value are normalized, the space is saved on one hand, and the transfer of the sensing data is facilitated on the other hand.
In a tabular dataset of sensory data, each row represents a time period, and each column represents data for a particular sensor for which the sensor type and location of the generated value are the same. For example, taking a continuous day as the time of investigation, each time interval is 5 minutes, so there are 288 time periods in the day. The terminal device of the internet of things has 2 device types (numbered 1 and 2) and 3 positions (numbered 1, 2 and 3) for generating values, so 6 columns are shared.
Table 1 is an example of a tabular dataset for sensory data
(1,1) | (1,2) | (1,3) | (1,1) | (2,2) | (2,3) | |
Time point 00: 00 | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value |
Time point 00: 05 | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value |
Time point 00: 10 | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value |
... | ... | ... | ... | ... | ... | ... |
Time point 23: 50 | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value |
Time point 23: 55 | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value | Sensing a value |
In the numerical content area of table 1, row 2, column 2 indicates that at time point 00: and 05, the equipment type is 1, and the position of the generated value is 2, wherein the sensing value is uploaded by the terminal equipment of the internet of things.
Step S2b, the second level edge node allocates a frequency value for each sensing data in the cache, the initial value of the frequency value is 0, and the frequency value is larger than or equal to 0. Every fixed period of time (e.g., 5 minutes), if the sensing data is not accessed, the frequency value of the sensing data is decreased by 1 until the frequency value is decreased to 0; otherwise, the frequency value of the sensing data is increased by 1. The sensing data and the frequency value are in one-to-one correspondence.
Table 2 is an example of frequency values for sensed data
(1,1) | (1,2) | (1,3) | (1,1) | (2,2) | (2,3) | |
Time point 00: 00 | 0 | 1 | 3 | 0 | 3 | 4 |
Time point 00: 05 | 0 | 2 | 4 | 0 | 3 | 5 |
Time point 00: 10 | 1 | 3 | 4 | 0 | 5 | 6 |
... | ... | ... | ... | ... | ... | ... |
Time point 23: 50 | 2 | 3 | 6 | 1 | 4 | 4 |
Time point 23: 55 | 1 | 2 | 2 | 2 | 3 | 5 |
Step S2c, when the terminal device of the internet of things uploads new sensing data to the second level edge node, the second level edge node first determines whether the cache of the edge node is full: if the cache is not full, the second-level edge node stores the new data in the cache of the second-level edge node, and assigns a frequency value for the new sensing data, wherein the initial value of the frequency value is 0; if the cache is full (namely the size of the new sensing data is larger than or equal to the maximum available cache size), deleting the historical sensing data of the terminal equipment of the Internet of things with the minimum frequency value from the cache; if a plurality of sensed data have the same frequency value, the oldest sensed data will be deleted.
Step S2d, the second-level edge node uploads new sensing data of the terminal equipment of the Internet of things to the first-level edge node, and the first-level edge node stores all the sensing data uploaded by the second-level edge node in communication connection with the first-level edge node; similarly, the first-level edge node uploads new sensing data of the terminal equipment of the internet of things to the cloud end, and the cloud end stores all the sensing data uploaded by the first-level edge node in communication connection with the cloud end. That is, all the sensing data will be stored in the cloud.
Because the buffer space of the second-level edge node is limited, the freshness characteristic of the data of the internet of things is considered, and a data replacement mode is adopted, the sensing data with high freshness characteristic and the sensing data with high access rate are preferentially reserved, so that the traffic load of part of the network and the access delay of users are reduced.
In step S3, edge prefetching of the second-level edge node.
More often, the user sends multiple request data in a short time to form a request data sequence, rather than a single request data. Also, there are specific relationships, such as location correlation, time correlation, and type correlation, between multiple request data in the same request data sequence. For example, a user located at place a, who is more interested in sensory data generated at place a, than place B; the user just requests to access the local temperature sensing data at the moment, and then the user is likely to request to access the local humidity sensing data at the moment; after the user accesses the heart rate sensing data of the user, the user can want to access the sensing data related to the body health type, such as blood pressure or blood sugar.
In addition, some areas of request data have periodic variability. Taking an environmental sensor as an example, 6: 00 to 9: 00, many users want to know the temperature, humidity and air quality, and the frequency of accessing these sensory data peaks the day.
In summary, the request data sequence reflects that there is a specific relationship among a plurality of request data, such as location correlation, time correlation and type correlation, that is, reflects the interest and trend of all users requesting access in a certain location area. Therefore, by analyzing the request data sequence, the second-level edge node actively prefetches the sensing data with high user interest to the first-level edge node, so that the sensing data can be fed back in time when the subsequent user requests the sensing data.
Step S3a, the second level edge node establishes a set of type relationships.
Specifically, the second-level edge node collects request data of all internet of things terminal devices obtained within a period of time (for example, 5 minutes) into 1 request data sequence, and the request data at least comprises 4 fields, namely, the type of a sensor requested by a user, the position of the sensor data requested by the user, the time requested by the user, and the time for generating the sensor data; and extracting the request data from each request data sequence according to each association rule and forming a typed request data sequence.
The association rules are as follows:
classifying all the request data in each request data sequence, counting the number of the request data of the same type, and dividing the number of the request data of the same type by the total number of the request data in the request data sequence to obtain the proportion of the request data of the same type. When the proportion of the request data of the same type is larger than or equal to the proportion threshold value, grouping all the request data of the type into 1 request data sequence of the same type; there may be a plurality of request data sequences of the same type extracted from each request data sequence. And sequentially processing all the request data sequences to obtain k request data sequences of the same type (k is more than or equal to 2 and is less than or equal to the total number of the types of the request data).
Namely: type (A) = number of request data of the same type/total number of request data in the request data sequence, and type (A) ≧ proportion threshold. Wherein, A is a classification type of request data, and type (A) represents the proportion of A type request data.
And 2, calculating the association degree of each Type to other types, wherein the association degree (A → B) = Type (A ^ B)/Type (A), and A and B are both classification types of the request data.
The degree of association (a → B) indicates the possibility that the request data of type B appears after the request data of type a appears.
And presetting a confidence coefficient, and when the association degree (A → B) is higher than the confidence coefficient, pairing the two request data sequences of the same type A and B and including the two request data sequences of the same type A and B into a type relation set.
Step S3b, when the second level edge node receives a new data request, it will search the sensing data corresponding to the data request in the cache; if the sensing data is not in the cache, the condition of cache missing data occurs, and edge prefetching is triggered:
if the proportion of the same type of request data of the new data request is larger than or equal to the proportion threshold value, finding other same type of request data associated with the same type of request data according to the type relation set, and then adding the same type of request data sequence of the new data request and the associated other same type of request data sequences into a prefetching list;
otherwise, the same type of request data sequence in which the new data request is located is added to the prefetch list.
The sensing data corresponding to the same type of request data in the prefetch list has high possibility of being visited.
Step S3c, the second level edge node finds out the request data of cache loss according to all the request data in the prefetch list, and then sends the request data of cache loss in the prefetch list to the first level edge node;
the first-level edge node receives the request data of the second-level edge node, acquires the sensing data lost by the cache from the cloud if the request data lost by the cache exists, stores the sensing data in the current-level cache, and then sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node; and the first-level edge node receives the request data of the second-level edge node, and sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node if the request data of the cache loss does not exist.
And the second-level edge node stores all the sensing data corresponding to the request data in the pre-fetching list in the current-level cache, and sends the sensing data corresponding to the new data request to the terminal equipment of the Internet of things.
And the second-level edge node stores all associated sensing data into the cache thereof through edge prefetching and sends the sensing data with cache loss to the user. Thus, if other users also request such data, the local second level edge node can satisfy the services without having to obtain the sensed data from the upper layers.
And carrying out simulated index evaluation on the scheme.
Firstly, simulation setting is carried out on the multi-level edge node architecture of the Internet of things. In the architecture, there are 3 first-level edge nodes (numbered 0, 1 and 2, respectively), and each first-level edge node is communicatively connected with 3 second-level edge nodes (numbered 0, 1 and 2, respectively). Each second-level edge node is in communication connection with 5 internet of things terminal devices (sensors), so that the architecture has 15 internet of things terminal devices in total.
The update frequency of the internet of things terminal devices is between two minutes and one hour, and all the internet of things terminal devices maintain the same update frequency in order to simplify the model.
The physical cache of the first level edge node is about 20MB and the physical cache of the second level edge node is about 2 MB.
In order to test the access delay of the architecture more accurately, the communication time between adjacent layers needs to be tested, and the access delay of a user is measured by taking the communication time as a reference. The cloud uses the Aliskiren cloud, and cloud servers are automatically distributed. The first level edge node uses a server near the company that is connected to the cloud over a wired network. The second-level edge node uses a Raspberry pi-3 (Raspberry Pis-3) single-board computer which is arranged near the terminal equipment of the Internet of things and is connected to the first-level edge node through a wired network.
The communication time is the time period from the time the device sends the request to the time it receives the reply, and the processing time of the request is ignored in order to simplify the model. The average access delay from the user to the second level edge node is 5 milliseconds, the average access delay from the second level edge node to the first level edge node is 48 milliseconds, the average access delay from the first level edge node to the cloud is 38 milliseconds, and the average access delay from the user to the cloud is 120 milliseconds.
In order to make a more comprehensive evaluation of the performance of the present solution, 3 additional control examples were set up.
First group of comparative examples: and (5) cloud architecture. The architecture only comprises the Internet of things terminal equipment and the cloud. And the terminal equipment of the Internet of things is directly in communication connection with the cloud. The terminal equipment of the Internet of things directly uploads the sensing data to a cloud database. The cloud receives and services the user's request.
Second group comparative example: an edge node architecture. The architecture only comprises the Internet of things terminal equipment, the edge nodes and the cloud. The cloud has a database that holds all the data. The edge node is connected with the cloud end through a wired network, and a cache is arranged on the edge node. The terminal equipment of the Internet of things directly uploads the sensing data to the edge node, and the edge node receives the sensing data and the request data.
Third group comparative example: a multi-level edge node architecture without prefetching. The framework comprises an Internet of things terminal device, a second edge layer, a first edge layer and a cloud end. The second edge layer, the first edge layer, has a step of edge caching, but does not perform edge prefetching.
In order to test the performance index of the scheme, the true data is needed. Data of a plurality of open real internet-of-things sensors can be collected to serve as sensing data, however, user request records are difficult to obtain, and therefore request data are lacked. Therefore, the user request record is simulated to be used as the request data, and the user request record is ensured to accord with the real user behavior.
First group of comparative examples: cloud architecture with an average latency of 120 milliseconds.
Second group comparative example: and the average delay of the edge node architecture is 54 milliseconds.
Third group comparative example: a multi-level edge node architecture without prefetching. If the first-stage edge node and the second-stage edge node are connected by adopting a wired network, the average delay is 14 milliseconds. If the first-level edge node and the second-level edge node are connected by adopting a wireless network, the average delay is 21 milliseconds.
In this embodiment: a multi-level edge node architecture for edge prefetching. If the first-stage edge node and the second-stage edge node are connected by adopting a wired network, the average delay is 6.1 milliseconds. If the first-level edge node and the second-level edge node are connected by adopting a wireless network, the average delay is 6.9 milliseconds.
Therefore, by adopting the prefetching scheme of the embodiment, the delay improvement rate is up to 94.9% compared with the cloud architecture of the first comparative example.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.
Claims (2)
1. A caching and prefetching method based on multi-level edge nodes of the Internet of things is characterized by comprising the following steps:
step S1, establishing an Internet of things multi-level edge node architecture; the multi-level edge node architecture of the Internet of things comprises an Internet of things terminal device, a second edge layer, a first edge layer and a cloud end;
the terminal equipment of the Internet of things comprises user equipment and a sensor;
the second edge layer is provided with a second-level edge node with a cache; each Internet of things terminal device is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wireless network, a wired network or a cellular network;
the first edge layer is provided with a first-level edge node with a cache; each second-level edge node is in communication connection with at least 1 second-level edge node, and the communication connection adopts a wired network or a cellular network;
the cloud is provided with a database for storing all past data; all the first-stage edge nodes are in communication connection with the cloud end, and the communication connection adopts a wired network or a cellular network;
step S2, caching edges of the sensing data; the sensing data with high freshness and high access rate is preferentially reserved by adopting a data replacement mode;
step S3, performing edge prefetching of the second-level edge node; the second-level edge node stores all associated sensing data which are lost in the cache into the cache;
step S2 further includes the steps of:
step S2a, sensing new sensing data by the terminal equipment of the Internet of things at intervals, and uploading the new sensing data to a second-level edge node of a second edge layer;
step S2b, the second-level edge node allocates a frequency value for each sensing data in the cache, the initial value of the frequency value is 0, and the frequency value is more than or equal to 0; every fixed time period, if the sensing data is not accessed, the frequency value of the sensing data is reduced by 1 until the frequency value is reduced to 0; otherwise, increasing the frequency value of the sensing data by 1; the sensing data and the frequency values are in one-to-one correspondence;
step S2c, when the terminal device of the internet of things uploads new sensing data to the second level edge node, the second level edge node first determines whether the cache of the edge node is full: if the cache is not full, the second-level edge node stores the new data in the cache of the second-level edge node, and assigns a frequency value for the new sensing data, wherein the initial value of the frequency value is 0; if the cache is full, deleting the historical sensing data of the terminal equipment of the Internet of things with the minimum frequency value from the cache; deleting the oldest sensing data if the plurality of sensing data have the same frequency value;
step S2 further includes the steps of:
step S2d, the second-level edge node uploads new sensing data of the terminal equipment of the Internet of things to the first-level edge node, and the first-level edge node stores all the sensing data uploaded by the second-level edge node in communication connection with the first-level edge node; the method comprises the steps that a first-level edge node uploads new sensing data of terminal equipment of the Internet of things to a cloud, and the cloud stores all the sensing data uploaded by the first-level edge node in communication connection with the cloud;
step S3 further includes the steps of:
step S3a, the second-level edge node establishes a type relation set; the second-level edge node collects the request data of all the terminal equipment of the Internet of things obtained within a period of time into 1 request data sequence, and the request data at least comprises 4 fields, namely the type of a sensor requested by a user, the position of the sensing data requested by the user, the time requested by the user and the time for generating the sensor data; extracting the request data from each request data sequence according to each association rule and forming a typed request data sequence;
step S3b, when the second level edge node receives a new data request, it will search the sensing data corresponding to the data request in the cache; if the sensing data is not in the cache, the condition of cache missing data occurs, and edge prefetching is triggered:
if the proportion of the same type of request data of the new data request is larger than or equal to the proportion threshold value, finding other same type of request data associated with the same type of request data according to the type relation set, and then adding the same type of request data sequence of the new data request and the associated other same type of request data sequences into a prefetching list;
otherwise, adding the same type request data sequence where the new data request is located into the prefetching list;
step S3c, the second level edge node finds out the request data of cache loss according to all the request data in the prefetch list, and then sends the request data of cache loss in the prefetch list to the first level edge node;
the first-level edge node receives the request data of the second-level edge node, acquires the sensing data lost by the cache from the cloud if the request data lost by the cache exists, stores the sensing data in the current-level cache, and then sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node; the first-level edge node receives the request data of the second-level edge node, and if the request data of the cache loss does not exist, the first-level edge node sends the sensing data corresponding to the request data of the second-level edge node to the second-level edge node;
and the second-level edge node stores all the sensing data corresponding to the request data in the pre-fetching list in the current-level cache, and sends the sensing data corresponding to the new data request to the terminal equipment of the Internet of things.
2. The Internet of things multi-level edge node-based caching and prefetching method according to claim 1,
in step S3a, the association rule is as follows:
1, classifying all request data in each request data sequence, counting the number of the request data of the same type, and dividing the number of the request data of the same type by the total number of the request data in the request data sequence to obtain the proportion of the request data of the same type; when the proportion of the request data of the same type is larger than or equal to the proportion threshold value, grouping all the request data of the type into 1 request data sequence of the same type; a plurality of request data sequences of the same type are extracted from each request data sequence; sequentially processing all the request data sequences to obtain k request data sequences of the same type, wherein k is more than or equal to 2 and is less than or equal to the total number of the types of the request data;
namely: type (A) = number of request data of the same type/total number of request data in the request data sequence, and type (A) is more than or equal to a proportion threshold value; wherein A is a classification type of request data, and type (A) represents the proportion of A type request data;
calculating the association degree of each Type to other types, wherein the association degree (A → B) = Type (A ^ B)/Type (A), and A and B are both classification types of the request data;
the degree of association (a → B) indicates the possibility that the type B request data appears after the type a request data appears;
and presetting a confidence coefficient, and when the association degree (A → B) is higher than the confidence coefficient, pairing the two request data sequences of the same type A and B and including the two request data sequences of the same type A and B into a type relation set.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110553109.8A CN113037872B (en) | 2021-05-20 | 2021-05-20 | Caching and prefetching method based on Internet of things multi-level edge nodes |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110553109.8A CN113037872B (en) | 2021-05-20 | 2021-05-20 | Caching and prefetching method based on Internet of things multi-level edge nodes |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113037872A CN113037872A (en) | 2021-06-25 |
CN113037872B true CN113037872B (en) | 2021-08-10 |
Family
ID=76455418
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110553109.8A Active CN113037872B (en) | 2021-05-20 | 2021-05-20 | Caching and prefetching method based on Internet of things multi-level edge nodes |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113037872B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114691033A (en) * | 2022-02-17 | 2022-07-01 | 阿里巴巴(中国)有限公司 | Data copying method, data storage system control method, device, equipment and medium |
CN115065645B (en) * | 2022-07-25 | 2022-12-13 | 南京曦光信息科技研究院有限公司 | Internet of things terminal sensing function control node |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102439934A (en) * | 2011-01-24 | 2012-05-02 | 华为技术有限公司 | Method and system for managing multilevel caches of edge server in cdn |
CN105849724A (en) * | 2013-12-22 | 2016-08-10 | 交互数字专利控股公司 | Web application acceleration using personalized caching or prerendering |
CN111552564A (en) * | 2020-04-23 | 2020-08-18 | 中南大学 | Task unloading and resource optimization method based on edge cache |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10356197B2 (en) * | 2016-11-21 | 2019-07-16 | Intel Corporation | Data management in an information-centric network |
CN107277125A (en) * | 2017-06-13 | 2017-10-20 | 网宿科技股份有限公司 | File prefetched instruction method for pushing, device and file pre-fetching system |
US10972576B2 (en) * | 2018-11-14 | 2021-04-06 | International Business Machines Corporation | Content acceleration for cross-site resources by CDN |
US11196837B2 (en) * | 2019-03-29 | 2021-12-07 | Intel Corporation | Technologies for multi-tier prefetching in a context-aware edge gateway |
CN110784881B (en) * | 2019-10-23 | 2023-05-02 | 深圳供电局有限公司 | Method, equipment and medium for actively caching multi-level edge nodes of Internet of things terminal |
CN110933692B (en) * | 2019-12-02 | 2021-06-01 | 山东大学 | Optimized cache system based on edge computing framework and application thereof |
CN112256429B (en) * | 2020-10-22 | 2023-09-22 | 莫毓昌 | Data request processing method based on computational graph and industrial cloud edge system |
-
2021
- 2021-05-20 CN CN202110553109.8A patent/CN113037872B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102439934A (en) * | 2011-01-24 | 2012-05-02 | 华为技术有限公司 | Method and system for managing multilevel caches of edge server in cdn |
CN105849724A (en) * | 2013-12-22 | 2016-08-10 | 交互数字专利控股公司 | Web application acceleration using personalized caching or prerendering |
CN111552564A (en) * | 2020-04-23 | 2020-08-18 | 中南大学 | Task unloading and resource optimization method based on edge cache |
Also Published As
Publication number | Publication date |
---|---|
CN113037872A (en) | 2021-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Sun et al. | Mobile data traffic prediction by exploiting time-evolving user mobility patterns | |
EP3032780B1 (en) | Method and apparatus for transmitting messages to users using trajectory-based clustering | |
CN105069703B (en) | A kind of electrical network mass data management method | |
EP3644581B1 (en) | Edge-cloud collaboration system for analyzing internet of things data and operating method thereof | |
CN109951463A (en) | A kind of Internet of Things big data analysis method stored based on stream calculation and novel column | |
CN109391629B (en) | Data processing method of rail transit comprehensive monitoring system | |
CN105307200B (en) | A kind of trajectory-based wireless sensor network multidimensional data rejecting outliers method | |
CN113037872B (en) | Caching and prefetching method based on Internet of things multi-level edge nodes | |
CN111277848B (en) | Method and device for processing interactive messages in live broadcast room, electronic equipment and storage medium | |
CN111629052B (en) | Content caching method, node, equipment and storage medium based on MEC | |
CN111460294A (en) | Message pushing method and device, computer equipment and storage medium | |
WO2022048050A1 (en) | Big data information collection system and usage method | |
Mahmoud et al. | The spatiotemporal data reduction (STDR): An adaptive IoT-Based data reduction approach | |
US20200053514A1 (en) | Collaborative geo-positioning of electronic devices | |
CN112115382B (en) | Data processing method and device, storage medium and electronic device | |
CN117010537A (en) | Target area prediction method, device, computer equipment and storage medium | |
CN113360094A (en) | Data prediction method and device, electronic equipment and storage medium | |
CN105430062B (en) | It is a kind of based on interest-degree of correlation mobile P 2 P network data prefetching method | |
Chen et al. | Prefetching and caching schemes for IoT data in hierarchical edge computing architecture | |
Verma et al. | Latent variables based data estimation for sensing applications | |
CN103944955B (en) | Mobile phone spatial Information Service method based on wireless network | |
CN112597146A (en) | Criminal investigation application system and method based on WIFI big data | |
Sato et al. | Creation of temporal model for prioritized transmission in predictive spatial-monitoring using machine learning | |
CN117992243B (en) | Load balancing method and device for middleware and computer equipment | |
Choi et al. | Locality-based Time Series Data Augmentation for Multi-Sensor Internet Of Things Terminal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |