WO2024021008A1

WO2024021008A1 - Data processing method, device and system, and storage medium

Info

Publication number: WO2024021008A1
Application number: PCT/CN2022/108901
Authority: WO
Inventors: 李佳徽; 王鹏鸿; 马梦瑶; 范晓鹏
Original assignee: 华为技术有限公司
Priority date: 2022-07-29
Filing date: 2022-07-29
Publication date: 2024-02-01

Abstract

Embodiments of the present application provide a data processing method, device and system, and a storage medium. The method may comprise: performing compression encoding processing on collected first initial data to obtain first data; receiving cross-node auxiliary information from a server or a base station; performing redundancy removal processing on the first data according to the cross-node auxiliary information to obtain second data, wherein the cross-node auxiliary information is information related to the first initial data collected by a first node and second initial data collected by a second node; and sending the second data. By using the means, the information related to the first initial data collected by the first node and the second initial data collected by the second node is removed, and then the data subjected to redundancy removal is sent to the server, so that the data transmission volume of the node can be reduced, thereby reducing the bandwidth resource consumption, and improving the transmission robustness.

Description

Data processing methods, devices, systems and storage media

Technical field

The present application relates to the field of communication technology, and in particular, to a data processing method, device, system and storage medium.

Background technique

Distributed single/multi-modal signal processing is a typical scenario in the field of task-oriented source channel joint coding/semantic communication. In this scenario, source acquisition units/nodes/terminals of the same or different modes need to process the collected information and send it to the server/base station for fusion processing to perform specific tasks. The source can be images, videos, audio, sensor parameters and other signals. For example, when one node collects video information and another node collects audio information, they can jointly perform audio-visual analysis tasks, including the detection and identification of audio, visual and audio-visual events, and determine which of these events are visible and possible. audible as well as visible and audible.

In the case of distributed wireless transmission, the sending end in the existing technology uses 2D and 3D deep residual network ResNet and VGGish models respectively to extract audio-visual features, and then extracts them through visual encoders and audio encoders respectively. The features are compressed and encoded, and finally sent out. The sent signal is transmitted to the remote server through the noise channel for processing. At the receiving end, a Transformer model structure is used to jointly decode the received audio and video features, and finally the probability of the event is output through a fully connected layer and the activation function Softmax.

Since this solution does not fully consider the correlation between different nodes and modal sources, for example, the video features collected by the first node and the audio features collected by the second node both correspond to the characteristics of the same environment. At this time, the video features There may be some correlation information between audio features. Then when two nodes independently encode and transmit their own data to the server, the server needs to process the two pieces of data. Since there is some related information in the two pieces of data, it will cause the server to process data redundantly, causing a certain degree of redundancy. Waste of transmission resources and loss of transmission performance.

Contents of the invention

This application discloses a data processing method, device, system and storage medium, which can reduce bandwidth resource consumption and improve the robustness of data transmission.

In the first aspect, embodiments of the present application provide a data processing method, applied to the first node, including:

Perform compression encoding processing on the collected first initial data to obtain the first data;

Receive cross-node assistance information from the server or base station;

The first data is de-redundantly processed according to the cross-node auxiliary information to obtain the second data. The cross-node auxiliary information is the first initial data collected by the first node and the second node. Information related to the second initial data collected;

Send the second data.

In this embodiment of the present application, the first node performs de-redundant processing on the first data based on the received cross-node auxiliary information from the server or base station, and combines it with the first initial data collected by the first node and the second data collected by the second node. Information related to the initial data is removed to obtain second data, and the second data is sent to the server. Using this method, by removing the information related to the first initial data collected by the first node and the second initial data collected by the second node, and then sending the de-redundant data to the server, the data transmission of the node can be reduced. amount, thereby reducing bandwidth resource consumption and improving transmission robustness.

In a possible implementation, the method further includes:

Receive first instruction information from the server or base station, where the first instruction information is used to instruct the first node to collect data in the first mode.

The first modality may be, for example, video, audio, image, etc.

In a possible implementation, the first initial data collected by the first node is data of the first modality.

In a possible implementation, the de-redundancy processing of the first data according to the cross-node auxiliary information to obtain the second data includes:

Both the cross-node auxiliary information and the first data are input into the first preset model for processing to obtain the second data, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, the training is triggered. The first default model.

By triggering the training model when the system's received signal-to-noise ratio change value exceeds the threshold, the model is continuously trained and updated, resulting in better model performance and higher transmission robustness during de-redundancy processing.

In the second aspect, embodiments of the present application provide a data processing method, applied to a server or base station, including:

Receive third data from the second node, where the third data is obtained by the second node after compressing and encoding the collected second initial data;

The third data is processed to obtain cross-node auxiliary information. The cross-node auxiliary information is information related to the first initial data collected by the first node and the second initial data collected by the second node. ;

Send the cross-node assistance information to the first node.

In this embodiment of the present application, the server processes the third data from the second node to obtain the cross-node auxiliary information between the second node and the first node, and then sends the cross-node auxiliary information to the first node. This helps the first node to perform de-redundant processing on the first data based on the received cross-node auxiliary information, which further helps the server to finally receive data from the first node as de-redundant data. Using this method can reduce the data transmission volume of nodes, thereby reducing bandwidth resource consumption, improving transmission robustness, and improving server processing efficiency.

In a possible implementation, the method further includes:

receiving second data from the first node;

Perform fusion processing on the second data and the third data.

In this solution, the data for fusion processing by the server is deredundant, which means that there is no duplicate or related information between the second data and the third data, which improves the processing efficiency of the server.

In a possible implementation, the method further includes:

Send first instruction information to the first node, where the first instruction information is used to instruct the first node to collect data in the first mode;

Send second instruction information to the second node, where the second instruction information is used to instruct the second node to collect data in the second mode.

In a possible implementation, the first initial data collected by the first node is data of the first modality, and the second initial data collected by the second node is data of the second modality. .

In a possible implementation, processing the third data to obtain cross-node auxiliary information includes:

The third data is input into a second preset model for processing to obtain the cross-node auxiliary information, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, training of the second preset model is triggered. .

By triggering the training model when the system's received signal-to-noise ratio change value exceeds the threshold, the model is continuously trained and updated, so that the model performs better during de-redundancy processing and has higher transmission robustness.

In a third aspect, embodiments of the present application provide a data processing device, including:

The first processing module is used to compress and encode the collected first initial data to obtain the first data;

A receiving module used to receive cross-node auxiliary information from the server or base station;

The second processing module is configured to perform de-redundant processing on the first data according to the cross-node auxiliary information to obtain the second data. The cross-node auxiliary information is the same as the third data collected by the first node. Information related to the first initial data and the second initial data collected by the second node;

A sending module, configured to send the second data.

In a possible implementation, the receiving module is also used to:

In a possible implementation, the second processing module is used to:

In a fourth aspect, this application provides a data processing device, including:

A receiving module, configured to receive third data from the second node, where the third data is obtained by the second node after compressing and encoding the collected second initial data;

A processing module, configured to process the third data to obtain cross-node auxiliary information, where the cross-node auxiliary information is related to the first initial data collected by the first node and the second data collected by the second node. Information related to initial data;

A sending module, configured to send the cross-node auxiliary information to the first node.

In a possible implementation, the receiving module is also configured to receive second data from the first node;

The processing module is also used to perform fusion processing on the second data and the third data.

In a possible implementation, the sending module is also used to:

In a possible implementation, the processing module is also used to:

In a fifth aspect, the application provides a data processing device, including a processor and a communication interface. The communication interface is used to receive and/or send data, and/or the communication interface is used to provide the processor with Output and/or output, the processor is used to call computer instructions to implement the method provided by any possible implementation manner of the first aspect, and/or to implement the method provided by any possible implementation manner of the second aspect method.

In a sixth aspect, this application provides a data processing system. The system includes a server or a base station, and also includes a first node, wherein:

The server or base station is configured to implement the method provided in any possible implementation manner of the second aspect; and the first node is configured to implement the method provided in any possible implementation manner of the first aspect.

In a seventh aspect, the present application provides a computer storage medium, including computer instructions. When the computer instructions are run on an electronic device, the electronic device causes the electronic device to execute any possible implementation manner and/or as in the first aspect. The method provided by any possible implementation of the second aspect.

In an eighth aspect, embodiments of the present application provide a computer program product. When the computer program product is run on a computer, it causes the computer to execute any possible implementation manner of the first aspect and/or any possible implementation method of the second aspect. Methods provided by the embodiments.

It can be understood that the above-mentioned device described in the third aspect, the device described in the fourth aspect, the device described in the fifth aspect, the system described in the sixth aspect, the computer storage medium described in the seventh aspect or the third aspect The computer program products described in the eight aspects are all used to execute the method provided in any one of the first aspects and the method provided in any one of the second aspects. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding methods, and will not be described again here.

Description of drawings

The drawings used in the embodiments of this application are introduced below.

Figure 1 is a schematic architectural diagram of a data processing system provided by an embodiment of the present application;

Figure 2 is a schematic flowchart of a data processing method provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a data processing method provided by an embodiment of the present application;

Figure 4 is a schematic diagram of another data processing method provided by an embodiment of the present application;

Figure 5 is a schematic diagram of another data processing method provided by the embodiment of the present application;

Figure 6 is a schematic diagram of a model training method provided by an embodiment of the present application;

Figure 7a is a schematic diagram of the framework structure of Attention _A provided by the embodiment of the present application;

Figure 7b is a schematic diagram of the framework structure of Attention _B provided by the embodiment of the present application;

Figure 8a is a schematic diagram of the frame structure of an Encoder _A1 provided by an embodiment of the present application;

Figure 8b is a schematic diagram of the frame structure of an Encoder _B2 provided by an embodiment of the present application;

Figure 9 is a schematic diagram of the frame structure of Encoder _B1 provided by the embodiment of the present application;

Figure 10a is a schematic diagram of the frame structure of another Encoder _A1 provided by the embodiment of the present application;

Figure 10b is a schematic diagram of the frame structure of another Encoder _B2 provided by the embodiment of the present application;

Figure 10c is a schematic diagram of the frame structure of another Encoder _B1 provided by the embodiment of the present application;

Figure 11 is a schematic structural diagram of a data processing device provided by an embodiment of the present application;

Figure 12 is a schematic structural diagram of another data processing device provided by an embodiment of the present application;

Figure 13 is a schematic structural diagram of yet another data processing device provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. The terms used in the implementation part of the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

Since the existing technology does not fully consider the correlation between different nodes and modal sources, the processing data received by the server is relatively redundant, which will cause a certain degree of waste of transmission resources and loss of transmission performance. In view of this, this application provides a data processing method, device, system and storage medium, which can reduce bandwidth resource consumption and improve the robustness of data transmission.

The system architecture of the embodiment of the present application will be introduced in detail below with reference to the accompanying drawings. Please refer to Figure 1. Figure 1 is a schematic diagram of a data processing system applicable to the embodiment of the present application. The system includes any one of node 1, node 2, server or base station.

The server is a device with centralized computing capabilities, which can be implemented through servers, virtual machines, clouds, or robots. Among them, servers include but are not limited to general computers, dedicated server computers (such as personal computers, servers, UNIX servers, or mid-range servers, etc.), blade servers, etc. When the server is implemented by a server, as shown in Figure 1, the number of servers it contains can be one or multiple (such as a server cluster). A virtual machine is a virtualized computing module. The cloud is a software platform that uses application virtualization technology, which allows one or more software and applications to be developed and run in an independent virtualized environment. Optionally, the cloud can be deployed on public cloud, private cloud, or hybrid cloud.

It should be noted that the embodiments of this application take a server as an example for explanation, and it can also be any other implementation of the server, and this solution does not limit this.

Access network equipment refers to the radio access network (RAN) node (or equipment) that connects terminals to the wireless network, and can also be called a base station. Currently, some examples of RAN nodes are: evolved Node B (gNB), transmission reception point (TRP), evolved Node B (evolved Node B, eNB), radio network controller, RNC), Node B (Node B, NB), base station controller (BSC), base transceiver station (BTS), home base station (for example, home evolved NodeB, or home Node B, HNB) , baseband unit (base band unit, BBU), or wireless fidelity (wireless fidelity, Wifi) access point (access point, AP), etc. In addition, in a network structure, access network equipment may include centralized unit (CU) nodes, distributed unit (DU) nodes, or RAN equipment including CU nodes and DU nodes. The RAN equipment including CU nodes and DU nodes separates the protocol layer of gnb in the NR system. Some of the protocol layer functions are centralized and controlled by the CU. The remaining part or all of the protocol layer functions are distributed in the DU and centralized by the CU. Control DU. The functions of CU can be implemented by one entity or by different entities. For example, the functions of the CU can be further divided, for example, the control plane (CP) and the user plane (UP) are separated, that is, the control plane (CU-CP) and the CU user plane (CU-UP) of the CU. For example, CU-CP and CU-UP can be implemented by different functional entities, and the CU-CP and CU-UP can be coupled with DU to jointly complete the functions of the base station.

In a possible implementation, node 1 and node 2 may be terminal devices. The terminal equipment (Terminal Equipment) in the embodiment of this application can also be called a terminal, user equipment (User Equipment, UE), mobile station (Mobile Station, MS), mobile terminal (Mobile Terminal, MT), etc. The terminal device can be a mobile phone (mobile phone), a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (Virtual Reality, VR) terminal device, an augmented reality (Augmented Reality, AR) terminal device, or an industrial control (Industrial Control) ), wireless terminals in Self Driving, wireless terminals in Remote Medical Surgery, wireless terminals in Smart Grid, wireless terminals in Transportation Safety Terminals, wireless terminals in Smart City, wireless terminals in Smart Home, etc.

In another possible implementation, node 1 and node 2 may be information source collection units. For example, both Node 1 and Node 2 can be used to collect signals such as images, videos, audios, and sensor parameters. Node 1 and node 2 are also used to encode the collected signals (data) and send the encoded data to the server/base station via the wireless transmission channel.

It should be noted that terminal equipment and terminal equipment may partially overlap. For example, mobile phones can collect images, videos, audio and other signals, and this solution does not limit this.

The server/base station stores and fuses the data received from node 1 and node 2, and then performs one or more tasks. For example, performing audio-visual analysis tasks based on the received video data and audio data, specifically including the detection and identification of audio, visual and audio-visual events, and determining which of these events are visible, audible, and both visible and audible; further, You can also determine the start time and end time of each event.

It should be noted that the embodiment of this application takes two nodes as an example for description. It can also be three nodes, four nodes, etc. This solution does not strictly limit this.

Furthermore, this solution can be oriented to future cellular standards and Wi-Fi standards, and the products involved can include future cellular and Wi-Fi related products, for example, it can be applied to: mobile phones; tablets/watches that access cellular and Wi-Fi /Notebooks/Other IoT devices; base stations/wireless routers and other devices.

In the embodiment of this application, the node obtains the first data by compressing and encoding the collected initial data; and then performs de-redundant processing on the first data based on the cross-node auxiliary information received from the server/base station, To obtain the second data, the cross-node auxiliary information is the relevant information between the initial data collected by the first node and the initial data collected by the second node; and send the second data to the server/base station. In this way, when the node transmits data, it does not need to transmit repeated and related information between the two nodes, which can reduce bandwidth resource consumption and improve the robustness of data transmission.

The architecture of the embodiment of the present application is described above, and the method of the embodiment of the present application is introduced in detail below.

Refer to FIG. 2 , which is a schematic flow chart of a data processing method provided by an embodiment of the present application. Optionally, this method can be applied to the aforementioned data processing system, such as the data processing system shown in Figure 1 . The data processing method shown in Figure 2 may include steps 201-211. It should be understood that, for convenience of description, this application is described through the sequence 201-211, and is not intended to limit the execution to the above sequence. The embodiments of the present application do not limit the execution sequence, execution time, number of executions, etc. of one or more of the above steps. In the following, the execution subject of steps 201-202 of the data processing method is the second node (such as the information source collection unit), the execution subject of steps 203-205, 210, and 211 is the server, and the execution subject of steps 206-209 is the first node (such as The information source acquisition unit) is described as an example, and this application is also applicable to other execution entities. Steps 201-211 are as follows:

201. The second node performs compression encoding on the collected second initial data to obtain the third data;

Among them, the coding methods of the compression coding process can be divided into three categories: (1) According to the statistical characteristics of the information source, methods such as predictive coding, transform coding, vector quantization coding, sub-band coding, and neural network coding are used. (2) According to the visual characteristics of the human eye, image coding based on directional filtering, image contour-ethical coding based on image, coding based on wavelet analysis and other methods are used. (3) According to the characteristics of the transferred scene: use methods such as fractal coding and model-based coding.

202. Send the third data to the server;

The second node may send the third data to the server through a wireless transmission channel.

203. The server receives the third data from the second node;

204. Process the third data to obtain cross-node auxiliary information. The cross-node auxiliary information is related to the first initial data collected by the first node and the second initial data collected by the second node. Information;

The cross-node auxiliary information can be understood as information related to the first initial data collected by the first node and the second initial data collected by the second node.

The relevant information can be understood as repeated information with the same value in the first initial data and the second initial data, or related information.

Optionally, a certain part of the data collected by node 1 and node 2 is the same, or is related. For example, node 1 collects the audio information of environment 1, and node 2 collects the video information of environment 1. This relevant information means that the data collected by node 1 and node 2 both correspond to the same environment. For another example, node 1 collects sky information during the day, and node 2 collects sky information at night. The relevant information is that the data collected by node 1 and node 2 both correspond to the sky.

In a possible implementation manner, cross-node auxiliary information is obtained by inputting the third data into a preset model for processing.

For example, the preset model is a trained model. Through neural network training, a model that can obtain cross-node auxiliary information is obtained.

In a possible implementation, when the change value of the system's received signal-to-noise ratio exceeds a threshold, training of the preset model is triggered. By continuously training the model, the model performs better.

For an introduction to this model, please refer to the records of the encoder Encoder _A2 in subsequent embodiments, and will not be described again here.

205. Send the cross-node auxiliary information to the first node;

The server sends the extracted cross-node auxiliary information between the first node and the second node to the first node, so that the first node can perform redundancy processing.

The server processes the third data from the second node to obtain cross-node auxiliary information between the second node and the first node, and then sends the cross-node auxiliary information to the first node. This helps the first node to perform de-redundant processing on the first data based on the received cross-node auxiliary information, which further helps the server to finally receive data from the first node as de-redundant data. Using this method can reduce the data transmission volume of nodes, thereby reducing bandwidth resource consumption, improving transmission robustness, and improving server processing efficiency.

206. The first node performs compression encoding on the collected first initial data to obtain the first data;

The first data is data obtained by compressing and encoding the initial data.

In a possible implementation, the execution order of step 206 may be before steps 202-205, which is not strictly limited in this solution.

In a possible implementation, before steps 201 and 206, the method further includes:

The server sends first instruction information to the first node, where the first instruction information is used to instruct the first node to collect data in the first mode;

Correspondingly, each node receives the instruction information sent by the server respectively. That is, the first initial data collected by the first node is data of the first modality, and the second initial data collected by the second node is data of the second modality.

Optionally, the data in the first mode or the data in the second mode may be, for example, one or more of audio signals, video signals, image signals, or environmental monitoring sensor signals.

Among them, the server can directly send the above instruction information, or send the instruction information based on the request sent by each node. This solution does not impose strict restrictions on this.

In a possible implementation, the first node sends a request to the server to instruct the server to perform initial configuration.

In another possible implementation, the server may perform initial configuration directly.

Wherein, the initial configuration may include the above-mentioned first instruction information to instruct the first node to collect data of the first mode.

The embodiment of this application only introduces the initial configuration including instructing the first node to collect data of the first mode as an example. The initial configuration may also include other information.

In one possible implementation, initial configuration may include:

a) Notify the compression parameters, including modal identification (which can indicate different modalities to achieve single/multi-modal processing), and the final layer network output dimension or length.

b) Feature transmission data type indication, that is, the specific format of the data code stream when each part of the feature is transmitted, such as directly sending the output parameters of the network (such as two adjacent real numbers forming a complex symbol, resulting in a string of complex symbols), Or the value after quantization (requires channel coding and modulation to obtain a series of complex symbols).

c) Indicate timestamp information, which can include specifying the maximum time difference (the corresponding data packet will be discarded when timeout occurs, a value of 0 means no setting), period information (0 means no period is set, greater than 0 means setting the period), etc.

d) The configured signaling can be in the following format (the node can actively set it and then request the server/base station, or the server/base station can configure it directly):

modal identifier

Output dimensions or length

type of data

maximum time difference

Period information

Quantization bit width

Optionally, the quantization bit width only needs to be sent when the data type is "quantized value". The above parameters can be sent in one signaling, or they can be encapsulated in different signaling and sent at different times. This solution does not limit this.

After the above configuration, the number of transmission resources required for a single inference can be calculated through the output dimension or length and data type, which can correspond to the number of symbols, for example.

For example, the data type is "directly sending the output parameters of the network", and the number of transmission resources = output dimension or length/2. For another example, if the data type is "quantized value", the number of transmission resources = output dimension or length * quantization bit width, or the number of transmission resources = output dimension or length * channel code rate, or the number of transmission resources = output dimension Or length*number of modulation bits, etc.

Among them, the general feature transmission format of the above b) ~ d) is:

Timestamp code stream

Characteristic data length

Characteristic data stream

For the timestamp code stream, adding a timestamp is used to indicate the current time of sending characteristic data, which facilitates data synchronization between multiple nodes. For example, the absolute time value t can be recorded (the time interval can be one or more time slots, subframes, symbols, etc., and is not specifically limited; the length of the interval between t and t-1 can be determined by the sending end and the receiving end through negotiation, or it can Determine the interval length by predefinition or preconfiguration), or set a certain period T0 (T0>0), and only record mod(t, T0), mod represents the remainder operation.

The timestamp can also be converted into binary data and then channel coded and modulated to obtain a complex signal. For example, low code rate channel coding and low-order modulation can be performed to ensure reliable transmission.

For the characteristic data code stream, according to the initially configured compression parameters and data type, the length of the processed code stream to be sent (a string of complex symbols) is consistent with the number of transmission resources. Optionally, a characteristic data length field (which can be channel coded and modulated together with the timestamp) can be added before the characteristic data code stream to indicate the actual length of the code stream.

The above corresponding code streams can be transmitted through PUSCH (applicable to nodes 1 and 2 → server/base station) or PDSCH (applicable to server/base station → nodes 1 and 2). In addition, they can also be sent at the physical layer after MAC packetization. .

Among them, if the data timestamp received by the server/base station or node 2 and the current time exceed the initially configured maximum time difference, the corresponding data packet will be discarded and no subsequent sending operation will be performed.

207. Receive cross-node auxiliary information from the server;

208. Perform redundancy processing on the first data according to the cross-node auxiliary information to obtain the second data;

The redundancy removal process may include, for example, removing cross-node auxiliary information from the first data, or removing information related to the cross-node auxiliary information from the first data, thereby obtaining second data after redundancy removal. That is to say, the second data and the first data have no duplicate information, or in other words, no related information.

In a possible implementation, the node inputs the received cross-node auxiliary information from the server and the first data into a first preset model for processing to obtain the second data.

The first preset model is obtained after training. When the change value of the system's received signal-to-noise ratio exceeds a threshold, training of the first preset model is triggered.

For an introduction to this model, please refer to the records of the encoder Encoder _B2 and Attention _B in subsequent embodiments, and will not be described again here.

209. Send the second data to the server;

Then, the first node sends the second data to the server through the wireless transmission channel, so that the server can process the corresponding task.

Wherein, the first node performs de-redundant processing on the first data based on the received cross-node auxiliary information from the server or base station, and correlates the first initial data collected by the first node and the second initial data collected by the second node. The information is removed to obtain second data, and the second data is sent to the server. Using this method, by removing the information related to the first initial data collected by the first node and the second initial data collected by the second node, and then sending the de-redundant data to the server, the data transmission of the node can be reduced. amount, thereby reducing bandwidth resource consumption and improving transmission robustness.

210. The server receives the second data from the first node;

211. Perform fusion processing on the second data and the third data.

Among them, the server obtains the cross-node auxiliary information from the third data sent by the second node, and sends the cross-node auxiliary information to the first node so that the first node can perform de-redundancy processing and obtain the second data, and then the server will send the cross-node auxiliary information from The second data from the first node and the third data from the second node are fused and processed, and corresponding tasks are performed. That is to say, the data finally processed by the server for fusion is deredundant data without duplicate or related information. This not only reduces the bandwidth resources used by the first node to transmit data, but also improves the processing efficiency of the server.

In this embodiment of the present application, the server processes the third data from the second node to obtain the cross-node auxiliary information between the second node and the first node, and then sends the cross-node auxiliary information to the first node. The first node performs de-redundant processing on the first data according to the received cross-node auxiliary information, removes the relevant information between the first node and the second node, obtains the second data, and sends the second data to the server. The second data, in turn, helps the server to finally receive the data from the first node as deredundant data. Using this method, the first node removes the relevant information between it and the second node, and then sends the de-redundant data to the server. This can reduce the data transmission volume of the node, thereby reducing bandwidth resource consumption and improving It improves transmission robustness and improves server processing efficiency.

Refer to FIG. 3 , which is a schematic diagram of a data processing method provided by an embodiment of the present application. Optionally, this method can be applied to the aforementioned data processing system, such as the data processing system shown in Figure 1 .

In a possible implementation, in the distributed single/multi-modal signal transmission scenario as shown in Figure 3, node 1 (for example, it can be the aforementioned second node) passes the data or features of the specific modality through the encoder. Encoder _A1 compresses and encodes the data and sends it to the server/base station through the wireless channel.

After the server/base station receives the data sent by Node 1, it obtains the cross-node auxiliary information in the data based on the encoder Encoder _A2 . The cross-node auxiliary information can be part of the data sent by Node 1, or it can be all the data. . Then, the server/base station sends the cross-node assistance information to node 2 (which may be the aforementioned first node, for example). Node 2 compresses and encodes the data or features of the specific modality through the encoder Encoder _B1 , decodes the received cross-node auxiliary information through the decoder _A2 , and then combines the data encoded by Encoder _B1 with the decoded data by Decoder _A2 . The cross-node auxiliary information is operated by the attention mechanism to guide the feature encoding, that is, the feature extraction shown in Figure 3 is performed to obtain the deredundant processed data, which is encoded by the encoder Encoder _B2 , and then the obtained The data is sent to the server/base station.

The server/base station also decodes the data sent by Node 1 through Decoder _A1 , and performs an attention mechanism operation on the decoded data to better restore the data sent by Node 1. Moreover, the server/base station decodes the received data from node 2, and then fuses the data of the two nodes to perform corresponding tasks.

After the above processing, compared with the current method of directly transmitting data from two nodes, in this solution one node only transmits data that is not relevant to the data of the other node, which can reduce the amount of feature transmission. and bandwidth resource occupancy. Further, when the total bandwidth occupancy is similar, the accuracy of the final task execution can be improved.

Refer to FIG. 4 , which is a schematic diagram of another data processing method provided by an embodiment of the present application. The difference between this example and the example shown in Figure 3 is that in the example shown in Figure 4, node 1 (for example, it can be the aforementioned second node) collects and transmits modal data with smaller dimensions (such as audio, environmental monitoring sensor signals), Node 2 (for example, it may be the aforementioned first node) collects and transmits larger-dimensional modal data (for example, images, video signals). Among them, the data collected by node 1 can be compressed into fewer symbols for transmission, so it only takes up less bandwidth resources. The server/base station receives the low-dimensional modal data, and then sends part or all of the data to node 2, such as a high-dimensional modal data collection terminal. Node 2 performs attention operations on the data sent by the server/base station and the high-dimensional modal data extracted by the node 2 to guide feature encoding, thereby effectively reducing the transmission volume and bandwidth resource occupation of high-dimensional modal data.

Based on the data processing methods shown in Figure 2, Figure 3 and Figure 4, embodiments of the present application also provide a data processing method. Among them, as shown in Figure 5, the method includes: first performing initial configuration. The initial configuration, for example, configures compression parameters, transmission data types, resource allocation, timestamp information, etc. For an introduction to this initial configuration, please refer to the records in the foregoing embodiments and will not be described again here.

Then, Node 1 compresses the collected data/features into smaller dimensions and transmits them to the remote server/base station through the noise channel. The remote server/base station receives the characteristics sent by node 1, obtains the cross-node auxiliary information, and then transmits the cross-node auxiliary information to node 2.

Node 2 compresses and encodes the collected data/features, decodes the received cross-node auxiliary information, and then performs attention calculation on the two to extract features that are not related to node 1, and converts the calculated node 2 Features are transmitted to the remote server/base station for further information fusion and calculation.

In a possible implementation, for node 1, based on the network output (real number) of Encoder _A1 , the characteristic data (complex signal) is obtained after processing according to the configured data type. By combining the characteristic data and channel coding and modulation The final timestamp code stream and characteristic data length are concatenated and sent to the server/base station.

For the server/base station, it is based on the network output (real number) of Encoder _A2 , which is processed according to the configured data type to obtain characteristic data (complex signal); the characteristic data and the timestamp code stream and characteristic data after channel coding and modulation are The length is concatenated and sent to node 2.

For node 2, based on the network output (real number) of Encoder _B2 , the characteristic data (complex signal) is obtained after processing according to the configured data type; the characteristic data is combined with the timestamp code stream and characteristic data length after channel coding and modulation. After splicing, it is sent to the server/base station.

For each corresponding encoder and decoder in the above data processing method, for example, the encoders Encoder _A1 , Encoder _A2 , Encoder _B1 , Encoder _B2 shown in Figure 3, and the decoders Decoder _A1 , Decoder _A2 , and Decoder _B2 can respectively correspond to Neural network model. It should be noted that the embodiments of the present application do not limit the number and types of models. As shown in Figure 6, this embodiment of the present application also provides a model training method. The training process includes: initial configuration, regularly sending pilot sequences to monitor changes in received signal-to-noise ratio (SNR), and model training/updating. It includes the following steps:

1. Initial configuration. It can mainly configure the compression parameters, transmission data types, resource allocation, timestamp information, training parameters, etc. of each model.

For initial configuration, such as node requesting from the server/base station, or direct configuration by the server/base station, this solution does not place strict restrictions on this.

This configuration can include:

a) Notify the compression parameters, including modal identification (which can indicate different modalities and achieve single/multi-modal processing), the final layer network output dimension or length, and the reverse gradient dimension or length;

b) Feature transmission data type indication, that is, the specific format of the data stream when each part of the feature is transmitted, such as directly sending the output parameters of the network (two adjacent real values form a complex symbol), or quantized values (requires channel coding and modulation before sending);

c) Indicate training timestamp information, mainly specifying the maximum time difference (the corresponding data packet will be discarded when timeout occurs, a value of 0 means no setting), period information (0 means no period is set, greater than 0 means setting the period), etc.;

d) Indicate training parameters, including conventional model training parameters (such as update rate, batch size/number, maximum number of iterations, loss function threshold, etc.) as well as SNR change threshold and monitoring period for monitoring model updates;

e) The configured signaling can be in the following format (the node can actively set it and then request it from the server/base station, or the server/base station can configure it directly):

f) The quantization bit width only needs to be sent when the data type is "quantized value";

g) The above parameters can be sent in one signaling, or they can be encapsulated in different signaling and sent at different times;

h) After the above configuration, the number of transmission resources required to train a single batch can be calculated through the output dimension or length and data type:

For example, the data type is "directly sending the output parameters of the network": number of transmission resources = sum of forward and reverse lengths/2;

For another example, the data type is "quantized value": number of transmission resources = forward and reverse length and * quantization bit width/channel code rate/number of modulation bits;

Among them: the sum of forward and reverse lengths = output dimension or length * batch size + gradient dimension or length.

2. Each node sends a pilot sequence to the server/base station respectively, and the server/base station sends a pilot sequence to node 2 to estimate the SNR of the relevant transmission link. The purpose of this processing is: under different SNR, the compression rate or coding rate of the transmitted data will become an important factor affecting the accuracy of the server or base station's task execution. Therefore, the compression rate or coding rate of the transmitted data can be adjusted by the calculated SNR. Code rate; in addition, different models can be selected for online inference based on different SNR.

3. Monitoring mechanism for model updates. Monitor the SNR estimated in step 2. When the SNR changes too much, trigger and update the training model in time.

For example, when the system's receiving SNR change value exceeds 3dB, 6dB, etc., model training is triggered. Of course, it can also be other values, and this solution does not impose strict restrictions on this.

For steps 2 and 3, the existing pilot sequence can be reused and regular detection can be performed according to the initially configured monitoring period.

In one possible implementation, by recording the initial SNR estimate value SNR0 and the current time SNR estimate value SNR1, if |SNR1–SNR0| ≥ the set SNR change threshold, an update of the model is triggered, and SNR0=SNR1 is updated. .

4. The server/base station sends training requests or instructions to node 1 and node 2. Each node prepares data after receiving the instructions, starts training after completing data synchronization, and sends a training completion instruction after the training is completed.

For model training/updating, where:

a) After the server/base station sends the training request/instruction, the data set is synchronized.

b) Training data interaction: mainly includes forward transmission (sending features) and reverse transmission (sending gradients).

Among them, the forward transmission includes: transmitting the characteristics of node 1 to the server/base station, the server/base station sending the characteristics of node 1 to node 2, and transmitting the characteristics of node 2 to the server/base station. For example, the following characteristic transmission format can be used:

Timestamp code stream

Characteristic data length

Characteristic data stream

Reverse transmission includes: the server/base station sends gradient information to node 2 (for updating the node 2 encoder), node 2 sends gradient information to the server/base station (for updating the cross-node auxiliary information encoder), the server/base station Send gradient information to node 1 (for node 1 encoder update).

c) The gradient information can be sent in the following transmission format, for example:

Training timestamp code stream

Gradient data length

gradient data stream

Among them, training timestamp code stream: used to indicate the time of the current gradient information to facilitate data synchronization between multiple nodes;

Gradient data code stream: According to the initially configured compression parameters and data type, the processed code stream to be sent (the optional processing method configured in the initial stage can be consistent with the corresponding processing method of the configured characteristic transmission data type: directly convert real numbers The gradient values are spliced into a complex signal; or quantized first, followed by channel coding and modulation to obtain a complex signal); optionally, the gradient data length field can be added before the gradient data code stream (channel coding and modulation are performed together with the timestamp ), indicating the actual length of the code stream;

The corresponding code stream can be transmitted through PUSCH (applicable to nodes 1 and 2 → server/base station) or PDSCH (applicable to server/base station → nodes 2 and 1). In addition, it can also be sent at the physical layer after being packaged by MAC.

The embodiment of the present application also provides an architecture of an encoder and a decoder. Figure 7a shows the frame structure of Attention _A shown in Figure 3, and Figure 7b shows the frame structure of Attention _B shown in Figure 3. Take the scenario where node 1 collects audio signals and node 2 collects video signals as an example. The network structure shown in Figure 3 contains multiple codecs and Attention. The input of Attention _A is the audio feature Y _A decoded by the server/base station. The data size can be 32*10*64, which is then calculated by self-attention and a feed-forward network (FFN). Get the output. Similarly, the input of Attention _B is the visual feature x _B of the video capture terminal and the decoded audio feature Z _A fed back to the video capture terminal (the data sizes of x _B and Z _A are both 32*10*64), and then through crossover Attention calculation and a feed-forward neural network get the output result (the size of the output result is 32*10*64).

Figure 8a shows a network structure of the audio encoder Encoder _A1 , and Figure 8b shows a network structure of the visual encoder Encoder _B2 . Among them, the audio encoder Encoder _A1 uses a residual fully connected network to reduce the loss of audio features during the compression process, where the data input dimension is 32*10*128. The visual encoder Encoder _B2 splices the encoded visual feature x _B and the x _B * after audio-visual attention calculation (the data dimensions of x _B * and x _B are both 32*10*64), and then Use a layer of fully connected network for compression (the dimension of the compressed data is 32*10*48). FC is the fully connected layer (Fully Connected, FC).

Figure 9 shows a network structure of the visual encoder Encoder _B1 . Encoder _B1 compresses and encodes the read video image features and timing features. The dimension size of the image features is 32*80*2048, and the dimension size of the timing features is 32*10*512. Its input includes image features and temporal features. First, an average pooling operation is performed on the image features (the data output after pooling is 32*10*2048), and then the data is compressed through a layer of full connection. By compressing the temporal features through a layer of fully connected layers, and then splicing the features on both sides, the final output (size 32*10*64) is obtained through a layer of fully connected layers.

For example, Decoder _A1 and Decoder _B2 can respectively use a layer of fully connected networks to expand the dimensions of audio features and video features respectively, so that Y _A * and Y _B have the same dimensions and are both 32*10*64 in size. Encoder _A2 transmits and encodes the feedback audio features. It can use a layer of fully connected network to calculate the input features. The input and output sizes are both 32*10*26. Decoder _A2 uses a layer of fully connected network to mainly implement dimension expansion of the delivered audio auxiliary information, that is, to make the dimension of Z _A consistent with the dimension of x _B for subsequent Attention operations (its input dimension is 32*10*26 , the output dimension is 32*10*64).

The embodiment of the present application also provides another encoder and decoder architecture. Take the scenario where node 1 collects image signals and node 2 collects video signals as an example. Figure 10a shows another frame structure of the encoder Encoder _A1 shown in Figure 3. Figure 10b shows the frame structure of the encoder Encoder _B2 shown in Figure 3. Figure 10c shows the encoder shown in Figure 3. The frame structure of Encoder _B1 . Among them, Encoder _A1 is a residual convolutional network, which compresses and codes the collected image features. The residual design is used to reduce the loss of information. Encoder _B2 splices the attention-operated information and visual features through a layer of convolutional network (for example, the convolution kernel size is 1*1) to achieve the fusion of multiple modal information. Encoder _B1 compresses and encodes the image features and timing features of the read video. The compression process is implemented by using a 3*3 convolution kernel and a 1*1 convolution kernel respectively. Optionally, the dimension size of the input data and output data of the above encoder is consistent with the previous embodiment.

Similarly, Decoder _A1 and Decoder _B2 can also use other network structures, such as long short-term memory (LSTM) networks to achieve compression and dimension expansion of input features. This solution does not impose strict restrictions on this.

It should be noted that in the various embodiments of this application, if there are no special instructions or logical conflicts, the terminology and/or descriptions between the various embodiments are consistent and can be referenced to each other. The technical features in different embodiments New embodiments can be formed based on their internal logical relationships.

The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below. It can be understood that in each device embodiment of the present application, the division of multiple units or modules is only a logical division based on functions and does not limit the specific structure of the device. In specific implementation, some of the functional modules may be subdivided into more small functional modules, and some of the functional modules may also be combined into one functional module. However, no matter whether these functional modules are subdivided or combined, the roughly what the device performs is The process is the same. For example, some devices include a receiving unit and a transmitting unit. In some designs, the sending unit and the receiving unit can also be integrated into a communication unit, and the communication unit can realize the functions realized by the receiving unit and the sending unit. Usually, each unit corresponds to its own program code (or program instruction). When the program codes corresponding to these units are run on the processor, the unit is controlled by the processing unit and executes the corresponding process to achieve the corresponding function. .

Embodiments of the present application also provide a device for implementing any of the above methods. For example, a data processing device is provided that includes modules (or means) for implementing each step performed by the first node in any of the above methods. As another example, another data processing device is also provided, including modules (or means) used to implement each step performed by the server or base station in any of the above methods.

For example, refer to FIG. 11 , which is a schematic structural diagram of a data processing device provided by an embodiment of the present application. The data processing device is used to implement the aforementioned data processing method, such as the data processing method executed by the first node shown in FIG. 2 .

As shown in Figure 11, the device may include a first processing module 1101, a receiving module 1102, a second processing module 1103 and a sending module 1104, specifically as follows:

The first processing module 1101 is used to compress and encode the collected first initial data to obtain the first data;

The receiving module 1102 is used to receive cross-node auxiliary information from the server or base station;

The second processing module 1103 is configured to perform de-redundant processing on the first data according to the cross-node auxiliary information to obtain second data, where the cross-node auxiliary information is related to the data collected by the first node. Information related to the first initial data and the second initial data collected by the second node;

The sending module 1104 is used to send the second data.

In a possible implementation, the receiving module 1102 is also used to:

In a possible implementation, the second processing module 1103 is used to:

For a detailed introduction to the processing corresponding to this module, please refer to the corresponding records in the foregoing embodiments, and will not be described again here.

Refer to FIG. 12 , which is a schematic structural diagram of another data processing device provided by an embodiment of the present application. The data processing device is used to implement the aforementioned data processing method, such as the data processing method executed by the server shown in FIG. 2 .

As shown in Figure 12, the device may include a receiving module 1201, a processing module 1202 and a sending module 1203, specifically as follows:

The receiving module 1201 is configured to receive third data from the second node, where the third data is obtained by the second node after compressing and encoding the collected second initial data;

The processing module 1202 is configured to process the third data to obtain cross-node auxiliary information, where the cross-node auxiliary information is related to the first initial data collected by the first node and the third data collected by the second node. 2. Information related to initial data;

Sending module 1203, configured to send the cross-node auxiliary information to the first node.

In a possible implementation, the receiving module 1201 is also configured to receive second data from the first node;

The processing module 1202 is also used to perform fusion processing on the second data and the third data.

In a possible implementation, the sending module 1203 is also used to:

In a possible implementation, the processing module 1202 is also used to:

It should be understood that the division of each module in each device above is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or it can also be physically separated. In addition, the modules in the data processing device can be implemented in the form of the processor calling software; for example, the data processing device includes a processor, the processor is connected to a memory, instructions are stored in the memory, and the processor calls the instructions stored in the memory to achieve the above. Any method or function of each module of the device is implemented, where the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the modules in the device can be implemented in the form of hardware circuits, and some or all of the unit functions can be implemented through the design of the hardware circuits, which can be understood as one or more processors; for example, in one implementation, The hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units through the design of the logical relationships of the components in the circuit; for another example, in another implementation, the hardware circuit is It can be realized by programmable logic device (PLD), taking field programmable gate array (FPGA) as an example, which can include a large number of logic gate circuits, and the logic gate circuits are configured through configuration files. connection relationships, thereby realizing the functions of some or all of the above units. All modules of the above device may be fully implemented by the processor calling software, or all may be implemented by hardware circuits, or part of the modules may be implemented by the processor calling software, and the remaining part may be implemented by hardware circuits.

Refer to FIG. 13 , which is a schematic diagram of the hardware structure of another data processing device provided by an embodiment of the present application. The data processing device 1300 shown in FIG. 13 (the device 1300 may specifically be a computer device) includes a memory 1301, a processor 1302, a communication interface 1303 and a bus 1304. Among them, the memory 1301, the processor 1302, and the communication interface 1303 implement communication connections between each other through the bus 1304.

The memory 1301 may be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM).

The memory 1301 can store programs. When the program stored in the memory 1301 is executed by the processor 1302, the processor 1302 and the communication interface 1303 are used to execute various steps of the data processing method in the embodiment of the present application.

The processor 1302 is a circuit with signal processing capabilities. In one implementation, the processor 1302 can be a circuit with the ability to read and run instructions, such as a central processing unit (CPU), a microprocessor, a graphics processor (graphics processor) processing unit (GPU) (can be understood as a microprocessor), or digital signal processor (digital signal processor, DSP), etc.; in another implementation, the processor 1302 can implement certain functions through the logical relationship of the hardware circuit , the logical relationship of the hardware circuit is fixed or reconfigurable, for example, the processor 1302 implements a hardware circuit for an ASIC or a programmable logic device PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading the configuration file and realizing the hardware circuit configuration can be understood as the process of the processor loading instructions to realize the functions of some or all of the above modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing Unit (deep learning processing unit, DPU), etc. The processor 1302 is used to execute relevant programs to implement the functions required to be performed by the units in the data processing device in the embodiment of the present application, or to execute the data processing method in the method embodiment of the present application.

It can be seen that each module in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA , or a combination of at least two of these processor forms.

In addition, all or part of the modules in the above device may be integrated together, or may be implemented independently. In one implementation, these modules are integrated together and implemented as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each module of the device. The at least one processor may be of different types, such as a CPU and an FPGA, or a CPU and an artificial intelligence processor. CPU and GPU etc.

The communication interface 1303 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 1300 and other devices or communication networks. For example, data can be obtained through communication interface 1303.

Bus 1304 may include a path that carries information between various components of device 1300 (eg, memory 1301, processor 1302, communication interface 1303).

It should be noted that although the device 1300 shown in Figure 13 only shows a memory, a processor, and a communication interface, during specific implementation, those skilled in the art will understand that the device 1300 also includes other devices necessary for normal operation. . At the same time, based on specific needs, those skilled in the art should understand that the device 1300 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device 1300 may only include components necessary to implement the embodiments of the present application, and does not necessarily include all components shown in FIG. 13 .

Embodiments of the present application also provide a data processing system. The system includes a server or a base station, and also includes a first node, wherein: the server or base station is used to implement the data processing method provided in the second aspect. One or more steps; the first node is used to implement one or more steps in the data processing method provided by the first aspect.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores instructions, which when run on a computer or processor, cause the computer or processor to execute one of the above methods. or multiple steps.

An embodiment of the present application also provides a computer program product containing instructions. When the computer program product is run on a computer or processor, the computer or processor is caused to perform one or more steps in any of the above methods.

It should be understood that in the description of this application, unless otherwise stated, "/" indicates that the related objects are in an "or" relationship. For example, A/B can mean A or B; where A and B can be singular numbers. Or plural. Furthermore, in the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions thereof refers to any combination of these items, including any combination of a single item (items) or a plurality of items (items). For example, at least one of a, b, or c can mean: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple . In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as “first” and “second” are used to distinguish identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the number and execution order, and words such as "first" and "second" do not limit the number and execution order. At the same time, in the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in the embodiments of the present application is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner that is easier to understand.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the division of this unit is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not used. implement. The mutual coupling, direct coupling, or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical, or other forms.

A unit described as a separate component may or may not be physically separate. A component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means A website site, computer, server or data center for transmission. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The available media may be read-only memory (ROM), random access memory (RAM), or magnetic media, such as floppy disks, hard disks, tapes, disks, or optical media, such as , digital versatile disc (digital versatile disc, DVD), or semiconductor media, such as solid state drive (solid state disk, SSD), etc.

The above are only specific implementation modes of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present application shall be covered by this application. within the protection scope of the application embodiment. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

A data processing method, applied to the first node, is characterized by including:

Perform compression encoding processing on the collected first initial data to obtain the first data;

Receive cross-node assistance information from the server or base station;

The first data is de-redundantly processed according to the cross-node auxiliary information to obtain the second data. The cross-node auxiliary information is the first initial data collected by the first node and the second node. Information related to the second initial data collected;

Send the second data.
The method of claim 1, further comprising:

Receive first instruction information from the server or base station, where the first instruction information is used to instruct the first node to collect data in the first mode.
The method of claim 2, wherein the first initial data collected by the first node is data of the first modality.
The method according to any one of claims 1 to 3, characterized in that: performing redundancy processing on the first data according to the cross-node auxiliary information to obtain the second data includes:

Both the cross-node auxiliary information and the first data are input into the first preset model for processing to obtain the second data, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, the training is triggered. The first default model.
A data processing method, applied to servers or base stations, is characterized by including:

Receive third data from the second node, where the third data is obtained by the second node after compressing and encoding the collected second initial data;

The third data is processed to obtain cross-node auxiliary information. The cross-node auxiliary information is information related to the first initial data collected by the first node and the second initial data collected by the second node. ;

Send the cross-node assistance information to the first node.
The method of claim 5, further comprising:

receiving second data from the first node;

Perform fusion processing on the second data and the third data.
The method according to claim 5 or 6, characterized in that, the method further includes:

Send first instruction information to the first node, where the first instruction information is used to instruct the first node to collect data in the first mode;

Send second instruction information to the second node, where the second instruction information is used to instruct the second node to collect data in the second mode.
The method of claim 7, wherein the first initial data collected by the first node is data of a first modality, and the second initial data collected by the second node is data of a second modality. Modal data.
The method according to any one of claims 5 to 8, characterized in that processing the third data to obtain cross-node auxiliary information includes:

The third data is input into a second preset model for processing to obtain the cross-node auxiliary information, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, training of the second preset model is triggered. .
A data processing device, characterized in that it includes:

The first processing module is used to compress and encode the collected first initial data to obtain the first data;

A receiving module used to receive cross-node auxiliary information from the server or base station;

The second processing module is configured to perform de-redundant processing on the first data according to the cross-node auxiliary information to obtain the second data. The cross-node auxiliary information is the same as the third data collected by the first node. Information related to the first initial data and the second initial data collected by the second node;

A sending module, configured to send the second data.
The device according to claim 10, characterized in that the receiving module is also used to:

Receive first instruction information from the server or base station, where the first instruction information is used to instruct the first node to collect data in the first mode.
The device according to claim 11, wherein the first initial data collected by the first node is data of the first modality.
The device according to any one of claims 10 to 12, characterized in that the second processing module is used for:

Both the cross-node auxiliary information and the first data are input into the first preset model for processing to obtain the second data, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, the training is triggered. The first default model.
A data processing device including:

A receiving module, configured to receive third data from the second node, where the third data is obtained by the second node after compressing and encoding the collected second initial data;

A processing module, configured to process the third data to obtain cross-node auxiliary information, where the cross-node auxiliary information is related to the first initial data collected by the first node and the second data collected by the second node. Information related to initial data;

A sending module, configured to send the cross-node auxiliary information to the first node.
The device according to claim 14, wherein the receiving module is further configured to receive second data from the first node;

The processing module is also used to perform fusion processing on the second data and the third data.
The device according to claim 14 or 15, characterized in that the sending module is also used to:

Send first instruction information to the first node, where the first instruction information is used to instruct the first node to collect data in the first mode;

Send second instruction information to the second node, where the second instruction information is used to instruct the second node to collect data in the second mode.
The device according to claim 16, characterized in that the first initial data collected by the first node is data of a first modality, and the second initial data collected by the second node is second Modal data.
The device according to any one of claims 14 to 17, characterized in that the processing module is also used to:

The third data is input into a second preset model for processing to obtain the cross-node auxiliary information, wherein when the change value of the system's received signal-to-noise ratio exceeds a threshold, training of the second preset model is triggered. .
A data processing device, characterized in that it includes a processor and a communication interface, the communication interface is used to receive and/or send data, and/or the communication interface is used to provide output to the processor and/or Output, the processor is used to call computer instructions to implement the method described in any one of claims 1-4, and/or to implement the method described in any one of claims 5-9.
A data processing system, characterized in that the system includes a server or a base station, and also includes a first node, wherein:

The server or base station is used to implement the data processing method as described in any one of claims 5-9; the first node is used to implement the data processing method as described in any one of claims 1-4.
A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and the computer program is used to implement the method described in any one of claims 1-4, and/or, implement The method according to any one of claims 5-9.