CN118331931B

CN118331931B - Data tracing method, device and equipment

Info

Publication number: CN118331931B
Application number: CN202410771147.4A
Authority: CN
Inventors: 李常宝; 艾中良; 刘忠麟; 王书龙; 顾平莉; 袁媛; 李云鹏
Original assignee: CETC 15 Research Institute
Current assignee: CETC 15 Research Institute
Priority date: 2024-06-14
Filing date: 2024-06-14
Publication date: 2024-08-30
Anticipated expiration: 2044-06-14
Also published as: CN118331931A

Abstract

The embodiment of the specification discloses a data tracing method, a device and equipment, comprising the following steps: determining a tracing task type based on the acquired tracing task request, wherein the tracing task type comprises a data level tracing and a record level tracing; initializing data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link, wherein the data to be traced is a file to be traced or a library table to be traced; based on the initialization tracing link, accessing the current level tracing source data and sequentially extracting the upper level tracing source metadata of the current level tracing source data; and if the upper-level tracing source metadata of the current-level tracing source data are extracted, taking the next-level tracing source data of the current-level tracing source data as updated current-level tracing source data until all tracing sources of the data to be traced are accessed, and forming a constructed tracing link, wherein the constructed tracing link comprises a constructed data-level tracing link and a constructed record-level tracing link.

Description

Data tracing method, device and equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data tracing method, apparatus, and device.

Background

At present, the mainstream data processing mode mostly adopts a mode of data+flow or data+model, and the data result after multistage multi-round processing cannot be usually interpreted or traced, which affects the credibility of the result. Especially in the professional data processing fields of information analysis, protocol analysis and the like, the credibility of the processing result directly influences the feasibility of service operation. Therefore, it is important to perform data tracing.

Aiming at data tracing, a data blood-source method is mainly adopted at present. The data blood-edge method is mainly used for tracing the upstream and downstream relation in the data treatment flow, only supports the tracing granularity of file/library table levels, the tracing content is limited to basic conversion operations such as data extraction, fusion and screening, and the data analysis, studying and judgment processes are not involved, so that the data blood-edge method does not have the interpretability of data processing results, cannot realize tracing of fine granularity such as record-level granularity, cannot timely find the deficiency and change of upstream and downstream basis in a link, and has low tracing reliability.

Based on this, a data tracing method is required.

Disclosure of Invention

The embodiment of the specification provides a data tracing method, a device and equipment, which are used for solving the following technical problems: in the prior art, the data tracing method does not have the interpretability of the data processing result, cannot realize tracing of fine granularity such as recording grade granularity, cannot timely find the deficiency and change of upstream and downstream references in a link, and has low tracing reliability.

In order to solve the above technical problems, the embodiments of the present specification are implemented as follows:

The embodiment of the specification provides a data tracing method, which comprises the following steps:

determining a tracing task type based on the acquired tracing task request, wherein the tracing task type comprises a data level tracing and a record level tracing;

initializing data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link, wherein the data to be traced is a file to be traced or a library table to be traced;

based on the initialization tracing link, accessing the current level tracing source data and sequentially extracting the upper level tracing source metadata of the current level tracing source data;

And if the upper-level tracing source metadata of the current-level tracing source data are extracted, taking the next-level tracing source data of the current-level tracing source data as updated current-level tracing source data until all tracing sources of the data to be traced are accessed, and forming a constructed tracing link, wherein the constructed tracing link comprises a constructed data-level tracing link and a constructed record-level tracing link.

The embodiment of the specification also provides a data tracing device, which comprises:

The determining module is used for determining the type of the traceability task based on the acquired traceability task request, wherein the traceability task type comprises data-level traceability and record-level traceability;

The initialization module is used for initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialization tracing link, wherein the data to be traced is a file to be traced or a library table to be traced;

the access module is used for accessing the current level tracing source data based on the initialization tracing link and sequentially extracting the upper level tracing source metadata of the current level tracing source data;

And the first construction module of the traceability link is used for taking the next-stage traceability data of the current-stage traceability data as updated current-stage traceability data if the upper-stage traceability metadata of the current-stage traceability data are extracted, and forming a constructed traceability link until all traceability sources of the data to be traced are accessed, wherein the constructed traceability link comprises a constructed data-stage traceability link and a constructed record-stage traceability link.

The embodiment of the specification also provides an electronic device, including:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executable by the at least one processor to enable the at least one processor to:

The data tracing method provided by the embodiment of the specification is based on the organization and utilization methods of tracing source data and tracing cue data, records and associates key processing cues in the data processing process, can support the generation basis of specific processing results, performs multi-level link restoration, can trace the generation basis of the data processing results under the condition that normal data processing flow is not affected, realizes tracing of data processing results with different granularity at data level and record level, can be suitable for a macroscopic processing process restoration scene and a specific result precise tracing scene, and judges the deletion and change of the upper-level tracing source data and tracing cue data through the tracing verification of each level of tracing source data and tracing cue data in the tracing process, and ensures the credibility and the availability of a tracing link, thereby realizing the tracing of credible and precise data processing results.

Drawings

In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some of the embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a system architecture of a data tracing method according to an embodiment of the present disclosure;

fig. 2 is a schematic flow chart of a data tracing method according to an embodiment of the present disclosure;

Fig. 3 is a schematic diagram of a process for forming to-be-traced data according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of a forming process of yet another to-be-traced data according to an embodiment of the present disclosure;

Fig. 5 is a frame diagram of a data tracing method provided in the embodiment of the present disclosure;

fig. 6 is a schematic diagram of a data tracing device according to an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

The data blood-edge method is mainly used for recording and restoring the upstream and downstream relations in the data treatment process to form a data flow path and supports visual display analysis, so that data management staff is helped to better understand the data processing process, track data circulation, trace data sources and the like. However, the traceability of the data blood-edge method is limited to basic conversion operations such as data extraction, fusion and screening, and the analysis and judgment process of the data is not involved, so that the method has no interpretability of the data processing result.

Based on the above, the embodiment of the specification provides a data tracing method, which is based on organization and utilization methods of tracing source data and tracing cue data, records and associates key processing cues in a data processing process, and traces the generation basis of a data processing result under the condition that a normal data processing flow is not affected, so that the tracing of a reliable and accurate data processing result is realized.

Fig. 1 is a schematic diagram of a system architecture of a data tracing method according to an embodiment of the present disclosure. As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The terminal devices 101, 102, 103 interact with the server 105 via the network 104 to receive or send messages or the like. Various client applications can be installed on the terminal devices 101, 102, 103. Such as tracing of data processing.

The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be a variety of special purpose or general purpose electronic devices including, but not limited to, smartphones, tablets, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services) or as a single software or software module.

The server 105 may be a server providing various services, such as a back-end server providing services for client applications installed on the terminal devices 101, 102, 103. For example, the server may perform tracing based on the data processing so as to display the tracing result of the data processing on the terminal device servers 101, 102, 103, or may perform tracing of the data processing so as to display the tracing result of the data processing on the terminal devices 101, 102, 103.

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When server 105 is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or as a single software or software module.

Fig. 2 is a flow chart of a data tracing method according to an embodiment of the present disclosure. From the program perspective, the execution subject of the flow may be a program installed on an application server or an application terminal. It is understood that the method may be performed by any apparatus, device, platform, cluster of devices having computing, processing capabilities. As shown in fig. 2, the data tracing method includes:

Step S201: and determining the type of the traceability task based on the acquired traceability task request, wherein the traceability task type comprises data-level traceability and record-level traceability.

In the embodiment of the present disclosure, the traceable task request is a request for tracing the data processing result. Specifically, the data processing result is a processing result of numeric data or a processing result of text data, and may be converted into a processing result of numeric data or may be converted into a processing result of text data.

In the embodiment of the present specification, the data level tracing refers to tracing of the data itself, and the record level tracing refers to tracing of each record included in the data.

In one embodiment of the present disclosure, the trace-source task request includes a trace-source task type field, and the trace-source task type may be determined according to the trace-source task type field in the trace-source task request.

Step S203: and initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link, wherein the data to be traced is a file to be traced or a library table to be traced.

Initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link, and specifically comprising the following steps:

If the type of the tracing task is the data-level tracing, determining the file to be traced or the library table to be traced, and forming the initialization tracing link;

and if the trace-source task type is the record-level trace-source, determining the file to be traced or the library table to be traced and the result record to be traced, and forming the initialization trace-source link.

In this embodiment of the present disclosure, the data to be traced is numeric data or text data, and may be converted into numeric data or data that may be converted into text data;

The data to be traced at least comprises: tracing source and trace source clues;

The tracing head comprises: the method comprises the following steps of data content description, data structure description, data access address, data generator, data version number and version description, data storage format, data tracing source access address and superior tracing source metadata;

the trace-source cue comprises: the system comprises a tracing result chain, a tracing check code and a tracing information body access address.

In the embodiment of the present specification, the trace header is used to describe data-level summary information of the data itself. The content of the tracing source H is as follows: h= < info, str, data, auth, ver, ston, lnk, sup >, wherein: info is the description of the content of the data; str is the description of the data structure (field name, field type, field definition, field meaning); data is the access address of the data; auth: the data generator; ver is the version number and version description of the data; the sto is the data storage format; lnk is the access address of the source of the data trace; sup is metadata of a superior tracing source, and comprises a superior tracing head replicator, a superior tracing head check code and a superior data tracing source access address, wherein a plurality of organization modes of { < H _i,H_i check code, H _i access address >, …, < H _j,H_j check code and H _j access address > are adopted.

In the embodiment of the present disclosure, the trace source thread L is used for describing record-level abstract information of a data recording process, where the record-level abstract information includes a trace source result chain and trace source check code-level trace source information body access addresses.

In the present description embodiment, it is assumed that there are m result records in total:

L= { L ₁,l₂,…,l_i,…,l_m},l_i is the trace-source cue of the ith result record v _i;

The tracing clues of the ith result record v _i include: l _i = < cha, code, addr >, wherein: cha is a traceability result chain, which is used for recording traceability information of the result record v _i, and the organization mode is as follows: < result record v _i, trace number w, trace chain ：{<H₁,k₁>,…,<H_i,k_j>…,<H_m,k_n>}>,H_i is trace header where trace information body is located, k _j is specific trace information in trace source, in specific embodiment, result record v _i is numerical data or text data);

The code is a tracing check code for checking whether tracing clues are changed. In one embodiment of the present disclosure, code=md5 (k ₁,k₂,…,k_j), and of course, the traceability check code may be MD5 or other check forms;

addr is a tracing information body access address and is used for recording the specific position of the tracing information body input by the result record v _i, the organization mode is {<H₁,k₁,p₁>,…,<H_j,k_j,p_k>…,<H_m,k_n,p_v>},H_i is a tracing head of the tracing information body, k _j is specific tracing information in the tracing source, p _k is the position information of k _j in H _i, the two conditions are divided into a file and a database, the file can adopt offset position information, and the database can adopt column sequence number information.

In the embodiment of the present disclosure, the data to be traced is data formed in the data processing process. Specifically, the data to be traced is formed in the recording process of the file/library table.

To further understand the data to be traced mentioned in the embodiments of the present specification. The detailed process of forming the data to be traced will be described below in connection with the recording process of the original/result file or library table. Fig. 3 is a schematic diagram of a process for forming data to be traced according to the embodiment of the present disclosure, as shown in fig. 3, where the process for forming data to be traced is implemented based on two judging steps in a data processing process, specifically, in the data processing process, whether a trace header file exists is checked; if the trace source file exists, the trace source data is updated, and if the trace source file does not exist, the trace source data is created; further determining whether a trace source file exists; if the trace source cue file exists, the trace source cue data is updated, and if the trace source cue file does not exist, whether the trace source cue file is an original input file is further judged; and if the trace source file is the original input file, finishing the record of the data to be traced. And if the trace source cue file is not the original input file, the trace source cue data is created.

In order to further understand the process of forming the data to be traced provided in the embodiment of the present disclosure, fig. 4 is a schematic diagram of another process of forming the data to be traced provided in the embodiment of the present disclosure.

As shown in fig. 4, in the embodiment of the present disclosure, during a data processing process, it is checked whether a trace header file exists, and if the trace header file exists, the trace header data is updated, which specifically includes: if the trace source data exist, updating content description, structure description, author information, update version, storage format and access address information of the trace source file of the file/library table corresponding to the data processing process based on the trace source file of the file/library table corresponding to the data processing process, and updating the upper-level trace source metadata to finish trace source data updating and storage.

In the embodiment of the present disclosure, during a data processing process, whether a trace header file exists is checked, and if the trace header file does not exist, trace source data is created, which specifically includes: if the trace source head file does not exist, the trace source head file of the file/library table corresponding to the data processing process is built, the content description, the structure description, the author information, the updated version, the storage format and the access address information of the trace source head file of the file/library table corresponding to the data processing process are built, and meanwhile, the upper-level trace source metadata are built, so that the trace source data are built and stored.

In this embodiment of the present disclosure, after constructing trace source data or updating trace source data, it is further determined whether a trace source file exists, and if the trace source file exists, the trace source file updating is performed, which specifically includes:

and checking whether a trace source cue file exists in the original/result file or the library table in the data processing process, if the trace source cue file exists, automatically updating a trace source result chain, a trace source information body access address and a trace source check code based on each processing result in the original/result file or the library table in the data processing process, and finishing trace source cue data updating and storing.

Checking whether a trace source cue file exists or not according to an original/result file or a library table in the data processing process, and if the trace source cue file does not exist, further judging whether the trace source cue file is an original input file or not; and if the trace source file is the original input file, finishing the record of the data to be traced. If the trace source file is not the original input file, the trace source file is created, which specifically includes: and automatically creating a traceability result chain and a traceability information body access address based on the original/result file in the data processing process or each processing result in the library table, and generating a traceability check code to finish the creation and storage of traceability cue data.

Step S205: based on the initialized tracing link, the primary tracing source data are accessed, and the superior tracing source metadata of the primary tracing source data are sequentially extracted.

In the embodiment of the specification, the sup field in the tracing source, namely the upper-level tracing source metadata, is extracted by initializing the tracing head included in the tracing link.

In this embodiment of the present disclosure, based on the initialization tracing link, access the present level tracing source data and sequentially extract the upper level tracing source metadata of the present level tracing source data, which specifically includes:

based on the initialized tracing link, judging whether the data to be traced completes the access of all tracing sources;

If the data to be traced does not finish all tracing source access, the current level tracing source data of the data to be traced is accessed, and the upper level tracing source metadata of the current level tracing source data are sequentially extracted.

And judging whether the upper-level tracing source metadata of the current-level tracing source data are extracted.

In this embodiment of the present disclosure, based on the initializing a tracing link, determining whether the data to be traced completes access of all tracing sources includes:

And traversing all superior tracing heads of the current level tracing source corresponding to the data to be traced based on the initialized tracing link, and judging whether the data to be traced completes the access of all tracing sources.

In this embodiment of the present disclosure, the superior tracing source refers to all superior tracing heads of the present-level tracing source, and specifically includes: the last level of this level is traced to the source, and the last second level of this level is traced to source.

In the embodiment of the specification, judging whether the extraction of the upper-level tracing metadata of the present-level tracing data is completed refers to judging whether to traverse all the upper-level tracing metadata of the present-level tracing data, i.e. whether to traverse all sup of the present-level tracing data.

Step S207: and if the upper-level tracing source metadata of the current-level tracing source data are extracted, taking the next-level tracing source data of the current-level tracing source data as updated current-level tracing source data until all tracing sources of the data to be traced are accessed, and forming a constructed tracing link, wherein the constructed tracing link comprises a constructed data-level tracing link and a constructed record-level tracing link.

In this embodiment of the present disclosure, if the extraction of the metadata of the previous level tracing source data of the present level tracing source data is completed, the next level tracing source data of the present level tracing source data is used as updated present level tracing source data until all the tracing sources of the data to be traced are accessed, so as to form a constructed tracing link, where the constructed tracing link includes a constructed data level tracing link and a constructed record level tracing link. That is, from step S201 to step S207, the same method is adopted regardless of whether a data-level trace-source link is constructed or a record-level trace-source link is constructed.

In this embodiment of the present specification, the next-level tracing data refers to next-level tracing head data adjacent to the present-level tracing head data, that is, tracing head data of the next level of the present-level tracing head data.

Step S209: if the metadata of the upper-level tracing source of the current-level tracing source data are not extracted, accessing the upper-level tracing source to form a tracing link node, and adding the tracing link node to a tracing link;

If the type of the tracing task is data level tracing, determining whether a file/library table corresponding to the current level tracing source of the tracing link is original input data tracing source, and determining updated current level tracing source data until all tracing sources of the data to be traced are accessed, and completing construction of the data level tracing link;

And if the trace task type is record level trace source, determining updated home level trace source data based on whether trace source cue data of the trace source link is extracted and whether the trace source cue data is original input data trace source or not until all trace source sources of the data to be traced are accessed, and completing construction of the record level trace source link.

In this embodiment of the present disclosure, if the metadata of the upstream trace source of the present-stage trace source data is not extracted, the upstream trace source is accessed to form a trace source link node, and the trace source link node is added to a trace source link, which specifically includes:

If the upper-level tracing source metadata of the current-level tracing source data are not extracted, accessing the corresponding upper-level tracing head based on the access address of the upper-level tracing source metadata of the current-level tracing source data, and after the access is successful, checking through the upper-level tracing head check code corresponding to the access address of the upper-level tracing source, if the check is successful, forming a tracing link node, and adding the tracing link node to a tracing link; if the verification is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data; and if the access is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data to form a tracing link node, and adding the tracing link node to a tracing link.

In this embodiment of the present disclosure, if the metadata of the previous level trace source of the present level trace source data is not extracted, the previous level trace source is accessed to form a trace source link node, and the trace source link node is added to a trace source link, where the trace source link node formed includes a data level trace source link node and/or a record level trace source link node. For data-level tracing, if the metadata of the upper-level tracing source of the current-level tracing source data are not extracted, accessing the upper-level tracing source to form a data-level tracing link node, and adding the data-level tracing link node to a data-level tracing link; and for record level tracing, if the metadata of the upper level tracing source of the current level tracing source data is not extracted, accessing the upper level tracing source to form a record level tracing link node, and adding the record level tracing link node to the record level tracing link.

In this embodiment of the present disclosure, if the type of the trace task is data level tracing, determining whether a file/library table corresponding to a current level trace source of the trace link is an original input data trace source, and determining the updated current level trace source data until all trace sources of the data to be traced are accessed, so as to complete construction of the data level trace link, including:

If the tracing task type is data level tracing and the current level tracing source data is original input tracing head data, the current level tracing source data is used as updated current level tracing source data.

And if the trace task type is data level trace source, and the trace source data of the present level is not trace source data of original input, taking the trace source data of the next level of trace source data of the present level as the trace source data of the updated present level.

In this embodiment of the present disclosure, if the trace task type is record level trace, determining the updated trace source data based on whether trace source cue data of the trace source link is extracted and whether the trace source data is an original input data trace source until all trace sources of the data to be traced are accessed, and completing construction of the record level trace source link, including:

If the type of the tracing task is record level tracing and the tracing cue data of the tracing link is already extracted, determining the updated current level tracing source data based on whether the current level tracing source data of the tracing link is original input tracing head data or not until all tracing sources of the data to be traced are accessed, and completing construction of the record level tracing link;

if the type of the tracing task is record-level tracing and the tracing cue data of the tracing link is not extracted, sequentially accessing each tracing information body of the tracing cue data of the tracing link, checking tracing check codes corresponding to the tracing cue data of the tracing link, if successful, forming a data-level tracing link node, adding the tracing link node to the tracing link, and accessing the current-level tracing cue data of the tracing link until the fact that the tracing cue data of the tracing link are all extracted is determined; and if the trace source link fails, accessing the trace source cue data of the trace source link until the trace source cue data of the trace source link are completely extracted.

In this embodiment of the present disclosure, if the type of the trace task is record level tracing and trace data of the trace link has been extracted, determining the updated trace source data based on whether the trace source data of the trace link is original input trace head data, until all trace sources of the data to be traced are accessed, and completing construction of the record level trace link, including:

if the type of the tracing task is record level tracing and the tracing cue data of the tracing link is already extracted and the current level tracing source data is original input tracing head data, the current level tracing source data is used as updated current level tracing source data until all tracing sources of the data to be traced are accessed, and the construction of the record level tracing link is completed;

If the trace source data of the trace source link is extracted, and the trace source data of the current stage is not the trace source data of the original input, the trace source data of the next stage of trace source data of the current stage is used as updated trace source data of the current stage until all trace sources of the data to be traced are accessed, and the construction of the record stage trace source link is completed.

In order to further understand the data tracing method provided in the embodiments of the present disclosure, fig. 5 is a frame diagram of the data tracing method provided in the embodiments of the present disclosure.

As shown in fig. 5, the type of the trace-source task is determined according to the trace-source task request. If the data level tracing is the data level tracing, initializing a data level tracing link; based on the initialized data-level tracing link, judging whether the data to be traced completes the access of all tracing sources; if the data to be traced finishes the access of all tracing sources, the construction of the data-level tracing link is finished; if the data to be traced does not finish all tracing source access, the current level tracing source data of the data to be traced is accessed, and the upper level tracing source metadata of the current level tracing source data are sequentially extracted. Judging whether the upper-level tracing source metadata of the current-level tracing source data are extracted or not; if the upper-level tracing source metadata of the current-level tracing source data are extracted, the next-level tracing source data of the current-level tracing source data are used as updated current-level tracing source data until all tracing sources of the data to be traced are accessed, and a constructed data-level tracing link is formed; if the upper-level tracing source metadata of the present-level tracing source data are not extracted, accessing the corresponding upper-level tracing source based on the access address of the upper-level tracing source metadata of the present-level tracing source data, and after the access is successful, checking through the upper-level tracing source check code corresponding to the access address of the upper-level tracing source, if the check is successful, forming a data-level tracing link node, and adding the tracing link node to a data-level tracing link; if the verification is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data; and if the access is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data to form a data-level tracing link node, and adding the tracing link node to the data-level tracing link. After the data-level tracing link node is added to the data-level tracing link, further judging whether the current-level tracing source data is original input tracing head data or not, and if the current-level tracing source data is the original input tracing head data, taking the current-level tracing source data as updated current-level tracing source data. And if the current level tracing source data is not the original input tracing source data, taking the next level tracing source data of the current level tracing source data as the updated current level tracing source data.

If the record level tracing is the record level tracing, initializing a record level tracing link; based on the initialized record-level tracing link, judging whether the data to be traced completes the access of all tracing sources; if the data to be traced finishes the access of all tracing sources, finishing the construction of a record-level tracing link; if the data to be traced does not finish all tracing source access, the current level tracing source data of the data to be traced is accessed, and the upper level tracing source metadata of the current level tracing source data are sequentially extracted. Judging whether the upper-level tracing source metadata of the current-level tracing source data are extracted or not; and if the upper-level tracing source metadata of the current-level tracing source data are extracted, taking the next-level tracing source data of the current-level tracing source data as updated current-level tracing source data until all tracing sources of the data to be traced are visited, and forming a constructed record-level tracing link. If the upper-level tracing source metadata of the current-level tracing source data are not extracted, the upper-level tracing source is accessed, and if the upper-level tracing source is not accessible, the next-level tracing source metadata are sequentially extracted until the upper-level tracing source metadata of the current-level tracing source data are not extracted. If the metadata of the upper-level tracing source of the current-level tracing source data are not extracted and the upper-level tracing source is accessible, after the access is successful, checking through an upper-level tracing head check code corresponding to an access address of the upper-level tracing source, if the check is successful, forming a tracing link node, and adding the tracing link node to a tracing link; if the verification is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data; and if the access is unsuccessful, sequentially extracting the upper-level tracing source metadata of the current-level tracing source data to form a tracing link node, and adding the tracing link node to a tracing link.

Further, whether the tracing cue data of the tracing link are extracted is judged. If the extraction is finished, determining the updated home level tracing source data based on whether the home level tracing source data of the tracing link is original input tracing head data or not until all tracing source accesses of the data to be traced are completed, and completing construction of a record level tracing link.

If the trace source cue data of the trace source link are not extracted, each trace source information body of the trace source cue data of the trace source link is sequentially accessed, the trace source check codes corresponding to the trace source cue data of the trace source link are checked, if the trace source check codes are successful, a trace source link node of the data level is formed, the trace source link node is added to the trace source link, and the current trace source cue data of the trace source link is accessed until the fact that the trace source cue data of the trace source link are all extracted is determined. And if the trace source link fails, accessing the trace source cue data of the trace source link until the trace source cue data of the trace source link are completely extracted.

The foregoing details a data tracing method, and accordingly, the present disclosure further provides a data tracing device, as shown in fig. 6. Fig. 6 is a schematic diagram of a data tracing device provided in an embodiment of the present disclosure, where the data tracing device includes:

The determining module 601 determines a tracing task type based on the obtained tracing task request, wherein the tracing task type comprises a data level tracing and a record level tracing;

The initialization module 603 initializes the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialization tracing link, wherein the data to be traced is a file to be traced or a library table to be traced;

the access module 605 accesses the current level tracing source data and sequentially extracts the upper level tracing source metadata of the current level tracing source data based on the initialization tracing link;

and the first traceability link construction module 607 is configured to take the next traceability source data of the current traceability source data as updated current traceability source data if the previous traceability source metadata of the current traceability source data is extracted, until all traceability sources of the data to be traced are accessed, and form a constructed traceability link, where the constructed traceability link includes a constructed data-level traceability link and a constructed record-level traceability link.

In this embodiment of the present disclosure, the data tracing apparatus further includes:

The trace source link second construction module 609 accesses the upper level trace source to form trace source link nodes if the upper level trace source metadata of the present level trace source data is not extracted, and adds the trace source link nodes to the trace source links.

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, electronic devices, non-volatile computer storage medium embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to the description of the method embodiments.

The apparatus, the electronic device, the nonvolatile computer storage medium and the method provided in the embodiments of the present disclosure correspond to each other, and therefore, the apparatus, the electronic device, the nonvolatile computer storage medium also have similar beneficial technical effects as those of the corresponding method, and since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, the electronic device, the nonvolatile computer storage medium are not described here again.

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable GATE ARRAY, FPGA)) is an integrated circuit whose logic functions are determined by user programming of the device. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler (logic compiler)" software, which is similar to the software compiler used in program development and writing, and the original code before being compiled is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but HDL is not just one, but a plurality of kinds, such as ABEL（Advanced Boolean Expression Language）、AHDL（Altera Hardware Description Language）、Confluence、CUPL（Cornell University Programming Language）、HDCal、JHDL（Java Hardware Description Language）、Lava、Lola、MyHDL、PALASM、RHDL（Ruby Hardware Description Language）, and VHDL (Very-High-SPEED INTEGRATED Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing one or more embodiments of the present description.

It will be appreciated by those skilled in the art that the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is by way of example only and is not intended as limiting the application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims

1. The data tracing method is characterized by comprising the following steps of:

Determining a tracing task type based on the acquired tracing task request, wherein the tracing task type comprises a data level tracing and a record level tracing, the data level tracing is the tracing of the data, and the record level tracing is the tracing of each record included in the data.

Initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link, wherein the data to be traced is a file to be traced or a library table to be traced, and the data to be traced at least comprises: tracing source and trace source clues, the formation process of the data to be traced includes: in the data processing process, checking whether a source tracing header file exists or not; if the trace source file exists, the trace source data is updated, and if the trace source file does not exist, the trace source data is created; further determining whether a trace source file exists; if the trace source cue file exists, the trace source cue data is updated, and if the trace source cue file does not exist, whether the trace source cue file is an original input file is further judged; if the trace source file is the original input file, finishing the record of the data to be traced; if the trace source cue file is not the original input file, the trace source cue data is created;

Based on the initialization tracing link, accessing the primary tracing source data and sequentially extracting the superior tracing source metadata of the primary tracing source data, wherein the superior tracing source metadata are sup fields in tracing sources included in the initialization tracing link, and the superior tracing source data refer to all superior tracing source data of the primary tracing source;

If the upper-level tracing source metadata of the current-level tracing source data are extracted, taking the next-level tracing source data of the current-level tracing source data as updated current-level tracing source data until all tracing sources of the data to be traced are accessed, and forming a constructed tracing link, wherein the constructed tracing link comprises a constructed data-level tracing link and a constructed record-level tracing link, and the next-level tracing source data refer to next-level tracing source data adjacent to the current-level tracing source data;

If the upper-level tracing source metadata of the current-level tracing source data are not extracted, accessing the upper-level tracing head to form a tracing link node, and adding the tracing link node to a tracing link; if the type of the tracing task is data level tracing, determining whether a file/library table corresponding to the current level tracing source of the tracing link is original input data tracing source, and determining updated current level tracing source data until all tracing sources of the data to be traced are accessed, and completing construction of the data level tracing link; and if the trace task type is record level trace source, determining updated home level trace source data based on whether trace source cue data of the trace source link is extracted and whether the trace source cue data is original input data trace source or not until all trace source sources of the data to be traced are accessed, and completing construction of the record level trace source link.

2. The data tracing method of claim 1, wherein said method further comprises:

If the metadata of the upper-level tracing source of the current-level tracing source data are not extracted, accessing the upper-level tracing source to form a tracing link node, and adding the tracing link node to a tracing link;

3. The data tracing method according to claim 2, wherein if the metadata of the upper level tracing source of the present level tracing source data is not extracted, accessing the upper level tracing source to form a tracing link node, and adding the tracing link node to a tracing link, specifically comprising:

4. The data tracing method according to claim 2, wherein if the tracing task type is data level tracing, determining whether a file/library table corresponding to a current level tracing source of the tracing link is an original input data tracing source, determining the updated current level tracing source data, and completing the construction of the data level tracing link until all tracing sources of the data to be traced are accessed, includes:

If the type of the tracing task is data-level tracing and the current-level tracing source data is original input tracing head data, the current-level tracing source data is used as updated current-level tracing source data.

5. The data tracing method according to claim 2, wherein if the tracing task type is record level tracing, determining the updated home level tracing source data based on whether tracing cue data of the tracing link is completely extracted and whether the tracing cue data is an original input data tracing source, until all tracing source accesses of the data to be traced are completed, and completing construction of a record level tracing link, specifically comprising:

6. The data tracing method according to claim 5, wherein if the trace task type is record level tracing and trace cue data of the trace link has been extracted, determining the updated current level tracing source data based on whether the current level tracing source data of the trace link is original input trace head data or not until all tracing sources of the data to be traced are accessed, and completing construction of the record level trace link, specifically comprising:

7. The data tracing method according to claim 1, wherein initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialized tracing link specifically includes:

8. The data tracing method according to claim 1, wherein the data to be traced is numerical data or text data, and data which can be converted into numerical data or text data;

9. The utility model provides a data traceability device which characterized in that, data traceability device includes:

The initialization module is used for initializing the data to be traced corresponding to the tracing task request according to the tracing task type to form an initialization tracing link, wherein the data to be traced is a file to be traced or a library table to be traced, and the data to be traced at least comprises: tracing source and trace source clues, the formation process of the data to be traced includes: in the data processing process, checking whether a source tracing header file exists or not; if the trace source file exists, the trace source data is updated, and if the trace source file does not exist, the trace source data is created; further determining whether a trace source file exists; if the trace source cue file exists, the trace source cue data is updated, and if the trace source cue file does not exist, whether the trace source cue file is an original input file is further judged; if the trace source file is the original input file, finishing the record of the data to be traced; if the trace source cue file is not the original input file, the trace source cue data is created;

The access module is used for accessing the primary tracing source data and sequentially extracting the superior tracing source metadata of the primary tracing source data based on the initialization tracing link, wherein the superior tracing source metadata are sup fields in tracing sources included in the initialization tracing link, and the superior tracing source data refer to all superior tracing source data of the primary tracing source;

The first construction module of the traceability link takes next-stage traceability data of the current-stage traceability data as updated current-stage traceability data if the upper-stage traceability metadata of the current-stage traceability data are extracted, and the constructed traceability link is formed by completing the access of all traceability sources of the data to be traced, wherein the constructed traceability link comprises a constructed data-stage traceability link and a constructed record-stage traceability link, and the next-stage traceability data refer to next-stage traceability data adjacent to the current-stage traceability data;

10. An electronic device, comprising:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,