CN109032536B - Data storage method, device, system and equipment based on distributed cluster system - Google Patents
Data storage method, device, system and equipment based on distributed cluster system Download PDFInfo
- Publication number
- CN109032536B CN109032536B CN201811011203.5A CN201811011203A CN109032536B CN 109032536 B CN109032536 B CN 109032536B CN 201811011203 A CN201811011203 A CN 201811011203A CN 109032536 B CN109032536 B CN 109032536B
- Authority
- CN
- China
- Prior art keywords
- data
- pool
- size information
- target
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000007246 mechanism Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 2
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage method based on a distributed cluster system, which comprises the following steps: writing target data into the data pool by the client; receiving first data size information of written target data and second data size information described in metadata of the written target data; judging whether the first data size information and the second data size information are consistent with the original data size information or not; if yes, the target data is successfully written; if not, rewriting the target data. Therefore, after the data are written into the data pool, whether the size of the data written into the data pool is correct or not and whether the size of the data recorded by the metadata of the data written into the data pool is correct or not need to be determined; if the data are correct, the data are judged to be successfully written, so that the consistency of the data is ensured in the mode, and the reliability of the distributed cluster system is improved; the invention also discloses a data storage device, a system and equipment based on the distributed cluster system, and the technical effects can be realized.
Description
Technical Field
The present invention relates to the field of distributed storage technologies, and in particular, to a data storage method, apparatus, system, and device based on a distributed cluster system.
Background
The distributed storage system is used for storing data on a plurality of independent devices in a distributed mode. The traditional network storage system adopts a centralized storage server to store all data, the storage server becomes the bottleneck of the system performance, is also the focus of reliability and safety, and cannot meet the requirement of large-scale storage application. The distributed network storage system adopts an expandable system structure, utilizes a plurality of storage servers to share the storage load, and utilizes the position server to position the storage information, thereby not only improving the reliability, the availability and the access efficiency of the system, but also being easy to expand.
At present, a distributed cluster storage system is widely applied in various scenes of current mass data, for example: high performance scenes, video monitoring scenes, broadcast and television assets and the like. When distributed cluster storage is performed, if data inconsistency occurs in the data storage process, the data reliability of the distributed cluster system is reduced.
Therefore, how to improve the reliability of the distributed cluster system is a problem to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a data storage method, a data storage device, a data storage system and data storage equipment based on a distributed cluster system, so as to improve the reliability of the distributed cluster system.
In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:
a data storage method based on a distributed cluster system comprises the following steps:
writing target data into the data pool by the client;
the client receives first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service;
the client judges whether the first data size information and the second data size information are consistent with the original data size information of the target information;
if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
Before the client writes the target data into the data pool, the method further comprises the following steps:
and the client sends a metadata request to a metadata service so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
Wherein, this scheme still includes:
the metadata service detects whether the log event is successfully landed to the data pool;
if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
The method for writing target data into the data pool by the client comprises the following steps:
the client writes data into the data pool in sequence in a segmented mode; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
A data storage device based on a distributed cluster system, the data storage device comprising, based on a client:
the data writing module is used for writing target data into the data pool;
an information receiving module, configured to receive first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service;
the judging module is used for judging whether the first data size information and the second data size information are consistent with the original data size information of the target information; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent with the target data, triggering the data writing module to write the target data into the data pool again.
Wherein, this scheme still includes:
and the data request sending module is used for sending a metadata request to a metadata service before writing target data into the data pool, so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
The data writing module is specifically used for sequentially writing data into the data pool in a segmented manner; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
A data storage system based on a distributed cluster system comprises a client, a metadata service and a data pool;
the client is used for writing target data into the data pool; receiving first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service; judging whether the first data size information and the second data size information are consistent with the original data size information of the target information or not; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
Wherein the metadata service is further to: detecting whether the log event is successfully landed to the data pool; if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
A data storage device based on a distributed cluster system, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the data storage method based on the distributed cluster system when executing the computer program.
As can be seen from the foregoing solutions, a data storage method based on a distributed cluster system provided in an embodiment of the present invention includes: writing target data into the data pool by the client; the client receives first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service; the client judges whether the first data size information and the second data size information are consistent with the original data size information of the target information; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
Therefore, in the scheme, after data is written into the data pool, whether the size of the data written into the data pool is correct or not and whether the size of the data recorded by the metadata of the data written into the data pool is correct or not are determined by using a metadata synchronization mechanism and a data synchronization mechanism; if the data are correct, the data are judged to be successfully written, so that the consistency of the data is ensured in the mode, and the reliability of the distributed cluster system is improved;
the invention also discloses a data storage device, a system and equipment based on the distributed cluster system, and the technical effects can be realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data storage method based on a distributed cluster system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data storage device based on a distributed cluster system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data storage system based on a distributed cluster system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a data storage method, a data storage device, a data storage system and data storage equipment based on a distributed cluster system, and aims to improve the reliability of the distributed cluster system.
Referring to fig. 1, an embodiment of the present invention provides a data storage method based on a distributed cluster system, including:
s101, writing target data into a data pool by a client;
the method for writing target data into the data pool by the client comprises the following steps:
the client writes data into the data pool in sequence in a segmented mode; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
Specifically, in this embodiment, the target data written into the data pool may be a complete segment of data, or may be a segment of complete data divided into a plurality of data segments in a segmentation manner, where one data segment is the target data in the scheme; the way of processing the complete data segment can respectively confirm the consistency of each segment of data, and if an error occurs, only the data segment with the error needs to be rewritten; if the complete data is directly written into the data pool, if the complete data has a problem, all the data needs to be rewritten, which wastes a lot of time.
In the scheme, after data is written into a data pool, if the data is determined to be successfully written through S102-S103, the next section of data is continuously written into the data pool, and the subsequent steps are continuously executed; if the data is not successfully written, the data needs to be rewritten until the next section of data can be written after the data is judged to be successfully written.
S102, the client receives first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through metadata service;
specifically, after the target data is written into the data pool, first data size information and second data size information need to be fed back to the client, where the first data size information is specifically the size of a storage space occupied after the target data is written into the data pool, and can be determined by an algorithm.
The second data size information is size information of the target data recorded in metadata in the target data in the data pool after the target data is written in the data pool, and if the size information of the target data is successfully recorded in the metadata, the second data size information should be the same as the original data size information of the target data, and if the size information of the target data is different from the original data size information, the metadata error is indicated.
S103, the client judges whether the first data size information and the second data size information are consistent with the original data size information of the target information; if yes, executing S104; if not, re-executing S101;
and S104, judging that the target data is successfully written.
Specifically, the original data size information in this scheme refers to: before the target data is not stored in the data pool, the original data size information is the comparison reference and is correct information in the client. It should be noted that, this embodiment actually discloses two mechanisms, one is a data synchronization mechanism, and the other is a metadata synchronization mechanism, in this scheme, the data synchronization mechanism is used to determine whether data is correctly written into the data pool, that is, whether data is correctly written is determined by size information of target data written into the data pool, and if data is correctly written, size information of the first data is the same as size information of the original data. And the metadata synchronization mechanism is used for determining whether the metadata in the data pool correctly counts the size of the written target data, and if the second data size information is the same as the original data size information, the metadata is correct.
When the target data is successfully written into the data pool, the first data size information, the second data size information and the original data size information should be the same, if any two of the first data size information, the second data size information and the original data size information are different, the writing failure is indicated, the target data needs to be rewritten, that is, the step S101 is executed again until the first data size information, the second data size information and the original data size information of the rewritten data are the same.
Specifically, if a client initiates a file write operation, the correctness of data and metadata needs to be respectively confirmed through a data synchronization mechanism and a metadata synchronization mechanism; for example: the data to be written is a 1GB file, 4MB data is written in a segmented mode, wherein 4MB is original data size information; after writing 4MB size data into a data pool, receiving returned first data size information and second data size information, if the first data size information is 4MB, the data written into the data pool is correct, otherwise, the data is incorrect; if the second data size information is 4MB, the metadata corresponding to the written data is tangent, otherwise it is incorrect; if either is incorrect, the client resends the write operation.
In conclusion, the metadata synchronization mechanism of the client side is used for ensuring the consistency of the metadata, and the data synchronization mechanism of the client side is used for ensuring the consistency of the data, so that the risk of data loss in a fault scene can be avoided, and the reliability of the data is fundamentally ensured. Of course, the data storage manner in the present scheme may also be applied to unified storage, and is not particularly limited herein.
Based on any of the foregoing embodiments, in this embodiment, before the client writes the target data into the data pool, the method further includes:
and the client sends a metadata request to a metadata service so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
The metadata service detects whether the log event is successfully landed to the data pool;
if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
Specifically, when a client performs operations such as creation, modification and the like on a Data pool, a metadata request needs to be initiated (creation, modification and the like) to a metadata service (Meta Data Server, MDS); after receiving the metadata request, the MDS generates a corresponding log event and takes the log event to the disk; the log event is only used to identify the operation type of the client on the data pool, and the log event may be understood as an event that records the next operation performed on the data pool, for example, the event that writes the target data into the data pool in the above embodiment, and similarly, the log event in this embodiment falls to the data pool, that is, the client falls to the data pool in the above embodiment.
Further, after the client sends the metadata request to the MDS, the MDS sends a response that the log event is successfully landed to the data pool after the target data is successfully landed, if the log event is not landed successfully, the MDS does not send response information to the client, which indicates that a fault occurs when the log event is landed, at this time, the MDS can replay the metadata request through a metadata log event replay mechanism, and if the request is not executed, that is, the data write operation or the data modification operation corresponding to the request is not executed, the processing is continued.
In the following, the data storage device provided by the embodiment of the present invention is introduced, and the data storage device described below and the data storage method described above may be referred to each other.
Referring to fig. 2, an embodiment of the present invention provides a data storage device based on a distributed cluster system, where based on a client, the data storage device includes:
a data writing module 101, configured to write target data into a data pool;
an information receiving module 102, configured to receive first data size information of the written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service;
a judging module 103, configured to judge whether both the first data size information and the second data size information are consistent with original data size information of the target information; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent with the target data, triggering the data writing module to write the target data into the data pool again.
Wherein, this scheme still includes:
and the data request sending module is used for sending a metadata request to a metadata service before writing target data into the data pool, so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
The data writing module is specifically used for sequentially writing data into the data pool in a segmented manner; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
In the following, the data storage system provided by the embodiment of the present invention is introduced, and the data storage system described below and the data storage method described above may be referred to each other.
Referring to fig. 3, the data storage system based on the distributed cluster system according to the embodiment of the present invention includes a client 100, a metadata service 200, and a data pool 300;
the client 100 is configured to write target data into the data pool; receiving first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service; judging whether the first data size information and the second data size information are consistent with the original data size information of the target information or not; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
Wherein the metadata service is further to: detecting whether the log event is successfully landed to the data pool; if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
Wherein the client is further configured to: before target data is written into a data pool, a metadata request is sent to a metadata service, so that the metadata service generates a corresponding log event according to the metadata request and drops to the data pool.
The client is specifically used for sequentially writing data into the data pool in a segmented mode; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
The embodiment of the invention also discloses a data storage device based on the distributed cluster system, which comprises:
a memory for storing a computer program;
and the processor is used for implementing the steps of the data storage method based on the distributed cluster system in any method embodiment when executing the computer program.
The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the data storage method based on the distributed cluster system in any method embodiment are realized.
Wherein the storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data storage method based on a distributed cluster system is characterized by comprising the following steps:
writing target data into the data pool by the client;
the client receives first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service;
the client judges whether the first data size information and the second data size information are consistent with the original data size information of the target data or not;
if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
2. The data storage method of claim 1, wherein before the client writes the target data to the data pool, the method further comprises:
and the client sends a metadata request to a metadata service so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
3. The data storage method of claim 2, further comprising:
the metadata service detects whether the log event is successfully landed to the data pool;
if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
4. The data storage method according to any one of claims 1 to 3, wherein the writing of the target data to the data pool by the client comprises:
the client writes data into the data pool in sequence in a segmented mode; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
5. A data storage device based on a distributed cluster system, wherein the data storage device comprises, based on a client:
the data writing module is used for writing target data into the data pool;
an information receiving module, configured to receive first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service;
the judging module is used for judging whether the first data size information and the second data size information are consistent with the original data size information of the target data or not; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent with the target data, triggering the data writing module to write the target data into the data pool again.
6. The data storage device of claim 5, further comprising:
and the data request sending module is used for sending a metadata request to a metadata service before writing target data into the data pool, so that the metadata service generates a corresponding log event according to the metadata request and drops the log event to the data pool.
7. The data storage device according to claim 5 or 6, wherein the data writing module is specifically configured to sequentially write data to the data pool in a segmented manner; and after the client judges that the data is successfully written, writing the next section of data into the data pool.
8. A data storage system based on a distributed cluster system is characterized by comprising a client, a metadata service and a data pool;
the client is used for writing target data into the data pool; receiving first data size information of written target data fed back by the data pool and second data size information recorded in metadata of the written target data fed back by the data pool through a metadata service; judging whether the first data size information and the second data size information are consistent with the original data size information of the target data or not; if the target data are consistent, judging that the target data are successfully written; and if the target data are inconsistent, rewriting the target data into the data pool.
9. The data storage system of claim 8, wherein the metadata service is further configured to: detecting whether the log event is successfully landed to the data pool; if not, the metadata service utilizes the log event to play back the corresponding metadata request according to a log event playback mechanism, and generates a log event to be downloaded to the data pool.
10. A data storage device based on a distributed cluster system, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the distributed cluster system based data storage method according to any of claims 1 to 4 when executing said computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811011203.5A CN109032536B (en) | 2018-08-31 | 2018-08-31 | Data storage method, device, system and equipment based on distributed cluster system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811011203.5A CN109032536B (en) | 2018-08-31 | 2018-08-31 | Data storage method, device, system and equipment based on distributed cluster system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109032536A CN109032536A (en) | 2018-12-18 |
| CN109032536B true CN109032536B (en) | 2021-08-10 |
Family
ID=64622592
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811011203.5A Active CN109032536B (en) | 2018-08-31 | 2018-08-31 | Data storage method, device, system and equipment based on distributed cluster system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109032536B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110096358A (en) * | 2019-04-11 | 2019-08-06 | 上海交通大学 | Chain drive remote center distributed storage and distributed computing method |
| CN113835643B (en) * | 2021-11-23 | 2022-03-08 | 苏州浪潮智能科技有限公司 | Data storage method and device, electronic equipment and readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101888395A (en) * | 2009-05-13 | 2010-11-17 | 华为技术有限公司 | Method for synchronizing data, widget terminal and server |
| CN102223382A (en) * | 2011-07-18 | 2011-10-19 | 中国科学院深圳先进技术研究院 | Cloud safety method and system with data distribution characteristics as password |
| CN103809147A (en) * | 2012-11-12 | 2014-05-21 | 苏州工业园区新宏博通讯科技有限公司 | AC ammeter fault self diagnosis method |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN104246767A (en) * | 2012-04-05 | 2014-12-24 | 微软公司 | Telemetry system for a cloud synchronization system |
| CN104932953A (en) * | 2015-06-04 | 2015-09-23 | 华为技术有限公司 | Data distribution method, data storage method, and relevant device and system |
| CN105025106A (en) * | 2015-07-28 | 2015-11-04 | 焦点科技股份有限公司 | Breakpoint resuming method based on segmentation and meta-information |
| CN105141685A (en) * | 2015-08-18 | 2015-12-09 | 浪潮(北京)电子信息产业有限公司 | File read-write system and meta data memory thereof as well as method and device for reading and writing files |
| CN106527958A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Data writing and obtaining methods and devices used for distributed storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102142006B (en) * | 2010-10-27 | 2013-10-02 | 华为技术有限公司 | File processing method and device of distributed file system |
-
2018
- 2018-08-31 CN CN201811011203.5A patent/CN109032536B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101888395A (en) * | 2009-05-13 | 2010-11-17 | 华为技术有限公司 | Method for synchronizing data, widget terminal and server |
| CN102223382A (en) * | 2011-07-18 | 2011-10-19 | 中国科学院深圳先进技术研究院 | Cloud safety method and system with data distribution characteristics as password |
| CN104246767A (en) * | 2012-04-05 | 2014-12-24 | 微软公司 | Telemetry system for a cloud synchronization system |
| CN103809147A (en) * | 2012-11-12 | 2014-05-21 | 苏州工业园区新宏博通讯科技有限公司 | AC ammeter fault self diagnosis method |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN104932953A (en) * | 2015-06-04 | 2015-09-23 | 华为技术有限公司 | Data distribution method, data storage method, and relevant device and system |
| CN105025106A (en) * | 2015-07-28 | 2015-11-04 | 焦点科技股份有限公司 | Breakpoint resuming method based on segmentation and meta-information |
| CN105141685A (en) * | 2015-08-18 | 2015-12-09 | 浪潮(北京)电子信息产业有限公司 | File read-write system and meta data memory thereof as well as method and device for reading and writing files |
| CN106527958A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Data writing and obtaining methods and devices used for distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109032536A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11614867B2 (en) | Distributed storage system-based data processing method and storage device | |
| US8762772B2 (en) | Method for a plurality of RAID systems and data storage system thereof | |
| CN102033786B (en) | Method for repairing consistency of copies in object storage system | |
| CN107506266B (en) | Data recovery method and system | |
| CN111506251B (en) | Data processing method and device, SMR storage system and storage medium | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| US20150213100A1 (en) | Data synchronization method and system | |
| CN106933843B (en) | Database heartbeat detection method and device | |
| CN102955720A (en) | Method for improving stability of EXT (extended) file system | |
| CN111797172B (en) | Data migration method, device, equipment, distributed system and storage medium | |
| CN107315659B (en) | Metadata redundancy backup method and device | |
| CN108595119B (en) | Data synchronization method and distributed system | |
| US10346074B2 (en) | Method of compressing parity data upon writing | |
| CN111857602A (en) | Data processing method, data processing device, data node and storage medium | |
| CN109032536B (en) | Data storage method, device, system and equipment based on distributed cluster system | |
| CN106815094B (en) | Method and equipment for realizing transaction submission in master-slave synchronization mode | |
| CN113204530B (en) | Data writing method, device, equipment and medium of distributed file system | |
| US20130103645A1 (en) | Method and Device for Inputting Data in Multimedia Service Database on Embedded Device | |
| CN115167782B (en) | Temporary storage copy management method, system, equipment and storage medium | |
| CN105892954A (en) | Data storage method and device based on multiple copies | |
| CN106933707B (en) | Data recovery method and system of data storage device based on raid technology | |
| CN107045426B (en) | Multi-copy reading method and system | |
| CN110928945B (en) | Data processing method and device for database and data processing system | |
| CN107153589B (en) | Reading and writing method of mirror image data | |
| CN112527561B (en) | Data backup method and device based on Internet of things cloud storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |