Article

Free access

Latency management in storage systems

Authors:

Rodney Van Meter,

Minxi GaoAuthors Info & Claims

OSDI'00: Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4

Article No.: 8

Published: 22 October 2000 Publication History

PDF eReader Publisher Site

Abstract

Storage Latency Estimation Descriptors, or SLEDs, are an API that allow applications to understand and take advantage of the dynamic state of a storage system. By accessing data in the file system cache or high-speed storage first, total I/O workloads can be reduced and performance improved. SLEDs report estimated data latency, allowing users, system utilities, and scripts to make file access decisions based on those retrieval time estimates. SLEDs thus can be used to improve individual application performance, reduce system workloads, and improve the user experience with more predictable behavior.

We have modified the Linux 2.2 kernel to support SLEDs, and several Unix utilities and astronomical applications have been modified to use them. As a result, execution times of the Unix utilities when data file sizes exceed the size of the file system buffer cache have been reduced from 50% up to more than an order of magnitude. The astronomical applications incurred 30-50% fewer page faults and reductions in execution time of 10-35%. Performance of applications which use SLEDs also degrade more gracefully as data file size grows.

References

[1]

{ADN⁺95} Thomas E. Anderson, Michael D. Dahlin, Jeanna M. Neefe, David A. Patterson, Drew S. Roselli, and Randolph Y. Wang. Serverless network file systems. In Proc. 15th ACM Symposium on Operating Systems Principles, pages 109-126. ACM, December 1995.]]

Digital Library

[2]

{AOG92} David P. Anderson, Yoshitomo Osawa, and Ramesh Govindan. A file system for continuous media. Trans. on Computing Systems, 10(4):311-337, November 1992.]]

Digital Library

[3]

{BFD97} William J. Bolosky, Robert P. Fitzgerald, and John R. Douceur. Distributed schedule management in the Tiger video fileserver. In Proc. 16th ACM Symposium on Operating Systems Principles, pages 212-223. ACM, October 1997.]]

Digital Library

[4]

{CFKL96} Pei Cao, Edward E. Felten, Anna R. Karlin, and Kai Li. Implemention and performance of integrated application-controlled file caching, prefetching, and disk scheduling. ACM Transactions on Computer Systems , 14(4):311-343, November 1996.]]

Digital Library

[5]

{Dig76} Digital Equipment Corporation. decsys-tem10 Monitor Calls, June 1976.]]

[6]

{FGBA96} Armando Fox, Steven D. Gribble, Eric A. Brewer, and Elan Amir. Adapting to network and client variability via on-demand dynamic distillation. In Proc. ACM Seventh International Conference on Architectural Support for Programming Languages and Operating Systems, pages 160-170. ACM, October 1996.]]

Digital Library

[7]

{GA95} James Griffioen and Randy Appleton. Performance measurements of automatic prefetching. In Proceedings International Conference on Parallel and Distributed Computing Systems, September 1995.]]

[8]

{HP89} Robert L. Henderson and Alan Poston. MSS-II and RASH: A mainframe UNIX based mass storage system with a rapid access storage hierarchy file management system. In Proc. 1989 USENIX Winter Technical Conference, pages 65-84. USENIX, 1989.]]

[9]

{HS96a} Bruce Hillyer and Avi Silberschatz. On the modeling and performance characteristics of a serpentine tape drive. In Proc. ACM Sigmetrics Conference on Measurement and Modeling of Computer Systems, pages 170-179. ACM, May 1996.]]

Digital Library

[10]

{HS96b} Bruce K. Hillyer and Avi Silberschatz. Random I/O scheduling in online tertiary storage systems. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, pages 195-204. ACM, June 1996.]]

Digital Library

[11]

{Kot94} David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74. USENIX Association, November 1994. Updated as Dartmouth TR PCS-TR94-226 on November 8, 1994.]]

Digital Library

[12]

{Kro00} Thomas M. Kroeger. Modeling File Access Patterns to Improve Caching Performance. PhD thesis, University of California Santa Cruz, March 2000.]]

[13]

{LLJR99} Per Lysne, Gary Lee, Lynn Jones, and Mark Roschke. HPSS at Los Alamos: Experiences and analysis. In Proc. Sixteenth IEEE Symposium on Mass Storage Systems in co-operation with the Seventh NASA Goddard Conference on Mass Storage Systems and Technologies, pages 150-157. IEEE, March 1999.]]

[14]

{LT96} Edward K. Lee and Chandramohan A. Thekkath. Petal: Distributed virtual disks. In Proc. ACM Seventh International Conference on Architectural Support for Programming Languages and Operating Systems , pages 84-92. ACM, October 1996.]]

Digital Library

[15]

{MS96} Larry McVoy and Carl Staelin. Lmbench: Portable tools for performance analysis. In Proceedings of the USENIX 1996 Annual Technical Conference, pages 279-294. USENIX, January 1996.]]

Digital Library

[16]

{NAS00} NASA. Lheasoft. Goddard Space Flight Center Laboratory for High Energy Astrophysics, http://rxte.gsfc.nasa.gov/lheasoft/, April 2000.]]

[17]

{NSN⁺97} Brian D. Noble, M. Satyanarayanan, Dushyanth Narayanan, James Eric Tilton, Jason Flinn, and Kevin R. Walker. Agile application-aware adaptation for mobility. In Proc. 16th ACM Symposium on Operating Systems Principles, pages 276-287. ACM, October 1997.]]

Digital Library

[18]

{PA94} Joseph Pasquale and Eric Anderson. Container shipping: Operating system support for I/O-intensive applications. IEEE Computer , 27(3):84-93, March 1994.]]

Digital Library

[19]

{PGG⁺95} R. Hugo Patterson, Garth A. Gibson, Eka Ginting, Daniel Stodolsky, and Jim Zelenka. Informed prefetching and caching. In Proc. 15th ACM Symposium on Operating Systems Principles, pages 79-95. ACM, December 1995.]]

Digital Library

[20]

{RW94} Chris Ruemmler and John Wilkes. An introduction to disk drive modeling. Computer, 27(3):17-28, March 1994.]]

Digital Library

[21]

{Sat90} Mahadev Satyanarayanan. Scalable, secure, and high available distributed file access. IEEE Computer, pages 9-21, May 1990.]]

Digital Library

[22]

{Sch00} Marc Schaefer. The migration filesystem. web page, April 2000. http://www-internal. alphanet.ch/~schaefer/mfs.html.]]

[23]

{SDH⁺96} Adam Sweeney, Doug Doucette, Wei Hu, Curtis Anderson, Mike Nishimoto, and Geoff Peck. Scalability in the XFS file system. In Proc. 1996 USENIX Technical Conference , pages 1-14. USENIX, January 1996.]]

Digital Library

[24]

{Shi98} Jamie Shiers. Building a database for the large hadron collider (LHC): the exabyte challenge. In Ben Kobler, editor, Proc. Sixth NASA Goddard Conference on Mass Storage Systems and Technologies in Co-operation with Fifteenth IEEE Symposium on Mass Storage Systems, pages 385-396, March 1998.]]

[25]

{SM99} Olav Sandstå and Roger Midstraum. Low-cost access time model for serpentine tape drive. In Proc. Sixteenth IEEE Symposium on Mass Storage Systems in cooperation with the Seventh NASA Goddard Conference on Mass Storage Systems and Technologies, pages 116-127. IEEE, March 1999.]]

[26]

{Ste97} David C. Steere. Exploiting the nondeterminism and asynchrony of set iterators to reduce aggregate file I/O latency. In Proc. 16th ACM Symposium on Operating Systems Principles, pages 252-263. ACM, October 1997.]]

Digital Library

[27]

{Van97} Rodney Van Meter. Observing the effects of multi-zone disks. In Proc. USENIX '97 Technical Conference, pages 19-30. USENIX, January 1997.]]

Digital Library

[28]

{Van98} Rodney Van Meter. SLEDs: Storage latency estimation descriptors. In Ben Kobler, editor, Proc. Sixth NASA Goddard Conference on Mass Storage Systems and Technologies in Cooperation with Fifteenth IEEE Symposium on Mass Storage Systems, pages 249-260, March 1998.]]

[29]

{vI99} Catherine van Ingen. Storage in Windows 2000. presentation, February 1999.]]

[30]

{WC95} R. W. Watson and R. A. Coyne. The parallel I/O architecture of the high-performance storage system (HPSS). In Proc. Fourteenth IEEE Symposium on Mass Storage Systems, pages 27-44. IEEE, September 1995.]]

Digital Library

[31]

{WGP94} Bruce L. Worthington, Gregory R. Ganger, and Yale N. Patt. Scheduling for modern disk drives and non-random workloads. Technical Report CSE-TR-194-94, University of Michigan, March 1994.]]

[32]

{WGSS95} John Wilkes, Richard Golding, Carl Staelin, and Tim Sullivan. The HP AutoRAID hierarchical storage system. In Proc. 15th ACM Symposium on Operating Systems Principles , pages 96-108. ACM, December 1995.]]

Digital Library

Cited By

Yang SHarter TAgrawal NKowsalya SKrishnamurthy AAl-Kiswany SKaushik RArpaci-Dusseau AArpaci-Dusseau RMiller EHand S(2015)Split-level I/O schedulingProceedings of the 25th Symposium on Operating Systems Principles10.1145/2815400.2815421(474-489)Online publication date: 4-Oct-2015
https://dl.acm.org/doi/10.1145/2815400.2815421
Traeger AZadok EJoukov NWright C(2008)A nine year study of file system and storage benchmarkingACM Transactions on Storage10.1145/1367829.13678314:2(1-56)Online publication date: 28-May-2008
https://dl.acm.org/doi/10.1145/1367829.1367831
Silva DKrieger OWisniewski RWaterland ATam DBaumann A(2006)K42ACM SIGOPS Operating Systems Review10.1145/1131322.113133340:2(34-42)Online publication date: 1-Apr-2006
https://dl.acm.org/doi/10.1145/1131322.1131333
Show More Cited By

Index Terms

Latency management in storage systems
1. General and reference
  1. Cross-computing tools and techniques
    1. Performance
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Operational analysis
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Communications management
        Buffering
        Input / output
        File systems management
        Memory management
        Allocation / deallocation strategies

Recommendations

Storage management in large distributed object-based storage systems
Storage management for large scale systems
A data management method for databases using hybrid storage systems

When applications require high I/O performance, solid-state drives (SSDs) are often preferable because they perform better than traditional hard-disk drives (HDDs). Therefore, database system response time can be improved by moving frequently used data ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OSDI'00: Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4

October 2000

355 pages

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems
USENIX Assoc: USENIX Assoc
IEEE Technical Committee on Operating Systems (TCOS)

Publisher

USENIX Association

United States

Publication History

Published: 22 October 2000

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)6

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang SHarter TAgrawal NKowsalya SKrishnamurthy AAl-Kiswany SKaushik RArpaci-Dusseau AArpaci-Dusseau RMiller EHand S(2015)Split-level I/O schedulingProceedings of the 25th Symposium on Operating Systems Principles10.1145/2815400.2815421(474-489)Online publication date: 4-Oct-2015
https://dl.acm.org/doi/10.1145/2815400.2815421
Traeger AZadok EJoukov NWright C(2008)A nine year study of file system and storage benchmarkingACM Transactions on Storage10.1145/1367829.13678314:2(1-56)Online publication date: 28-May-2008
https://dl.acm.org/doi/10.1145/1367829.1367831
Silva DKrieger OWisniewski RWaterland ATam DBaumann A(2006)K42ACM SIGOPS Operating Systems Review10.1145/1131322.113133340:2(34-42)Online publication date: 1-Apr-2006
https://dl.acm.org/doi/10.1145/1131322.1131333
Arpaci-Dusseau AArpaci-Dusseau RBurnett NDenehy TEngle TGunawi HNugent JPopovici FScott MPeterson L(2003)Transforming policies into mechanisms with infokernelProceedings of the nineteenth ACM symposium on Operating systems principles10.1145/945445.945455(90-105)Online publication date: 19-Oct-2003
https://dl.acm.org/doi/10.1145/945445.945455
Arpaci-Dusseau AArpaci-Dusseau RBurnett NDenehy TEngle TGunawi HNugent JPopovici F(2003)Transforming policies into mechanisms with infokernelACM SIGOPS Operating Systems Review10.1145/1165389.94545537:5(90-105)Online publication date: 19-Oct-2003
https://dl.acm.org/doi/10.1145/1165389.945455
Burnett NBent JArpaci-Dusseau AArpaci-Dusseau R(2002)Exploiting Gray-Box Knowledge of Buffer-Cache ManagementProceedings of the General Track of the annual conference on USENIX Annual Technical Conference10.5555/647057.713850(29-44)Online publication date: 10-Jun-2002
https://dl.acm.org/doi/10.5555/647057.713850
Arpaci-Dusseau AArpaci-Dusseau R(2001)Information and control in gray-box systemsACM SIGOPS Operating Systems Review10.1145/502059.50204035:5(43-56)Online publication date: 21-Oct-2001
https://dl.acm.org/doi/10.1145/502059.502040
Arpaci-Dusseau AArpaci-Dusseau RMarzullo KSatyanarayanan M(2001)Information and control in gray-box systemsProceedings of the eighteenth ACM symposium on Operating systems principles10.1145/502034.502040(43-56)Online publication date: 21-Oct-2001
https://dl.acm.org/doi/10.1145/502034.502040

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents