extended-abstract

Approximating stationary measures of structured continuous-time Markov models using matrix diagrams

Authors:

Gianfranco Ciardo,

Andrew S. Miner,

Min Wan,

Andy Jinqing YuAuthors Info & Claims

ACM SIGMETRICS Performance Evaluation Review, Volume 35, Issue 3

Pages 16 - 18

https://doi.org/10.1145/1328690.1328697

Published: 01 December 2007 Publication History

Get Access

Abstract

We consider the stationary solution of large ergodic continuous-time Markov chains (CTMCs) with a finite state space S, i.e., the computation of π as solution of π · Q = 0 subject to ∑_iεsπ[i] = 1, where Q coincides with transition rate matrix R except in its diagonal elements, Q[i, i] = - ∑_jεs R [i, j].

References

[1]

V. Amoia, G. De Micheli, and M. Santomauro. Computer-oriented formulation of transition-rate matrices via Kronecker algebra. IEEE Trans. Rel., 30:123--132, June 1981.

Crossref

Google Scholar

[2]

P. Buchholz, G. Ciardo, S. Donatelli, and P. Kemper. Complexity of memory-efficient Kronecker operations with applications to the solution of Markov models. INFORMS J. Comp., 12(3):203--222, 2000.

Digital Library

Google Scholar

[3]

G. Ciardo, R. L. Jones, A. S. Miner, and R. Siminiceanu. Logical and stochastic modeling with SMART. Perf. Eval., 63:578--608, 2006.

Digital Library

Google Scholar

[4]

G. Ciardo and A. S. Miner. A data structure for the efficient Kronecker solution of GSPNs. Proc. 8th Int. Workshop on Petri Nets and Performance Models (PNPM'99), pages 22--31, 1999. IEEE Comp. Soc. Press.

Digital Library

Google Scholar

[5]

P. Fernandes, B. Plateau, and W. J. Stewart. Efficient descriptor-vector multiplication in stochastic automata networks. J. ACM, 45(3):381--414, 1998.

Digital Library

Google Scholar

[6]

A. S. Miner. Efficient solution of GSPNs using canonical matrix diagrams. Proc. 9th Int. Workshop on Petri Nets and Performance Models (PNPM'01), pages 101--110, 2001. IEEE Comp. Soc. Press.

Digital Library

Google Scholar

[7]

A. S. Miner, G. Ciardo, and S. Donatelli. Using the exact state space of a Markov model to compute approximate stationary measures. Proc. ACM SIGMETRICS, pages 207--216, 2000. ACM Press.

Digital Library

Google Scholar

[8]

A. Srinivasan, T. Kam, S. Malik, and R. K. Bray-ton. Algorithms for discrete function manipulation. In Int. Conference on CAD, pages 92--95. IEEE Comp. Soc. Press, 1990.

Crossref

Google Scholar

[9]

W. J. Stewart. Introduction to the Numerical Solution of Markov Chains. Princeton University Press, 1994.

Google Scholar

Cited By

View all

Ciardo GMiner A(2023)Research Directions in Decision Diagram Technology2023 IEEE John Vincent Atanasoff International Symposium on Modern Computing (JVA)10.1109/JVA60410.2023.00018(46-53)Online publication date: 5-Jul-2023
https://doi.org/10.1109/JVA60410.2023.00018
Shruti SRakhee R(2020)Analysis of Spectrum Sensing and Spectrum Access in Cognitive Radio Networks with Heterogeneous Traffic and p-Retry BufferingIEEE Transactions on Mobile Computing10.1109/TMC.2020.3042836(1-1)Online publication date: 2020
https://doi.org/10.1109/TMC.2020.3042836
Suliman ILehtomäki JUmebayashi K(2015)On the effect of false alarm rate on the performance of cognitive radio networksEURASIP Journal on Wireless Communications and Networking10.1186/s13638-015-0474-32015:1Online publication date: 10-Nov-2015
https://doi.org/10.1186/s13638-015-0474-3
Show More Cited By

Index Terms

Approximating stationary measures of structured continuous-time Markov models using matrix diagrams
1. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computations in finite fields
      2. Computations on matrices
  2. Probability and statistics
    1. Probabilistic representations
      1. Markov networks
    2. Stochastic processes
      1. Markov processes
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Markov decision processes

Recommendations

Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

This paper deals with continuous-time Markov decision processes in Polish spaces, under an expected discounted reward criterion. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates ...
Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

This paper introduces and develops a new approach to the theory of continuous time jump Markov decision processes (CTJMDP). This approach reduces discounted CTJMDPs to discounted semi-Markov decision processes (SMDPs) and eventually to discrete-time ...
Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

We consider continuous-time Markov decision processes in Polish spaces. The performance of a control policy is measured by the expected discounted reward criterion associated with state-dependent discount factors. All underlying Markov processes are ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMETRICS Performance Evaluation Review

ACM SIGMETRICS Performance Evaluation Review Volume 35, Issue 3

December 2007

73 pages

ISSN:0163-5999

DOI:10.1145/1328690

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2007

Published in SIGMETRICS Volume 35, Issue 3

Check for updates

Qualifiers

Extended-abstract
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
68
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ciardo GMiner A(2023)Research Directions in Decision Diagram Technology2023 IEEE John Vincent Atanasoff International Symposium on Modern Computing (JVA)10.1109/JVA60410.2023.00018(46-53)Online publication date: 5-Jul-2023
https://doi.org/10.1109/JVA60410.2023.00018
Shruti SRakhee R(2020)Analysis of Spectrum Sensing and Spectrum Access in Cognitive Radio Networks with Heterogeneous Traffic and p-Retry BufferingIEEE Transactions on Mobile Computing10.1109/TMC.2020.3042836(1-1)Online publication date: 2020
https://doi.org/10.1109/TMC.2020.3042836
Suliman ILehtomäki JUmebayashi K(2015)On the effect of false alarm rate on the performance of cognitive radio networksEURASIP Journal on Wireless Communications and Networking10.1186/s13638-015-0474-32015:1Online publication date: 10-Nov-2015
https://doi.org/10.1186/s13638-015-0474-3
BAIER CHAHN EHAVERKORT BHERMANNS HKATOEN J(2013)Model checking for performabilityMathematical Structures in Computer Science10.1017/S096012951200025423:04(751-795)Online publication date: 8-Jul-2013
https://doi.org/10.1017/S0960129512000254
Ciardo GMiner AWan M(2009)Advanced features in SMARTACM SIGMETRICS Performance Evaluation Review10.1145/1530873.153088536:4(58-63)Online publication date: 25-Mar-2009
https://dl.acm.org/doi/10.1145/1530873.1530885

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Comments

Published In

Publisher

Publication History

Check for updates

Qualifiers

Funding Sources

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations