Stochastic Processes and Queueing Theory (Theoretical Course)
Introduction
1. Overview
• Definition of Probability, Random Variable, Stochastic Process
• Classification of Stochastic Processes
• Overview of Queueing Theory
Part I: Stochastic Processes Theory
1. Conditional Probability and Conditional Expections
• Math Definition
• Applications
2. Markov Processes and Poisson Process
• Definition
• Chapman-Kolmogorov Equations
• Limiting Probability
• Time Reversibility
• Markov Decision Process
• Kolmogorov Forward and Backward Equation
• Definition of Exponential Distribution
• Properties of Exponential Random Variable
• Convolutions of Exponential Random Variable
• Defintiation of Counting Process, Poisson
• Properties of Poisson Process
• Variations of Poisson Process (nonhomogenous, compound, conditional)
3. Renew Processes, Random Walk, Brownian Motion
• Definition of Renewal Process
• Distribution of N(t)
• Wald's Equatioin
• Insights of Renewal
• Definition of Random Walk
• Duality of Random Walk
• Analyze Random Walk through Martingale
• Applications to G/G/1 Queue
• Definition of Brownian Motion Process
• Hitting Times, Maximum Variable, and Arc Sine Law
• Variations on Brownina Motion
• Absorbed Brownian Motion
• Reflected Brownian Motion
• Geometric Brownian Motion
• Integrated Brownian Motion
• Brownian Motion with drift
• Analyze Brownian Motion through Martingale
• Kolmogrov Differential Equations for Brownian Motion
4. Martingale Processes, Stationary Processes
• Supper Martiginale, Sub Martingale
• Fundamental Martingale Inequalities
• Doob's Martingale Convergence Theorem
• Definition of Stationary Process
• Limiting Theorems and Ergodic Theory
Part II: Queueing Theory
1. M/M/1, M/M/C, etc (Chapter 3)
2. M/Er/1, Er/M/1, etc (Chapter 4)
3. M/G/1 (Chapter 5)
4. G/M/1 (Chapter 6)
5. Priority Queue
6. G/G/1 (Chapter 8)
7. Queueing Networks (Jackson Networks, Wittle Networks)
Project Review Questa paper
Final Exam Cover All
Textbooks TBD
References
1. "Probability (Graduate Texts in Mathematics), second Edition", by A. N. Shiryaev
2. "Probability and Measure Theory, 3rd Edition", by Patrick Billingsley
3. "Probability and Measure Theory, second Endition", by Robert Ash. and C. A. Doleans
4. "Probability: Theory and Examples, 3rd Edition", Richard Durrett
5. "Introduction to Probability Models, 9th Edition", by Sheldon Ross
6. "Stochastic Processes", by Sheldon, Ross
7. "A First Course in Stochastic Processes", by Samuel Karlin and Howard M. Taylor
8. "A Second Course in Stochastic Processes", by Samuel Karlin and Howard M. Taylor
9. "Stochastic Processes", by J. L. Doob
10. "Principles of Random Walk", by Frank Spitzer
11. "Probability, Random Processes and Ergodic Properties", by Robert M. Gray
12. "Markov Chains and Stochastic Stability", by S. P. Meyn and R. L. Tweedie
13. "Queueing Systems, Vol I", by L. Kleinrock
14. "Queueing Systems, Vol II", by L. Kleinrock
15. "Queueing Analysis Vol 1: Vacation and Priority Systems", by Hideaki Takagi
16. "Queueing Analysis Vol 2: Finite Systems ", by Hideaki Takagi
17. "Queueing Analysis Vol 3:Discrete-Time Systems", by Hideaki Takagi
18. "Stochastic Differential Equations: An Introduction with Applications (Universitext)", by Bernt Oksendal
Other Resources