This is a PhD level course on stochastic process.

Course Description

We introduce the general concepts and properties of (mostly discrete-time) stochastic processes which lays the foundation for more detailed studies of various types of stochastic processes encountered in business research. We will focus on the classical renewal theory and Markov chain theory. Towards the end, we unify them in the framework of Markov renewal theory. We will spend the last week or two to discuss applications, with a particular emphasis on Markov decision process and reinforement learning.

Schedule

Week	Topic
1	Laplace Transform
2	Stopping Time
3	Renewal Theory
4	Markov Chain
5	Semi-Markov Process
6	Markov Decision Process
7	Reinforcement Learning