Skip to document
Welcome to StudocuSign in to access the best study resources
Guest user
0followers
HomeMy LibraryAsk AI
  • You don't have any recent items yet.
My Library
  • You don't have any modules yet.
  • You don't have any books yet.
  • You don't have any Studylists yet.

MA20225-sheet 6-2022 - help with exercise sheets for the given lecture notes and problems

help with exercise sheets for the given lecture notes and problems
Module

Probability 2A (MA20224)

13 Documents
Students shared 13 documents in this course
Academic year: 2020/2021
Uploaded by:
Anonymous Student
This document has been uploaded by a student, just like you, who decided to remain anonymous.
University of Bath

Comments

Please sign in or register to post comments.

Students also viewed

Related documents

Preview text

MA20225 Probability 2B: Example Sheet 6

Questions 1–2 are to be discussed in the tutorials. Questions 3–4 are homework. Questions 5–7 will be discussed in the Friday problems class.

Question 1LetPbe the transition matrix of a simple random walk on the half-lineI={ 0 , 1 , 2 ,.. .}. That is, pi,i+1=p, i≥ 0 pi,i− 1 = 1−p, i≥ 1 p 0 , 0 = 1−p. (i) Show that if 0 < p < 1 , thenPis irreducible. (ii) Determine, depending on the value ofp, whether the random walk is positive recurrent, null recurrent, or transient. Determine the invariant distribution, when it exists.

(iii) Findlimn→∞p(iin)fori= 0, 1 , 2 ,.. ., in the positive recurrent case. (iv) Find the expected return time to stateifori= 0, 1 , 2 ,.. .. (v) Complete parts (ii) and (iv) for simple random walk onZas well.

Question 2 Consider a simple random walk on the vertices of an octahedron (see Sheet 4 for a picture). That is, if the walk is currently at vertexi, it chooses one of the incident edges uniformly at random, and moves along that edge.

(i) Find the transition matrixP. (ii) IsPirreducible? Show that ifπis the uniform distribution on the vertices, thenπis invariant, and it is the unique invariant distribution. (iii) Find the expected return time to each state. (iv) Find the period.

(v) Determinelimn→∞p(ijn)for any pair of statesi, j.

Recall from the lectures that in any irreducible Markov chain on afinitestate space, all states are positive recurrent and there is a unique invariant distributionπ. The following question explores what happens when the state space is finite but the Markov chain is not irreducible.

Question 3 LetPbe a transition matrix on afinitestate spaceI.

(i) Show that there is at least one recurrent communicating class.

(ii) LetTandRdenote the set of transient and recurrent states, respectively. LetPRdenote the restriction ofPtoR, i. the matrix (PR)ij=pij, i, j∈R. Show thatPRis a transition matrix onR.

(iii) LetHR= inf{n≥0 :Xn∈R}be the time the Markov chain entersR. Show that for all i∈Ione has Pi[HR<∞, Xn∈Rfor alln≥HR] = 1.

(iv) Show that any invariant distributionπofP is supported on the setR, i.πi= 0for all i∈T. Conclude that if there is only one recurrent class, then there exists a unique invariant distribution.

(v) Show that if there is more than one recurrent class, then there is more than one invariant distribution. Give an example of a Markov chain that has two recurrent classes and one transient class.

Question 4 Consider the Markov chain on the state spaceI={ 1 , 2 , 3 }with transition matrix

P=

1 2 0

1 2 1 0 0 1 2

1 2 0

.

(i) Starting in state 1 , find the expected return time to state 1.

(ii) Findlimn→∞Pi[Xn=j]fori, j= 1, 2 , 3.

Question 5 A knight moves around on the 8 × 8 chessboard, each time taking one of the available moves uniformly at random. (A knight move is L-shaped, having length 2 squares in one direction, and 1 square in a perpedicular direction.) If the knight starts at the lower left corner of the board, find the expected number of steps it takes until it returns to the same square.

Question 6 LetPbe a transition matrix onI. Define the transition matrixPonI:=I×Iby the formula

p

(

(i, j),(k, ℓ)

)

:=pikpjℓ, i, j, k, ℓ∈I.

Show that ifPis irreducible and aperiodic, thenPis irreducible. (This is a claim used in Lecture 12.)

Question 7 LetPbe an irreducible transition matrix onI. Complete the proof of Theorem 1 from Lecture 11 using the steps below, referring to Proposition 1 from the notes.

(a) Show that if some stateiis positive recurrent, then there exists an invariant distribution. Hint: Use thatγ(i)P=γ(i), and putπj=γj(i)/mi,j∈I. Why ismi<∞?

(b) Show that if there exists an invariant distributionπ, then all states are positive recurrent. Hint: AssumingπP=πandπi> 0 , putηj=πj/πi,j∈I. ThenηP=ηand henceη≥γ(i).

Was this document helpful?

MA20225-sheet 6-2022 - help with exercise sheets for the given lecture notes and problems

Module: Probability 2A (MA20224)

13 Documents
Students shared 13 documents in this course

University: University of Bath

Was this document helpful?
MA20225 Probability 2B: Example Sheet 6
Questions 1–2 are to be discussed in the tutorials. Questions 3–4 are homework. Questions 5–7 will
be discussed in the Friday problems class.
Question 1 Let Pbe the transition matrix of a simple random walk on the half-line I={0,1,2, . . . }.
That is,
pi,i+1 =p, i ≥0pi,i−1= 1 −p, i ≥1p0,0= 1 −p.
(i) Show that if 0< p < 1, then Pis irreducible.
(ii) Determine, depending on the value of p, whether the random walk is positive recurrent, null
recurrent, or transient. Determine the invariant distribution, when it exists.
(iii) Find limn→∞ p(n)
ii for i= 0,1,2, . . . , in the positive recurrent case.
(iv) Find the expected return time to state ifor i= 0,1,2, . . . .
(v) Complete parts (ii) and (iv) for simple random walk on Zas well.
Question 2
Consider a simple random walk on the vertices of an octahedron (see Sheet 4 for a picture). That
is, if the walk is currently at vertex i, it chooses one of the incident edges uniformly at random, and
moves along that edge.
(i) Find the transition matrix P.
(ii) Is Pirreducible? Show that if πis the uniform distribution on the vertices, then πis invariant,
and it is the unique invariant distribution.
(iii) Find the expected return time to each state.
(iv) Find the period.
(v) Determine limn→∞ p(n)
ij for any pair of states i, j.
Recall from the lectures that in any irreducible Markov chain on a finite state space, all states are
positive recurrent and there is a unique invariant distribution π. The following question explores
what happens when the state space is finite but the Markov chain is not irreducible.
Question 3
Let Pbe a transition matrix on a finite state space I.
(i) Show that there is at least one recurrent communicating class.
(ii) Let Tand Rdenote the set of transient and recurrent states, respectively. Let PRdenote the
restriction of Pto R, i.e. the matrix
(PR)ij =pij , i, j ∈R.
Show that PRis a transition matrix on R.

Students also viewed

Related documents