Skip to document
Welcome to StudocuSign in to access the best study resources
Guest user
0followers
HomeMy LibraryAsk AI
  • You don't have any recent items yet.
My Library
  • You don't have any modules yet.
  • You don't have any books yet.
  • You don't have any Studylists yet.

MA20225-sheet 5-2022 - help with exercise sheets for the given lecture notes and problems

help with exercise sheets for the given lecture notes and problems
Module

Probability 2A (MA20224)

13 Documents
Students shared 13 documents in this course
Academic year: 2021/2022
Uploaded by:
Anonymous Student
This document has been uploaded by a student, just like you, who decided to remain anonymous.
University of Bath

Comments

Please sign in or register to post comments.

Students also viewed

Related documents

Preview text

MA20225 Probability 2B: Example Sheet 5

Questions 1–2 are to be discussed in the tutorials. Questions 3–4 are homework. Questions 5–6 will be discussed in the Friday Problems Class.

Question 1(§1, §1) A coin with probability 1 / 2 of heads is tossed repeatedly, giving the sequence of results ξ 1 , ξ 2 , ξ 3 ,... where eachξiis eitherH (head) orT (tail). Forn≥ 0 , defineXnto be the pattern{ξn+1ξn+2}. Thus, if the coin sequence isHT T H... thenX 0 =HT, X 1 =T T, X 2 =T Hand so on.

(i) Show thatXis a Markov chain and determine the state space and the one-step tran- sition matrix.

(ii) How many tosses does it take on average to first getHT?

(iii) What is the expected number of tosses to get the first run of two identical tosses? (For example this gives 2 if the sequence isT T HT H... and 5 if it isT HT HH.. ..)

Question 2(§1)

(i) Determine the communicating classes for the transition matrix

P ̃=

1 /4 1/4 1/ 2

1 /3 1/3 1/ 3

0 0 1

.

(This is from Sheet 4, Question 2.)

(ii) For each state in part (i), determine if it is recurrent or transient.

Question 3(§1) LetCN = { 0 , 1 ,... , N− 1 }denote the vertices of a cycle of length N. Each vertex can be either vacant or occupied by a particle. At each step, a pair of neighbouring vertices is chosen at random, and their status is exchanged (i. if both were occupied or both vacant, there is no change, and if exactly one of them was occupied, the particle there moves to the other vertex). Determine the state space and the communicating classes of this Markov chain.

Question 4(§1) Let(Xn)n≥ 0 be a Markov chain on the countable state spaceI.

(i) LetC ⊂I be a recurrent class. Show that if the Markov chain starts inC, it never leaves it. That is, for alli∈Cwe havePi[Xn∈Cfor alln≥0] = 1.

(ii) LetC ⊂I be a communicating class, and assume thatC is finite. Show that if it is impossible to leaveC, then all states inCare recurrent. That is, if for alli∈C,j6∈C we havei6→j, andCis finite, thenCis recurrent.

(iii) Deduce that in a finite irreducible Markov chain every state is recurrent.

Question 5 LetG= (V, E)be a finite graph (multiple and loop-edges allowed). (i) Show that simple random walk onGis irreducible if and only ifGis connected. (ii) Show that simple random walk onGhas either period 1 or 2. Can you characterise these two cases?

Recall the following notation from Lecture 9. Given a Markov chain∑ (Xn)n≥ 0 , let Vi = ∞ n=0 1 Xn=idenote the number of visits to statei(including any visit at time 0 ).

Question 6(§1) Let(Xn)n≥ 0 be an irreducible Markov chain on the state spaceI. DenoteGij :=Ei[Vj], i, j∈I. Recall that we showed in Lecture 9 thatGii= 1 − 1 Fii,i∈I, whereFij=Pi[Tj<∞], and ifFii= 1, then 1 / 0 is interpreted as∞.

(i) Show that fori 6 =jwe haveGij=Fij 1 − 1 Fjj.

(ii) Show that eitherGij=∞for alli, j∈I, orGij<∞for alli, j∈I. What do these two cases correspond to?

(iii) Using the fact thatGii=

∑∞

n=0p

(n) ii , deduce that ifi∈Iis transient, thenlimn→∞p

(n) ii = 0.

(iv) Show that ifi∈Iis recurrent, thenlimn→∞p(iin)can be either 0 or> 0 , depending on the Markov chain: on the one hand, consider a two-state example, on the other, consider simple random walk onZwithp= 1/ 2. For the latter use Stirling’s formula: n!∼

√

2 πnn n enasn→∞.

Was this document helpful?

MA20225-sheet 5-2022 - help with exercise sheets for the given lecture notes and problems

Module: Probability 2A (MA20224)

13 Documents
Students shared 13 documents in this course

University: University of Bath

Was this document helpful?
MA20225 Probability 2B: Example Sheet 5
Questions 1–2 are to be discussed in the tutorials. Questions 3–4 are homework. Questions
5–6 will be discussed in the Friday Problems Class.
Question 1 (§1.1, §1.3)
A coin with probability 1/2of heads is tossed repeatedly, giving the sequence of results
ξ1, ξ2, ξ3, . . . where each ξiis either H(head) or T(tail). For n≥0, define Xnto be the
pattern {ξn+1ξn+2}. Thus, if the coin sequence is HT T H . . . then X0=HT ,X1=T T ,
X2=T H and so on.
(i) Show that Xis a Markov chain and determine the state space and the one-step tran-
sition matrix.
(ii) How many tosses does it take on average to first get HT ?
(iii) What is the expected number of tosses to get the first run of two identical tosses?
(For example this gives 2if the sequence is T T HT H . . . and 5if it is T HT HH . . . .)
Question 2 (§1.4)
(i) Determine the communicating classes for the transition matrix
˜
P=
1/4 1/4 1/2
1/3 1/3 1/3
0 0 1
.
(This is from Sheet 4, Question 2.)
(ii) For each state in part (i), determine if it is recurrent or transient.
Question 3 (§1.4)
Let CN={0,1, . . . , N −1}denote the vertices of a cycle of length N. Each vertex can
be either vacant or occupied by a particle. At each step, a pair of neighbouring vertices is
chosen at random, and their status is exchanged (i.e. if both were occupied or both vacant,
there is no change, and if exactly one of them was occupied, the particle there moves to
the other vertex). Determine the state space and the communicating classes of this Markov
chain.
Question 4 (§1.4)
Let (Xn)n≥0be a Markov chain on the countable state space I.
(i) Let C⊂Ibe a recurrent class. Show that if the Markov chain starts in C, it never
leaves it. That is, for all i∈Cwe have Pi[Xn∈Cfor all n≥0] = 1.

Students also viewed

Related documents