Bounding the Number of Calls to PS(M)

We've seen:
For every $l_I>\frac{\ln{\varepsilon(1-\gamma)}}{\ln\gamma}$,

$$\Pr( \left\Vert\widehat{v}_{l_I}-v^*\right\Vert \geq \varepsi... ... \frac{\gamma^{l_I}}{1-\gamma})}{\gamma(1-\gamma^{l_I})}}^2}$$ (22)

Let's find m_I and l_I such that

$$\Pr( \left\Vert\widehat{v}_{l_I}-v^*\right\Vert \geq \varepsilon) \leq \delta$$ (23)

By the previous claims, it suffices to find m_I and l_I s.t.:

1.

$l_I > \frac{\ln{(\varepsilon(1-\gamma))}}{\ln\gamma}$

2.

$\delta = l_I\cdot\ \vert S\vert\cdot \vert A\vert\cdot 2e^{-2\cdot m_I\cdot (... ...repsilon - \frac{\gamma^{l_I}}{1-\gamma}) } { \gamma (1-\gamma^{l_I}) } })^2}$

Note that $log_{\gamma}{2} > 0$ (since $0<\gamma<1$). Then we can set (taking upper value if necessary):

  $$\begin{eqnarray*} l_I &=& \frac{\ln{(\varepsilon(1-\gamma))}}{\ln\gamma} + ... ... log_{\gamma}{2}\\ &=& \log_\gamma(2\varepsilon(1-\gamma)) \end{eqnarray*}$$

The number of calls to PS(M) is m_I:

Here, again, m_I and l_I must be natural numbers, but taking their upper value won't change the asymptotic complexity, so, the number of calls to PS(M) by the indirect algorithm is:
$O(\frac{1}{\varepsilon^2} \cdot \ln{(\frac{\vert S\vert\cdot \vert A\vert}{\delta})} + \frac{1}{\varepsilon^2} \cdot \ln\ln\frac{1}{\varepsilon} )$