  Extended Dynamic Mode Decomposition (EDMD) is an algorithm that approximates
the action of the Koopman operator on an $N$-dimensional subspace of the space
of observables by sampling at $M$ points in the state space. Assuming that the
samples are drawn either independently or ergodically from some measure $\mu$,
it was shown that, in the limit as $M\rightarrow\infty$, the EDMD operator
$\mathcal{K}_{N,M}$ converges to $\mathcal{K}_N$, where $\mathcal{K}_N$ is the
$L_2(\mu)$-orthogonal projection of the action of the Koopman operator on the
finite-dimensional subspace of observables. In this work, we show that, as $N
\rightarrow \infty$, the operator $\mathcal{K}_N$ converges in the strong
operator topology to the Koopman operator. This in particular implies
convergence of the predictions of future values of a given observable over any
finite time horizon, a fact important for practical applications such as
forecasting, estimation and control. In addition, we show that accumulation
points of the spectra of $\mathcal{K}_N$ correspond to the eigenvalues of the
Koopman operator with the associated eigenfunctions converging weakly to an
eigenfunction of the Koopman operator, provided that the weak limit of
eigenfunctions is nonzero. As a by-product, we propose an analytic version of
the EDMD algorithm which, under some assumptions, allows one to construct
$\mathcal{K}_N$ directly, without the use of sampling. Finally, under
additional assumptions, we analyze convergence of $\mathcal{K}_{N,N}$ (i.e.,
$M=N$), proving convergence, along a subsequence, to weak eigenfunctions (or
eigendistributions) related to the eigenmeasures of the Perron-Frobenius
operator. No assumptions on the observables belonging to a finite-dimensional
invariant subspace of the Koopman operator are required throughout.
