Vanessa recommenda handbook of markov decision processes. Havent read it. Looks interesting. Cant imagine it's the best resource