Abstraction augmented Markov models