Standard version of recurrent neural network language model (RNNLM) has shown modest results in language modelling of Russian. In this paper we present a special modification of RNNLM making separate predictions of lemmas and morphology. New model shows superior results compared to Knesser-Ney language model both in perplexity and in ranking experiment. At the same time morphology integration has not shown any improvement.