The Probalistic Latent Semantic Indexing model, introduced by T. Hofmann (1999), has engendered applications in numerous fields, notably document classification and information retrieval. In this context, the Fisher kernel was found to be an appropriate document similarity measure. However, the kernels published so far contain unjustified features, some of which hinder their performances. Furthermore, PLSI is not generative for unknown documents, a shortcoming usually remedied by "folding them in" the PLSI parameter space. This paper contributes on both points by introducing: - a new, rigorous development of the Fisher kernel for PLSI, addressing the role of the Fisher Information Matrix; we show how it relates to the kernels proposed so far; - a novel and theoretically sound document similarity, which avoids the problem of "folding in" unknown documents through model identification. For both aspects, experimental results are provided on a large information retrieval evaluation set.