Background Oral health in nursing home residents is poor. Robust, mandated assessment tools such as the Resident Assessment Instrument – Minimum Data Set (RAI-MDS) 2.0 are key to monitoring and improving quality of oral health care in nursing homes. However, psychometric properties of RAI-MDS 2.0 oral/dental items have been challenged and criterion validity of these items has never been assessed. Methods We used 73,829 RAI-MDS 2.0 records (13,118 residents), collected in a stratified random sample of 30 urban nursing homes in Western Canada (2007–2012). We derived a subsample of all residents (n = 2,711) with an admission and two or more subsequent annual assessments. Using Generalized Estimating Equations, adjusted for known covariates of nursing home residents’ oral health, we assessed the association of oral/dental problems with time, dentate status, dementia, debris, and daily cleaning. Results Prevalence of oral/dental problems fluctuated (4.8 %–5.6 %) with no significant differences across time. This range of prevalence is substantially smaller than the ones reported by studies using clinical assessments by dental professionals. Denture wearers were less likely than dentate residents to have oral/dental problems (adjusted odds ratio [OR] = 0.458, 95 % confidence interval [CI]: 0.308, 0.680). Residents lacking teeth and not wearing dentures had higher odds than dentate residents of oral/dental problems (adjusted OR = 2.718, 95 % CI: 1.845, 4.003). Oral/dental problems were more prevalent in persons with debris (OR = 2.187, 95 % CI: 1.565, 3.057). Of the other variables assessed, only age at assessment was significantly associated with oral/dental problems. Conclusions Robust, reliable RAI-MDS 2.0 oral health indicators are vital to monitoring and improving oral health related quality and safety in nursing homes. However, severe underdetection of oral/dental problems and lack of association of well-known oral health predictors with oral/dental problems suggest validity problems. Lacking teeth and not wearing dentures should be considered an indicator for urgent oral/dental treatment needs. Electronic supplementary material The online version of this article (doi:10.1186/s12963-016-0108-y) contains supplementary material, which is available to authorized users.