Bioaerosols can affect human and plant health as well as climate. New automatic bioaerosol monitors capable of detecting and classifying pollen and fungal spores in real time have recently been developed, revolutionizing the way how data are collected, analyzed and distributed to the public. However, the technologies, still being very new, have not been adequately characterized and the instruments' performance is poorly understood. Here, we developed a general method for evaluating the performance of both the hardware (particle detector) and software (machine learning algorithm) of automated bioaerosol monitors. For the first time, number concentration measurements were carried out for particle sizes up to 70 μm. To do this, three different reference methods were combined: a custom-made reference optical particle counter, an inkjet aerosol generator (IAG) and particle tracking velocimetry (PTV). The size-dependent counting efficiency and unit-to-unit variability of five different SwisensPoleno Jupiter bioaerosol monitors was thus determined in a traceable manner over almost the entire pollen and fungal spore size range. The classification efficiency of the supervised machine learning (ML) algorithm developed by MeteoSwiss, which is currently being used by various research institutes and monitoring stations in Europe, was determined by delivering well-known pollen taxa to the Poleno monitor under controlled laboratory conditions. The influence of factors, such as environmental conditions and geographic location of the tree, on the classification efficiency was quantified, and recommendations are made for improving ML algorithm training in the future. The methods outlined in this study aim to establish a traceable framework to ensure that real-time bioaerosol measurements, despite the measurement challenges related to large micrometre-sized particles at low concentrations (a few hundred particles per m 3), are carried out at the same level of accuracy as legislated air-quality measurements. This is particularly important as a step toward their integration into European legislation.