Performance differences in the growing number of electromagnetic (EM) sensing systems designed to predict soil water content from dielectric permittivity estimates suggests the need for a standardized characterization methodology. The objectives of this study were to i) develop a methodology for characterizing sensing system performance using well defined and readily available reference dielectrics and ii) suggest a standard for comparison of sensing systems estimating permittivity. Standards are based on fluids of known or measurable frequency-dependent permittivity, which provide a homogeneous system for immersion of sample-scale EM probes. Measurement frequencies for broadband sensing systems can be inferred from correlated network analyzer and sensing system measurements or from manufacturer suggestions. Fluids were selected to provide surrogate soil-related effects (e.g. relaxation) occurring both within and outside of the effective measurement frequency range of common systems. Test conditions included dielectrically relaxing (R) and non-relaxing (NR) as well as electrically conducting (C) and non-conducting (NC) liquids and combinations thereof. No suitable combination of relaxing and conducting (R-C) dielectric fluid was found in this study but remains a goal of future work because it represents the more difficult and often common condition for EM sensing system measurements in soils containing relaxing and conducting elements. Criteria for qualitative assessment of sensing systems include measurement frequency and accuracy, susceptibility to variations in salinity, dielectric relaxation and temperature and spatial variation in probe sampling area. The approach outlined here was applied to seven different EM systems with results being presented in a companion oral presentation by Blonquist et al.