Public cloud platforms might start with homogeneous hardware; nevertheless, because of inevitable hardware upgrades, or adding more capacity, the initial homogeneous platform will gradually evolve into heterogeneous as time passes by. The consequent performance heterogeneity is of concern to cloud users. In this article, we evaluate performance variations from hardware heterogeneity and scheduling mechanisms of public clouds. Amazon Elastic Compute Cloud (Amazon EC2) and Rackspace Cloud are used as the representatives because of their relatively long record and wide usage among small and medium enterprises (SMEs). A comprehensive set of micro-benchmarks and application-level macro-benchmarks have been used to investigate performance variation. Several major contributions have been made. Firstly, we find out that heterogeneous hardware is a commonality among the relatively long-lasting cloud platforms, although the level of heterogeneity varies. Secondly, we observe that heterogeneous hardware is the primary culprit of performance variation of cloud platforms. Thirdly, we discover that varied CPU acquisition percentages and different virtual machine scheduling mechanisms exacerbate the performance variation problem, especially for network related operations. Finally, based on the observations, we propose cost-saving approaches and analyze Nash equilibrium from cloud user perspective. By using a simple "trial-and-error" approach, i.e., keep good-performing instances and discard bad-performing instances, cloud users can achieve up to 30% cost saving.