It has been theoretically predicted and experimentally shown that circular coaxial aperture arrays have higher transmissivities with respect to simple circular ones. This observation is mainly attributed to the propagating waveguide modes supported by the circular coaxial unit cell. In this letter, we investigate extraordinary light transmission in simple rectangular and coaxial rectangular aperture arrays through decaying TE waveguide modes at mid-infrared wavelengths. We demonstrate enhanced transmissions for the rectangular coaxial aperture arrays with respect to simple ones indicating that the enhancement of extraordinary light tranmission in coaxial structures can not be simply explained by the presence of propagating waveguide modes. Using 3-D FDTD simulations and experimental analysis of the localized plasmons at the aperture rims of the individual apertures, the nature and the enhancement of extraordinary light transmission for the coaxial apertures are shown. Shape anisotropy of the apertures is utilized for polarization control of the transmitted light through the total suppression of the desired polarizations. Depolarization ratios larger than the commercially available holographic wire grid polarizers are obtained. The reported results indicate the underlying physics of enhanced extraordinary transmission in coaxial aperture arrays is intricate and merits further scientific attention while practical applications are possible through the controlling of the aperture shapes.