Polarization codes were recently introduced by Ar\i kan. They are explicit code constructions which achieve the capacity of arbitrary symmetric binary-input discrete memoryless channels (and even extensions thereof) under a low complexity successive decoding strategy. The original polarization construction is closely related to the recursive construction of Reed-Muller codes and is based on the $2 \times 2$ matrix $\bigl[ \begin{smallmatrix} 1 &0 \ 1& 1 \end{smallmatrix} \bigr]$. It was shown by Ar\i kan and Telatar that this construction achieves an error exponent of $\frac12$, i.e., that for large enough blocklengths the error probability decays exponentially in the square root of the blocklength. It was already mentioned by Ar\i kan that in principle larger matrices can be used. The fundamental question then is to see whether there exist matrices with exponent strictly exceeding $\frac12$. Based on a recent result by Korada and \c Sa\c so\u glu, which shows that any $\ell \times \ell$ matrix none of whose column permutations is upper triangular polarizes symmetric channels, we first characterize the exponent of a given square matrix. We then derive upper and lower bounds on the exponents of matrices. Using these bounds we show that there are no matrices of size less than $15$ with exponents exceeding $\frac12$. Further, we give a general construction based on BCH codes which for large $n$ achieves exponents arbitrarily close to $1$ and which exceeds $\frac12$ for size $16$.