A novel approach is presented for constructing polynomial chaos representations of scalar quantities of interest (QoI) that extends previously developed methods for adaptation in Homogeneous Chaos spaces. In this work, we develop a Bayesian formulation of the problem that characterizes the posterior distributions of the series coefficients and the adaptation rotation matrix acting on the Gaussian input variables. The adaptation matrix is thus construed as a new parameter of the map from input to QoI, estimated through Bayesian inference. For the computation of the coefficients' posterior distribution, we use a variational inference approach that approximates the posterior with a member of the same exponential family as the prior, such that it minimizes a Kuilback-Leibler criterion. On the other hand, the posterior distribution of the rotation matrix is explored by employing a Geodesic Monte Carlo sampling approach, consisting of a variation of the Hamiltonian Monte Carlo algorithm for embedded manifolds, in our case, the Stiefel manifold of orthonormal matrices. The performance of our method is demonstrated through a series of numerical examples, including the problem of multiphase flow in heterogeneous porous media.