Charge-dependent anisotropy Fourier coefficients ($v_n$) of particle azimuthal distributions are measured in pPb and PbPb collisions at $ \sqrt{\smash[b]{s_{_{\mathrm{NN}}}}} = $ 5.02 TeV with the CMS detector at the LHC. The normalized difference in the second-order anisotropy coefficients ($v_2$) between positively and negatively charged particles is found to depend linearly on the observed event charge asymmetry with comparable slopes for both pPb and PbPb collisions over a wide range of charged particle multiplicity. In PbPb, the third-order anisotropy coefficient, $v_3$, shows a similar linear dependence with the same slope as seen for $v_2$. The observed similarities between the $v_2$ slopes for pPb and PbPb, as well as the similar slopes for $v_2$ and $v_3$ in PbPb, are compatible with expectations based on local charge conservation in the decay of clusters or resonances, and constitute a challenge to the hypothesis that the observed charge asymmetry dependence of $v_2$ in heavy ion collisions arises from a chiral magnetic wave.