We studied the changes in the optical properties of the RVO3 series (R = Sr, Ca, La, Y) using band structure calculations. These oxides present a transition from a non-magnetic metallic phase in SrVO3-CaVO3, to an antiferromagnetic insulator state in LaVO3-YVO3. The standard LDA and GGA approach to DFT fails to reproduce the observed band gap in the LaVO3-YVO3 insulating compounds. We show here that the use of the modified Becke-Johnson exchange potential in the calculation of the optical properties solves this problem. In particular, the optical conductivity of the metallic SrVO3-CaVO3 oxides exhibits a Drude contribution, whereas the insulating LaVO3-YVO3 materials reflects the opening of the optical gap. We also show that the calculated optical conductivity of these materials is in good agreement with previous experimental results.