The phase transition behaviors of (1-x)Pb(Yb1/2Nb1/2)O3-xBa(Yb1/2Nb1/2)O-3 (PBYN) solid solution system (0 less than or equal tox less than or equal to0.30) have been investigated by X-ray diffraction, permittivity and E-P hysteresis loop measurements. In the solid solution, for x less than or equal to0.10, the transition shows a normal first-order paraelectric (PE)-antiferroelectric (AFE) behavior and its AFE crystal structure is orthorhombic. The transition temperature gradually decreases with the increasing Ba2+ concentration. On the composition range 0.16 less than or equal tox less than or equal to0.30, a typical relaxor behavior is displayed. The low temperature crystal structure is pseudocubic in this composition range. (C) 2001 Published by Elsevier Science Ltd.