A boundary variation method for the forward modeling of multilayered diffraction optics is presented. The approach permits fast and high-order accurate modeling of periodic transmission optics consisting of an arbitrary number of materials and interfaces of general shape subject to plane-wave illumination or, by solving a sequence of problems, illumination by beams. The key elements of the algorithm are discussed, as are details of an efficient implementation. Numerous comparisons with exact solutions and highly accurate direct solutions confirm the accuracy, the versatility, and the efficiency of the proposed method. (C) 2004 Optical Society of America.