When stimulated with a pulse from an exogenous WNT pathway activator, small aggregates of mouse embryonic stem cells (ESCs) can undergo embryo-like axial morphogenesis and patterning along the three major body axes. However, these structures, called gastruloids, currently lack the anterior embryonic regions, such as those belonging to the brain. Here, we describe an approach to generate gastruloids that have a more complete antero-posterior development. We used hydrogel microwell arrays to promote the robust derivation of mouse ESCs into post-implantation epiblast-like (EPI) aggregates in a reproducible and scalable manner. These EPI aggregates break symmetry and axially elongate without external chemical stimulation. Inhibition of WNT signaling in early stages of development leads to the formation of gastruloids with anterior neural tissues. Thus, we provide a new tool to study the development of the mouse after implantation in vitro, especially the formation of anterior neural regions.