We present the first results from the CANDELS-Area Prism Epoch of Reionization Survey (CAPERS), utilizing PRISM observations with the JWST/NIRSpec microshutter assembly in the PRIMER-UDS field. With just 14% of the total planned data volume, we spectroscopically confirm two new bright galaxies (MUV ∼ −20.4) at redshifts z = 10.562 ± 0.034 and z = 11.013 ± 0.028. We examine their physical properties, morphologies, and star formation histories, finding evidence for recent bursting star formation in at least one galaxy thanks to the detection of strong rest-frame equivalent width (EW0 ∼ 70 Å) Hγ emission. Combining our findings with previous studies of similarly bright objects at high z, we further assess the role of stochastic star formation processes in shaping early galaxy populations. Our analysis finds that the majority of bright (MUV ≲ −20) spectroscopically confirmed galaxies at z > 10 were likely observed during a starburst episode, characterized by a median star formation rate (SFR) ratio of SFR10/SFR100 ∼ 2, although with substantial scatter of 0.24 dex. Our work also finds tentative evidence that z > 10 galaxies are more preferentially in a bursting phase than similarly bright z ∼ 6 galaxies. We finally discuss the prospects of deeper spectroscopic observations of a statistically significant number of bright galaxies to quantify the true impact of bursting star formation on the evolution of the bright end of the UV luminosity function at these early epochs.