Recently, Anshu et al. introduced "partially" smoothed information measures and used them to derive tighter bounds for several information-processing tasks, including quantum state merging and privacy amplification against quantum adversaries [IEEE Trans. Inf. Theory 66, 5022 (2020)]. Yet, a tight second-order asymptotic expansion of the partially smoothed conditional min-entropy in the i.i.d. setting remains an open question. Here we establish the second-order term in the expansion for pure states, and find that it differs from that of the original "globally" smoothed conditional min-entropy. Remarkably, this reveals that the second-order term is not uniform across states, since for other classes of states the second-order term for partially and globally smoothed quantities coincides. In view of the tight bounds on the entanglement cost of state merging in terms of the partially-smoothed conditional min-entropy by Anshu et al., this indicates that the second-order asymptotic rate for that task will not separate so neatly into a term depending on the state (the variance) and a term depending on the error (the quantile). Finally, by relating the task of quantum compression to that of quantum state merging, our derived expansion allows us to determine the second-order asymptotic expansion of the optimal rate of quantum data compression. This closes a gap in the bounds determined by Datta and Leditzky [IEEE Trans. Inf. Theory 61, 582 (2015)], and shows that the straightforward compression protocol of cutting off the eigenspace of least weight is indeed asymptotically optimal at second order.