The paper aims at minimizing DC voltage oscillations in offshore multiterminal high-voltage direct current (HVDC) grids based on modular multilevel converters (MMCs). The DC voltage stability is a crucial factor in multiterminal HVDC networks since it is associated with the grid power balance. Furthermore, DC voltage oscillations can cause the propagation of significant disturbances to the interconnected AC grids. This paper proposes an optimal control technique based on semidefinite programming to improve the DC voltage stability margins under the worst-case perturbation scenario. A centralized optimal linear feedback controller is introduced to achieve this goal while ensuring compliance with the control inputs' and state variables' constraints. Furthermore, the methodology is adapted to develop a decentralized optimal linear feedback controller with naturally decoupled constraints on the control inputs and state variables. It is shown that the proposed centralized and decentralized optimal linear controllers can minimize the DC voltage oscillations under the worst-case perturbation scenario in the presence or absence of the droop control gain. The performance of these controllers is verified via eigenvalue stability analysis and time-domain simulations of the MMC-based four-terminal HVDC test grid. Finally, a DC voltage oscillation index is introduced as a potential decision-support criterion. Its applicability is exemplified by identifying, among several options, the HVDC link that gives minimum DC voltage oscillations between independent point-to-point networks while considering the wind intermittency effect.