A measurement of the differential branching fraction of the decay B-0 -> K* (892)(0) mu(+)mu(-) is presented together with a determination of the S-wave fraction of the K+ pi(-) system in the decay B-0 -> K+ pi-mu(+)mu(-). The analysis is based on pp-collision data corresponding to an integrated luminosity of 3 fb(-1) collected with the LHCb experiment. The measurements are made in bins of the invariant mass squared of the dimuon system, q(2). Precise theoretical predictions for the differential branching fraction of B-0 -> K* (892)(0) mu(+) mu(-) decays are available for the q(2) region 1.1 < q(2) < 6.0 GeV2/c(4). In this q(2) region, for the K+pi(-) invariant mass range 796 < m(K pi) < 996MeV/c(2), the S-wave fraction of the K+pi(-) system in B-0 -> K+pi(-)mu(+)mu(-) decays is found to be F-S - 0.101 +/- 0.017(stat) +/- 0: 009(syst), and the differential branching fraction of B-0 -> K* (892)(0) mu(+)mu(-) decays is determined to be dB/dq(2) = (0.392(-0.019)(+ 0.020)(stat) +/- 0.010(syst) +/- 0.027(norm)) x 10(-7) c(4)/GeV2. The differential branching fraction measurements presented are the most precise to date and are found to be in agreement with Standard Model predictions.