Swimming coaches provide regular timed and technical feedback to swimmers and guide them efficiently in training sessions. Due to the complexity of swimmers' performance, which is not visible in qualitative observation, quantitative and objective performance evaluation can better assist the coach in this regard. Inertial measurement units (IMUs) are used in swimming for objective performance evaluation. In this study, we propose a new performance evaluation feedback (SmartSwim) using IMU and investigate its effects on the swimmer's weekly progress. Measurements were conducted each week with 15 competitive swimmers for 10 weeks using a Sacrum IMU. The SmartSwim report included a comprehensive representation of performance based on goal metrics of each phase extracted from the IMU signals. The swimmers were divided into two groups: the experimental and control groups. The SmartSwim report for each swimmer in the experimental group was given to the coach, who used it to adjust the training accordingly. The results showed that the experimental group outperformed the control group when comparing each swimmer, each session and the whole sessions. At the level of each individual, more members of the experimental group showed significant downward trend of average lap time (Mann-Kendall trend test, 95% confidence level). While comparing the sessions, the experimental group showed significantly lower lap time than the control group from the sixth session onwards (p-value < 0.05 from t-test). Considering all sessions, the experimental group showed significantly higher progress, lower average lap time, and more consistent records (Mann-Whitney U test at 95% confidence level) than the control group. This study demonstrated that SmartSwim can assist coaching by quantitatively assessing swimmers' performance, leading to more efficient training.