In this research, a generalized machine learning (ML) framework is proposed to estimate the fatigue life of epoxy polymers and additively manufactured AlSi10Mg alloy materials, leveraging their failure surface void characteristics. An extreme gradient boosting algorithm-based ML framework encompassing Synthetic Minority Over-sampling TEchnique (SMOTE), categorical data encoding, and external loop cross-validation is developed to evaluate the fatigue life across materials. The influence of different training strategies based on materials, input features, encoding method, and data standardization on the model performance is explored. Additionally, the importance of anti-data-leakage and anti-overfitting measures over the ML model performance is addressed. The result shows that the data-leakage-free, external loop cross-validated model can estimate the fatigue life of selective epoxy polymers and metal alloys with an average R2 of 0.71 ± 0.06 using a mere 12 to 27 experimental data points per material category. Whereas the model trained with data-leakage and overfitting results in high R2 of 0.9.