Blood pressure (BP) is an important indicator for prevention and management of cardiovascular diseases. Alongside the improvement in sensors and wearables, photoplethysmography (PPG) appears to be a promising technology for continuous, non-invasive and cuffless BP monitoring. Previous attempts mainly focused on features extracted from the pulse morphology. In this paper, we propose to remove the feature engineering step and automatically generate features from an ensemble average (EA) PPG pulse and its derivatives, using convolutional neural network and a calibration measurement. We used the large VitalDB dataset to accurately evaluate the generalization capability of the proposed model. The model achieved mean errors of -0.24 +/- 11.56 mmHg for SBP and -0.5 +/- 6.52 mmHg for DBP. We observed a considerable reduction in error standard deviation of above 40% compared to the control case, which assumes no BP variation. Altogether, these results highlight the capability to model the dependency between PPG and BP.