The ever-increasing demand for rehabilitation of existing structures and construction of new ones, along with environmental issues of the construction sites represent a heavy burden for society in terms of economy and environment. Meanwhile, Ultra-High Performance Fiber Reinforced Concrete (UHPFRC) has proved its potential to be one of the solutions to contain the explosion of the maintenance costs, because of its extremely low permeability and outstanding mechanical properties. Even though using UHPFRC already significantly contributes to decreasing the environmental impact of rehabilitation construction sites by up to 50% when compared to using conventional concretes, further optimization of their mix design with replacement of high environmental cost components like steel fibers and clinker, would be another step towards sustainability. The main objective of this study was to further improve the already established concept of UHPFRC in two main directions: firstly, to develop new strain hardening UHPFRC mixes with reduced environmental impact by replacing the steel fibers with synthetic ones and by replacing 50% vol. clinker with cheap and widely available Supplementary Cementitious Materials, and secondly, to investigate the mechanical properties, delayed response, protective function, and environmental impact of the newly developed material for structural applications. An advanced packing density model was developed to optimize the powder mix that consisted of six different powders. An extensive experimental campaign with a focus on the tensile behavior was carried out to investigate the factors influencing the mechanical properties of the UHPFRC mixes and validate their performances. The achieved new mix named PE-UHPFRC has on average a tensile elastic limit of 7.7 MPa, a tensile ultimate strength of 11.7 MPa, a tensile deformation capacity of more than 3.5%, and a compressive strength of 120 MPa, well adapted for cast-onsite applications of rehabilitation, or strengthening associated to rebar. The delayed response of PE-UHPFRC was investigated with a focus on autogenous deformations and associated eigenstresses under full restraint conditions, using a Temperature Stress Testing Machine. The results showed more than 70% reduction in the eigenstresses in a 50 mm thick PE-UHPFRC layer when compared to those observed in conventional UHPFRC with steel fibers, which is highly beneficial for rehabilitation applications. An original capillary absorption setup was developed to investigate the protective properties of the PE-UHPFRC under tensile loading, at different strain levels ranging from 0.15 to 20‰. Before reaching the tensile elastic limit, the capillary absorption coefficient was extremely low, at 24 g/m2√h, however, a sudden increase in the capillary absorption was observed after the elastic limit under tensile load. Finally, the environmental benefits of PE-UHPFRC combined with rebar were characterized at the structural level, for the strengthening of a bridge with a span of 34 m, and compared with those of: (1) a reconstruction with prestressed concrete, and (2) a strengthening with conventional UHPFRC with steel fibers combined with rebar. The LCA results showed on average 55% and 34% decrease in the environmental impact for the strengthening method with R-PE-UHPFRC compared with the reconstruction with prestressed concrete and strengthening with conventional R-UHPFRC, respectively.