A measurement is performed of the cross section of top quark pair production in association with a W or Z boson using proton-proton collisions at a center-of-mass energy of 13 TeV at the LHC. The data sample corresponds to an integrated luminosity of 35.9 fb$^{-1}$, collected by the CMS experiment in 2016. The measurement is performed in the same-sign dilepton, three- and four-lepton final states. The production cross sections are measured to be $\sigma(\mathrm{t}\overline{\mathrm{t}}\mathrm{W})= 0.77 ^{+0.12}{-0.11}\text{(stat)} ^{+0.13}{-0.12}\text{(syst)}$ pb and $\sigma(\mathrm{t}\overline{\mathrm{t}}\mathrm{Z}) = 0.99 ^{+0.09}{-0.08}\text{(stat)} ^{+0.12}{-0.10}\text{(syst)}$ pb. The expected (observed) signal significance for the $\mathrm{t}\overline{\mathrm{t}}\mathrm{W}$ production in same-sign dilepton channel is found to be 4.5 (5.3) standard deviations, while for the $\mathrm{t}\overline{\mathrm{t}}\mathrm{Z}$ production in three- and four-lepton channels both the expected and the observed significances are found to be in excess of 5 standard deviations. The results are in agreement with the standard model predictions and are used to constrain the Wilson coefficients for eight dimension-six operators describing new interactions that would modify $\mathrm{t}\overline{\mathrm{t}}\mathrm{W}$ and $\mathrm{t}\overline{\mathrm{t}}\mathrm{Z}$ production.