Why can't I maintain a erection with the woman I want to have a relationship with? I'm 58

I'm a 58 yr old healthy man whom is in love with a lovely 56 yr. old woman. We have been in a relationship for the past couple years.  Everything is great with us exept the 2 times we have attemped sex I started out great but I lost my erection. I've never had that problem before. Because of the 2 failures I have had sex with a few prostitutes and everything went great (not something I want to continue). They raved about my sexual performance. The woman I love was very dry even after fore-play and when I tried to enter her I just couldn't get it in. After about 3-4 minutes I felt it was me and I lost my erection and I had that "fight ot flight" reaction and I couldn't get the erection back. I'm embarraced, feeling less than a man and inadequate. I know that I can perform but with the one I really want I or we haven't been able to consimate that part of the relationship. I really feel that this could be the deciding factor of our relationship going forward or not. It is a shame if we can't because of my disfunction (if ii is a disfuction...I don't know). I even took a ED pill with my love one, as well as with the prostitutes. 

I don't know what to do. Can someone give me some sound advise???