Trouble with erection

(please dont check my spelling) Ok so heres this problem ive been having for about a year now. Getting straight to the point, I have trouble keeping it up during sex much more often then i should. And sometimes it doesnt even get completley hard. Like I can usually start off ok but after a little time it starts to go soft and I have to really try hard to keep it from going soft. not every time but often. If I try and insert it into her and she isnt very wet yet, and there is some effort involved in getting it in all the way, it goes soft immediatley. I can usually get it back quickly if she touches me again, but its embaressing that she has to do that. And also I havent been lasting nearly as long as I used to. I used to last at least 7 to 10 minutes every time and now im lucky if I can last 4. If i try to fight off orgasm too many times ill just completley lose the erection. I am 24 and i go to the gym at least twice a week, and im in pretty good shape, so I dont know why this would be happening. This is waring heavily on my mind and especially my ego. Ive never ever had this problem with my ex-girlfriend whom I was with for 5 years. Ive been sexually active since I was 17 and it always made me feel good that I was able to perform above average every time, but now im lucky if i can keep it up for a full 2 or 3 minutes. It used to give me great confidence knowing i was good at sex, not my confidence level and self esteem has plummeted.I have been with my current girlfriend for over 9 months now and since we have been together I have had this problem. Im running out of excuses. I want to show her a great time like i did with other girls but i just cant anymore. My ex-girlfriend and i used to dabble in drug use so for a while i thought that had something to do with it, but i havent done any drugs in a long time and this problem still persists. I realize that im rambling at this point but im very close to my wits end. I dont know what to do. Im thinking maybe I damaged the muscle tissue in my penis because for a while I used this masturbation technique where I would simply rub it somewhat hard thru my pants. I did this a lot for quite a while and sometimes it would leave my penis feeling sore. I seem to be comming up with a lot of strange reasons for this problem but I doubt that Ive figured it out. Im terrified of talking about this with my doctor because although I trust him, he is the phisycian for my entire family and it just makes me feel very strange. In a nutshell, my penis isnt working right and im very depressed because of it... So, I thank you for taking the time to read this rambling nonsense, and would greatly appreciate some serious advice.