Do men really care about a womans breast size?.....asked the women with low BREAST-esteem

I know that this is pretty normal for women but for me its way different, ones a C and the other is pretty much an A. when I was 7 yr old I had cancer. A tumor was found on my collar bone it was removed just fine but to be safe Doc's gave me chemo and radiation therapy. All that said, ever since going thru puberty I've been really self-conscious because its a little noticeable even with my clothing on. I always, always have on a padded bra to help "even" things out. I'm 24 now and it makes me self conscious with guys. I insist on wearing my bra even during sex. I've thought about getting breast implants to even things out, my only worry is that I remember asking my Doctor when I was 17 about implants but He said he would not recommend it because the Cancer could come back and the implants would make it hard to detect it.

Even though I've been Cancer free and healthy for 17 years and counting :) I still always wondered if I could do something to fix my breast or if I should just build a bridge and get over it? I just always have a fear that I will get into a serious relationship with a guy but when things get a hot an heavy and he sees my breast he'll get weirded out and i'd never hear from him again or he'd just end up cheating on my with a more "proportional" woman.

My main question is are guys really that concerned about breast? should I get a few more options from doctors on the idea of getting implants? or could hormone shots work.... don't guys who want to become women take hormone injections to make them grow breast and stuff?