Pap Test and Virginity

I'm rather over due for a pap test, but my first and only experience was just... awful for me, mentally. I was and am a virgin. There's been masturbation here and there, but not penetration. I use tampons and I used to ride horses. Not to mention I was a rough and tumble kid. So. Here's my dilemma:

After we did the wonderful pap test, my doctor asks me when I started to have sex. I was confused, because I had already told her that I didn't, so I say "I haven't." She looks at me again, this is the part that made me uncomfortable, the sheer condescending disbelief. "When did you have sex?" So I tell her again, I have not. She then says "You are not a virgin." Just thinking about it makes me cry :p Just the humiliation and confusion and the look she was giving me like I was hiding some massive secret or that I was an id**t. Anyways, she goes on to list that 'masturbation' can be the cause, and some other things like childhood curiousity, because, really, I only know of tampons ever being inserted.

I'm not going back to her for my test, but what I really want to know is how can doctors tell if you're a virgin? I've been trying to find an answer and, unless you recently had sex, the doctor can't really tell unless your hymen is still intact. However, a lot of hymens break despite not having sex from various things. Ugh. Can anyone just ease my mind?