He had semen on his hands and he fingered me right after wiping it off with a tissue

If he still had any wet cum on his fingers than there is a small possibility. It will greatly depend where you are at on your menstrual cycle. Mid cycle is normally more risky for pregnancy then very beginning or towards the end. (I.E. in a 28 day cycle days 8 to 20 are considered risky)