I had few hours period, later I had brown discharge

I had sex with my boyfriend on feb 14 with a condom. I was not on the pill at this point, but picked it up a couple of days later. On 22nd feb (i think) I got what I thought was my period, so took my first pill. It was brown-ish discharge and there was a very small amount of blood. But weirdly, that was basically all the blood there was? It just stopped after a few hours. Now I'm taking my pill every day, so my period wouldn't come anyway, but I just had a small amount of the same brown discharge so now I basically don't know what's going on. (I know bleeding between is common when starting on the pill, but the lack of period thing isn't).

I don't think I'm pregnant, I have generalised anxiety disorder and health/body related things are a massive trigger for me, so I'm thinking my freaking out is probably down to this. The weird one day period thing has happened to me once before, and I just got the proper one about 2 weeks later, so as I'm quite irregular usually this isn't necessarily a cause for concern.

However, I also saw something about implantation bleeding being similar to what I descibed, and I've just gone into panic mode. 

We've always been really safe.. Do I come off the pill now and see if I get my period? Help..