About 4 weeks ago, my left labia minora became very swollen after being itchy. I was rather freaked out by it so i decided to go to the doctors, who prescribed me a thrush duo treatment. I took the treatment and the itchyness went away, although there was still some swelling. I decided to go back to the doctors, which then they prescribed me another cream (Fucidin H) to use. I used this and the swelling went away gradually.
I have avoided sex, and any sexual contact with my bf since the swelling happened. I keep good hygiene by bathing everyday, wear cotton knickers and avoid using any soaps or gels that could irritate 'my area'. 
BUT on monday, I felt some mild discomfort and realised that my left labia minora had become slightly swollen again (not as severe as last time). I haven't touched down there so i don't know what could have triggered it. 
The only thing i can think of is stress levels causing hormone imbalance for it to happen as when it has occured, I have been emotionally stressed on the occasions. 
Does anyone know what could be causing this or had any similar problems? 

P.S My doctor gave me the all clear on any infections.