We had anal sex 5 times then waited 3 weeks and now it hurts??

So my hubby and I were finally able to have anal sex, which was enjoyable for us both, but then we got busy and did not do it for about 3 weeks, we tried the other night and it really hurt, I was completely relaxed and wanted it, and my hubby felt bad hurting me. So if we have done it 5 times already why did it hurt, does the anus get tight again. ?And how can I go back to doing it with him? And how often is best to do it so it does not hurt, but doesn't cause me problems??