If he already has HSV 1 orally, then you giving him oral sex would be extremely unlikely to give him oral sex since he already has antibody and an immune response to HSV 1 of his own. Yes, you are exactly right about that. If I was in your situation, I wouldn’t worry at all about this.
Were you swab tested to know that you symptoms were caused by HSV 1?
Terri