Has he ever been tested to know if he is positive or not? Maybe he should be if he hasn’t been so you both know his status.
He may have come in contact with the virus by giving you oral sex, but that doesn’t insure that he will get it orally. If he does, he could get what looks like a cold sore on his lip but it is very unlikely that he will get it all over his face. If he gets a sore on his lip or face, I recommend having it swab tested for herpes.
If he has not sores and the kids touch his face, they won’t get it. If he has a sore on his face or lip, he should simply avoid kissing the kids on their face.