well, there are a few possibilities here. Since you have had no antibody testing done, we don’t know if you might have had herpes for years and didn’t know it. Rough sex may or may not have triggered an outbreak, that is not a given. You need a type specific IgG antibody test. A swab test taken from a lesion a couple of days after it appeared would likely be positive, but not always. A PCR is far more sensitive than a culture, do you know which one was used for the swab test? See if you can find out.
If the new partner had recently been infected with HSV 2, his blood work may not be accurate, correct. Did you actually see the blood work of these people? Do you know if it was an IgG or IgM test? Only IgG should be used.
It is also possible that the lesion could be HSV 1, was everyone tested for that? If you are tested, it should be for both HSV 1 and 2. Have you ever had a cold sore on your lip?
Another option of course is that it isn’t herpes at all.