Sometimes you don't know until later.
I have ended a few ST relationships based on this as it generated all kinds of bad feeling. One girl had had a FB for 8 years even though it was only once a year. She wanted to invite me to one of his house parties - I told her to fuck off - it just shows a lack of respect for the person you are with I reckon. "Hi, my name's xxx and I used to bang the chick you were with for sexual release once a year." "Hi, nice to meet you, want a beer so we can compare notes?"
Of course she tried explaining that she suppresses any attraction for this guy as she was with me and felt more for me but I generally end relationships with girls like that as it just leads to bad situations.