Is it wrong to continue a friendship with a past lover. He is married and so am I

We live in different states but we talk on the phone and email. We flirt and talk about how special our affair was. He was married then too. I feel like we are falling in love again but maybe not.