What should I do? :(

My crush kissed me a few days ago, even though he's dating another girl, I know some of you might find that wrong and I respect that, but he's in love with me and her. Him and I are trying to hide the fact that we kissed and almost made out... I feel really bad and so does he, but we want to keep kissing and being together, we have to hide it from everyone and we both want to be seen in public together, I don't know if I should stay with him and stay happy and guilty or leave him and feel heart-broken and soulless. What do you think I should do?? =(