People forget things. I agree in your case there was a difference, but personally, in a friendly casual game, I would be fine with it, especially if it's someone whom I enjoy playing with. I would rather play many more games with them in the future, than only play one and have them think me a poor sportsman, right or wrong.