should i stop letting my friend use my house to have sex because it makes my gf uncomfortable

im gonna be honest it doesnt bother me. when i was like 19 (im24 now ) my friend use to let me use his house for sex. since his family moved back to his house ive returned the favor and let him bring his gf and use my guestroom. he always calls before and he even brings me beer as like a thank you whenever he does it(never asked just something he does) i literally dont care but my gf told me that it weirds her out since this is gonna be the house our kids grow up in(weve been dating a year) obviously if my kids are around it wouldnt happen but i dont think this is something i should make a fuss over now. im basically returning the favor from years ago and this friend is basically my brother. he washes the sheets and everything before he leaves(which is actually more than i did when i was 19. i would feel like an asshole because honestly this is the least i could do for him. its not an everyday thing and i never even notice he was there.

my question for you is should i put a stop to it? my gf has always been annoyed by my friend so i kind of think this is her trying to stick it to him. i doubt wed break up if i told her no but i want a third opinion

edit:thanks for the help. ive decided to pretty much do nothing. i honestly dont care and this is a stupid thing to worry about. im not gonna tell her that he still does it