When I say play it down I mean play the going out part down, not the compliment. Don't make it a big deal, sound too serious etc etc. Keep it casual and there's no reason someone should say no other other than genuinely not being free.