I'm no expert but maybe you are on the right track by switching to a boring conversation, talking about where you live where you work and so so is pretty dull.

Maybe go back to the guessing game of where you live and being a bit playful with it you could maybe keep the banter going?