just remember that you are not their entertaining monkey for the night, silences can be powerful-- sure be awesome and lay down the banter until youve hit a level of comfort with them, but dont fear a silence - see if they fill it themselves--if not they dont give a shit about the intereaction and you, so eject
Me 'im bored now im off'

remember you are high value they should contribute to the intereaction