Yep! I fully admit that this was one of the biggest fears I had, and still sometimes have, when talking to women. Just yesterday I was sat next to a smoking hot indian girl in college, and we were having a nice little chat about whatever, and it just came to a point where I had nothing else to say! Simple as! So I just turned round and started to get on with my work, and after no more than a few seconds she restarted the convo with a question about where I lived or something, and bang it was back on!
So because she could clearly see that I was completely relaxed with the silence and wasn't really trying at all to keep her attention, she naturally felt attracted to me. Awesome - if only I had known what I know now all them years ago!