Two ways usually. One is not until I've throughly plumbed it. I usually overstay my time there too, unwilling or unable to see the writing on the wall and leave, because I'm in a serious situation that I couldn't easily leave (professional job etc). Then part of my psyche stages a revolt (I throw the interview, or the solo, or test or whatever) and I have to analyze and get a perspective on what is going on. I find the next thing holistically. It's a feeling that strikes me, Eureka! I want to do this next. This was true for Physics and Music, where I reached professional stature in both when I left.
With other things it becomes clear sooner, usually within one to four years. I get good enough to realize I'll never go farther, or that I've "got it", or that it'll never really satisfy. But I got a good workable knowledge of the field. This was true for art, writing, photography and a bunch of others.
Programming/engineering was different. I had a moment in the first case (I knew I didn't want to work at a job forever), but when faced with what I'd do instead (the same thing - programming/engineering) I came back to the vocation. So it goes in a category on its own.
These examples are for serious interests only, there are many more occasional ones. Also, now having been around the block I take things easier. I've gotten into film - art cinema, foreign film and film history. It's a great way to relax from the day, but knowing myself I try not to go too crazy on it and just have fun. Mostly seems to be working, with a few fits and starts.