I totally hear you. I have had problems with this my entire life. I have a job now where I can get into "the zone" for eight or so hours per day, so that helps me tremendously. It is not a passion, per se, but I enjoy it enough to get my mind completely engaged in the task at hand. I used to want much more from a job that "that", but after a lot of trial and error, I'm happy to have found something I can do that will get me into the zone and still allows me to pay my bills.
I also understand the social interaction component. For me, that will make or break my life experience. If I have one or two cool people to interact with, it's like the routine doesn't even come into play. If i only have to deal with obnoxious/rude types, then I'm left ruminating on things I wouldn't have cared about otherwise. I think we INTPs need an interesting friend or two. Too much isolation is not so good for us.
I just bought a piano and it's been sitting in the shipping box for six weeks. I love music and consider it a passion of mine (listening, though not necessarily creating) but I figure I'll give it a go and see if I'm any good.