I base someones "smarts" on their wisdom, intelligence, and knowledge. Intelligence comes naturally to me, so I expect similar of others: whether that is true or not. Wisdom is another easy one, so I half expect that in everyone, too. However, having squandered my intelligence and wasted my time, anyone with intensely deep knowledge in any subject gains my respect as really smart.
Looking at those three things, I must actually disagree with most people's opinions that IQ tests are just a number. Conventionally, intelligence is how well you learn: not what you know, or how you use it, etc. As far as strict general intelligence, the standardized IQ tests (the ones that ask about puzzle-solving and pattern-recognition, not the fake online knowledge-based "IQ Tests") do a decent job of measuring.* Of course, not to 100% precision, but to a decent level: the "high IQ" trend among generally-intelligent people isn't coincidence: the tests do actually give a decent idea of one's intelligence.
*puzzle-solving and pattern-recognition are key abilities in learning.
As for the idea of multiple intelligences, I partially disagree with the theory. A person who is good at recognizing patterns with words will most likely be just as good at recognizing patterns with numbers, music, pictures, etc. We merely have our own interests which affect our attention and drive.
