One has to be careful about the word, "infinity." Does is really exist?
There is an exact mathematical solution for Pi. It involves an infinite series, but it exists.
This is basically correct. There are some problems with interpretations of the word "infinity," but, it's still basically correct.
There are formulas (hundreds of them, actually--see the above post from Wolfram about that) that will generate pi. But, as pi is an irrational number, is is expressed as a non-repeating, non-terminating decimal. Among other things, this just means that there is no final, finitely-numbered decimal position to the number. This doesn't mean that pi isn't exact. Pi is exact. But, if by "exact" you demand a rational number, then your demand is inappropriate.
Many people have trouble wrapping their minds around what irrational numbers mean, but, they're every bit as important, real, functional (pardon all the unintended equivocations on those terms and their math definitions) as rational numbers.
So, in part because pi has "infinitely many" decimal places, the formulas that can be used to calculate it can only provide you with more and more accurate approximations. If you cannot carry out all of the infinite iterations of the formula, then you cannot get to the "last" decimal place. (How handy that there is no last decimal place, right?)
I think the most fun idea here is the juxtaposition that 1) pi has a "definite" value, and that 2) pi has an "indefinite" number of decimal places.
Dave