underground more likely, in practice you can only go so far up, tho as seen in cities of today outward and upward would tend to be the norm for centuries more I'd say.... plenty of planet surface left, and only island-locked metropoli will likely face this problem, though as these cities tend to be economic hubs with high real-estate values and perhaps cost of living, they are less likely to breed a populous underclass who are desperate for lebensraum--if they can't survive, most will probably emigrate to areas where life and space are cheaper.
Underground/upward construction already happening to an extent in cities like Singapore. The problem comes in with sewer, electrical, communication and transport systems that are in the way, as well as aquifers/oceans. In a city prone to flooding, are vast underground living/working spaces a good idea? Maybe with adequate hatches...
Would almost say that with this mentality its better to design a new city or district from scratch, depending on the level of development of the city ofc. Consider that such digging operations in a city area are going to bring a halt to areas probably for safety/practical reasons. Drainage and sewage systems will have to be designed to take the sub-terra population in mind. Add the danger audits governments will likely conduct to consider the stability of buildings above/near the excavations and the expenses mount even more.
then there's the 40k arcology-turned hive option, where the city has stood for thousands of years and so many new layers and spires have been constructed that the original layers have become sunken, some sinking into the acidic cesspits of runoff from countless centuries of industry.
This isn't anything that we will see soon, perhaps in the far future if such concepts are even architecturally possible... considering that in most civilized places buildings are demolished to make place for newer ones, this fictional conception of human hives might be naive... my experience is that with a considerable underclass and enough poverty, good metal rarely goes to waste if it can be stripped for scrap, a potentiality which somewhat counters my love for sunken fabricatums of jagged rusting metal filled with eldritch science horrors of ages past.
*EDIT: Oh I guess if you drill the initial tunnel deep enough, you can cut under any city without having to go to the trouble of working in the city, which to me is a logistical nightmare considering the scale needed to make underground areas on par with above ground ones. What this will mean for seismic activity tho idk, there are many instances of mines and mining activity causing quakes and tremors, and building human habitats/factories underground will entail far more digging than are done in mines imo. Subsurface colonies under Sahara/Antartica etc. maybe, where excavations won't cause damage to existing settlements?