Are all volcanoes mountains?

Not necessarily. There are several extinct volcanoes which are more or less holes in the ground at this point:
https://en.wikipedia.org/wiki/Yellowstone_Caldera
https://en.wikipedia.org/wiki/Lake_Toba

Since a volcano is a feature which erupts lava at the earth's surface, a volcano does not necessarily have to be a mountain. The lava does tend to pile up and produce a hill or mountain, though. The afformentioned volcanoes, however, exploded such a huge amount of magma at once that they essentially left gigantic craters.

The most common perception of a volcano is of a conical mountain, spewing lava and poisonous gases from a crater at its summit. This describes just one of many types of volcano, and the features of volcanoes are much more complicated. The structure and behavior of volcanoes depends on a number of factors. Some volcanoes have rugged peaks formed by lava domes rather than a summit crater, whereas others present landscape features such as massive plateaus. Vents that issue volcanic material (lava, which is what magma is called once it has escaped to the surface, and ash) and gases (mainly steam and magmatic gases) can be located anywhere on the landform. Many of these vents give rise to smaller cones such as Puʻu ʻŌʻō on a flank of Hawaii's Kīlauea.


Lakagigar fissure vent in Iceland, source of the major world climate alteration of 1783-84.
Skjaldbreiður, a shield volcano whose name means "broad shield"
January 2009 image of the rhyolitic lava dome of Chaitén Volcano, southern Chile during its 2008-2009 eruption.
Holocene cinder cone volcano on State Highway 18 near Veyo, Utah.
Mayon, near perfect stratovolcano in the Philippines.
The Lake Toba volcano created a caldera 100 km long
Pillow lava (NOAA)
Herðubreið, one of the tuyas in Iceland
Mud volcano on Taman Peninsular, RussiaOther types of volcano include cryovolcanoes (or ice volcanoes), particularly on some moons of Jupiter, Saturn and Neptune; and mud volcanoes, which are formations often not associated with known magmatic activity. Active mud volcanoes tend to involve temperatures much lower than those of igneous volcanoes, except when a mud volcano is actually a vent of an igneous volcano.

https://en.wikipedia.org/wiki/Volcano

Yellowstone National Park in the US is one big supervolcano, and though there are areas which feature elevations, the park itself is mostly a big crator.