I would agree that, if you designed a map which would work at VHard, than yes, other difficulties wouldn't do much more than speed up the completion (by this I mean the build, not the map and win-conditions, which are often limited by monuments that do not, inter alia, speed up relative to difficulty).
However, I'd say the better way to view difficulties is, like most games, mistakes are less forgiving. Money is tighter so you have less flexibility. Design flaws are more likely to be caught out. You can build a housing block that works on VHard and it will work on all difficulties. But at Normal or Hard, you could build a block which works at that difficulty but not on VHard due to the increased fire/disease/malaria risk.
Also, the changes in Culture requirements can need noticeably more buildings and more employees, especially when high ratings are needed.
I'd say that the gap in difficulty between Hard-VHard feels about the same as going from VEasy-Easy or Easy-Hard. Going from Easy-Normal or Normal-Hard doesn't present as much of a change as the other difficulty gaps do.
If you are good enough to play VHard, try not to just complete the map but add your own challenges too like large populations, high Culture, fast completion or lots of scribal housing. When you start playing around and finding your own personal challenges you like, you'll find yourself pushing maps to the limits and then that's when the difficulty changes become more apparent.
Eagles may soar, but weasels do not get sucked into jet engines
Two Eskimos sitting in a kayak were chilly, so they lit a fire in the craft. Unsurprisingly it sank, Proving once again that you can't have your kayak and heat it too[This message has been edited by evil_live_vile (edited 01-10-2019 @ 03:53 PM).]