"But isn't it the GM's game?"
At first, yes. The GM starts the creative process by setting the stage for the players to tell their story, but it's the collaboration between the players and the GM that move the game forward. Without this interaction, the players may as well be playing craps in the corner while the GM expounds on the glories of the world in his imagination.
That's the point: The world needs to exist in the players' imaginations, not just the GM's. Without this investment, this willing suspension of disbelief to immerse the players in the world through their characters, the game will fall flat.
This concept dances along a sword's edge through the entire game. Too much detail and the GM freezes the action in perpetual description. Too little description and the players have no grasp of the world so they can't immerse themselves. Too little direction and the characters grow restless and make their own trouble. Too much direction and the players will rebel with cries of "Railroading!"
Who has the power at the table? The GM creates the setting, but the players drive the action through their characters. Ideally, the GM and players form a positive feedback loop that drive a great story, but the communication can break down in innumerable places and in twice as many ways.
Being a GM is far from a cakewalk. Fiat ain't just an auto maker, but it can just as effectively drive your players away. Anyone who has never been a GM thinking they can pick it up in 5 minutes and do better is obviously smoking crack. If you come across one of these people, I recommend joining their game and doing everything you can't stand players doing just to see how they react. Bicker about the rules. Pointedly ignore the adventure hook hitting you in the head. Push boundaries of what your character knows. Explore off the edge of the map.
A great GM should be able to handle these situations with calm grace and still make a fun game as if he had scripted the entire night's entertainment. Where preparation and improvisation collide, that's where you'll find a great GM.
But a word to the wise: Engage your players before you get the game in gear. Listen to them or they'll take your carefully-crafted plans and crush them beneath the boots of the berserker horde they've managed to raise in defense of their town. Invite them into the creative process of your game or you'll find yourself sitting at an empty table obsessively correcting your notes.
Your setting and the rules always take a back seat to the fun you experience in creating the story with your players.