The Narrator is the medium through which storytelling occurs, and the writer is the man behind the curtain that pretends to be the former. The characters in a story should know as much about the two as an ant is required to understand thermodynamics to live and die.
What does this mean? It means that it is perfectly fine for the characters in a story to be blissfully unaware of things the narrator is telling. Telling to whom? To the reader, of course. The most intimate relationship in a novel is not between the narrator and a character (except, say, in first-person POV, of course) but between the Narrator and the Reader. The first is the one who starts saying, “[I’m going to tell you a story about] A long, long time ago, in a kingdom far, far away…” The protagonist might not even know he is in any kingdom.
Why does this matter? Because three of the classic genres of literature can hardly exist (or reach its full potential) with a Narrator that is identical (knows and sees the same) and is tied to the main character’s perceptions. Those are Tragedy, Comedy, and some types of Horror.
Tragedy is achieved when the viewer, that is, the reader, knows that the main character is doomed but can do nothing to stop it and knows that neither can the character. So, the reader knows that the protagonist is actually in love with his mother or that the man he is about to kill is, in fact, his father. And the readers knows that, eventually, the protagonist will discover the awful truth. How does he know? Because the Narrator told him.
Contemporary literature (especially genre literature) has reduced the Narrator to a security camera, so such things are mostly explained through “twists.” A twist can be defined as a moment of revelation when everybody, obviously the characters involved, but also the Narrator and the Reader, discovers the truth. Such a sudden revelation reduces tragedy to surprise and its related emotions (e.g., sudden disgust, not foreboding or dread.) That fits a whodunit or certain genres, but not tragedy. The ponderous feeling of doom that Tragedy aims for is hard to achieve if you rely on twists.
On the other extreme but somewhat related, comedy usually (but not always) relies on the viewer knowing things the Fool (the protagonist, which is also the victim of the jokes) doesn’t know. A twist where a character suddenly slips on a banana peel can be surprising but hardly funny. If you see the banana peel, but the character doesn’t, humor can arise when you realize what this implies. Comedy and Humor are, after all, quite cruel, so it shouldn’t surprise they depend on you knowing things the victim doesn’t. Even the lowly romantic comedies do that, as when they show the comedy of errors of two potential romantic partners and the viewers laugh, both because it is funny but also because they know that, eventually, the confusion will be dispelled (otherwise, if the protagonists are doomed to unnecessary misery through ignorance, it would be a cruel joke, or a tragedy.)
Some types of horror stories also share a similar logic because you don’t need to scare so much the main character as the Reader. Horror can build up even with a character who is completely happy and content and is unaware of the impending doom or the terror surrounding his (or the world’s) condition. The protagonist doesn’t have to be scared even once, as long as the Reader feels dread. That is something most writers of fantastic horror don’t understand, as they think they have to terrify the protagonist with jump scares or monsters and that this, somehow, will trickle down to the Reader. Maybe, or maybe not.
On the other hand, you could focus your attention and literary mind-tricks on the Reader. Whether the protagonist is scared or not might not matter, as it is possible to terrify the Reader just by implication. That is one of the goals of Cosmic Horror, although it is rarely achieved.
This applies to other genres, of course. Keeping in mind that the Narrator is not the equivalent of a movie’s camera will help you avoid common literary traps.
Imagine you have a Bond-type of a scene, where the main hero and the villain meet, the former having resurfaced after having been thought dead. In a movie, you would be tempted to let the villain engage in a soliloquy where he explains what he has gone through and how much he hates the hero, and so on. More competent directors would avoid this using indirect means of narration and storytelling, but the (usually) lack of Narrator in a movie means the temptation to engage in such villainous expository speeches is too strong. Not so in a written story.
All the exposition can be told by the Narrator to the Reader, bypassing any Mr. Exposition character. You don’t even have to say that this is what the villain is thinking; you can just explain it. The only thing the villain ever needs to say to the hero is “Time to die, Mr. Bond.”
Many “relatable” villains fail (among many other reasons) because the writer is working under the assumption that the relatability has to be public, that both the hero and the villain need to share their personal stories like in some type of psychotherapy that, incidentally, has to drag the demoralized reader too, somehow. But it is perfectly possible to have a Hero and a Villain who are understood by the Reader, but who don’t understand each other or who never speak to each other.
To sum up, your Narrator is a distinct entity in a story, closely related to the Reader (whom he tries to
manipulate emotionally influence,) and who gives the story its distinct voice and style. He can know things none of the characters know, and he can tell them only to you, the Reader, even if only to make you feel worse (as is the goal of Horror,) better (as in Comedy,) or both, alternating, as tension and catharsis/release (in Tragedy.)