The attitudes toward Jedi in the OT really don't jive very well with the prequels and associated EU. Like you said, Tarkin talks about Jedi like they're backwards weirdos. Except for Vader (and, later, as we find out, Palpatine), the Empire constantly goes for bigger machines. There's no sign that anyone else in the Empire places any stock in the Force.
Plus, you have plenty of guys like Han, who are "Force atheists." People simply don't seem to believe in the Force. Luke's never even heard of it. At all. Or a lightsaber.
The only people who refer to the Force are Vader and Obi-Wan (who clearly use and put a lot of stock in it), Motti (who mockingly calls it Vader's "sorcerer ways" until he gets a demonstration), and Dodonna (who's older than dirt). IIRC, Leia doesn't even mention it.
And there's obviously no one who Luke can talk with about the Force. At all. He picks up a few tricks on his own before he learns about Yoda, but not much. Certainly, if he'd had any exposure (or, heck, something like a training manual), he should be a little better at it after three years, especially if he can get so full-on Jedi by the end of ESB (after presumably a few weeks on Dagobah).
So, it seems like the Jedi worked in obscurity, seen mostly as legends by people who didn't know of their deeds defending the Republic, who mostly were completely ignorant of the Force or saw it as an outdated belief system.
And, now, we learn that, just a mere eighteen years before, the Jedi were essentially celebrities, and the Force was used openly and widely. According to several sources, Anakin Skywalker and Obi-Wan Kenobi were, because of their exploits in the Clone Wars, two of the best-known individuals in the entire galaxy. Even Yoda was a household name. That's like saying that the American troops in the Gulf War were all 70-foot tall cyborgs who transformed into Porsches and guest-hosted Saturday Night Live, but no one believes that now.
No one ever said to Luke growing up, "Hey, Skywalker--any relation to that hotshot pilot Jedi dude from the Clone Wars?" He must've told someone at some point about his training on Dagobah, even if it just Leia. Surely her dad would've mentioned the galaxy-famous leader of the Jedi, right? Hell, Yoda was friends with Chewbacca! And Luke was babbling about Yoda when Han found him on Hoth--did no one ever put two and two together? "Yoda? You mean a short dude, about yay high, like this he talks? I knew that guy!"
The Empire's propaganda machine simply could not be that effective. Not after just eighteen years, with so clearly a limited reach on so many worlds.
And, if there are so many non-Sith Dark Side users, why not simply replace the old Jedi Order with a new one, and make it look like nothing has changed?