It only took about five minutes, but I realized something very quickly as I settled into the Season 4 premiere of Justified: I’d missed these sons of bitches.

Is there any other great (or very good) drama series on the air that’s as fun to hang out in as this one? Mad Men is too cool, too baroque. Breaking Bad is a hell of a lot of fun, but who would want to live there? Game of Thrones? Nah. Moving to Homeland for a week or two would just make you buy one of those 64 packs of Crayola crayons and start marking up the walls. But Justified is special. I can’t really say I’d want to live in the show’s version of Harlan County, Kentucky, but I’d definitely love to live inside its vibe, inside the show’s atmosphere. And if the opportunity came to just sit back and hang out with Raylan or Boyd or even poor, underused Rachel, I’d snap it up in a second. Justified has somehow made the hangout drama great art; there are some interesting themes and ideas peeking around the corner of the show, but the most fun you’ll have watching it is in figuring out what these people — so much more clever and cool than you are — are going to say and do next.

The problem with hangout shows is that they often disappear inside their own atmospheres. They grow so devoted to making everything within their confines enjoyable that they become theme-park rides, devoid of real menace. Justified has somehow managed to create a world that viewers would love to walk around in, even if that world is full of things that bite. (In the case of the snake-handling church that Boyd stumbles upon late in the episode, those things will bite quite literally.) The one constant here is Raylan Givens. At any given time, we in the audience can be relatively certain he’ll walk out of whatever situation he’s in unscathed, because he’s just that kind of guy. Thus, the show has to spend most of its time beating up on him emotionally, and that’s something it’s gotten very good at.

Any Justified season premiere is going to be bursting with new story elements that might not immediately add up. What’s pleasing about “Hole in the Wall” is that, ultimately, so much of it does add up to something more. The only story line that feels more or less disconnected from everything else is Ava’s attempt to keep her prostitution ring running smoothly, and even that ultimately gets tied into Boyd’s investigation of the new church. At first, the two teenagers who break into Arlo’s house to dig around for copper wiring seem to be some sort of case of the week that’s just there to complicate Raylan’s day, but once they find something very strange inside the wall — an old bag containing identification belonging to one Waldo Truth — everything starts to spin off its axis.

One of the things that made this show’s third season less satisfying than its near-perfect second season was the fact that it became gravely imbalanced by the sheer number of things going on. There were multiple villains, a bunch of characters trying to fill the power vacuum left by the death of Mags Bennett at the end of Season 2, and Raylan barely able to keep his head above water. It was a highly ambitious structure, but it also meant the show was struggling to keep its head above water. Everything came together in a terrific finale, but the whole thing was so top-heavy that I don’t remember much about the season off the top of my head, outside of some gorgeous character moments in the final episode.

Showrunner Graham Yost has already admitted in print (well, specifically, in one of those Entertainment Weekly TV preview pieces I can’t keep myself from reading) that this season will be less about a major villain and more about a major mystery, a case that Raylan just can’t crack. This is probably the right call, as having Raylan go after yet another villain could’ve gotten very formulaic, very quickly — and who’s to say the mystery won’t lead to another diabolical mastermind or two? What makes Justified work is its gradual accumulation of local texture, the growing sense that the world it takes place in is a real one that’s just a few degrees off of our own reality. So it makes sense that that world would have its own hidden secrets and urban legends, like the image that opens the episode: a man lying dead on the ground, parachute billowing from his back, bags of cocaine ripped open around him. This almost certainly has something to do with whoever Waldo Truth is — and whoever it is that wants Arlo to give up what he knows about the stash that somehow wound up in his walls. There’s not much here about this mystery, but I’m already intrigued.

Instead, we get to watch Raylan try to deal with his new reality. Winona’s off somewhere, pregnant with his child, and he’s trying to do the responsible thing by making a little extra scratch to kick her way, so he doesn’t end up an absentee father and provoke the kind of rusted-out parent-child relationship he has with his own dad. Generational legacies weigh heavily on Justified, and for the first time, Raylan is being forced to consider his place in the world as someone who’ll be imparting wisdom, not just receiving it. This, of course, has led him to bounty-hunter work, and when the episode opens, he’s in pursuit of a murderer the police need hauled in — at least until he gets a call from the man he’s deputized to protect Arlo’s house, a local official named Constable Bob (played by Patton Oswalt).

Constable Bob is one of those instantly great, immediately recognizable Justified bit players, in no small part thanks to the actor playing him, who’s perfectly cast. (Oswalt even throws on a hint of a Kentucky accent, and it suits him just fine.) He was elected to his position, and he’s mostly in that role because he was the only one who wanted it. He wastes most of his money on lights for his car and other things meant to make him look official, and he’s constantly trying to get people to “draw” on him, that he might show how effective he could be with a knife should the situation present itself. He’s a humorous character — c’mon; he’s played by Patton Oswalt! — but the show takes his desires seriously enough to understand just how flummoxed he would be by the fact that he’s not Raylan Givens and is, instead, a dumpy little town constable. The best thing about Justified is how it will take the smallest of guest stars — Mike the hardware store owner, for instance — then make them feel as if they could easily be the lead of their own spinoff, should Yost and his writers get their dander up. Constable Bob goes above and beyond this. I kind of want the show to just turn into Bob and Raylan driving around, chasing juvenile delinquents and talking about the old days.

Of course, it can’t. Raylan and Bob corner the two kids who stole the former’s car (with his bounty locked up in the trunk). There’s a confrontation, and Bob saves the day by stabbing a teenage girl in the foot. This wasn’t exactly how any of this was supposed to play out, but the reason Raylan always survives is that he’s willing to roll with the punches. And, of course, he realizes there can be only one reason those kids were digging around in Arlo’s walls: The old man sent them there to find that bag, and now that news of the bag is out, things can only get worse. It’s fitting that, in a season when Raylan turns his thoughts to what kind of a father he’ll be, he has to deal so closely with the man who raised him, the man he hates more than anyone in the world, most likely. Whatever that bag is — much as I want to know — is almost immaterial. The real fun is going to come because it brings these two back into each other’s orbit.

Todd VanDerWerff (@tvoti) is the TV editor of The A.V. Club. His writing has also appeared at the L.A. Times, Hitfix.com, and The House Next Door.