Earlier this summer, while stuck — or, if my girlfriend asks, thrilled to be — in Syracuse for a wedding, I went to see the Toledo Mud Hens play the Syracuse Chiefs. The Chiefs, the Triple-A affiliate of the Washington Nationals, featured few promising prospects. Their starter that day was Shawn Hill, an injury-prone pitcher who helped save my fantasy season in 2007 and had hardly been seen since. Toledo, the Tigers affiliate, had players with similar stories: low-ceiling hitters and pitchers who’d followed the Peter Principle up the minor league ladder and found they could climb no further, and bitter veterans who still believed they should be in the big leagues. But the Mud Hens also had the most interesting man on the field: 21-year-old outfielder Nick Castellanos, the Tigers’ top prospect and, according to Baseball Prospectus and Baseball America, one of the 20 best prospects in baseball.

I watched Castellanos closely that day, willing myself to see something that would set him apart from the pack and mark him as the major leaguer he was destined to be. Instead, I saw him go 0-for-4 with four strikeouts and left wondering whether the real Castellanos had swapped uniforms with another nonprospect. If I hadn’t been briefed before the game or peeked at his full-season stat line, there’s no way I would’ve known there was something special about him. And it wouldn’t have helped if I’d gone back the next day, when he went 0-for-5 with another 4 K’s. Yet six weeks later, Castellanos got the call to Detroit. At 0-for-9 or not, he was major league material.

Even certified scouts can be deceived; see a guy on a bad day and it’s hard not to file a negative report. But the best scouts can look beyond the box score and focus on whether a player’s process is likely to lead to long-term results. Even as Castellanos collected back-to-back golden sombreros, there was probably some silver lining that I couldn’t see. A scout might’ve picked up on a positive trait in his stance or his swing, or liked what he did on defense, or approved of the way he held himself on his way back to the dugout. There had to be some sign he wouldn’t go 0-for-4 forever.

I’m never going to be an experienced scout, but I do write about baseball, and it bothers me that there’s so much I’m missing. So I’m doing something about it: I’m going to Scout School.

Known more formally as the Scout Development Program, Scout School is a 12-day course designed to teach the basics of baseball scouting. It takes place in Phoenix each fall, and it’s run by the Major League Scouting Bureau, an organization (now under the domain of Major League Baseball) that has supplemented the scouting efforts of MLB clubs since 1974. More than 1,200 people have participated in the first 28 sessions of Scout School, and many of them have gone on to hold high positions with teams, including some general managers whose names you know.

Aspiring scouts and player development personnel make up most of the graduates, but in recent years other front-office members have begun flocking to Phoenix to be exposed to the scouting perspective. Progressive organizations have increasingly sought to combine statistics and scouting information in their efforts to evaluate players. As then–Baseball Prospectus columnist Dayn Perry once put it, asking whether a team should be run with scouts or statistics is like asking someone to choose between beer or tacos. The best answer is “Both, you fool.”

To be invited to Scout School, you have to be sponsored by a major league team, which means that most attendees have already broken into baseball. And it’s serious business for those who hope to pursue scouting as a career, since their future employment could depend on getting a good grade. Last Sunday, I arrived in Arizona with 65 fellow members of Scout School’s 29th class, many of whom once played professionally and were scouted themselves. Some, like me, have no scouting experience; others have already been hired as scouts but are still seeking the bureau’s seal of approval, which can come only from the 14 instructors whose task it is to condense a combined 325 years of experience into a 12-day program designed to create scouts from scratch. As I progress through that program, I’ll be studying the bureau’s scouting manual, putting my new knowledge to use on Phoenix’s blistering fields, and writing reports on players. And I’ll also attempt to pass on some of the insights I’ve gained.

You might wonder why all this effort is necessary, since technology has made it easier than ever to analyze baseball without a scout’s skill set. Glance at your smartphone, look at the scoreboard, or soon, flick your eyeballs upward, and you can access a website that will tell you what a pitcher throws and when and how hard he throws it. The same sites tell us how often hitters swing at both balls and strikes, how they hit against curveballs, and whether they keep the ball on the ground or hit a lot of line drives. But it takes time before we can trust some of those stats, and they don’t necessarily tell us whether a player’s performance is sustainable. Are a pitcher’s mechanics endangering his arm? Is a hitter’s bat speed slowing? Has a defender lost a step, and if so, might he have to switch positions?

FIELDf/x could offer some clues, as could the high-speed cameras at ASMI and other biomechanics facilities. But most of us don’t have those tools at our disposal, and competition at most professional and amateur levels below the major leagues is still, relatively speaking, a statistical wasteland, at least as far as the public sector is concerned. When stats are scarce or deceptive, how do you decide which players are projectable? How do you know which tools are tied to success, and whether a player has the mental makeup to translate his tools to production? And when you think you’ve spotted a prospect, how will you explain why he caught your eye?

For a week, I’ve been searching for answers, wearing a stopwatch and a Scout School–issued straw hat and applying SPF 100 sunscreen. And if all goes according to plan, I’ll be ready for the next Nick Castellanos.

Think back to the longest seminar you ever sat through in college. Now imagine it three times as long and 10 times as interesting, and you’ll have some sense of what the first day of Scout School was like. For eight hours, 66 acolytes sat in orderly rows, listening and taking notes as a conga line of instructors stepped up to the podium to brief us on their scouting specialties. By the end, my hand was cramped and my brain was burned out, but I felt like I’d leveled up. It was the closest I’ve come to that scene from The Matrix when Keanu learns kung fu.

In the morning, we tore through throwing, fielding, and catching. There is no way to cover all the ins and outs of each area in an hour or two, but the basics are simple. The key to throwing is the action of the arm — is it loose, easy, and effortless, or does it look a little funky? Arm slot — the angle at which a player releases the ball — isn’t set in stone, but arm action can’t often be altered. Those with suspect arm action go by a number of evocative names, each of which describes a certain sort of malfunction — hooking, wrapping, stabbing, slinging, or short-arming — that may be tough to pick up. Arm strength is a little easier to assess. When grading out arm strength, we were told, ask yourself how far the ball carries and what its trajectory looks like. When it’s halfway to the target, is the ball rising or falling, or is it traveling parallel to the ground?

Next we heard that the secret to effective fielding is a live lower body. Scouts want to see flexibility, agility, and mobility below the hips. It also helps to have soft, supple hands, though moisturizing won’t make you better at baseball. When it came time to cover catching, Jim Walton — who’s been in baseball for close to 60 years and looks exactly like Hollywood says grizzled scouts are supposed to — stressed the importance of a wide crouch when a runner might be about to go.

After a break for some coleslaw, we shifted our focus to hitting and pitching. For hitters, scouting starts with the approach — not the mental approach (which is also important), but the stance and the stride. From there, the scout moves on to the swing — does he chop down, swing level, or uppercut? — the timing, the feel for hitting, and the result (or how the ball comes off the bat). Projecting whether a player will hit is notoriously tough, and the instructors acknowledged that even they still struggle with it.

Scouting pitchers is simpler, though there are still several factors in play. There’s the windup, the arm angle and action, and the delivery, not to mention the control (ability to throw strikes), the command (ability to hit the target), and the repertoire of pitches. Radar guns are helpful, but velocity doesn’t tell the whole tale. Movement can make a slow pitch play up, and a straight fastball can be hit hard even if it touches triple digits.

In every area, the instructors emphasized the importance of a “proper” approach. Some players succeed with unorthodox tactics, but the bureau believes that there is an ideal way to do things. However, some present impropriety can be corrected; above all else, a scout values athleticism, a quality that makes it easier to fix flaws. The players who improve are the ones who make their actions look easy. If they’re stiff, then they’d better be strong.

As they walked us through each topic, the instructors told us the best places to stand when we put our book and lecture learning into practice at the park. Cruelly, you can’t sit in one place while scouting, even if it’s hot out and you’ve finally found some shade. You have to see a pitcher from behind home plate to rate his movement, but you also have to see him from the side to tell how his arm action is. On the bright side, moving halfway down the line from behind home plate is nothing if you’ve already driven for hours to get to the ballpark. One of the many scouting maxims I’ve heard so far says, “If you’re early you’re on time, and if you’re on time you’re late.” Unless you love scouting, the lifestyle sucks.

On Monday, we got a good theoretical grounding; on Tuesday, we took it outside, riding a bus to the Peoria Sports Complex for an Instructional League game between Reds and Mariners minor leaguers. The Instructional League, more commonly known as “Instructs,” runs from mid-September to mid-October and gives hand-picked players a chance to see some extra action, focusing on specific skills or just getting some work in after an injury.

Scouting amateurs is simple; most games won’t have more than a player or two with pro potential, which makes it easy to get a good look at anyone interesting. But at the professional level, every player has been scouted and signed, which means they’re all at least nominally prospects. The game goes so fast and there’s so much to see that it can quickly become overwhelming for a fledgling scout.

This was the deep end, but our water wings were on: Our instructors were with us, and they gave us some guidance. Our mission was to gather information and write reports on two particular players: Seattle’s starting pitcher and Cincinnati’s shortstop. Circulating in small groups, each one shepherded by an instructor, we watched the teams take infield practice, then observed both players from a few vantage points once the game got under way. In this context, tunnel vision was something to strive for. An experienced scout might form a holistic picture of a player and derive the details from there, but a scrub has to start by collecting clues until the pattern becomes clear. If I’d looked at each player like I normally would, I would’ve missed the forest and the trees. I had to narrow my field of vision to a player’s feet or hands or shoulders, going body part by body part until I’d seen and recorded it all.

As we committed the two players to memory, we focused on their tools, or particular talents. Tools (hitting ability, hitting for power, fielding, running, and throwing) are rated on the standard 2-8 (or 20-80) scouting scale, where 2 is poor, 8 is excellent, and 5 is major league average. On bureau reports, everything from pitch types to instincts to aggressiveness gets graded on the same spectrum. The system is understood by all scouts, but some of its quirks can make it confusing to those outside the industry. For one thing, not all 8s are equally rare: There are more 8 runners than there are 8 hitters. Some grades loosely correspond to big league performance: Someone with 7 power, for instance, can hit 27 to 34 homers at the major league level. But it doesn’t always work that way: a 7 fastball is 94 or 95 mph, but a heater that hard could be a 6 if it has lousy life. And then there’s the distinction between “present” and “future” tools — present is what grade you’d give it now (that’s the easy part), and future is what you project it to be at the player’s peak.

While writing reports, we were reminded repeatedly that “words equal numbers, and numbers equal words.” You can’t describe something as “good” unless you grade it as a 6. If you give it a 5, it’s just average.

For scouts, average is everything, the benchmark that all evaluations are based on. Every scout knows what average looks like, so when he’s assessing a tool, he pictures its average equivalent and adjusts upward or downward from there. The biggest problem for Scout School students is that they haven’t yet built up a catalogue of tools to compare. There’s no substitute for seeing thousands of games, but as we watched our first one, our instructors did their best to bring us up to speed, muttering “that’s average” after each applicable throw or time to first. Before long, it began to sink in.

The Seattle starter, a right-hander with impressive poise for his age, had a decently paced delivery, an arm action free of red flags, and made some adjustments after getting hit hard in the first. But his control was inconsistent and his stuff was somewhat limited. His fastball ranged from 88 to 92, sitting at 89-90; it was an average pitch without a ton of projection. The breaking ball, falling somewhere between a slider and a curve, was a little loose, with a visible “hump” out of the hand that made it easy to pick up. It was a 3 pitch, with a 4 future projection. And his changeup appeared to be in the early developmental stages. To calculate the Overall Future Projection (OFP) for a pitcher, you add the future grades for the fastball, curveball, slider, and fourth pitch, add a zero, and divide by the number of categories used. The bureau doesn’t count changeups as an “other pitch” for OFP purposes, so the Seattle starter’s OFP was just his future fastball (50), plus his future curveball/slider (40), divided by two: 45. Fifty is average, so a 45 OFP put him somewhere between the bullpen and the back of the rotation — possibly a swingman.

The Reds shortstop was tall and lean, with a projectable frame but little present power. He could play the position, showing adequate range and an average arm, but he didn’t look graceful. He ran from home to first in 4.33 seconds — an average time for a right-handed hitter — but his gait wasn’t smooth. The most important tool for a position player is the hit tool; without that, only a defensive wizard has any hope of occupying an everyday role. And the position player in question had a fringy hit tool at best: He was hyper-aggressive on fastballs early in the count, which suggested that he was hoping to avoid breaking balls. His OFP (the sum of his five future tools times two) was 46, also below average. He might be a bench bat, but he doesn’t have the stick for a starting role.

I can’t show you video of the shortstop or starter we watched, but I can show you this footage from a Monday Instructs start. This is 2013 Diamondbacks draftee Jimmy Sherfy, a college righty who made it to the Midwest League after his professional debut:

Jimmie Sherfy, RHP, Arizona Diamondbacks Fall Instructional League – September 30, 2013 from Jason Cole on Vimeo.

Sherfy’s delivery is a “semi windup.” He hardly raises his hands before they separate. And look how far apart the hands get once they do break apart: He brings the one with the ball way back behind him. That’s a long arm action, in this case off-center, and could qualify as a “wrap” or a “hook.” Because he brings the throwing hand back, he has to rush to complete the delivery, which his good arm speed allows him to do. But his mechanics aren’t balanced or efficient; there’s a lot of effort there, which makes you wonder whether he can repeat the delivery and finish consistently enough to maintain control.

Sherfy’s mechanical flaws make him a reliever, a guy who can come in for short bursts but isn’t suited for multiple-inning outings. But look at the stuff. The fastball is mid-90s with a lot of life, varying between boring in on righties and cutting to the opposite side of the plate. And the nasty slider starts out looking like the fastball, then drops off the table and gets hitters to chase. Every hitter looks uncomfortable. The fastball is a 7; the slider is a 6. And if Sherfy’s health holds up, he’s going to be in the big leagues.

In a sense, we’re all scouts. When you interview a prospective employee, you try to project how he or she would perform in a new and maybe more challenging environment. Before you insult the belligerent guy at the bar, you size him up to see which one of you would win a physical fight. And even in second grade, you knew the difference between your coordinated and uncoordinated classmates and could sense when it would hurt to get hit during dodgeball. But the lower the athletic level, the greater the variation in talent, and the easier it is to tell the good from the bad. Baseball scouts aren’t trying to distinguish the gifted from the ungifted; they’re looking at players with a certain 99th-percentile skill set and trying to pick out the ones who belong in the 99.9th. Scout School should last a lot longer.

The next step on the Scout School tour: Taking off the training wheels.