The Hall of Fame jockey usually tries to occupy himself with a glass of his favorite wine or by watching football, anything to quiet his frantic mind. He doesn't want to think about Zenyatta's winning streak, now 19-0.
But eventually he caves and opens the Daily Racing Form , the industry's book of past performances and handicapping.
"Then the floodgates open. What if this happens, what if this happens. It just never stops," he said recently. "As blessed believe me as I am, and I'm not complaining by no means, to ride her also becomes a lot of responsibility."
Smith will feel the pressure again Saturday, when Zenyatta enters the starting gate for the $5 million Breeders' Cup Classic at Churchill Downs.
"If we're blessed enough to finish this thing 20-0, how do you even describe it?" he said. "It would just be incredible."
Last year, she beat the boys to become the first female to win the Classic. Now she'll take on the best male horses in the world again. A victory would likely make Zenyatta the Horse of the Year, an honor she lost out on last year when Rachel Alexandra won.
"It's the first Classic ever that everybody is going to be rooting for one horse," said trainer Bob Baffert, who will saddle Preakness winner Lookin At Lucky in the race.
Zenyatta has that effect on people, drawing many new fans, especially women. She drew the largest crowd in nine years -- 25,837 -- to Hollywood Park for her most recent race last month.
As beloved as she is, Zenyatta has detractors. They point out that 17 of her 19 wins have come on the controversial synthetic surfaces at Southern California's tracks.
Saturday's Classic is on dirt.
"She beats them in the big one and they still want to complain," Smith said dismissively. "Thirty-three months, she's still running this way, that's incredible. No one's ever done this."