Michael Roth of the Gamecocks threw a two-hitter against Kent State earlier in the day to keep the Gamecocks' hopes alive.
South Carolina (48-18) and Arkansas (46-21) will play again Friday night to decide which team goes to the best-of-three finals against Arizona starting Sunday.
"For the second time today we got an absolute wonderful game out of our lefty," Gamecocks coach Ray Tanner said. "Jordan followed up what Michael did earlier today and was outstanding. He's had some good games for us this year. And this was the best one."
Arkansas beat South Carolina 2-1 on Monday, snapping the Gamecocks' record 22-game winning streak in the NCAA tournament.
The Gamecocks' runs Thursday came in the second inning on Tanner English's double and Chase Vergason's single.
Montgomery (6-1) struck out six and walked one in his first outing since June 3.
"Definitely pitching in front of 20,000 people maybe, plus, had me a little nervous in the bullpen," Montgomery said. "But after that first inning and I guess the way I settled in and the way the team was playing behind me, I was just kind of staying on cruise control and just went."
South Carolina is the third team in the 63-year history of the CWS to win two full games in the same day. The last to do it was Holy Cross in 1952, when it beat Western Michigan and Penn State en route to the title. Tennessee won two in a day in 1951.
South Carolina pitchers faced 57 batters in Thursday's two games — three over the minimum.
Roth came up big after Wednesday night's elimination game against Kent State was postponed because of rain. Tanner decided to start Roth because the rainout gave the staff ace a fourth day of rest.
Montgomery, who had been the scheduled starter on Wednesday, was quite efficient when his turn came up against the Razorbacks. He threw 89 pitches, 52 for strikes, before giving way to Price to start the ninth. Price earned his 13th save.
"He got the ball inside. We hit a lot of balls in the middle of the bat. We weren't driving the ball," Arkansas coach Dave Van Horn said of Montgomery. "And we hit some balls off the end of the bat on his changeup. He just did a tremendous job."
Three of Arkansas' four batters to reach base against Montgomery were erased. Joe Serrano was caught stealing in the first, Bo Bigham was thrown out at third in the fifth and Matt Vinson was doubled off first on a line drive to shortstop in the sixth.
Arkansas had handed Montgomery his only loss of the season last month. He was tagged for nine hits and five runs in 5 1-3 innings that night.
Three singles, a walk and a hit batsmen were all the Hogs could muster against him Thursday.
Arkansas' staff came into the game having allowed just two runs over its previous 32 1-3 innings.
Randall Fant (2-3) lasted only 1 1-3 innings, allowing four hits and getting charged for both runs. Brandon Moore pitched 5 2-3 innings of two-hit relief, but his effort went unrewarded because the Hogs couldn't crack Montgomery.
Arkansas might have cost itself a run in the sixth when Bigham tried to go from first to third on Derrick Bleeker's hit to left. English threw him out to end the inning.
South Carolina has allowed four runs or fewer in all eight of its NCAA tournament games this season.
The Gamecocks have allowed four runs or fewer in 18 consecutive CWS games since 2004.