They will fight Oct. 3 at Atlantic City's Boardwalk Hall, Williams promoter Dan Goossen told The Associated Press on Wednesday after finishing the negotiations.
The bout is a risk for Pavlik (35-1, 31 KOs), who rebounded from his one-sided loss to Bernard Hopkins last year by stopping Marco Antonio Rubio in February for his second defense of his WBC and WBO 160-pound titles.
"There's a lot of money out there for him to make, and sometimes making that type of money makes you a lot braver," Goossen said of Pavlik.
Few prominent champions are willing to fight Williams (37-1, 27 KOs), an uncommonly versatile fighter who creates awkward matchups. Though he considers himself a welterweight, Williams claims he has no trouble moving between 147 and 168 pounds.
"There's not a lot of great names at 160, you know?" said Carl Moretti of Top Rank, which promotes Pavlik. "Paul Williams arguably is a middleweight, even though he hasn't won the middleweight title yet. You're looking at two rising stars."
Williams has won four straight fights since his only defeat in February 2008, most recently earning a lopsided unanimous decision over Winky Wright last April. Wright also wanted a fight with Pavlik, but the champion chose Williams for a bout to be shown on HBO.
"I've said all along that from 147 to 168 pounds, Paul is willing to fight anybody, which is a tremendous oddity in our sport, in any sport," Goossen said. "You don't have it in virtually any man-to-man confrontations. This is really another testament, that he's willing to show why we believe he's the most feared man in the world."
Williams and Pavlik both are tall, lanky fighters with surprising power. Williams has held WBO belts at 147 and 154 pounds, but looks most like a middleweight in the ring.
The bout will be among the most anticipated of an already loaded fall for boxing, with Floyd Mayweather Jr.'s return against Juan Manuel Marquez on Sept. 19 and Manny Pacquiao's meeting with Miguel Cotto on Nov. 14 both happening in Las Vegas.