Proceeds from a sale of the property and its contents could be used to help reimburse those who lost billions of dollars investing with Mr. Madoff before he confessed to running a Ponzi scheme.
U.S. Marshal Joseph Guccione said the marshals arrived at the property at noon with a court order permitting them to take custody of the apartment and to make anyone living there move out.
Mr. Guccione said Mr. Madoff's wife, Ruth, had been advised in advance of the marshals' plans and was leaving the residence and surrendering all personal property.
"She will be leaving," he said at midday. "Restitution for the victims is the government's top priority."
Typically, the U.S. Marshals Service changes all locks and secures a property when it seizes a location.
By about 1 p.m., the 67-year-old Ruth Madoff had left. It was not immediately clear where she went to live.
"Ruth moved out voluntarily pursuant to the prior agreements we reached with the government," said her laywer, Peter Chavkin.
The 71-year-old Mr. Madoff was sentenced Monday to 150 years in prison. He pleaded guilty in March to charges that his investment advisory business was a multibillion-dollar Ponzi scheme that wiped out thousands of investors and ruined charities.
Authorities said Mr. Madoff had carried out the fraud for at least two decades before confessing to his sons in December that his investment business was a fraud and that he had lost as much as $50 billion.
Last week, Mrs. Madoff agreed to give up all of her possessions in return for a promise that federal prosecutors would not pursue
$2.5 million not tied to the fraud. The money, though, is not protected from civil legal actions that might be pursued by a court-appointed trustee liquidating Mr. Madoff's assets or by investor lawsuits.
Mrs. Madoff broke her silence Monday when she said in a statement that her husband "stunned us all with his confession and is responsible for this terrible situation in which so many now find themselves."
Before she agreed on a deal with the government to resolve her finances a week ago, Mrs. Madoff had indicated through lawyers that she planned to try to keep the penthouse and an additional $62 million in assets as unrelated to the fraud.
Before the fraud was exposed, the Madoffs had homes in Palm Beach, Fla., the south of France and the tip of Long Island along with the midtown Manhattan penthouse. They also traveled by private jet and yacht.
The couple met at their Queens high school and married in 1959. Mrs. Madoff worked with her husband when he started his financial business in 1960, and she reportedly still had an office near his when the fraud was exposed.
Mr. Madoff has said he operated his fraud without the knowledge of his family.