Richland County Sheriff Leon Lott said during a news conference that he didn't have enough physical evidence to charge the swimmer, but he defended his investigation.
"Michael Phelps is truly an American hero ... but even with his star status, he is still obligated to obey the laws of our state," Sheriff Lott said.
The photo showed Mr. Phelps smoking from a marijuana pipe at a party in November when he visited the University of South Carolina. The sheriff said he seized the marijuana pipe, known as a bong, that was in the photo but couldn't prove that Mr. Phelps had smoked from it.
"We had a photo and him saying he was sorry for inappropriate behavior. He never said, 'I smoked marijuana.' We didn't have physical evidence," Sheriff Lott said.
Mr. Phelps has called his judgment bad and said he would learn from his mistake.
USA Swimming suspended Mr. Phelps for three months in the wake of the photo, and Kellogg Co. said it would not renew its endorsement deal with him.
The photo surfaced in a British newspaper, News of the World, on Feb. 1. The swimmer, who won a record eight gold medals at the Beijing Games, did not dispute its authenticity.
News of the World said the picture was taken during a party while Mr. Phelps was visiting the university. During that trip, he attended a football games and received a big ovation when introduced to the crowd.
The sheriff's office said seven people arrested during the investigation would still face a charge of simple possession of marijuana, which carries a maximum penalty of 30 days in jail or a $575 fine. Another person was arrested for driving under suspension.
The sheriff's office wouldn't identify those charged, but an attorney for three of them said they should be handled just like any other marijuana possession charge. Attorney Dick Harpootlian said he expects his clients to either have the charge dismissed or to get a conditional discharge, which allows offenders to avoid punishment as long as they comply with certain conditions for six months and stay out of trouble.
The party occurred nearly three months after the Olympics, and Mr. Phelps' actions should have no impact on the eight gold medals he won at Beijing. He has never tested positive for banned substances.
This isn't the first embarrassing episode for Mr. Phelps after an Olympic triumph. In 2004, a few months removed from winning six gold and two bronze medals in Athens, the swimmer was arrested on a drunken driving charge at age 19. He pleaded guilty and apologized for the mistake.