The protesters breached the usually tight security around the embassy and reached the compound grounds but did not enter the main building housing the offices. Once inside the compound, they brought down the U.S. flag, burned it and replaced it with a black banner bearing Islam's declaration of faith — "There is no God but Allah."
Before storming the grounds, demonstrators removed the embassy's sign on the outer wall, set tires ablaze and pelted the compound with rocks.
It was similar to an attack on the U.S. Embassy in the Egyptian capital of Cairo on Tuesday night. A mob of Libyans also attacked the U.S. consulate in the eastern city of Benghazi on Tuesday, killing American Ambassador Chris Stevens and three other Americans.
In Egypt, protesters were clashing with police near the U.S. Embassy in the capital Cairo for the third day in a row.
The violence has raised worries that further protests could break out around the Muslim world as anger spreads over the movie.
Yemeni security forces who rushed to the scene fired in the air and used tear gas to disperse the demonstrators and were eventually able to drive them out of the compound. It was not immediately clear whether anyone was inside the embassy at the time of the attack.
The Yemeni Embassy in Washington condemned the attack and vowed to ensure the safety of foreign diplomats and to step up security measures around their missions in the country.
Yemen is home to al-Qaida's most active branch and the United States is the main foreign supporter of the Yemeni government's counterterrorism campaign. The government on Tuesday announced that al-Qaida's No. 2 leader in Yemen was killed in an apparent U.S. airstrike, a major blow to the terror network.
The movie, "Innocence of Muslims," came to attention in Egypt after its trailer was dubbed into Arabic and posted on YouTube. The video-sharing website blocked access to it Wednesday. The trailer depicts Muhammad as a fraud, a womanizer and a madman in an overtly ridiculing way, showing him having sex and calling for massacres.