MacArthur's agent, Richard Lewis, said the actor died in Florida of "natural causes," but no direct cause was specified.
In a career that spanned more than four decades, MacArthur was most recognized for his role as Detective Danny "Danno" Williams on "Hawaii Five-0," which aired from 1968 to 1980. Episodes often ended with detective Steve McGarrett, the lead character, uttering what became a pop culture catch phrase: "Book 'em, Danno."
Jack Lord, who starred as McGarrett, died in 1998.
MacArthur quit the role of McGarrett's sidekick a year before the program's final season.
"Quite frankly, I grew bored," he explained on his website. "The stories became more bland and predictable and presented less and less challenge to me as an actor."
"Hawaii Five-O," one of the longest running crime shows in TV history with 278 episodes, was shot on location in the Hawaiian islands. It was the first Hawaii-based national TV series.
The drama has been remade by CBS with a new cast this season.
MacArthur, born Dec. 8, 1937, seemed destined to become an actor. He was the adopted son of playwright Charles MacArthur and Helen Hayes, an award-winning actress often referred to as "First Lady of the American Theatre." Silent film star Lillian Gish was his godmother.
"They did teach me a lot about the theatre just through my life with them," he said of his parents in a 1957 interview in Teen Life magazine. "They never pushed me in any direction. Any major decision has always been my own to make."
James MacArthur made his stage debut at age 8 in a summer stock production of "The Corn is Green."
His breakout role was in the 1957 "Climax!" television series production of "The Young Stranger," in which he starred as the 17-year-old son of a movie executive who has a run-in with the law.
He entered Harvard that same year, but dropped out in his sophomore year to pursue an acting career.
As a young actor, MacArthur appeared in the Walt Disney movies "Kidnapped," ''Third Man on the Mountain," ''Swiss Family Robinson" and "The Light in the Forest."
He also had roles in "The Interns, "Spencer's Mountain," ''Battle of the Bulge" and "Hang 'Em High," as well as many guest roles on TV series such as "Gunsmoke."
He performed in many stage plays, including the lead role of Hildy Johnson in a 1981 production of "The Front Page," which was co-written by his father in the late 1920s, at the Stanford Community Theatre in Palo Alto, Calif.
His live acting career won him the 1961 Theatre World Award for best new actor for his performance in "Invitation to a March."
MacArthur said that one of his favorite "Hawaii Five-O" episodes was a 1975 segment called "Retire in Sunny Hawaii Forever" because it marked one of the rare times that he worked on screen with his mother. Hayes played Danno's Aunt Clara, who visits Hawaii and helps the detectives solve a murder.
Asked by the Hawaii Star Bulletin newspaper in 2003 about his fondest memories about working on "Hawaii Five-O," MacArthur replied: "Living in Hawaii."