He won a national championship in football, then retired from coaching to become the school's athletic director. He had so much success in that position that he kept working when others his age were retiring.
But his stint as an athletic director in the Southeastern Conference came to an end when his athletic department got bogged down by turmoil and negative publicity.
Former longtime Georgia coach and athletic director Vince Dooley has an understanding for what Arkansas athletic director Frank Broyles is going through, because the two legends have shared similar career paths.
"It's just the reality of it, you get questioned," Dooley said. "But if have been a football coach like Frank has and myself, you are a little better conditioned to that type of criticism than if you didn't have that experience."
After months of criticism over his handling of Arkansas' football program, Broyles, 82, announced Feb. 17 that he'll retire as the school's athletic director at the end of the year. He will remain in the position until Dec. 31, then move into a consulting role with the university.
Arkansas Chancellor John White will head the search for a new athletic director. In the meantime, Broyles intends to enjoy his 50th year at the university. That's 10 years longer than Dooley spent at Georgia as either the football coach or athletic director (1964-2004).
"Frank, I think, did it much longer than most anybody in modern times. I could have stayed on for a couple of more years, but I would have retired within the next two or three years anyway," the 74-year-old Dooley said. "But I knew at some point in time it was going to happen, but I just did not know when."
The difference between Broyles' situation and that of Dooley's is that Broyles appears to be leaving on his own terms.
Broyles has always had a good indication for how Arkansas' Board of Trustees felt about athletics, and he knew the time was right for him to retire.
Dooley, however, was forced out in 2004 by University of Georgia president Michael Adams after a rash of high-profile incidents within the athletic department, including the football team.
As was the case with Dooley, Broyles has been criticized by some fans who have questioned the athletic director's leadership and wondered whether he's getting too old to do the job effectively.
Dooley said such criticism should be expected, even for former coaches like himself and Broyles who each won national championships at their respective schools (Broyles in 1964 and Dooley in 1980).
"That is part of the reality of life and the way it is. The memories are very short, the fans are very fickle," Dooley said.
"I remember (former Georgia and Minnesota Vikings quarterback) Fran Tarkenton. He said probably the toughest thing he ever had to endure for all these things he's done in pro ball, his last year or two, is getting ready to run out and they introduced him and (the fans) booed him."
Dooley has known Broyles for at least 40 years. As Auburn's quarterbacks coach in the early 1960s, Dooley wrote Broyles a letter to inquire if there was a possibility of joining Arkansas' coaching staff.
A few years later, Dooley said Broyles called him to see if he would be interested in being Arkansas' backfield coach. By then, Dooley was being considered for the Georgia head coaching job. But Broyles' interest helped convince then Georgia athletic director Joel Eaves to hire Dooley.
"He's probably indirectly or maybe directly responsible for me getting the job at Georgia," Dooley said of Broyles.
Their working relationship continued into the 1990s as they both served as athletic directors at SEC schools. But the days of an individual staying at one school -- first as a coach and then as the athletic director -- might be over.
"Frank and I basically learned on the job," Dooley said.
Retirement hasn't been so bad for Dooley. He still has an office at Georgia that overlooks the football practice field, and he has more time for traveling.
So does he have any retirement advice for Broyles, who's an avid golfer?
"I don't know that I will, except there is a lot of things to do out there. It's not a bad thing to have to not be responsible for things that go on," Dooley said.
"I think there is a great life out there. I don't want to advice him to look at (retirement) as a negative thing. There are just too many opportunities to do too many things.
"It's a big world out there."