Smit's appointment on Tuesday signalled the end of his long playing career and will take him back to the team he played for for 13 years.
Smit, who led the Springboks to the 2007 World Cup and is South Africa's most capped international, will leave English premiership leaders Saracens next month after two years with the club.
The 35-year-old hooker and prop will work alongside current Sharks chief Brian van Zyl who will mentor him in his new role until February next year.
"We have every confidence that he will embrace his new role with the same leadership, spirit and vigour that he displayed as the most capped Springbok captain and player of all time," said Sharks chairman Stephen Saad said in a statement.
Smit played 186 times for the Sharks between 1998 and 2011 and won 111 caps for the Springboks. He captained his country 83 times.