The Sun reported that Elliott, 48, texted Rufus the words: "Ur (sic) a stupid man n*****. You dog. Ur history my friend."
Elliott, who received a CBE for his services to equality and diversity in football earlier this month, also allegedly added: "This will follow you scumbag."
The paper further claimed that Elliott insists the term was not offensive because both men involved are black.
FA chairman David Bernstein said in a statement that Elliott's position was "untenable" after the matter was reported by press earlier this week.
Elliott was a leading light and trustee of the Kick It Out campaign, a position from which he has also stepped down, and served on the FA's judicial panel and as a nominated member of UEFA committees.
A statement read: "The FA has today accepted the resignation of Paul Elliott from all roles representing the organisation.
"This follows a conversation in which discriminatory abusive comments were made to Richard Rufus."
Bernstein said: "I wish to thank Paul for his dedicated and unstinting work, particularly in the area of anti-racism.
"I am saddened by this turn of events and it is with regret that we accept Paul's resignation.
"However, the use of discriminatory language is unacceptable regardless of its context and in effect has made Paul's position untenable."