Castleford, who reached last year's semi-finals, never recovered from the shock of conceding three tries in an eight-minute spell as they were beaten by part-time opponents for the second time in three years.
Featherstone, who conceded 60 points at home to Sheffield Eagles just seven days earlier, produced a superb performance to spark a pitch invasion and rekindle memories of their glory days.
Alan Agar, who coached Rovers to the last of their three cup triumphs at Wembley in 1983, was among the 4,165 crowd at the Bigfellas Stadium to witness a memorable victory by the underdogs amid a wonderful atmosphere.
Tangi Ropati, Kyle Briggs, Ian Hardman and Greg Worthington touched down for the hosts, with Daryl Clark, Josh Griffin and Nick Youngquest crossing for the Tigers.
St Helens survived a rousing Widnes fightback to seal a place in the fifth round of the Carnegie Challenge Cup with a 40-38 success.
Saints looked set for a comfortable victory when they led 40-22 before Kurt Haggerty went over for Widnes.
Then Saints hooker James Roby was sin-binned for a professional foul and Widnes hit back with three late tries from Hep Cahill, Patrick Ah Van and Lloyd White. Widnes would have drawn had Ah Van's attempted conversion to his own try not hit the post four minutes from time.
Both teams scored seven tries, with St Helens duo Josh Jones and Paul Wellens and Widnes' Ah Van grabbing two each.
Halifax ran in 15 tries in an 84-28 rout of South Wales Scorpions in the fourth round of the Carnegie Challenge Cup at the Shay.
The Scorpions, who conceded home advantage due to the unavailability of their ground, took the lead after just two minutes when full-back David James sprinted through a gap in the home defence for the opening try and Scott Leatherbarrow kicked the first of four goals.
But Halifax quickly demonstrated their class against their lower-division opponents to establish a 36-12 interval lead and ran riot in the second half with nine further tries.
Swinton also enjoyed a big win as they crushed Gateshead 70-10.