World road race champion Cavendish and Tour de France winner Wiggins have expressed interest in being part of the track squad in Rio de Janeiro, with the team pursuit their best hope of securing a spot.
However, British Cycling director of performance David Brailsford warned both riders that the gold medal won by the existing squad at the London Games meant there would be no room for sentiment when the four-man team is chosen.
"I think if they both want to do it, they've got to train hard to do it, because it won't be easy to get in that team in four years' time and I think you've got to get in that on merit," Brailsford told reporters on Wednesday.
Before hitting the road and becoming Britain's first Tour champion in July and then claiming the London Olympic time trial title, Wiggins won six Games medals on the track including a team pursuit gold in Beijing four years ago.
Cavendish, last year's Tour green jersey winner, also won two world titles in the madison track event before concentrating on the road.
To return to the track, Wiggins and Cavendish would have to give up road cycling for the best part of the season leading up to the 2016 Games.
Welshman Geraint Thomas, their team mate at Team Sky, did exactly that this year.
"If they want to come and have a go, they can come and have a go but they'll have to prepare properly and be one of the four fastest then great but if not...," explained Brailsford, also boss of Wiggins and Cavendish in the Team Sky professional road cycling outfit.
"Frankly I think it would be very difficult to prepare. If you look at how Geraint Thomas and Peter Kennaugh prepared, to break the world record twice here that took some doing.
"That's how we'll approach the team pursuit. We want the fastest guys for the job."
Thomas, Kennaugh, Steven Burke and Ed Clancy won the team pursuit gold in London by breaking their own world record with a perfectly oiled ride over 4,000 metres.
It did not come from nowhere.
"They both (Thomas and Kennaugh) rode the Giro, the start of the season was dedicated to the (track cycling) World Cups and team pursuiting," said Brailsford.
"They went from the Giro (Italy's road cycling tour) into total team pursuiting, which meant no Tour de France. If you're willing to make that kind of level of sacrifice, then you'll be considered. If not, I think it's implausible."
Wiggins, however, faces a more immediate challenge.
Having won the Tour and with his victory in the time trial giving him a seventh Olympic medal, the 32-year-old will need to cope with increased media and fan attention.
"It will be very difficult. I would go as far as saying that most people, when they win something, a big major, career defining victory, the period after that...," said Brailsford, hinting champions struggle to raise their game after success.
"Bradley will get pulled this winter from award ceremonies to media things to all the things that normally aren't there in life and enables him to train, he will inevitably get asked to do a lot of that this winter.
"The challenge with a lot of guys who hit a peak, they go through all of that and while everyone else is training really really hard, they're not. Not because they don't want to, it's just difficult to fit it all in."
Success is one thing. Repeating success is another story.
Going back to training, staying fit throughout the close season is a tough assignment when you are pulled left, right and centre.
"The guys who had repeated success on something like the Tour de France, like (Lance) Armstrong, it's a phenomenal achievement, in that sense, to manage your life and to have the discipline to manage your life," said Brailsford.
"And that's the challenge for Bradley really now, is to manage that portion."