The tenants fear the missile base above their heads could make them the focus for an attack but a judge ruled on Tuesday they did not have an arguable case, the Press Association reported.
The Ministry of Defence (MoD) says there is no credible threat and that the siting of the missiles is both "legitimate and proportionate".
The block is one of six sites in the capital where missiles, including rapier and high-velocity systems, will be deployed to protect Games venues.
The residents applied for permission to seek judicial review, saying there had been a "disproportionate interference" with their human rights, and that they were not consulted properly.
Their lawyers argued during a one-day hearing that those who wanted to move out should at least be relocated in hotels by the MoD for the duration of the Games, or a gantry should be erected away from the block to take the missile system.
But Mr Justice Haddon-Cave said in his ruling on Tuesday: "The law and the facts militate against the claim for judicial review.
"In my judgment the MoD's voluntary engagement with the community and residents in this matter were immaculate".
On Monday Marc Willers, representing the residents, told the court: "It is the unprecedented siting of a military base or missile site in peace time on English soil that brings us to this court."
He said of the residents: "They have a fully justified fear that installation or deployment of the missile system on the roof of the ... tower gives rise to the additional risk that the tower itself may become the focus of a terrorist attack."
