After a lot of thought and because of the ongoing crisis we reached the conclusion that a change at this point could help us turn things around urgently," Leverkusen CEO Michael Schade said in a statement.
The 40-year-old former Finland international, who took over at Leverkusen in 2012 is to be replaced until the end of the season by interim coach Sascha Lewandowski, with whom he had shared coaching duties in his first season in charge.
Lewandowski returned to the youth team set up at the start of the season, leaving the former Liverpool defender in sole charge.
Leverkusen, who reached the Champions League knockout stage this season had again been strong contenders for a spot in Europe's premier club competition for the first half of the season.
However, the team has imploded since the winter break, and have dropped from second to fourth with five games remaining.
Borussia Moenchengladbach are a further two points back and Mainz one more behind with each of Leverkusen's closest rivals having one more game to play.
"It was a very difficult decision because Sami did a great job the past two years," Leverkusen sporting director Rudi Voeller said.
"The dramatic developments in the past few weeks, however, gave us no real choice than to part ways. We want and must try everything to rescue this season."