The concept of atonement is found throughout Scripture, and so of course we Orthodox do believe in it. There was in fact a feast in the Old Testament called " The Day of Atonement," which in Hebrew is called Yom Kippur. This was the only fast day specifically called for in the Law of Moses, and was "a most holy sabbath [Shabbat Shabbaton]" (Leviticus 16:31). This is the fast that was alluded to in Acts 27:9, which states that "sailing was now dangerous, because the fast was now already past..."
The English word "atonement" was coined by William Tyndale, and means "to make one" or literally "at one-ment" (taking the two words "at" and "one" and adding the suffix "-ment." This well translated the meaning of the Hebrew word "Kippur," which means "reconciliation" -- specifically reconciling sinful men with a Holy God.
Another term which William Tyndale brought into English is "Mercy Seat". William Tyndale based his translation on Luther's translation into German: "Gnadenstuhl," which literally means the seat of grace or mercy. However, there is nothing in the Hebrew term, Kapporet, which suggests "mercy" or "seat." "Kapporet" is a form of the word "Kippur", and literally means "the place of reconciliation". This was the lid of the Ark of the Covenant, and the Mercy Seat was the place upon which the blood of the sacrifice on the day of Atonement was sprinkled, and by which reconciliation between God and men was brought about. The Greek translation for Mercy Seat was "ἱλαστήριον, hilasterion." And we find this word used in Romans 3:24-25: "Being justified freely by his grace through the redemption that is in Christ Jesus: Whom God hath set forth to be a propitiation through faith in his blood, to declare his righteousness for the remission of sins that are past, through the forbearance of God." Interestingly we find in that text the word "redemption", which could be translated as "ransom", and the Hebrew word for "ransom" (Koper) is from the same root word as Kippur -- a word used in reference to the Old Testament sacrifices, and which clearly has the connotation of "payment."
Church Tradition directly connects the Cross wih the Ark of the Covenant, because the Ark and the Mercy Seat was the place of atonement, and the Ark is referred to as "the place where His feet have stood" (Psalm 131:7 lxx) and the Cross is the place were Christ's feet stood, when he made atonement for our sins (see Christopher Veniamin, trans. Saint Gregory Palamas: The Homilies (Waymart, PA: Mount Thabor Publishing, 2009) p. 86).
There are many contemporary Orthodox writers who wish to deny or downplay a number of concepts that relate to our redemption. They will argue we don't believe Christ had to die in our place, or that His blood needed to be shed to pay the penalty for our sins. They will deny the legitimacy of legal terms, in favor of the idea that the Church is a spiritual hospital. The problem is not that the Church is not a spiritual hospital, but rather that in emphasizing one set of images used to explain our salvation, they deny a whole set of equally valid images that are clearly Biblical. It is true that in the west there was an over emphasis on legal imagery, but the solution to such an imbalance is not a new imbalance in the opposite direction. We can and should speak of sin as an illness, but when we die, we do not go before the final medical exam -- we face the final judgment, which is a legal image if ever there was one. And so we can also speak of sin as a transgression of the Law of God, and of our need to be justified by God, even as we speak of sin in terms of an illness that we need to be healed of.
We reject the idea that Christ's death was a ransom paid to the devil, but that it was a ransom in some sense is confirmed by the Lord Himself, and elsewhere in Scripture (Matthew 20:28; Mark 10:45; 1 Timothy 2:6). So we simply have to understand that verbal images point to a reality, but are not the reality itself, and we get a better idea of that reality by considering all the Biblical images that point to it -- not by focusing on one or two to the exclusion of the rest, and certainly not by pressing those images beyond the point that they are intended to make.
St. Gregory Palamas, in his Sixteenth Homily (delivered on Holy Saturday: "About the Dispensation According to the flesh of our Lord Jesus Christ and the Gifts of Grace Granted to Those Who Truly Believe in Him"), speaks quite a bit about the need for Christ to die in our place. The entire homily is well worth reading, but here are some excerpts:
"Man was led into his captivity when he experienced God's wrath, this wrath being the good God's just abandonment of man. God had to be reconciled with the human race, for otherwise mankind could not be set free from the servitude. A sacrifice was needed to reconcile the Father on high with us and to sanctify us, since we had been soiled by fellowship with the evil one. There had to be a sacrifice which both cleansed and was clean, and a purified and sinless priest" (Christopher Veniamin, trans. Saint Gregory Palamas: The Homilies (Waymart, PA: Mount Thabor Publishing, 2009) p. 124).
"Christ overturned the devil through suffering and His flesh which He offered as a sacrifice to God the Father, as a pure and altogether holy victim -- how great is His gift! -- and reconciled God to our human race" (p.125).
"For this reason the lord patiently endured for our sake a death He was not obliged to undergo, to redeem us, who were obliged to suffer death, from servitude to the devil and death, by which I mean death both of the soul and of the body, temporary and eternal. Since He gave His blood, which was sinless and therefore guiltless, as a ransom for us who were liable to punishment because of our sins, He redeemed us from our guilt. He forgave us our sins, tore up the record of them on the Cross and delivered us from the Devil's tyranny (cf. Col 2:14-15)"( p. 128f)."
As is often the case, the proper Orthodox perspective on this question is one of balance. We should proclaim the whole counsel of God (Acts 20:27), and not just the parts that we find most appealing. Nor should we overreact to the imbalances of heterodox theologians, and thus fall into a new error, by rejecting important aspects of our Tradition.