Mark 1:21-28 is the first of four exorcism stories in the gospel of Mark; the other three are found in 5:1-20 (The Gerasene Demoniac), 7:24-30 (The Syro-phoenician woman's daughter) and 9:14-29 (The exorcism at the foot of the mount of Transfiguration). An aspect of Jesus' person is highlighted in each of these stories. In Mark 1:21-28, Jesus' teaching authority is underlined.
Jesus has just recruited four fishermen (1:16-20) whom he now brings with him into Caphernaum. The fame that Jesus earns from this episode prepares for his successful miracle and teaching tour later on (1:29-39). From the time Jesus begins to proclaim his message (1:14-15), the rapid succession of events (the use of "and immediately": 1:18,20,21,23) brings him face to face with an unclean spirit. This encounter occurs after he teaches in a synagogue.
The synagogue is a feature of Jewish religious practise that developed because of the destruction of the Temple in 587 BC. It probably dates back to Ezra or at the most to the Babylonian exile. By the time of Jesus, it was already an established institution, with a building of its own and a hierarchy led by synagogal elders who were also the elders of the community. It was a layman's institution; a priest was treated as part of the congregation.
A synagogue is not, like the Temple, a dwelling place for God. It was rather a house of prayer and study; sacrifice had no place in it. Services were held on the Sabbath and on feast days. Here is a description of a synagogue service from McKenzie
Men and women were segregated iin the seating. The services began with a recitation of the Shema (Deut. 6:4ff), the Jewish profession of faith. This was followed by a long prayer recited by one of the congregation; originally improvised, the prayers acquired a set form which was copied in manuals. A section of the law was read in Hebrew, followed by a translation or paraphrase into the vernacular (Targum) and then by a homily upon the passage read, again improvised by a member of the congreation. The reading of the prophets was added early to the reading of the Law. If a priest were present, the services concluded by the recitation of the sacerdotal blessing (Numbers 6:24-26). 1
In Mark's story, Jesus was a guest who was asked to give the sermon after the reading. The reaction of his listeners was amazement for it was different from that of the scribes. This is the first time in Mark's gospel where scribes are mentioned. In succeding episodes, the scribes will emerge as Jesus' antagonists. What distinguishes Jesus' teaching with the scribes is his authority and its newness (see v. 27). It is an authority that he will manifest later on over an unclean spirit.
At the beginning of Mark's Gospel, we are told that Jesus is the Son of God (v. 1), a title that the voice at Jesus' baptism confirms (see 1:11). Going into the exorcism story of 1:21-28 then, the reader already knows something about Jesus that the people in the synagogue do not know: Jesus comes with the authority of the Son of God. Scribes taught giving a list of their sources, often prefacing their words with "Rabbi X said...". The freshness of Jesus' teaching ("new teaching", v. 27) derives from his special relationship with the Father.
This teaching does not come from any school. It is radically different from what can be larned in schools. It is not the kind of explanation or interpretation that is taught there. It is different; it is interpretation "with authority" ...
Jesus' teaching is not the product of human learning, of whatever kind. It originates from immediate contact with the Father, from "face-to-face" dialogue -- from the vision of the one who rests close to the Father's heart. It is the Son's word. Without this inner grounding, his teaching would be pure presumption. That is just what the learned men of Jesus' time judged it to be, and they did so precisely because they could not accept its inner grounding: seeing and knowing face-to-face.2
Mark uses the phrases "unclean spirit" (1:23.26.27;3:11.30;5:2.8.13;6:7;7:25;9:25) and "demons" (1:34.39;3:15.22;6:13;7:26.29.30;9:38;16:9.17) to mean the same thing: hostile spirits. Their hostility is made manifest in cases of possession. The proclamation of God's reign involves the exorcism of such spirits and so we also find Jesus later on giving his apostles authority over demons. The New Testament is in continuity with the religious literature contemporary to it and those preceding it. But it also introduces something new: Jesus' authority over such spirits. Mark had already mentioned that Jesus was tempted by Satan for forty days (1:13-14).
Satan is the equivalent of Qumran's Belial, the leader of demonic spirits (see Mark 3:26) Unlike Matthew and Luke, he does not tell us how Jesus fared in that meeting. We know however from Jesus' triumphs over unclean spirits that in the forty-day meeting he had with Satan, he was like the stronger man who bound up the master of the household (cf. Mark 3:27) so that afterwards, demons would leave wherever he goes.
To note here is that the demon, in an attempt to get over the control of the situation, calls Jesus by a secret name, "the Holy One of God". Later, the Gerasene demoniac would do the same thing by calling him "Son of the Most High". It was believed that the one who calls one by one's name has control over him. By calling Jesus "Holy One of God" the demon reveals something about Jesus' that was not immediately evident to the people in the synagogue. At Jesus' baptism, he was revealed by the Voice as "the Son, the Beloved, the one in whom I am well-pleased". This introduction will be repeated later on in the Transfiguration (Mark 9). Jesus has authority over the demons because of His relationship with the Father. He comes with the authority of the Son and so is able to make the demon shut up and get out of the man.
The hostility of the unclean spirit is manifested in his raised voice (v. 23) and loud shout (v. 26). This is in contrast in Jesus who quietly but firmly orders him out of the man. Is this perhaps an allussion to Jesus' identity as the Servant of Yahweh, an allussion already made by the Voice at his baptism?
Behold my servant,
I will uphold him: my elect, my soul delighteth in him:
I have given my spirit upon him, ...
He shall not cry, nor have respect to person, neither shall his voice be heard abroad. (Isaiah 42:1-2)
Jesus' first encounter with a demoniac is also the first time that he amazes a synagogal gathering with his "new teaching". It is a teaching that is distinguished from those of the scribes because it is a teaching with authority. That authority is that of the Son, "the Holy One of God" who has come to inaugurate God's reign. Wherever he goes, the kingdom of Satan recedes.

