What does Mark 9:28 mean?
We aren't told which house they are entering, but if the mountain on which Jesus was transfigured was Mt. Meron, it's possible they have returned to Andrew and Peter's house, since Capernaum is a seven-hour hike away. If so, it looks like they tour around Galilee one more time (Mark 9:30) before stopping by Capernaum (Mark 9:33) before they head south to Judea and Perea (Mark 10:1).The disciples would have good reason to wonder why they seem to have no authority over the demon. Some time before, they traveled around Galilee, healing and casting out demons (Mark 6:13). While in Mark Jesus explains that they did not pray, Matthew 17:20 reveals that they did not have enough faith.
Faith and prayer are closely related. The more faith in God we have, the more we realize how much we need Him. Even if we have faith that He will work all things together for good (Romans 8:28), real faith keeps its eyes on the One in whom the faith is placed, not the situation in which the Faithful One acts. That attention is partially expressed as prayer. It's possible the disciples are displaying the same ironic hubris that Jesus condemns the Pharisees of: they're filled with self-importance because they're associated with God, but they allow that self-importance to feed the belief they have an autonomous power and authority outside of God.
Before long, the disciples will meet someone who better understands how faith and God's power are related. Instead of learning from the example, they will try to stop it (Mark 9:38). Eventually, with the help of the Holy Spirit, they will understand (see Acts 1:1–10).