I asked my brother a question, but he didn’t reply me.
I asked my brother a question, but he didn’t reply to me.
I asked my brother a question, but he didn’t answer me.
Use reply to before a person or message: ‘Please reply to my email.’
Use answer without to: ‘Please answer me.’
See our lesson on reply, answer and other speaking verbs.
See ANSWER