× Speaking English is very difficult for my.
√ Speaking English is very difficult for me.
My is a possessive adjective (e.g. my car), and not a pronoun. After my we need a noun. ‘I love my house.’ ‘My name’s Eric.’
After a preposition, we use an object pronoun (me, you, him, her, it, you, us, them). ‘Is this for me?’ ‘I like talking about him.’ ‘I am tired of them.’