My handwriting was atrocious until I was in high school when it suddenly got better. I got a compliment on my handwriting the other day which floored me because I grew up with such terrible handwriting. I have no idea what caused it to be so bad for so much longer than other kids. It looked like a five year old write things I was doing in middle school. My high frequency hearing is good. I have very steep reverse slope loss.
The thing that sounds concerning about his handwriting trouble is his fretting over it. If he has been being made fun of or unintentionally shamed by a teacher because of it, it could make him unwilling to practice or put any effort into it. I think it's worth a conversation with him about why he hates writing so much.
As for not using ASL with you, I think that's pretty standard for kids whose parents don't speak a language as well as they do. I grew up moving around a lot and heard the same thing about kids who learned other languages and then refused to speak them with their parents after they left that country. Apparently, I spoke French once upon a time and would just laugh at my parents when they tried to speak French with me. Maybe see if there are some fluent signers who would be willing to act as mentors to him. Or maybe there are other D/deaf/hoh kids in your area and you could form a play group.
