Some of that sounds like me as well. In the paragraph where you speak of understanding in quiet situation with hearing aids on and even in situations with noise with your hearing aids on or not. I know i have low frequency hearing loss and it was mild moderate but i find myself struggling a lot more than i was Things seem more muffled and fuzzy.i describe myself as hard of hearing. I have a mild to moderate hearing loss in both ears, starting at around the 30 db mark in low frequencies that steadily worsens in the middle and high frequencies going down to around 55 to 60 db as the tones get higher. I've also got tinnitus but my hearing aids help with that during the day.
As for hearing with and without hearing aids. I'm getting now that I struggle to understand speech in quiet situations with more than one person if I don't wear my hearing aids and depending on the listening conditions, background noise for example, I can find it difficult in that situation to both hear and understand speech with my hearing aids on.
I'm not a natural lip-reader either, but like all of us who are deaf or hard of hearing it is easier when people recognise the fact that I don't hear well and have the decency to look at me when they speak.
You might be lipreading more than you think. I am uncomfortable when listening to someone with heavy face-hair, or when I can't see their lips. One great help is using closed-captioning with the sound on the TV turned way down. Since your hearing loss is progressive (?), it would be a good idea to work on this.Yes I don't lip read, but I do use the face to try work out what a person is saying.
Sometimes just a slight movement of the mouth can make you understand a word or 2 that you missed.
I also suffer from social anxiety so looking at people is not always the easiest for me. I can normally tell you what shoes that a person was wearing but I don't remember faces or names to well.
Going to my local deaf centre today too after I receive my hearing aids (if they are open)