Yes, because you had not developed an identity as either one. A child at this age can identify what they are not, as it is a much easier process that possessing an identity of who one is.
An interesting note, a hearing parent raising a Deaf child in a bilingual bicultural environment rarely realizes the unspoken messages that the kids internalize from that environment. My son, until the age of 5, thought that I was deaf, as well. I always used sign with him, and he knew that people that were not "deaf", in his experience, used spoken language. Therefore, because I used ASL, he assumed that I was deaf. I saw him tell another child that he was "Deaf same mommy". I corrected him that I was not deaf, I was hearing. It was very difficult for him to grasp, because at that age, a child's knowledge of what "deaf" even means is limited to their experience. If they have never heard, telling them that they are deaf because they can't hear is not even something they can grasp fully. Imagine them trying to sort through the specifics of Cultural Deafness.
There is a very good reason that we use age appropriate language and comparisons when we explain complicated concepts to children. You would not try to explain sex to a 5 year old in the same way you would a 15 year old. There is a reason for that.
Shel and I have discussed topics like this in the past. As a M.Ed., she has studied child development and applies that knowledge to students on a daily basis. She is in agreement with me regarding the developmentally determined capabilities of a child.