Yes, because as you go back, villages were smaller. Even neighboring villages might have people you’re already related to without knowing.
I think in 30 generations, the number becomes just over 2.14 billion family members. Bad thing is if I remember correctly, earth didn’t get a billion...