I am in a lot of group chats with people whose number I do not have saved. Additionally, I tend to delete phone contacts if they are people who I had met but they never answered over time and had no other connection to them. So when I connected the iMessage sync, it pulled in hundreds on random phone numbers as additional contacts, and made it difficult to sort through, so I had to disconnect the sync and archive all those phone numbers.