The speaker, that is, the old man wants to say that he has no family but his pet animals. He spends all his time by taking care of those animals.
The old man says, “There were three animals altogether”. There were two goats and a cat. Then there were four pairs of pigeons too.