In a small village, a newly appointed priest decided to call on his parishioners. In one modest home, teeming with children, he was greeted by the lady of the house. He asked her how many children she and her husband had. 'Ten,' she replied, 'five pairs of twins.' The priest asked, 'You always had twins?' to which the woman replied, 'No, Father, sometimes we had nothing.'

from "The Strangest Man - the Hidden Life of Paul Dirac, Mystic of the Atom," by Graham Farmelo

A slightly different form appears in "Quips, Quotes, and Quanta: an Anecdotal History of Physics," by Anton Z. Capri (Thanks to Google):

A new priest in a small village on his first tour of his parish visits a somewhat modest home overflowing with small children. To his inquiry as to how many children the couple has, the reply is, "Ten; five pairs of twins." The astonished priest asks, "Have you always had twins?" The lady of the house replies, "No, Father, sometimes we had nothing."

Cheers,
Joe