Connecting Labels to their Demographic Base