Going Deeper Than Skin Color