We examined task switching to different attributes of faces (gender, emotion, occupation) when an irrelevant aspect of the face could also change (e.g., the facial emotion could change when participants alternated every second trial between gender and occupation decisions). The change in the irrelevant attribute either coincided with a repetition or a switch in the explicit task. The results indicated disruptive effects of changing the facial emotion and gender of the face when it was irrelevant to the main task, but no effect of changing the occupation of the person.The data are consistent with the implicit processing of facial emotion and gender but not of higher-order semantic aspects of faces (the person’s occupation), unless those aspects are task-relevant.