Face recognition systems are becoming a prevalent authentication solution on smartphones. This work is the first to deploy a poisoning attack against an authentication system based on a state-of-the-art face recognition technique. The attack is executed against the underlying SVM learning model that classifies face templates extracted by the FaceNet deep neural network. We demonstrate how an intelligent attacker can undermine the reliability of the authentication system through injecting a single intelligently crafted adversarial image to its training data. The most successful attacks within our evaluation framework trigger an authentication error of more than $50\%$. Our research illustrates the urge to evaluate and protect face authentication against adversarial machine learning.