Machine acceptance is a vital part of the manufacturing process, especially for 5-axis machine tools prevalent in the aerospace industry. It is currently done by skilled workers using their experience and knowledge to iteratively improve the machine tool until it is able to manufacture a test piece that meets the required quality standards. This process is time consuming, requires a lot of expertise, and is not easily transferable to new workers. In this paper, we propose a system that uses machine control signals to detect anomalies during the manufacturing of the test piece and classify them by their cause, like an onset of chatter, positional errors, or others. For this, the machine signals are segmented using a sliding window approach. Multiple strategies to reduce the dimensionality of the segments are evaluated, including autoencoders based on a Convolutional Neural Network or a Long-Short Term Memory Network as well as manually designed features. The reduced segments are then classified using a Random Forrest. The results show that the proposed system is able to detect anomalies with high accuracy and classify them correctly.