Drilling of rivet holes in stacked materials consisting of CFRP and Ti-6Al-4V still represents unique challenges. It is common practice to drill the material layers using one single tool. When exiting the final metallic layer, formation of a drill cap usually leads to undesirable burrs and debris, requiring manual post-processing. In this work, a drilling process to avoid drill caps is presented. This is realized using low frequency high amplitude vibration assisted drilling with defined coupling of the rotational speed and the axial oscillations. Drilling parameters and their impact on the quality of the drilled hole in the metallic layer are also investigated. It is found that drill tip geometries and process parameters need to be tuned together to achieve a stable and repeatable process for drill cap avoidance. A highly sensorized experimental setup with multi-sensor systems including high-speed and thermal imaging was used for process evaluation.