The introduction of carbon nanotubes (CNTs) into fiber-reinforced polymer composites has been achieved predominantly via two routes: mixing CNTs entirely throughout the matrix (matrix modification) or attaching CNTs onto reinforcing fibers (interface modification). We studied unidirectional carbon fiber/epoxy composites where CNTs were introduced to enhance the hierarchical composites by two alternative strategies: mixing into the epoxy or attaching onto carbon fibers by electrophoretic deposition. Single-fiber fragmentation test combined with energy dispersive X-ray spectroscopy was creatively applied to calculate interfacial shear strength and improvements of 45.2% and 10.14% were achieved for hierarchical composites based on CNT-modified fibers (CF-CNTs/EP) and composites based on CNT-reinforced matrix (CF/EP-CNTs), respectively. Increases of 24.42% for CF-CNTs/EP and 10.41% for CF/EP-CNTs in tensile strength were achieved separately. The superiorities of CF-CNTs/EP were derived from that gradient interface layer formed due to the introduction of CNTs in interface and it was not obvious in CF/EP-CNTs.