Medicinal plants are used in a variety of ways in the pharmaceutical industry in many parts of the world to obtain medicines. They are traditionally used especially in developing countries, where they provide cost-effective treatments. However, accurate identification of medicinal plants can be challenging. This study uses a deep neural network and knowledge distillation approach based on a dataset of 4,026 images of 8 species of leaf-based Ethiopian medicinal plants. Knowledge from a ResNet50 teacher model was applied to a lightweight 2-layer student model. The student model, optimized for efficiency, achieved 96.91% accuracy and came close to the 98.98% accuracy of the teacher model on unseen test data. The training was built on optimization strategies, including oversampling, data augmentation, and learning rate adjustment. To understand the model decisions, LIME (Local Interpretable Model-agnostic Explanations) and degree Grad-CAM (Gradient-weighted Class Activation Mapping) post-hoc explanation techniques were used to highlight influential image regions that contributed to classification.