Background The emergence and spread of resistant microbes continue to be a major public health concern. Effective treatment alternatives, particularly from traditionally used medicinal plants, are needed. Objective The main objective of this study was to conduct phytochemical screening and antimicrobial activity evaluation of selected traditionally used medicinal plants in Ethiopia. Methods The ethnomedicinal use value frequency index (FI) was used to select twelve medicinal plants. Phytochemical classes of compounds were screened using different standard methods. Anti-microbial activities of plant extracts were evaluated against Klebsiella pneumoniae, Pseudomonas aeruginosa, Staphylococcus aureus, Escherichia coli, and Candida albicans. Minimum inhibitory concentrations were measured using the broth micro-dilution method. The data were analyzed using Statistical Package for the Social Sciences (SPSS) version 21.0 and the findings were presented descriptively and using non parametric one-way ANOVA analysis (Kruskal–Wallis/Ddunn’s test). Results The phytochemical constituents identified were flavonoids, alkaloids, glycosides, phenols, saponins, steroids, and terpenoids, with flavonoids, alkaloids, and phenols being the most abundant. The crude extracts and chloroform fractions of the extracts showed an activity against the tested strains. The crude extract of Thalictrum rhynchocarpum Quart.-Dill. and A.Rich root demonstrated superior activity against all the tested strains with the lowest minimum inhibitory concentrations of 0.48 μg/mL against Staphylococcus aureus and Escherichia coli; 0.98 μg/mL against Klebsiella pneumoniae, Pseudomonas aeruginosa; and 3.90 μg/mL against Candida albicans, which are even better than the reference drug, gentamicin and clotrimazole. Conclusion The majority of evaluated medicinal plants demonstrated remarkable activity against tested microbial strains, which can be attributed to the presence of secondary metabolites of different classes of compounds. The finding provided scientific evidence for the use of these traditionally used medicinal plants.