The main purpose of this study was to assess the validity of Grade 10 Math Model Exams by considering some selected secondary schools in Oromia Regional State. Thus, evidences on the content validity and criterion related validity of these exams were gathered and analyzed. In the study, mixed-methods research design was employed. The necessary data were collected from the selected secondary schools by using interview, questionnaire, check-list and document analysis. Questionnaire was administered to 31 teachers currently teach Grade 10 mathematics. Moreover, booklets of Grade 10 Math Model Exams were gathered from the schools and evaluated for appropriateness, relevance and representativeness of the items that were sampled by these exams. By using systematic sampling scores of 230 students on Grade 10 and 12 Math Model Exams, EGSECE and EUEE were also collected and analyzed. The criterion related validity evidences of Grade 10 Math Model Exams were analyzed through correlation and multiple regression analysis accordingly. The findings indicated that the Grade 10 Math Model Exams had moderate criterion related validity but suffered from weak content validity. The main reason for this weakness is that the purpose of the exam was not clearly documented and hence, this situation leads to unsystematic preparation of the exams which results in unintended practices in the education system. Therefore, the study pointed out some directions for possible solutions and suggested for further research in the field.