Sign language is a natural language mostly used by hearing impaired persons to communicate with each other. At present day, sign language explainers are used to eliminate the language obstacles between people who are hearing impaired and non-impaired one. However, they are very limited in number. So, automatic sign language recognition system is better to narrow the communication gap between hearing impaired and normal people. This thesis work dealts with development of automatic Amharic sign language translator, translates Amharic alphabet signs into their corresponding text using digital image processing and machine learning approach. The input for the system is video frames of Amharic alphabet signs and the output of the system is Amharic alphabets. The proposed system has four major components: preprocessing, segmentation, feature extraction and classification. The preprocessing starts with the cropping and enhancement of frames. Segmentation was done to segment hand gestures. A total of thirty-four features are extracted from shape, motion and color of hand gestures to represent both the base and derived class of Amharic sign characters. Finally, classification models are built using Neural Network and Multi-Class Support Vector Machine. The performance of each models, Neural Network (NN) and Support Vector Machine (SVM) classifiers, are compared on the combination of shape, motion and color feature descriptors using ten-fold cross validation. The system is trained and tested using a dataset prepared for this purpose only for all base characters and some derived characters of Amharic. Consequently, the recognition system is capable of recognizing these Amharic alphabet signs with 57.82% and 74.06% by NN and SVM classifiers respectively. Therefore, the classification performance of Multi-Class SVM classifier was found to be better than NN classifier.