Information communication between two peers can be done using various mediums. These mediums can be either linguistic or gestures. The development of procedure for realizing gestures into meaningful information plays a pivotal role in instances where linguistic feature cannot be taken as a basis and gestures can be used as the alternative for the conveying the same. This project basically presents a very effective and efficient approach for recognizing the hand gesture that represents numbers. The work basically represents the active and in-active fingers with Boolean true or false respectively, in different combination for representing different numbers. The method of representing the hand gesture in binary pattern contributes a lot for increasing the performance of classification process. The binary Support Vector Machine (SVM) is considered as a recognition tool.