This paper discusses the design and simulation of a lowercase handwritten text recognition algorithm that utilizes a fuzzy inference system. The algorithm allows a user to take photos of digital handwritten letters. Then it automatically detects features of the input image based on Maximally Stable Extremal Regions (MSER) algorithm. Next it filters out non-text features based on morphological differences between text and non- text regions to determine the stroke width of the letter. The image is then processed into different segments for calculations. The segmented image is utilized and implemented in a carefully designed fuzzy inference system to determine what each letter is in any given image.