Brain tumors require early MRI diagnosis, a task often done by specialists and known to be time-consuming. AI, particularly Convolutional Neural Networks (CNNs), allows for quicker and more accurate tumor detection. However, using AI models on cloud platforms can risk data leakage, have high latency, and incur significant hardware costs. Deploying AI on edge devices like Field-Programmable Gate Arrays (FPGAs) offers benefits like lower latency, faster processing, and better data security, but necessitates lightweight models due to FPGA constraints. This work proposes an optimized CNN model with reduced parameters and Post-Training Quantization (PTQ) for FPGA use. The quantized model is utilized in hardware design via High-Level Synthesis (HLS). The proposed system achieves a low hardware utilization ratio and a comparable performance to the software version, with a runtime of 0.118 s per input and an accuracy of 96.77% on the validation dataset.