Tabular data remains fundamental in diverse domains including finance, healthcare, multimedia, and industrial applications. While deep learning approaches have shown promise for analyzing tabular data, current architectures face high complexity and suboptimal performance when processing high-dimensional tabular datasets and handling missing values. Recent sequence modeling architectures like Mamba and Hyena have demonstrated computational efficiency advantages, but their direct application to tabular data is limited by the inherent heterogeneity of features and complex interdependencies. We propose two novel architectures, FT-Mamba and FT-Hyena, which enhance these sequence models with feature-aware components specifically designed for tabular data processing. These architectures incorporate specialized tokenization and encoding mechanisms that explicitly account for numerical and categorical feature types while maintaining computational efficiency. To address missing values, we introduce an innovative adaptive masking strategy that combines type-aware masking with dynamic probability adjustment based on feature-specific statistical properties. This is complemented by learnable missing value representations and a teacher-guided distillation approach that enables robust handling of incomplete data during inference without requiring separate imputation steps. Experimental evaluation across five real-world datasets demonstrates that our architectures, particularly FT-Hyena, achieve nearly twice the processing efficiency of existing approaches while maintaining competitive performance with state-of-the-art Transformer-based models. The adaptive masking strategy consistently outperforms conventional imputation methods while reducing computational overhead during inference. The code is available at https://github.com/Gudesoy/Models-for-Tabular-Data .