The automation of sensitive information classification using AI is critical for modern security, yet a singular focus on accuracy metrics dangerously obscures the risks posed by residual errors. False Negatives can lead to catastrophic data breaches, creating an unmanaged attack surface. This paper bridges the gap between formal security theory and applied machine learning by proposing a framework to operationalize principles of conditional secrecy. We demonstrate how the confidence scores from Transformer models can be used as a practical mechanism to enforce a principled secrecy policy, segmenting data into dynamic security levels. This approach allows organizations to manage the trade-off between automation coverage and risk exposure by treating low-confidence classifications as a controlled transfer to a trusted group of human reviewers. We validate this framework on the expert-annotated Monsanto Papers corpus, showing that Transformer-based classification, unlike traditional methods, provides the necessary discriminative power to make this principled approach viable. Our work offers a new, risk-aware methodology for the secure deployment of AI in high-stakes information governance.