Reverse AFM Height Map Search: Content-Based Topography Retrieval via Self-Supervised Deep Learning

Atomic force microscopy (AFM) is a unique and extensively adopted technique used to characterize various materials at nanoscale spatial resolution. 3M’s Corporate Research Analytical Laboratory has applied AFM to industry problems for over two decades, resulting in the production of tens of thousands of topographical image datasets along with unstructured contextualizing information. The ability to search this data could potentially provide immediate insight into current issues. However, the size and lack of structure of this aggregated dataset—and its collection over a long time period by multiple experts—makes locating data relevant to a newly pressing problem challenging. A tool with the ability to provide fast recall of meaningful historical data requiring only AFM data as input would therefore be invaluable. Content-based image retrieval (CBIR) applies computer vision techniques to the problem of retrieving images from a database on the basis of a search image rather than metadata or labels. However, the variety of approaches to CBIR, including recent developments in the application of deep learning, make determining an optimized path to solving the AFM search problem unclear. This work presents the results of applying a set of CBIR models utilizing both classical computer vision and deep learning approaches to AFM topographical data and defines a quantitative metric for evaluation of the models’ success in matching the opinions of expert AFM users. All models gave qualitatively good results, and the best performing deep learning approach according to the expert-informed metric was chosen to form the basis of a user-friendly, cloud-hosted, continuously updated web application, making content-based retrieval for historical AFM height map datasets an integral part of the 3M analytical scientist’s toolkit.