Overview of CALMS. CALMS uses a large language model in conjunction with conversational memory, document stores, and experimental tools to answer user queries or take action to drive an instrument. (Image: Prince et al. npj Comput Mater 10, 251 (2024))
Overview of CALMS. CALMS uses a large language model in conjunction with conversational memory, document stores, and experimental tools to answer user queries or take action to drive an instrument. (Image: Prince et al. npj Comput Mater 10, 251 (2024))
Upgrades to advanced scientific user facilities and instruments such as next-generation x-ray light sources, nanoscience centers, and neutron facilities are revolutionizing our understanding of materials across the physical sciences, from biology to microelectronics. However, these upgrades significantly increase complexity over previous generations of facilities and instruments. Driven by more exacting scientific needs, tools and experiments become more intricate each year, making it ever more challenging for domain scientists to design experiments that effectively leverage the capabilities of and operate on these advanced instruments. Large language models (LLMs) can perform complex information retrieval, assist in knowledge-intensive tasks across applications, and provide guidance on tool usage. Researchers led by Argonne National Laboratory explored the potential for Context-Aware Language Model for Science (CALMS) to assist scientists with instrument operations and complex experimentation.
With the ability to retrieve relevant information from documentation, CALMS can answer simple questions on scientific capabilities and other operational procedures. Designed to interface with software tools and experimental hardware, CALMS can conversationally operate scientific instruments. By making information more accessible and acting on user needs, LLMs could expand and diversify scientific facilities’ users and accelerate scientific output.
The core of CALMS consists of four components: an LLM, conversational history that allows follow-on queries, semantic search over document stores to retrieve the most relevant context given a question from the user, and instrument tools that the LLM can use when instructed to by the user. The team compared responses from two state-of-the-art LLMs, OpenAI’s GPT-3.5 Turbo and an open-source model Vicuna, over questions related to experimental planning assistance, operation, and ability to drive an instrument successfully, noting that the CALMS framework is independent of the chosen LLM.
In a paper published in npj Computational Materials, the researchers demonstrated that when provided with appropriate context and tools, CALMS can either answer complex technical questions of which the LLM has no prior knowledge, or it can execute an experiment or perform a computation.
The researchers anticipate LLMs capable of leveraging decades of information recorded in e-logs, as well as fully autonomously extracting code or commands and executing experimental workflows. Context- and tool-aware language models such as CALMS have the potential to streamline the transfer of knowledge between instrument experts and domain users, with the time saved enabling researchers to accomplish more and explore new science opportunities. This could enable a paradigm shift in the way users interact with the beamline instruments, paving the path to full automation and handling of exceptions based on past experiences recorded in the form of e-logs.
Prince, M. H., H. Chan, A. Vriza, T. Zhou, V. K. Sastry, Y. Luo, M. T. Dearing, R. J. Harder, R. K. Vasudevan, and M. J. Cherukara. “Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilities,” npj Computational Materials (November 2024), Springer Nature.
https://doi.org/10.1038/s41524-024-01423-2
