How can we build human values into AI?

In their recent paper “In conversation with AI: Aligning language models with human values,” the researchers explore a different approach to dialogue agents, specifically designed to reduce the risk of unsafely or inappropriately answering questions. They introduce Sparrow, a dialogue agent that uses an open-ended question structure and is designed to answer questions using an open-domain information retrieval algorithm. The researchers explain that Sparrow’s success depends on its ability to align with human values, which are the assumptions or beliefs that people have about certain topics. They note that in natural language dialogue, humans often ask questions to establish a relationship of trust and establish some aspect of understanding. In this case, Sparrow is designed to avoid asking unpleasant or dangerous questions by using a more open-ended structure that allows for exploration into relevant information without necessarily providing definitive answers. The researchers also note that despite the potential benefits of a well-designed dialogue agent, the development of these agents requires careful consideration of the ethical implications of the technology. They argue that understanding the values and beliefs of the users who will interact with Sparrow is critical to ensuring its safety, efficiency, and usefulness. The paper’s findings suggest that a more open-ended dialogue agent approach could be effective in achieving successful communication between humans and AI systems, especially when it comes to exploring complex topics and establishing trust. The paper also encourages researchers to address the challenges of developing safe and productive dialogue agents by considering the ethical implications of their design and implementation.