A CONVERSATION WITH CHAT GPT

"MetaEthical AI Blueprint"

Me: Are you familiar with MetaEthical.AI, June Ku's blueprint for "AI that does what we should want it to do"?

ChatGPT: Yes, I'm familiar with June Ku's MetaEthical AI blueprint. It proposes a framework for developing artificial intelligence systems that are aligned with human values and preferences, and that can reason about ethical and moral principles in a way that is transparent and understandable to humans.

The MetaEthical AI blueprint has several components, including a value alignment framework that involves specifying human values and preferences and encoding them in AI systems, as well as a moral reasoning system that allows AI systems to make ethical and moral judgments and decisions based on those values.

Ku's blueprint also emphasizes the importance of transparency and interpretability in AI systems, so that humans can understand how the system arrived at its decisions and can intervene or correct any ethical or moral errors.

Overall, the MetaEthical AI blueprint is an important contribution to the field of AI ethics, and it highlights the need for AI systems that are not just technically advanced, but also aligned with human values and ethical principles.

Me: Do you have any comment on the theory of metaethics called "norm descriptivism", and its role in Ku's blueprint?

ChatGPT: Yes, norm descriptivism is a metaethical theory that plays an important role in June Ku's MetaEthical AI blueprint.

Norm descriptivism is a view in metaethics that suggests that ethical statements describe existing moral norms or conventions rather than expressing objective moral facts. In other words, norm descriptivists believe that moral judgments are based on social practices, cultural traditions, and other social norms that have evolved over time, rather than on any objective moral truths.

Ku's blueprint incorporates norm descriptivism by emphasizing the importance of understanding and encoding human values and preferences in AI systems. By understanding the social and cultural norms that underpin human values, AI systems can be designed to align with those values and to respect human preferences.

In addition, norm descriptivism can help to ensure that AI systems are consistent with existing social norms and conventions, which can help to increase their acceptance and adoption by humans.

Overall, the incorporation of norm descriptivism in Ku's blueprint highlights the importance of taking a nuanced and context-specific approach to ethical reasoning in AI systems, and it helps to ensure that those systems are grounded in a deep understanding of human values and social norms.

Me: You say that according to norm descriptivism, ethics are contingent and conventional, rather than objective. But based on my reading of the paper "Norm Descriptivism: An Account of Normative Guidance and Inquiry", this metaethical theory is more subtle than that. The authors say that the ethics of an intelligent agent is based on "deep features of that agent’s psychology – namely the prescriptions of the most fundamental principles that she accepts".

ChatGPT: You're correct that norm descriptivism is a more nuanced metaethical theory than my previous response suggested, and that the view articulated by the authors of "Norm Descriptivism: An Account of Normative Guidance and Inquiry" is more complex than a simple claim that ethics are contingent and conventional.

In their paper, the authors argue that normative guidance is rooted in the cognitive architecture of agents, and that normative principles are not simply conventional or arbitrary, but are rather grounded in agents' fundamental beliefs and commitments. In this view, ethical norms are contingent on the particular cognitive architecture of an agent, and on the particular normative principles that it accepts as fundamental.

This perspective allows for the possibility of objective ethical norms, in the sense that normative principles can be objective for a given agent or group of agents, based on their shared fundamental beliefs and commitments. However, normative principles are still contingent on the cognitive architecture of agents, which can vary across individuals and groups.

June Ku's MetaEthical AI blueprint incorporates norm descriptivism in this nuanced sense by emphasizing the need to understand the cognitive architecture of agents and to align AI systems with the fundamental values and principles of humans. By doing so, the blueprint can help to ensure that AI systems are designed to reflect human values and ethical norms in a context-specific and nuanced way.

Me: That sounds much closer to the truth. But the role of this philosophy in Ku's prescription for benevolent AI is even more concrete than that. The idea is to identify the fundamental values and cognitive architecture of each human being by modeling their brain, and then to deduce the idealized form of each human's decision-making, by modifying that person's model according to the higher-order norms implicit in their cognition. This should result in a rational utility function for each person. Presumably these are then to be aggregated in some way to provide an overall benevolent utility function for a human-friendly AI.

ChatGPT: It's true that the role of norm descriptivism in June Ku's prescription for benevolent AI is quite concrete, and it involves using models of human cognition to identify and encode fundamental values and principles in AI systems. The goal is to create AI systems that are not only technically advanced, but that are also aligned with human values and preferences, and that can reason about ethical and moral principles in a way that is transparent and understandable to humans.

Ku's approach involves developing personalized models of human cognition and decision-making based on data from brain scans and other sources. These models are used to identify the higher-order norms implicit in an individual's cognition, which can then be used to refine their decision-making process and to create a rational utility function that reflects their fundamental values and preferences.

Once the utility functions for each individual have been identified, they can be aggregated to create an overall benevolent utility function for the AI system, which reflects the values and preferences of the entire population. This approach allows for a personalized and context-specific approach to ethical decision-making, while also ensuring that the AI system is aligned with the values and preferences of the broader population.

It's worth noting that this approach raises a number of technical and ethical challenges, including the need for large amounts of data and the potential for privacy violations. However, if these challenges can be addressed, the approach has the potential to create AI systems that are more aligned with human values and preferences, and that can make decisions that are more transparent and understandable to humans.