AI alignment
How to ensure AI system behaviour is aligned with out values.
Value aware AI
-
value representation
-
value learning
-
value agreements
-
value-alignment of agent
-
value alignment of MAS
-
value-based expandability
Value definition
Values are abstract constructs, hence we use taxonomies to relate these concepts. They must be grounded by goals to allow for computation. No all values are as important: they need to be weighted or prioritized. Values are dynamic, and can change with additional context or over time.
Value learning
Values in the medical case
- Beneficence
- Non-maleficence
- Autonomy
- Justice
Case studies
Fire fighting
- Institutional values
- Values from culture
- Personal values
- Decisions under pressure are difficult, even after training and preparation
It was modelled as a multi-objective MDP.
Negotiation
Paradox of altruistic rejection.
Personal gain is not everything. Individual utility is a weighted sum of personal values, long term objectives, … Accounting for other’s utility is sometimes needed
Every time a proposal is rejected, each agent rates it according only to its own utility function, and then shares the list of all proposals sorted by preference with all the other agents. The goal is to allow other agents to learn about the preferences of each other, to make more informed decisions in future negotiations.
Theory of mind
It is about attributing intentions to others. We are not born with these capabilities, and we usually develop them around the age of 4. Autistic people struggle with this.
Classic example: Sally-Anne test. Sally puts a marble in a basket and leaves the room. Anne moves the marble to a box. When Sally comes back, where will she look for the marble? The correct answer is the basket, but autistic children often answer the box.
Based on empirical data, LLMs struggle with theory of mind, as soon as the recursive depth of reasoning increases. Hanabi is a cooperative card game that requires theory of mind reasoning, and LLMs struggle with it.