Microsoft has developed and deployed conversational AI ranging from FAQ chatbots to virtual assistants to assist users. Image credit: Microsoft
Microsoft participated in the Australian AI Ethics Principles pilot.
The company prepared this case study summarising its observations in applying the Australian AI Ethics Principles to conversational AI. The case study is an example of how the company considered the principles. The principles are voluntary and organisations can choose to apply them in different ways. The case study includes links to resources businesses can use when designing or deploying AI responsibly.
Microsoft is a platform provider and developer of computer software, hardware, and related products. One AI technology it has developed and deployed is conversational AI.
Conversational AI ranges from FAQ chatbots to virtual assistants who can assist users to complete tasks or help them make decisions based on data analysis. For example, someone might ask ‘why are sales in this city lower than others?’.
Microsoft provides the building blocks for conversational AI technology through tools like the Microsoft Bot Framework. Microsoft’s partners and customers then customise and deploy that technology across the intended applications where people will use the technology. Microsoft supports its partners and customers to responsibly use the conversational AI technologies it provides to them.
This case study on conversational AI is based on Microsoft’s experiences as both a developer and deployer across various applications. For the pilot, Microsoft:
Microsoft has found it is an ongoing journey to instil a company-wide culture and approach to operationalising responsible AI. It has a range of multi-disciplinary committees, working groups and champions including:
While Microsoft has established new governance mechanisms and processes, this may not work for other organisations. There is no one-size-fits-all approach. For example in financial services, Microsoft is aware some banks have augmented existing operational or ICT risks frameworks to consider AI ethics issues. Smaller organisations may also find it easier to change existing processes rather than establish new ones.
Microsoft’s starting point in applying the Australian AI Ethics Principles was to ask ‘Are the Australian AI Ethics Principles even relevant to conversational AI?’ On face value, people might assume conversational AI does not result in significant ethical impacts on people – and in many situations this is likely to be the case.
Potential impacts of using AI technologies should always be considered from the beginning, even where the AI application appears benign. Asking initial questions can help developers and deployers to focus on the most relevant ethical considerations. For example:
For example, customer service chatbots are a common conversational AI technology unlikely to cause severe harm. However if a business uses or manages chatbots inappropriately, it may cause negative impacts, such as:
Working through the Australian AI Ethics Principles assists organisations to make informed choices about how to avoid ethical impacts. Examples of broader design or deployment decisions include:
Decisions on how to design and deploy a chatbot so that it meets the Australian AI Ethics Principles will depend on the context. For example, handovers from a chatbot to a human are important where:
By contrast, handovers may be less important where customers encounter problems using AI to order takeaway food. Clearly this may still cause customer frustration, but that AI application has less significant impact compared to the government emergency example.
Microsoft’s core principles for responsible AI align with the Australian AI Ethics Principles. The company shared the following observations on applying them to conversational AI.
A chatbot’s purpose needs to be clearly defined. The team should assess whether:
Businesses should inform their customers at the start of an interaction that they are not interacting with a human. This ensures respect for users and prevents them feeling as though their autonomy has been infringed. Where there are potentially significant negative impacts, the chatbot should be able to redirect users to a human when requested. These ‘human hand-off mechanisms’ give humans control over bot functions and respect individual preferences for engaging with virtual chatbots.
Businesses should seek diverse views on an AI system’s benefits and risks early in its development and throughout its lifecycle. If users with a disability aren’t consulted for example, the team may overlook issues like designing chatbots in line with accessibility standards.
Useful practices:
See Microsoft’s Guidelines for Human-AI Interaction.
One way businesses can detect and mitigate bias is by considering input data and algorithm design, and putting audit systems in place. Microsoft provides an open-source toolkit to assist developers assess model fairness and mitigate negative impacts of bias (visit Microsoft’s FairLearn toolkit website). Businesses also need strategies to address bias and implement supporting governance processes.
Businesses also need to consider the potential for discrimination.
Useful practices to prevent chatbot and user interactions resulting in discrimination:
Microsoft’s own experience with Tay – a chatbot it deployed in 2016 – generated important lessons. Microsoft learnt how conversational AI systems may feed off positive and negative interactions with people. Malicious or ignorant users can train an AI-powered chatbot to exhibit negative behaviours.
Chatbot design needs to address discrimination technically and socially. The social lens needs to anticipate the wide ranging interactions society will have with the chatbot.
Chatbots can provide new opportunities for people with disabilities to access government services, interact with businesses, and work more efficiently.
To unlock this potential, businesses can make inclusive design choices allowing people with disabilities to more easily interact with chatbots. For example, a design team can build chatbots with voice activation capability to improve access for visually impaired persons. Procedures for referring customers back to humans also need to be accessible for people with disabilities.
Useful practices:
See Microsoft’s inclusive design toolkit.
British Broadcasting Corporation (BCC) partnered with Microsoft to build an end-to-end, customised digital voice assistant. BBC had to ensure its chatbot was multilingual to include its diverse workforce and audience or customers. It adjusted bot interactions with different local accents.
BBC also prioritised retaining trust with chatbot users through secure privacy controls. It worked with Microsoft to maintain complete control over the data use. This meant BBC could use data to improve services, while still maintaining transparency for users on how the data was used.
Chatbots can learn a great deal about its users, so privacy considerations are especially important. Businesses must obtain user consent to collect and process data. Businesses should keep users informed about what and how their data is used.
All Microsoft’s development teams must follow a privacy standard. Teams take steps to maintain privacy, through practices like data privacy impact assessments.
Useful practices:
Businesses should ensure AI systems operate as intended and that they identify and address performance outside expected parameters. To improve the accuracy of conversational AI, businesses may set clear metrics and closely monitor performance data for any issues. These key metrics can include:
The relevant error and reliability thresholds will vary according to the use case. A bot used for army recruitment should have a lower margin for error than one used to purchase socks. However, the ideal error rate for any use case should be one closest to zero.
Useful practices:
For Microsoft, being transparent means being able to provide meaningful and relevant information to stakeholders in different contexts. In Microsoft’s experience with conversational AI, both technical and non-technical transparency are necessary.
For example, Microsoft has developed non-technical transparency measures that highlight the ethical AI issues that can occur.
Businesses need to consider how they are transparent when making choices about:
Microsoft considers contestability especially relevant when the chatbot has the potential to have a significant impact on individuals, society or the environment. Even for less sensitive applications, Microsoft considers it important to have mechanisms in place for humans to be brought into the loop if needed.
Microsoft uses practices such as:
Businesses should tailor these practices to the use case depending on its sensitivity. For example, where a chatbot is designed to recommend retail purchases, it is unlikely a human needs to review this decision beforehand, unless a user requests a human assessment or has further questions.
Microsoft wants to see AI developers and deployers ask appropriate questions and ensure they use the AI system responsibly. All these businesses have complementary responsibilities on ethical impacts throughout an AI system’s lifecycle. As a developer, Microsoft ensures it understands:
Those deploying the AI systems should consider how the AI technology might result in negative impacts based on their intended use.
If a business designs and develops a bot for a third party to deploy, both parties need to have a shared understanding. They need to know who is responsible for what, and document that understanding .
Microsoft assists its partners and customers on responsible approaches by:
Through this pilot, Microsoft has learnt:
Microsoft strongly supports the government’s initiative to pilot the Australian AI Ethics Principles:
‘Translating AI principles into practice requires asking important questions and making complex choices.
‘We encourage further pilots between government agencies, industry and other stakeholders. These pilots allow for collaboration in identifying issues which can then be turned into improved tools for responsible AI.
‘We encourage the government to explore additional pilots with regulators or bodies, particularly in financial services or healthcare where AI is increasingly used.
‘Microsoft is committed to responsible AI in how we design, develop and deploy AI systems. Our goal is to make sure our AI systems are fair, reliable, safe and trustworthy.’
If you have any questions about this example or are interested to learn about Microsoft's ethical processes, see Microsoft’s responsible AI resources site.
Learn more with Microsoft’s:
