Skip to content

Commit fba0592

Browse files
authored
Merge pull request #5986 from MicrosoftDocs/pva-guidance
Pva-guidance
2 parents 03b85da + 5164ef0 commit fba0592

37 files changed

+1130
-31
lines changed

power-virtual-agents/TOC.yml

Lines changed: 20 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,14 @@
1010
href: admin-certification.md
1111
- name: Quotas, limits, and configuration values
1212
href: requirements-quotas.md
13-
- name: Use GPT capabilities in Power Virtual Agents
14-
items:
15-
- name: AI-based chatbot authoring overview (preview)
16-
href: nlu-gpt-overview.md
17-
- name: Quickstart guide for building bots with GPT (preview)
18-
href: nlu-gpt-quickstart.md
19-
- name: Boost conversations (preview)
20-
href: nlu-boost-conversations.md
21-
- name: Create (and edit) with Copilot (preview)
22-
href: nlu-authoring.md
13+
- name: What's new in Power Virtual Agents
14+
items:
15+
- name: Overview of what's new
16+
href: overview-preview.md
17+
- name: Quickstart for what's new
18+
href: quickstart-preview.md
19+
- name: Key concepts for Composer users
20+
href: composer-concepts-preview.md
2321
- name: Get started
2422
items:
2523
- name: Get access to Power Virtual Agents
@@ -30,26 +28,18 @@
3028
href: fundamentals-get-started.md
3129
- name: Community help and support
3230
href: fundamentals-support.md
33-
- name: What's new in Power Virtual Agents
34-
items:
35-
- name: Overview of what's new
36-
href: overview-preview.md
37-
- name: Quickstart for what's new
38-
href: quickstart-preview.md
39-
- name: Key concepts for Composer users
40-
href: composer-concepts-preview.md
4131
- name: AI capabilities in Power Virtual Agents
42-
items:
43-
- name: AI-based chatbot authoring overview
44-
href: nlu-gpt-overview.md
45-
- name: Quickstart guide for building bots with GPT
46-
href: nlu-gpt-quickstart.md
47-
- name: Generative answers
48-
href: nlu-boost-conversations.md
49-
- name: Generative answers with Search and summarize
50-
href: nlu-boost-node.md
51-
- name: Create (and edit) with Copilot
52-
href: nlu-authoring.md
32+
items:
33+
- name: AI-based chatbot authoring overview
34+
href: nlu-gpt-overview.md
35+
- name: Quickstart guide for building bots with GPT
36+
href: nlu-gpt-quickstart.md
37+
- name: Generative answers
38+
href: nlu-boost-conversations.md
39+
- name: Generative answers with Search and summarize
40+
href: nlu-boost-node.md
41+
- name: Create (and edit) with Copilot
42+
href: nlu-authoring.md
5343
- name: Administer and manage
5444
items:
5545
- name: Assign licenses
@@ -207,7 +197,6 @@
207197
href: admin-share-bots.md
208198
- name: Apply and configure data loss prevention policies
209199
items:
210-
211200
- name: Example 1 - Use DLP to block skills
212201
href: dlp-example-1.md
213202
- name: Example 2 - Use DLP to block HTTP requests
@@ -241,4 +230,4 @@
241230
- name: Requests to delete data
242231
href: gdpr-delete.md
243232
- name: Requests to discover data
244-
href: gdpr-discovery.md
233+
href: gdpr-discovery.md

power-virtual-agents/docfx.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,9 @@
6767
},
6868
"ms.service": {
6969
"**.*": "power-virtual-agents"
70+
},
71+
"ms.subservice": {
72+
"guidance/**/*.md": "guidance"
7073
}
7174
},
7275
"template": [],
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
- name: Power Virtual Agents guidance
2+
href: index.yml
3+
- name: Building chatbots
4+
items:
5+
- name: Topics best practices
6+
items:
7+
- name: Topics in Power Virtual Agents
8+
href: topics-overview.md
9+
- name: Triggering topics
10+
href: triggering-topics.md
11+
- name: Defining chatbot topics
12+
href: defining-chatbot-topics.md
13+
- name: Topic authoring best practices
14+
href: topic-authoring-best-practices.md
15+
- name: Using the Fallback topic
16+
href: fallback-topic.md
17+
- name: Trigger phrases best practices
18+
href: trigger-phrases-best-practices.md
19+
- name: Slot filling best practices
20+
href: slot-filling-best-practices.md
21+
- name: Measuring and improving chatbots
22+
items:
23+
- name: Measuring engagement
24+
href: measuring-engagement.md
25+
- name: Measuring outcomes
26+
href: measuring-outcomes.md
27+
- name: Deflection optimization
28+
items:
29+
- name: Deflection overview
30+
href: deflection-overview.md
31+
- name: Topic escalation analysis
32+
href: deflection-topic-escalation-analysis.md
33+
- name: Topic enrichment analysis
34+
href: deflection-topic-enrichment-analysis.md
35+
- name: Topic confusion analysis
36+
href: deflection-topic-confusion-analysis.md
37+
- name: Alternate escalation paths
38+
href: deflection-alternate-escalation-paths.md
39+
- name: Agent transcripts analysis
40+
href: deflection-transcripts-analysis.md
41+
- name: Custom analytics strategy
42+
href: custom-analytics-strategy.md
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: "Use custom analytics reports"
3+
description: "Guidance for extending the use and scope of analytics data in Power Virtual Agents by using a custom report template. Use your data with Dataverse, Azure data lake storage, and Power BI."
4+
author: athinesh
5+
ms.date: 5/11/2023
6+
ms.topic: conceptual
7+
ms.custom: guidance
8+
ms.author: athinesh
9+
ms.reviewer: iawilt
10+
ms.collection: virtual-agent
11+
---
12+
13+
# Custom analytics strategy
14+
15+
16+
Power Virtual Agents provides comprehensive [out-of-the-box analytics](/power-virtual-agents/preview/analytics-overview) that allow customers to understand a bot's usage and key performance indicators.
17+
18+
Customers can view reports related to:
19+
20+
- Performance and usage.
21+
- Customer satisfaction.
22+
- Session information.
23+
- Topic usage.
24+
- Billed sessions.
25+
26+
However, there are often scenarios where you'll need to create or use custom analytics. For example, you may need to:
27+
28+
- Share analytics with non-makers or users.
29+
- Report on conversation transcripts data for a period longer than the default last 30 days.
30+
- Design a report not covered by out-of-the-box analytics.
31+
32+
There are a few ways you can take the analytics data recorded by Power Virtual Agents and use it in customized reports.
33+
34+
## Power Virtual Agents analytics sample template report
35+
36+
To create custom analytics, our recommended approach is to start with the [Power Virtual Agents analytics sample template report](https://aka.ms/PVAAnalytics).
37+
The sample template report is a set of open-source assets, distributed through GitHub, greatly accelerating the time it takes to create a report that renders in Power BI.
38+
39+
> [!WARNING]
40+
> The solution is not part of the core Power Virtual Agents offering and will require configuration.
41+
>
42+
> The sample report is not supported by Microsoft, but you can raise issues in the GitHub repository to get help from the community.
43+
44+
### Dataverse
45+
46+
#### Conversation transcripts data
47+
48+
The analytics shown in the Power Virtual Agents portal come from a data service residing within Power Virtual Agents. Usage data is also written to the associated Dataverse environment, in the [conversation transcripts](/power-virtual-agents/analytics-sessions-transcripts) table.
49+
50+
#### Retention periods
51+
52+
By default, both sources have a data retention of 30 days, but customers can [change the retention period for conversation transcripts in Dataverse](/power-virtual-agents/analytics-sessions#change-the-default-period-of-session-transcript-retention).
53+
54+
#### Power Virtual Agents Dataverse tables
55+
56+
Power Virtual Agents uses the following tables for custom analytics in Dataverse:
57+
58+
- [**Chatbot**](/power-apps/developer/data-platform/reference/entities/bot) (`Bot`). This table includes details of each bot in an environment. Generally this is a small amount of data.
59+
- [**Chatbot Subcomponent**](/power-apps/developer/data-platform/reference/entities/botcomponent) (`BotComponent`). This table lists the topics, entities, and dialogs associated with the bot in your environment. Generally this is a small amount of data.
60+
- [**Conversation Transcripts**](/power-apps/developer/data-platform/reference/entities/conversationtranscript) (`ConversationTranscript`). This table contains detailed conversation data for all the chatbots in your environment. The size of the data in this table is related to the use of the bot and can be large.
61+
62+
### Azure Synapse Link for Dataverse (Azure Data Lake Storage Gen2)
63+
64+
When bots generate large volumes of conversation transcripts (roughly more than 10,000 sessions a month, representing more than 80 MB of transcript data), the recommended approach is to export bot data to an Azure Data Lake Storage Gen2 using the [Azure Synapse Link for Dataverse feature](/power-apps/maker/data-platform/export-to-data-lake). This approach has the benefit of offloading data from Dataverse database storage in a cheaper data lake storage. It also retains large volumes of conversation transcripts for long periods.
65+
66+
The export creates an incremental sync of configured Dataverse tables in the Azure data lake, using the Common Data Model format.
67+
68+
There are additional steps you'll need to take in addition to what is configured in the base template:
69+
70+
- [Create an Azure Data Lake Storage Gen2 and connect it to Dataverse](/power-apps/maker/data-platform/azure-synapse-link-synapse).
71+
- During configuration, select the *ConversationTranscript* table (_Chatbot_ and _Chatbot Subcomponent_ don't support incremental sync).
72+
- [Follow the guidance](https://github.com/microsoft/PowerVirtualAgentsSamples/tree/master/CustomAnalytics/DataFlowVersion) for setting up Power BI dataflows to process the incoming data.
73+
74+
> [!WARNING]
75+
> By default, Azure Synapse Link for Dataverse mirrors the configured table data from Dataverse to the Azure data lake. Therefore, any record that gets deleted in Dataverse (for example, by the recurring bulk delete job that deletes conversation transcripts older than 30 days by default) will also be removed from the Azure data lake. To work around this, you can create copies or snapshots of your data in the Azure data lake, or you can configure the synchronization to use the [append-only mode](/power-apps/maker/data-platform/azure-synapse-link-advanced-configuration#in-place-updates-vs-append-only-writes).
76+
77+
### Power BI
78+
79+
The custom analytics solution template includes a Power BI report that processes the raw transcript data (using Power Query) into a report that matches the Power Virtual Agents default analytics.
80+
81+
In addition, users of the report have access to:
82+
83+
- Data for all chatbots in an environment.
84+
- Data as far back as the feed provides (in Dataverse or in the Azure Data Lake Storage Gen2).
85+
- Raw data tables extracted from the Conversation Transcript table data – which can be used to create custom reports.
86+
- A transcript viewer, allowing users to see the actual conversations that occurred.
87+
88+
> [!WARNING]
89+
> This a complex report that performs significant transformations on the base data. Customers with Conversation Transcript tables greater than 80 MB should look to use the Azure Synapse Link for Dataverse and Power BI dataflows version of the report.
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
title: "Begin the bot topic design process"
3+
description: "You use bot topics in Power Virtual Agents to provide answers and information for your customers or bot users. Considering the types of questions and tasks that your bot users have assists in the development of your chatbot design plan."
4+
author: HenryJammes
5+
ms.date: 05/23/2023
6+
ms.topic: conceptual
7+
ms.custom: guidance
8+
ms.author: hejammes
9+
ms.reviewer: iawilt
10+
---
11+
12+
# Defining chatbot topics
13+
14+
15+
Defining the best topics for your bot requires an understanding of the questions users might ask or the tasks they try to accomplish, and the kind of information and automation you need to provide.
16+
17+
For example, a retail chatbot might start by asking the user to choose from four things they want to do: find a store, place an order, check the status of an order, or return a purchased product. Their answer could lead them into one of four topics, each with its own topic dialog.
18+
19+
An initial set of sources for this information include:
20+
21+
- Any existing frequently asked questions (FAQs) or knowledge bases (KB).
22+
- Common subjects raised by your employees or customers in internal or customer service scenarios. If you’re building a customer service bot, talk with existing service representatives to learn what the most common questions are and the order in which they’re typically asked.
23+
24+
## Different types of topics
25+
26+
Typically, bot users have a specific question or problem they want to address or an issue they want resolved (their "task"). The types of tasks undertaken by bot users (and, therefore, the types of bot topics you need to create) fit into three categories:
27+
28+
1. **Informational**: for example, "_What is…?_", "_When will…?_", "_Why…?_"
29+
2. **Task completion**: "_I want to…_", "_How do I…?_"
30+
3. **Troubleshooting**: _"Something isn’t working…_", "_I got an error message…_"
31+
32+
You may also need to create bot topics to handle ambiguous user questions, such as "_I need help_" or "_shopping_". These topics would ask the bot user for clarification so they can be routed to the correct topic.
33+
34+
## Topic design process
35+
36+
1. **Identify the topic**
37+
- Choose topics bot users ask about.
38+
- Start with high-impact topics.
39+
- Think about topics from an end-user's point of view. Keep in mind that they might be less familiar or knowledgeable about an area than you are.
40+
41+
1. **List all scenarios**
42+
- List all possible scenarios.
43+
- Categorize scenarios: **informational**, **task completion**, and **troubleshooting**.
44+
- Hierarchize topics: what are the initial questions a user asks?
45+
46+
1. **Design a high-level conversation tree**
47+
- Draw the conversation tree.
48+
- Define the hierarchy of questions asked inside each topic.
49+
- Define the fewest number of questions to understand the situation and provide the right solution.
50+
51+
1. **Validate and iterate on the design**
52+
- Read the dialog out loud before publishing to help identify if there are problems with the tone or words.
53+
- [Get analytics and read session transcripts](custom-analytics-strategy.md) for further optimization.
54+
- Continuously iterate on and refine your bot topics by observing your bot users' interactions with the bot after each authoring iteration.
55+
56+
> [!IMPORTANT]
57+
>
58+
> - Don’t just replicate what your website or app can already do – your customers will likely be familiar with your website or app and can accomplish common tasks themselves without needing to interact with a bot.
59+
> - Focus on creating topics for issues or scenarios which generate a high volume of chats or calls first. Work on the longer tail of other, less-critical issues over a period of time.
60+
> - Be as thorough as possible in your design and consider all of the potential scenarios that your users might ask or need help with.
61+
62+
## Plan for both single-turn and multi-turn interactions
63+
64+
For simple interactions, you create single-turn conversations with just one question and one answer. But more substantive topics require a multi-turn conversation with multiple back-and-forth interactions between the user and your bot.
65+
66+
For instance, if a user asks a retail bot whether there are any stores nearby, the chatbot might respond with a question such as _"What city do you live in?_" or "_What is your postal code?_" to narrow down its possible responses. The user’s response will determine the bot’s next response in the conversation.
67+
68+
## Use native generative AI capabilities
69+
70+
When designing a chatbot, you may not be able to anticipate all the types of questions your bot users ask. To help mitigate this, Power Virtual Agents incorporates a powerful AI-powered capability that uses the latest advancements in natural language understanding (NLU) models.
71+
72+
When you enable the [Boost conversations](/power-virtual-agents/nlu-boost-conversations) option for your bot and link your chatbot to a publicly available, Bing-indexed website, your bot can provide automatically generated, conversationally friendly, plain language responses without the need for you to create topics for every eventuality or "edge-case".
73+
74+
## Offload large volumes of single-turn question and answer pairs
75+
76+
Power Virtual Agents limits a chatbot to 1,000 topics.
77+
78+
If you have large numbers of FAQs or knowledgebases, you can avoid creating one topic for each question and answer pair by offloading them outside of Power Virtual Agents topics, while offering a single user experience in Power Virtual Agents by [using the Fallback topic](./fallback-topic.md).
79+
80+
> [!TIP]
81+
> [Question answering in Azure Cognitive Service for Language](/azure/cognitive-services/language-service/question-answering/overview) provides cloud-based NLP that allows you to create a natural conversational layer over your data. It is used to find the most appropriate answer for any input from your custom KB of information.
82+
> [Learn how to integrate question answering in Power Virtual Agents](/power-virtual-agents/integrate-with-question-answering).
83+
84+
> [!div class="nextstepaction"]
85+
> [Topic authoring best practices](topic-authoring-best-practices.md)
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
title: "Alternate Escalation Paths"
3+
description: "Set of best practices based on alternate escalation paths to improve the deflection rate of a Power Virtual Agents chatbot."
4+
author: athinesh
5+
ms.date: 05/23/2023
6+
ms.topic: conceptual
7+
ms.custom: guidance
8+
ms.author: athinesh
9+
ms.reviewer: iawilt
10+
---
11+
# Alternate escalation paths
12+
13+
> [!TIP]
14+
> There are a few strategies you can use to deflect the user from reaching the human agent when the bot user decides to escalate, without deprecating the user experience.
15+
16+
## Check for operating hours and queue size during escalation
17+
18+
In this approach, you can have a self-service action inside Power Virtual Agents, for example, a Power Automate cloud flow.
19+
20+
The action that checks for the operating hours of the human agent and agent queue size from the engagement hub using, for example, Dynamics 365 Omnichannel for Customer Service. It then transfers the chat only if the agent is available in the current hour or if the queue isn't full.
21+
22+
If the escalation happens outside the operating hours of the agent or the agent queue is full, then the bot can redirect the user to email support or schedule a call back.
23+
24+
This approach avoids unnecessary escalation and, in turn, improves the deflection rate.
25+
26+
## Provide an option to create a support ticket during escalation
27+
28+
When the user escalates to an agent, the chatbot can redirect through a self-service action to create a support ticket by providing the required details.
29+
30+
> [!div class="nextstepaction"]
31+
> [Agent transcript analysis](deflection-transcripts-analysis.md)

0 commit comments

Comments
 (0)