This article details the process to review DocsBot replies.
NOTE:
This should be performed every business day for the previous 24 hours. On Monday’s this will need to be performed for the period from Friday to Monday.
It’s recommend that this be performed at the same time each day, for example at 5pm ET.
Reviewing the data
- Ignore any rows with simple greetings with no other text or requests to speak with a live agent or human.
- Rows in a language other than English can be translated with Chrome’s translation extension. Just selected the text and right click to translate.
- Pay attention to multiple rows that have the same User, these represent conversations with DocsBot.
- Verify that any links included in the answer are the correct links.
How to fix issues and retrain the AI
If you see any incorrect answers, or answers that can be improved, follow these steps to retrain the AI:
- Click on the conversation to see details in the popup.
- Click the Revise answer link in the popup. Check the Question field to make sure this is the question you need to correct.
- Edit the text in the Answer field to provide the correct answer, then click Save.
The process below is not necessary as WP Simple Pay does not get many questions per day, around 2~6.
This is copied from AIOSEO.
Exporting the data
To get started with gathering the data for review, follow these steps:
1. Log into DocsBot at https://docsbot.ai/.
2. Click on Bots in the left-hand menu.
3. Click on the WPSP Support bot.
4. Click on the Questions icon.
5. Select the date range as the last period since you performed this process.
6. Click the Export Logs button to download the CSV file.
7. Repeat these steps for any other Bots.
Organizing the data
Next, we need to import the data in a Google Sheet and organize the data. To do this, follow these steps:
1. Open the DocsBot Reply Review sheet here – https://docs.google.com/spreadsheets/d/1-6Oe3IwFNU4-4ysVZ55Z8pQFl5hHAvp99Vz30XPT_uU/
2. Click on the Imported Data tab.
3. Upload the CSV file you downloaded from DocsBot and select Append to current sheet in the Import location drop-down.
4. Delete the ID, Question, Sources, and Referrer columns.
5. Rearrange the remaining columns into this order:
- Timestamp
- Alias
- Rating
- CouldAnswer
- StandaloneQuestion
- Answer
The sheet should match the example on the Template tab.
6. Select all of the data in the sheet and click Data > Sort Range > Advanced range sorting options.
7. Select the Data has header row checkbox and the sort on the Timestamp column in ascending order (so the oldest is at the top).
8. Delete any rows that were reviewed the previous day. For example, if you perform this task every day at 5pm, then you can delete any rows up to 5pm the previous day.
NOTE:
The timezone for this data is UTC (GMT-0).
Reviewing the data
Next, we’re going to review the data. To do this, follow these steps:
1. Start by locating any rows that contain text in a language other than English and translate the content to English so that you can review it.
2. Next, set up conditional formatting to highlight any references to “refund” or “cancel” in the StandaloneQuestion and Answer columns.
3. You can now find and delete any rows that have been highlighted as these contain requests for a refund or cancelation. We’re confident that the replies to these are correct, so they don’t need to be reviewed.
4. Next, delete any row that contains the following:
- A greeting such as Hi or Hello, with no other text.
- Any request to speak with a live agent or human.
5. You can now review the questions and answers that are left in the sheet. Pay attention to multiple rows that have the same Alias, these represent conversations that someone has had with DocsBot.
6. Verify that any links included in the answer are the correct links.
How to fix issues and retrain the AI
If you see any incorrect answers, follow these steps to retrain the AI:
1. Log into DocsBot at https://docsbot.ai/.
2. Click on Bots in the left-hand menu.
3. Click on the AIOSEO Support bot.
4. Click on the Questions icon and select the date range.
5. Search for the alias that corresponds to the incorrect answer.
6. Click on the conversation that matches the one you need to correct.
7. Click the Revise answer link in the popup. Check the Question field to make sure this is the question you need to correct.
8. Edit the text in the Answer field to provide the correct answer, then click Save.