DialoGPT adapts pretraining techniques to response generation using hundreds of Gigabytes of colloquial data. Like GPT-2, DialoGPT is formulated as an autoregressive (AR) language model, and uses a multi-layer transformer as model architecture. Unlike GPT-2, which trains on general text data, DialoGPT draws on 147M multi-turn dialogues extracted from Reddit discussion threads. The implementation is based on the huggingface pytorch-transformer and OpenAI GPT-2.

Commonsense question answering examples

|User | Bot |
|———|——–|
|who is the first president of the United States | George Washington |
|what is the boiling point of water? | I think it’s about 212 F. |
|which one is bigger, sun or moon? | The sun . |
|which animal has black and white stripes? | A black and white striped cat . |

Handling context that does not appear in training examples (note that training examples are sampled from 2005 to 2017)

|User | Bot |
|———|——–|
|The trading war between China and US is still happening . | I’m sure the Chinese government and the U . S . will be fine with that as long as they keep their arms on the table and the money flowing . |
|who won the world cup in 2018 ? | Germany , I believe . |
|Nvidia’s Titan RTX is really good . | I have this card and i love it . No complaints . |
|Can Porsche beat Tesla with its new Taycan EV ? | I don’t think so . The Tesla is more powerful than the porsche .

Our bot’s response to philosophy questions

|User | Bot |
|———|——–|
|What is the meaning of a good life ? | I think it’s that we’re all connected to our past lives and the meaning of life is to live the way you want to and how you want to . |
|Which is more important — being successful or being happy ? | Being successful is just endless . Being happy is the only thing that matters . |
|Is peace the only way to stop war ? | Not if the world is not willing to work with you . Peace is the ultimate goal , but peace requires more than that . |
| How much freedom should people have ? | The right amount of freedom is the freedom to do as you please , as long as you don’t hurt people or property .|

Finally

|User | Bot |
|———|——–|
|What do you think of the DialoGPT repo ? | It’s pretty good , but it’s very easy to find a bug . |
Please start a issue if you spot any :)
This project is a joint project between MSR AI and Microsoft Dynamics 365 AI Research team
Source: https://www.microsoft.com/en-us/research/project/large-scale-pretraining-for-response-generation/