We trained AI to generate Christmas stories

What comes to your mind when you hear “artificial intelligence”? One may think of an accurate machine capable of completing meticulous tasks, or intelligent software capable of tracking your digital footprint to predict further suggestions and keep you online. We like to think of AI as a creative genius, although the results show otherwise. 

We have seen AI writing Christmas songs, preparing recipes out of Christmas leftover food, and we even read AI-generated movie titles. That is why, inspired by experiments conducted by MIT Technology Review, The BBC, and the University of Toronto, we partnered up with our colleagues at Violet AI Labs to train an algorithm to generate Christmas stories. 

The approach

Humour and AI are two sides many researchers have tried combining, and many results were questionable. Of course, we wanted to join the fun! Our goal was for AI to generate fun and engaging Christmas stories. To do that, we decided to finetune a Generative Pre-trained Transformer 2 also known as GPT-2. 

GPT-2 was a model architecture designed and pre-trained by OpenAI in 2019 and is specifically designed for text generation purposes. We began by trying the GPT-2-base model without finetuning. Here is an example of one of the very first generated results:

It was the day before Christmas. I was sitting outside in the back of the living room, watching a movie and I was like, ‘oh my god, I just found that stuff. It’s all here. I’m really sick.’ I was like, ‘oh my god, I have to stop watching this.’ And I was like, ‘yessssssss.’ I was like, ‘oh my god, I don’t want to watch this.’ And it was like, ‘oh my god, what the hell’.

It is very obvious that the GPT-2 base model without finetuning didn’t produce the result we wanted. And so we tried again. 

‘O, dear! Christmas is coming in a fortnight, and I have got to think up presents for every body!’

‘No. I don’t want anything, no no, you don’t understand; I’m going to stay all winter in this door.’

‘Yes, you’re right. I suppose it is. But we’ll meet again in some days, when we’ll be happy in the summer.’

‘No, no, no. We have to get ready for Christmas, when the police are on our side, and I hope it will be a good time. Is there any need to stay in that condition?’

‘No. I don’t want to leave the city. I’m sure I won’t be in that condition. What would you like to do?’

‘I would like to stay, and we’ll have a good time together, and we’ll see how that goes.’

‘I can’t believe you’re going to stay in there.’

‘Well, I’m only looking for a fight. We must go to another house, and you must fight that one again. You cannot win this fight in the market place.

‘No, my girl, I know it. I’ve had enough of you. I know that you can’t get one more fight. But, you know, I don’t want to get into this fight. I’ve done enough and I know what you want to do. You need to know that I’m not going to allow you to win the fight. You need to know that the police are taking you seriously and your family is dying. I don’t want to take your life. I have no love for you. I don’t believe you’ll ever walk away from us. I don’t believe that you will walk away from us.’

It’s a complicated conversation. But the boy’s response is the same one that calls into question the price of his loyalty.

A woman pulled a bag over his head. The bag’s name was Wen Xue Ying. She had just paid him a visit.

“Wen Xue Ying? What’s wrong?” the woman asked.

“Why haven’t you been with me for a while? I can’t wait to take you away from us. It seems like you want to leave the country and join the Fuyi kingdom. My father isn’t going anywhere. In fact, my father’s almost gone. I don’t know what you’re going to do, but if I don’t go, you can try.

A bag over a woman’s head called Wen Xue Ying wants to join the Fuyi kingdom? A better result with a more linear narrative, but not really Christmas related.

We then set out to create a dataset of Christmas stories using americanliterature.com and scraping all possible Christmas-related short stories. The dataset was of 104 different shorter and longer Christmas stories.

We then tried finetuning a GPT-2 large model for 2 epochs over text from these 104 Christmas stories. 

After trying out a few different parameter settings we ended up with a parameter settings as such:

max_length = 500
min_length = 100
do_sample = true
top_p = 0.9
repetition_penalty = 1.5
temperature = 0.7

The lesson we learned

As with many of the previous experiments conducted, we realised it is not easy to make AI funny. Most of the results were rather random and followed no logical order. Names were messed up and thrown on every occasion, most of the sentences were too long and difficult to follow, and the sudden plot twist in some stories convinced us to not publish several of the results. AI is an expert at filling in words, but not at creating words. 

The results

After several attempts, the model was capable to generate stories and titles that related to the Christmas theme. Here are some of our favourite Christmas stories:

Rudolph -An Enthusiastic Character Who Loves Everything About Life

Once upon a time, in a place we all know and love, lived a little reindeer. Now, this reindeer was named Rudolph and he was very special.

That was to say, he had a great head and legs, beautiful brown eyes, big nostrils, and his whole body was a snow-white and golden one, with black fur and glowing horns; and the fur, when wet, made him look very beautiful. So this reindeer was, after all, not a snow-white, but was quite a reindeer, a snow-white in its own right. And so on and so forth! 

Some of Rudolph’s things are so beautifully wrought, such as a white fur coat, a tundra blanket, snow-white boots, fur-lined legs, an old woman’s hat, an ebony flower, a large red velvet stocking covered with purple ribbons and gold corduroys, and his own doll head. But it’s great work on some such rare precious objects as a large doll wig or a red velvet stocking wrapped up with silver chain links. And to his surprise, everybody loved him. Everybody loved him very much!

Beware: A story of St. Francis, Saint Joseph and others

St. Francesco da Nola, 17 December 2012

‘Twas the night before Christmas, when all through the house not a creature was stirring, not even a mouse. The stockings were hung by the chimney with care, in hopes that St Nicholas soon would be there. In the long darkness of midnight I crept quietly to my bed and shut myself up. What will happen next may perhaps surprise those who see. There’s many mysteries in God that will remain unsolved, until only then the soul finds itself completely free to live its own life under any form of divine law whatever.”You mean Christendom?” asked Mary suddenly.

“If I wanted to teach some children-of course-but I don’t need money for the latter. Only we Catholics think differently—it needs neither books nor prayer…For I wish only to keep their souls in order, but rather to put them on earth”.That pretty face told me everything: ‘Yes’, says mother, ‘we are Christians too!”.

I felt glad enough to hear Mary also cry aloud: “O dear, O sweet Angel!”, thought Christus. A young doctor named Daphna stood beside him. Mary held her hand, and spoke kindly to him. (We cannot find anywhere exactly where the account begins). And he answered:(Wherein it began!).

“The Reindeers are out of control” – The Daily Mail (UK) 

Once upon a time, in a place we all know and love, lived a little reindeer. Now, this reindeer was named Rudolph and he was very special. He loved to play with his friends on the road; when they were hungry or thirsty he would come up and give them some food, just like you do today! So that’s what made him so famous around here. But now it seems things have gone wrong for our dear friend…

He is not alone: “The Reindeers are out of control” – The Daily Mail (UK) Today I found myself wondering if there could be any hope left for my beloved reindeer at Christmas because last night he wasn’t feeling too well, but luckily he got better after dinner.

And then suddenly yesterday morning – well, actually quite late into the evening… After being asleep most of the day and only getting woken by herding animals from behind us to let me take care of him while shepherds brought milk from their cows, our favourite old lady came down to get some treats. We went over to the barn where she had been feeding herself until the dogs started barking about three hours ago.

When she returned home, she noticed something unusual which didn’t look right to anyone else who saw it first hand. She called the police. A couple minutes later the police arrived to find everything exactly how it should be.

This story appears in a new book titled ‘Reindeer Tales’ written by John Bunnett & Chris Bracken based off an article published in British newspapers including the BBC. It tells of another tale involving the reindeer, known as Fluffypony, one-time owner of Santa Claus, whose name is still a mystery to many people living near Rutherglow and its surrounding areas. In fact, according to Wikipedia, Fluffy has never really received much attention outside of Scotland, even though he did once make headlines back in 1999.

That year he wandered away during Easter holiday season along with two other sheep called Snowy Owl and Bunny Moon, both belonging to a family member named Sally Williams. They apparently ran away again soon afterwards.

At least four years earlier, however, Fluffy showed signs of having lost interest in following humans, especially those coming from England, although these events aren’t mentioned anywhere in the book. According to folklore, the reason why his owners kept bringing him to visit them each summer turned out to be a big mistake since, due to changes in weather conditions, he stopped returning visits altogether several times between 1998 and 2003.

As such, the authorities decided to remove him without warning. Since he died five months before the original disappearance, no trace can yet be located of Fluffy. However, legend holds that he may live forever within the confines of the North Highlands region itself thanks to a mysterious stone he carried, which is said to grant good luck whenever someone wants to meet him face-to-face. Thanks everyone!!!