Links

Demo live version here: demo

Source code : github

How does it work ?

This is one of those projects which seem really tough at first but then turn out quite straightforward to implement!

In general there are three main components at play here:

  1. Detecting and encoding faces and linking them to roasts
  2. Comparing the encoded face of the "roastee" to one of the encoded faces, then retrieving roasts linked to that encoding
  3. Scraping r/roastme pictures and comments

At the beginning we were thinking of training our own neural network to link faces to roasts, or even better, to generate brand new ones. However we quickly realised that there was already a perfect solution implemented for us - the python face-recognition library!

Re-inventing the wheel is always fun, but hackathons are really not the best place for it.

Enough talking, let's see some code:

Encoding Faces

One way you can quantize a face, is to somehow represent it as a feature vector - a vector of features, pretty self-explanatory however the exact way to do it is not that simple,here is a more in-depth article about this using the same library. Luckily this will be done for us, let's have a look at the main library function we'll need:

import face_recognition


# load the picture
image = face_recognition.load_image_file('victim.png')

# retrieve all faces and encode them
# this will be a numpy array
# this will also throw an exception if there is no face present!
face_encodings_in_image = face_recognition.face_encodings(image)

As you can see the library is a God-send, one line of code and we have our victim's face encoded! If you look at the result of this you'll see that it is a 128 dimensional vector - each component could represent some sort of feature of the face, perhaps either a dimension, color or maybe an abstract neural network feature. Anyhow - once we have vector representations of many faces, we can compare them.

Comparing Faces

Say the encodings are stored along with the top roasts already, then a user uploads their own face in hopes of getting roasted - what then?

well, it's very simple, we encode their face first, and then compare it to the encodings we've already stored. The encoding in our dataset which is the closest to the user's will be our result and the roast linked to it shown to the user.

How exactly can we tell how close two different face encodings are to each other? It's actually really trivial, by calculating the distance between them!

They're vectors! The Euclidian distance is the easiest distance calculation between two vectors and the library actually already provides a helper function which calculates it for us!

# load images
# these have to contain a recognizable face, or the snippet will throw an exception
known_image_1 = face_recognition.load_image_file('known_face1.png')
known_image_2 = face_recognition.load_image_file('known_face2.png')
victim_image = face_recognition.load_image_file('victim.png')


# find all faces and encode them
known_image_1_encoding = face_recognition.face_encodings(known_image1)[0]
known_image_2_encoding = face_recognition.face_encodings(known_image2)[0]
victim_image_encoding = face_recognition.face_encodings(victim_image)[0]

# find distances
encodings_dataset = [known_image_1_encoding, known_image_2_encoding]

# this will be a numpy array with distance values, smallest = closest to victims face
distances = face_recognition.face_distance(encodings_dataset, victim_image_encoding)

This comparison gives us all we need to find the face most similar to our victims, now all that's left is assembling the database (Probably the most laborious).

Scraping Images and Roasts

I used PRAW, and a wrapper around the reddit API to collect some top voted posts with their best comments. This was a little troublesome since Reddit's API has some weird quirks, like scrambling real upvote counts. After a while of messing with it though, I managed to successfully collect hundreds of data points to use with the RoastR!

Each post was added to a JSON file which contained the raw encoding and comments.

With the database ready, all that was left, was just connecting the dots and creating an application which performs all these steps and happily roasts its users.

Results

Sadly our amazing application did not grab any rewards in the hackathon. It certainly left a mark on the judges who all got personally slammed though.

Now for the part you've been waiting for, here are some example roasts:

(the pictures taken from new un-scraped posts from r/roastme)

Victim:

roastee-1

Roast: "In a 24 hour day, how long do you spend waiting under bridges?"

Victim:

roastee-2

Roast: "The five kids everyone looks at during the anti-bullying assembly"

Victim:

roastee-3

Roast: "When you smoke so much weed that you start seeing guinea pigs"

Not bad eh?

Some of these are not quite right, i.e. they reference multiple people, or things that are not present, this is because we did not perform any kind of filtering on our dataset - and doing so would be very difficult. But for an overnight project, this is not bad at all.

You can play with the demo live version here: demo

And have a look at the source code here: github