How I built an extension to download messenger conversations
As I was browsing the chrome web store, I saw that all the extensions to download messenger conversations were not working anymore. This is because facebook changed its API recently. I decided to create a new extension so that anybody could keep downloading conversations.
1. How to download messenger conversations
I tried to search for hours if facebook or messenger had a public API to download our own messages but I did not find anything convincing.
The only way to do it right now is to go to settings on facebook and ask to download your personal data. The problem is it’s very long and it’s just raw data, you do not have a nice HTML file.
I tried to see if I could replicate any XHR requests made by the browser when going on messenger.com but did not find any request made to the backend (do not hesitate to leave a comment if you know how to do this)
The last solution I could think of was a bit complicated but still possible, I had to scrap the messenger website and try to see if I could imitate a human behavior on the website to pretend i’m scrolling.
After scrolling for a certain amount of time, I would just have to try extracting the messages to a file and allow the user to download it. Looks simple right ?
2. Scrapping the messenger frontend
The first step was to find the container on the UI where the messages were stored (the parent div of all messages), it’s the one on the image below:
The next step was to find the scrollable div in order to scroll it to download as much messages as we want. This div was not hard to find because it was above the div containing all messages.
In order to imitate the human scroll, I just had to scroll to the top of the container every X seconds, you can do this easily in javascript with this line:
scrollable_container.scrollTop = 0;
Now I had everything in my hands to just create an infinite scroll. I coded a little algorithm that just loops until it has downloaded a predefined number of messages
Sometimes the scroll bar would just stop working, I had to add some logic to the algorithm to unlock the scroll bar if it was locked after trying to scroll up multiple times.
With this simple technique, the extension I created is able to download messages at a rate of 400 messages / minute. The download time is slower when you have to download old messages, messenger is slower to download messages that are 2, 3, 4 years old.
Look at how I start scrolling automatically when I click on “start”
3. Exporting the messages to HTML
As I wanted to export the conversation in a nice format, I had to find a way to take the messages from the UI and put them in a file.
I’ve found a really simple way to do this, I downloaded the messenger.com html page and I only kept the styling (CSS) and the scripts needed. Then, I injected the parent div containing all messages into this HTML file and I had a good enough result (see image below).
4. Avoiding messenger updates
After putting all this code into a chrome extension, I’ve seen that messenger was doing frequent updates on the UI and that it was constantly breaking my extension because the name of the classes was changing and I had to rely on some classes to find elements on the UI.
The best solution I’ve found so far is to store this configuration (the name of the classes) online so I could update them easily whenever messenger does an update (it happens almost every two weeks).
I decided to put this configuration file on AWS S3 with the property Cache-Control: no-cache because I did not want the browser to cache this config.
Below is an example of the configuration file, it allows me to find elements on the messenger frontend. When the frontend is updated I just have to update this file.
This file is downloaded each time you open the chrome extension.
5. Creating a chrome extension to automate this.
I tried to create the simplest chrome extension to automate this process, you just have to input the number of messages you want to download and click start.
Here is the github: https://github.com/cazabec/messenger-downloader-extension
Thanks for reading me !
