Introducing a Tool for Bulk Updating Account Images and What I Learned

I developed a tool, puppeteer-account-manager, that can update profile information (images, etc.) for various services like Google and Github all at once. I will introduce the purpose of development and the knowledge gained from it.

The repository is here.

https://github.com/silverbirder/puppeteer-account-manager

Why did I create it?

Web services like Github, Twitter, and Facebook allow you to register a profile picture. Due to my personality, I want to register with the same image for all services.

Therefore, when I get a good profile picture, I have to re-register the profile picture for all services, which is very troublesome. So, I created this tool to solve this problem.

Isn't Gravatar good enough?

This trouble might be solved by a web service called Gravatar.

http://gravatar.com/

This service provides a global profile picture. You can get the profile picture via the API.

However, due to the following problems, it was rejected.

• The profile picture size provided by gravatar is 80px × 80px
  • It's too small for some services
    • Although the image size can be enlarged, the quality is not good
• The profile items provided by gravatar are fixed
  • I wanted to register not only images but also profile items all at once
    • Depending on the service, the profile items do not match

So, I decided to use Contentful, an API-based CMS.

https://www.contentful.com/

In Contentful, you can freely decide the items. It's very convenient because you can get the items you created (images and introductions) via the API.

How did I create it?

It's a straightforward method. I used a library called Puppeteer that can automatically operate the Chrome browser. I just automated the process of "logging into each service from the Chrome browser and uploading a photo".

https://github.com/puppeteer/puppeteer

Wasn't there an API to update the profile picture?

There is for some services. For example, Twitter has the following API to update the profile picture.

https://developer.twitter.com/en/docs/accounts-and-users/manage-account-settings/api-reference/post-account-update_profile_image

However, not all services have such an API. Although it is correct to update using the API, I decided to automate it with Puppeteer to align the implementation methods of all services.

Is the password okay?

I made the Puppeteer-running node application and the Chrome browser operate within the same machine. Therefore, the password will not be intercepted during the execution of the node application. Also, the password is set to be injected from environment variables. Since it can operate in a Docker container, it can be run both locally and in a container service.

In the future, I would like to manage passwords in conjunction with services like Keepass or Lastpass.

https://github.com/keeweb/kdbxweb

Which services are supported?

This is limited in its use because I made it for my own convenience.

For more details, please check

https://github.com/silverbirder/puppeteer-account-manager/blob/master/src/index.ts.

What did I learn?

I encountered quite a few challenges.

Try not to write code that specifies selectors

The HTML returned by a web service does not always remain the same. There is no guarantee that a certain id or class html tag will always remain.

Therefore, I tried to operate the browser without specifying selectors as much as possible. For example,

• Instead of clicking buttons or links to navigate pages, go directly to the target page in the shortest possible way
  • Like https://medium.com/me.
• Instead of clicking the submit button, enter the enter key

By doing this, I was able to automate stably.

XPath is surprisingly useful

In Google or Medium, the id or class is a random value. Therefore, you cannot proceed by simply specifying an id or class.

However, you can specify a selector that contains the text "○○" with XPath. This was helpful.

Give up without forcing difficult logins

Amazon login triggers two-step authentication. You are asked to log in via text message or voice call, and there is nothing you can do with Puppeteer alone.

You can disable this two-step authentication feature, but it's not good for security, so I decided to give up without forcing it.

Execute parallel processing aggressively

To improve processing speed, I processed all services in parallel with Promise.all. I also made each one open in a secret window to process independently. However, Puppeteer sometimes crashes. The cause is influenced by the specifications of the machine running it (number of cores), but it can also be affected by the service side. Therefore, I handled errors so that it's okay if it crashes, and made it retry.

Also, I wanted to know what kind of screen it was if it failed, so I made it take screenshots.

Executable with Docker

I packed the modules required for Puppeteer into Docker and outsourced login information etc. from environment variables, I was able to create an execution environment that is not dependent on the environment. Therefore, by combining Pub/Sub and Container Engine, etc., You can update account information via Contentful's Webfook.

In conclusion

If my personality was more rough, I wouldn't have made this tool, but I couldn't help but worry... (laughs) Thank you for reading to the end.

If it was helpful, support me with a ☕!

Share

Related tags