Chromedp

It’s clear what we are trying to achieve, so let’s consider the requirements. Firstly, we need a tool to render web pages since JavaScript is commonly used nowadays. Secondly, we require an API to communicate with the headless browser. Lastly, saving the result can be challenging as browsers are designed to interact with rendered results rather than directly with the source code.

Headless browser

So we are looking for a headless browser. We are going to use Chromes headless-shell because it’s easy to use, and it’s based on Chromium . The most significant advantage is docker image, which we can efficiently run on our local machine or anywhere in the cloud.