One day you wake up and realize that you need Amazon pricing data which is not available in a structured form, so you need to figure out how to scrape it yourself. At first, it might sound easy, so you fire a bunch of HTTP requests using your favourite HTTP client… and you hit a wall: a CAPTCHA, merciless rate limiting, or a page full of JavaScript that your HTTP client can’t run.