Description
The aim of the project is to recover valuable concert and event tickets.
The technical side of this project is very interesting, as it deals with both levels of web and network security.
⇾ Connections were made using automated browsers (Selenium), proxies, requests (httpx) and multiprocessing.
⇾ Analysis were carried out using Burp-Suite and the browser's Network tab.
Proxy Manager
To avoid being banned because of spam requests we needed the use of proxies.
Initially we used proxies scraped free of charge from free proxy sites, which required a high-performance scraping and testing script due to the volatility of proxies found free of charge on the net.
We then used pools of proxies whose reputation had been improved using basic techniques such as opening mass emails.
For our proxy manager, we created a gett function, which aimed to replace the get function from requests by maximising the rate of good status codes.
Selenium Optimisation
Headless Browser: Browsers without a visual interface use fewer resources, so you can open more sessions at the same time.
Session Sharing: I created a session sharing script to retrieve a visual browser, when a headless browser retrieved an interesting cookie. You can find the script here.
Multiprocessing: Using Python's multiprocessing module, we could open the different browsers