Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revision | |
computing:managingbots [2025/04/06 19:31] – oemb1905 | computing:managingbots [2025/04/06 19:32] (current) – oemb1905 |
---|
------------------------------------------- | ------------------------------------------- |
| |
This tutorial is designed for Debian OS and LAMP stack users that want to track and or prohibit bot scraping (or other url requests) that might harm server performance and/or cause it to fail. In my case, I have a multi-site WordPress that includes my tech blog, poetry, and teaching blog. Additionally, I have a separate vhost on the same instance for my dokuwiki and another for file-share only Nextcloud. The same instance also runs my business’s email server with postgres and dovecot. Needless to say, I don’t want this virtual appliance to have down time. The first thing I did was create a script that would scrap and tally all the bots and how much they have done during the last day: | This tutorial is designed for Debian OS and LAMP stack users that want to track and or prohibit bot scraping (or other url requests) that might harm server performance and/or cause it to fail. In my case, I have a multi-site WordPress that includes my tech blog, poetry, and teaching blog. Additionally, I have a separate vhost on the same instance for my dokuwiki and another for file-share only Nextcloud. The same instance also runs my business’s email server with postgres and dovecot. Needless to say, I don’t want this virtual appliance to have down time. The first thing I did was create a script that would scrape and tally all the bots and how much they have done during the last day: |
| |
* [[https://repo.haacksnetworking.org/haacknet/haackingclub/-/blob/main/scripts/apache/bot-scrape-daily.sh?ref_type=heads|Daily Bot Scrape]] | * [[https://repo.haacksnetworking.org/haacknet/haackingclub/-/blob/main/scripts/apache/bot-scrape-daily.sh?ref_type=heads|Daily Bot Scrape]] |