Ask Me Anything How to track bots

[Graybeard]

/public_html/js/wyd.js << prepend your path ...
create a file that says:

you fuckin' dumb bot

read your server's access.log (for that domain) and see who, and who does not, go for the 'bait'

Googlebot requested that file <<< proof that a JavaScript will not fool Google. ( I have actually read this for a few years now -- no surprise.)

Dumb scraper bots do not request this file <caught >

 

[Graybeard]

And Facebook bot and googlebot took the bait ROFLMAO

 

[Graybeard]

Dumb scraper bots do not request this file <caught >

Newer Google headless bots using Selenium will request this file --> they are harder to detect because their user-agent name looks like the most current Chrome version when created/installed. However, these bots do not auto update like real peoples browser will > this is how many can be caught.

Happy bot shoot!

 

[T J Tutor]

Dumb scraper bots do not request this file <caught >

They're everywhere!

 

[Graybeard]

They're everywhere!

Newer Google headless bots using Selenium will request this file --> they are harder to detect because their user-agent name looks like the most current Chrome version when created/installed. However, these bots do not auto update like real peoples browser will > this is how many can be caught.

You can^t catch the newer types of bots with simple JavaScript traps --that was the point --

 

[T J Tutor]

You can^t catch the newer types of bots with simple JavaScript traps

There is a new era upon us and everything needs to be re-thought. Adapting to current structures of our industry are a constant battle with rules, regs, and oversights.

There is always an answer, many paths for a new approach are in development. In the end, there will be a shift, and it will likely be dramatic and create new battles.

 

[Graybeard]

Maybe, ICAAN Department of browser registration IDs --$10 per year
Every browser would have a public and a private key --that would have to be matched.
This is done with HTTPS (TLS) sort of but you would need some global huge database servers and probably quantum computing to deliver that volume in microseconds. Maybe the schema could be reduced somehow with a server side OCSP caching format.