Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for new instances #366

Open
petrospap opened this issue Sep 4, 2024 · 1 comment
Open

Request for new instances #366

petrospap opened this issue Sep 4, 2024 · 1 comment

Comments

@petrospap
Copy link

Hello!
I have a small request
when you add new crawler/s, is it possible to create a separated file with new instances, something like

[
  {
    "info": "Info",
    "created_date": "2024/09/02"
    "totalnewinstances": 1
  },
  {
    "pattern": "Googlebot-Image",
    "newinstances": [
      "Googlebot-Image/2.1"
    ]
  }
]

Reason for this request is, i made a php/javascript script to manage Crawler Agents, and this will help me to update database more easily
Script created by your crawler-user-agents.json and follows the same logic,
(if you see my page, they exist in category "crawler")

Script also can detect some malicious activity (in javascript)

you can see/download https://simply4all.net/agent-management

Thanks

@monperrus
Copy link
Owner

this could perhaps be done with an additional CI job, feel free to make a pull-request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants