Feature request: Skip History, Wanted options

[codefaux]

For users with exceptionally large libraries and histories stretching back multiple years on one instance, querying Wanted and History (which the user cannot erase from the *arr side) can take a long time.

Retruning (considerably) over ten thousand pages of History items AND (considerably) over one thousand pages of Wanted items with every query creates a GIANT processing / metrics storage footprint for no reason.

During processing these long-running API requests, the webui becomes unresponsive.

Prometheus seems to wholesale discard results from *arr-exporter instances which return extreme volumes of metrics in one blast -- I'm not fully convinced of this oen, but EVERY *arr-exporter instance on my system store metrics in Prometheus except sonarr-exporter and radarr-exporter, and both of these suffer from the extremely large History issue I mentioned, both hang for several seconds during polling those metrics, etc etc.

Please add an environment variable to skip Wanted and History exporting.

Please consider fully removing the sortKey=airDateUtc argument; it adds provider sorting time (*arr's sorting algorithm is NOT superbly optimized) which Prometheus and Grafana are explicitly designed for.

prometheus.yml clip:

  - job_name: "sonarr-exporter"
    scrape_interval: 600s
    static_configs:
      - targets: ["x:9707"]

docker-compose.yml clip;

  sonarr-exporter:
    image: ghcr.io/onedr0p/exportarr:latest
    container_name: sonarr-exporter
    command: ["sonarr"]
    network_mode: host
    environment:
      PORT: 9707
      URL: "http://x:8989"
      APIKEY: "x"
    restart: unless-stopped

Logs of one poll interval;

2024-05-08T20:15:12.909Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/series"}
2024-05-08T20:15:12.910Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/system/status"}
2024-05-08T20:15:12.910Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/queue?page=1"}
2024-05-08T20:15:12.910Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/health"}
2024-05-08T20:15:12.909Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/rootfolder"}
2024-05-08T20:15:12.910Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/history"}
2024-05-08T20:15:41.149Z        INFO    Sending HTTP request    {"url": "http://x:8989/api/v3/wanted/missing?sortKey=airDateUtc"}

Note the thirty second delay between sending the History request and sending the Missing request. The Missing request stalls for longer, because it's doing an arbitrary unneccessary sortKey process on top of returning all the data, even though there are 10% as many Missing items as History items.

This is on a 64-core 3ghz machine with 256GB of RAM with SAS media storage and NVME backed docker container storage, so please don't try to tell me my machine is just slow. This is an actual, problematic issue for folks with extremely large histories and libraries. It cannot be worked around in the source applications without wholesale deleting their databases and rebuilding, or manually opening their databases and deleting things by hand.

Please add an environment variable to skip Wanted and History exporting.

Please consider fully removing the sortKey=airDateUtc argument; it adds provider sorting time (*arr's sorting algorithm is NOT superbly optimized) which Prometheus and Grafana are explicitly designed for.

[onedr0p]

I can see this being less than ideal but I don't have time to work on this at the moment, PRs are accepted.

[codefaux]

Roger that.

Guess it's time to take a crack at learning Go, because now that I have the idea in mind I'm probably gonna fixate on it, lol.

Thanks for the honesty. I'll see what I can come up with.