Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Various updates towards v0.3.0 #23

Merged
merged 9 commits into from
Nov 16, 2024
Merged

Various updates towards v0.3.0 #23

merged 9 commits into from
Nov 16, 2024

Commits on Nov 11, 2024

  1. crawler: crawl: store result in a hash map, minor improvements

    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    c198e40 View commit details
    Browse the repository at this point in the history

  2. crawler: http: recycle http.Client among workers

    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    2870381 View commit details
    Browse the repository at this point in the history

  3. crawler: restore support for '--override-host' 

    When '--override-host <host>' is provided, the host in links of the
sitemap will be replaced by the one passed. This only affects links
at the first level of the sitemap.

Fixes #18
    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    48d7d45 View commit details
    Browse the repository at this point in the history

  4. crawler: http: Ensure HTTP response body is closed 

    Previously, the body could stay open in specific conditons,
causing increased memory usage.
    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    82422b4 View commit details
    Browse the repository at this point in the history

  5. tests: move tests and add new tests

    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    0521d42 View commit details
    Browse the repository at this point in the history

  6. Dockerfile: bump to golang:1.23

    @Pixep
    Pixep committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    8af6b2d View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2024

  1. tests: add benchmarking code

    @Pixep
    Pixep committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    e1e4085 View commit details
    Browse the repository at this point in the history

  2. Makefile: push both version and latest of docker image

    @Pixep
    Pixep committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    399aa95 View commit details
    Browse the repository at this point in the history

  3. crowlet: bump version for CLI

    @Pixep
    Pixep committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    38c8250 View commit details
    Browse the repository at this point in the history