r34 crawler Download - r34 crawler Source code download

r34 crawler

Other source code

v1.0.0

Download

r34-crawler

A simple, self-contained, CLI tool to fetch and download images from rule34.xxx (Attention: This site definetly contains NSFW content!).

You can download the latest release binaries from the releases page.

Alternatively, you can also install the tool using cargo when you have the Rust toolchain installed.

cargo install --git https://github.com/zekroTJA/r34-crawler r34-crawler

Then, just open up a terminal (bash or powershell) and execute the binary with the --help flag:

Usage: r34-crawler.exe [OPTIONS] [TAGS]...

Arguments:
  [TAGS]...  Image tags

Options:
  -o, --output         The output directory for downloaded images [default: output]
  -l, --limit           Number of images to be collected
  -O, --offset         Offset to be skipped in collected images
  -a, --after-id     Query posts created after the given post ID
  -p, --page-size   The page size used per request when listing images [default: 250]
      --overwrite              Force overwriting already downloaded images
  -t, --threads       Number of threads used for downloading images in parallel [default: 4]
  -m, --meta             Store image post metadata in the given file as JSON
  -h, --help                   Print help
  -V, --version                Print version

You can specify an ammount of threads with the --threads (or -w) flag. That means, if you specify 4 threads, for example, that 4 images will be downloaded in parallel. 4 threads is also the default value, if not further specified, because it yields the best results in my personal tests. Your mileage may vary depending on your system performance and your network speed.

Tests were executed on a 250 MBit/s downstream. Of course, the speeds are also depending on the image sizes and compression rates as same as the speed of the machine and drives.