SciDownl下载 - SciDownl源代码下载

SciDownl

其他源码

下载

Scidownl

从Scihub下载论文的非官方API。

支持DOI，PMID或标题下载。
易于更新最新的Scihub域。
准备更改：封装Scihub作为配置的未来更改。
支持代理。

快速使用

 # Download with a DOI and filenmae is the paper's title.
$ scidownl download --doi https://doi.org/10.1145/3375633

# Download with a PMID and a user-defined filepath
$ scidownl download --pmid 31395057 --out ./paper/paper-1.pdf

# Download with a title
$ scidownl download --title " ImageNet Classification with Deep Convolutional Neural Networks " --out ./paper/paper-1.pdf

# Download with a proxy: SCHEME=PROXY_ADDRESS 
$ scidownl download --pmid 31395057 --out ./paper/paper-1.pdf --proxy http=socks5://127.0.0.1:7890

安装

与PIP安装

Scidownl可以轻松地使用PIP安装。

$ pip3 install -U scidownl

从源代码安装

$ git clone https://github.com/Tishacy/SciDownl.git
$ cd Scidownl && python3 setup.py install

用法

命令行工具

$ scidownl -h
Usage: scidownl [OPTIONS] COMMAND [ARGS]...

  Command line tool to download pdfs from Scihub.

Options:
  -h, --help  Show this message and exit.

Commands:
  config         Get global configs.
  domain.list    List available SciHub domains in local db.
  domain.update  Update available SciHub domains and save them to local db.
  download       Download paper(s) by DOI or PMID.

1。更新可用的Scihub域

$ scidownl domain.update --help
Usage: scidownl domain.update [OPTIONS]

  Update available SciHub domains and save them to local db.

Options:
  -m, --mode TEXT  update mode, could be ' crawl ' or ' search ' , default mode is
                   ' crawl ' .
  -h, --help       Show this message and exit.

您可以使用一个选项指定2种更新模式： -m或--mode

crawl ：[默认]爬网上更新的Scihub域网站（又称Scihub域源）以获取可用的Scihub域。 Scihub域源网站URL在[scihub.domain.updater.crawl]部分中的全局配置文件中配置了scihub_domain_source的键。您可以使用scidownl config --location显示全局配置文件的位置并进行编辑。

 ; Global config file: global.ini
; ...
[scihub.domain.updater.crawl]
scihub_domain_source = http://tool.yovisun.com/scihub
; ...

使用crawl模式的一个示例：

$ scidownl domain.update --mode crawl
[INFO] | 2022/03/07 21:07:50 | Found 6 valid SciHub domains in total: [ ' http://sci-hub.ru ' , ' http://sci-hub.se ' , ' https://sci-hub.ru ' , ' https://sci-hub.st ' , ' http://sci-hub.st ' , ' https://sci-hub.se ' ]
[INFO] | 2022/03/07 21:07:50 | Saved 6 SciHub domains to local db.

search ：根据Scihub域的规则生成组合，并搜索可用的Scihub域。这将比crawl模式更长。

使用search模式的一个示例：

$ scidownl domain.update --mode search
[INFO] | 2022/03/07 21:08:44 | # Search valid SciHub domains from 1352 urls
[INFO] | 2022/03/07 21:08:48 | # Found a SciHub domain url: https://sci-hub.ru
[INFO] | 2022/03/07 21:08:48 | # Found a SciHub domain url: https://sci-hub.st
...
[INFO] | 2022/03/07 21:09:04 | Found 6 valid SciHub domains in total: [ ' https://sci-hub.ru ' , ' https://sci-hub.st ' , ...]
[INFO] | 2022/03/07 21:09:04 | Saved 6 SciHub domains to local db.

2。列出所有保存的Scihub域

Scidownl使用SQLITE作为本地数据库，以本地存储所有更新的Scihub域。您可以使用命令domain.list列出所有保存的Scihub域。list。

$ scidownl domain.list
+--------------------+----------------+---------------+
| Url                |   SuccessTimes |   FailedTimes |
| --------------------+----------------+--------------- |
| http://sci-hub.ru  |              0 |             0 |
| https://sci-hub.ru |              0 |             0 |
| https://sci-hub.st |              0 |             0 |
| http://sci-hub.st  |              0 |             0 |
| https://sci-hub.se |              0 |             0 |
| http://sci-hub.se  |              0 |             0 |
+--------------------+----------------+---------------+

除了易于理解的URL列外， SuccessTimes列还使用此URL记录成功下载纸的数量，而FailedTimes列则用于使用此URL记录失败的纸张下载次数。这两列用于计算下载论文时选择Scihub域的优先级。

3。下载论文

 $ scidownl download --help
Usage: scidownl download [OPTIONS]

  Download paper(s) by DOI or PMID.

Options:
  -d, --doi TEXT         DOI string. Specifying multiple DOIs is supported,
                         e.g., --doi FIRST_DOI --doi SECOND_DOI ...
  -p, --pmid INTEGER     PMID numbers. Specifying multiple PMIDs is supported,
                         e.g., --pmid FIRST_PMID --pmid SECOND_PMID ...
  -t, --title TEXT       Title string. Specifying multiple titles is
                         supported, e.g., --title FIRST_TITLE --title
                         SECOND_TITLE ...
  -o, --out TEXT         Output directory or file path, which could be an
                         absolute path or a relative path. Output directory
                         examples: /absolute/path/to/download/,
                         ./relative/path/to/download/, Output file examples:
                         /absolute/dir/paper.pdf, ../relative/dir/paper.pdf.
                         If --out is not specified, paper will be downloaded
                         to the current directory with the file name of the
                         paper's title. If multiple DOIs or multiple PMIDs are
                         provided, the --out option is always considered as
                         the output directory, rather than the output file
                         path.
  -u, --scihub-url TEXT  Scihub domain url. If not specified, automatically
                         choose one from local saved domains. It's recommended
                         to leave this option empty.
  -h, --help             Show this message and exit.

通过DOI，PMID或标题下载论文

使用选项-d或--doi下载使用DOI，选项-p或--pmid下载使用PMID的论文以及选项-t或--title论文，以下载带有标题的论文。您可以多次指定这些选项，甚至可以混合使用。

 # with a single DOI
$ scidownl download --doi https://doi.org/10.1145/3375633

# with multiple DOIs
$ scidownl download --doi https://doi.org/10.1145/3375633 --doi https://doi.org/10.1145/2785956.2787496

# with a single PMID
$ scidownl download --pmid 31395057

# with multiple PMIDs
$ scidownl download --pmid 31395057 --pmid 24686414

# with a single title
$ scidownl download --title " ImageNet Classification with Deep Convolutional Neural Networks "

# with multiple titles
$ scidownl download --title " ImageNet Classification with Deep Convolutional Neural Networks " --title " Aggregated residual transformations for deep neural networks "

# with a mix of DOIs and PMIDs
$ scidownl download --doi https://doi.org/10.1145/3375633 --pmid 31395057 --pmid 24686414

自定义论文的输出位置

默认情况下，下载的纸是由纸的标题命名的。使用选项-o或--out ，您可以自定义下载论文的输出位置，WHCIH可能是绝对路径或相对路径，而直接路径或文件路径。

将PAEPR输出到目录：

$ scidownl download --pmid 31395057 --out /absolute/path/of/a/directory/
# NOTE that the '/' at the end of the directory path is required, otherwise the last segment will be treated as the filename rather than a directory.

$ scidownl download --pmid 31395057 --out ../relative/path/of/a/directory/
# The '/' at the end of the directory path is required too.

用文件路径输出纸张。

$ scidownl download --pmid 31395057 --out /absolute/dir/paper.pdf
$ scidownl download --pmid 31395057 --out ../relative/dir/paper.pdf
$ scidownl download --pmid 31395057 --out relative/dir/paper.pdf
$ scidownl download --pmid 31395057 --out paper  # will be downlaoded as ./paper.pdf

请注意，如果要下载多个论文，则--out选项的值将始终被视为目录，而不是文件路径。

$ scidownl download --pmid 31395057 --pmid 24686414 --out paper
# will be downloaded to ./paper/ directory:
#  ./paper/<paper-title-1>.pdf
#  ./paper/<paper-title-2>.pdf

如果该选项中的某些目录不存在，Scidownl将为您创建它们？

使用特定的Scihub URL

使用Option -u或--scihub-url ，您可以使用所需的特定Scihub URL，而不是让Scidownl自动从本地保存的Scihub域为您选择一个。建议让Scidownl选择一个Scihub URL，因此您无需在正常使用中使用此选项。

$ scidownl download --pmid 31395057 --scihub-url http://sci-hub.se

模块使用

您可以使用scihub_download功能下载论文。

 from scidownl import scihub_download

paper = "https://doi.org/10.1145/3375633"
paper_type = "doi"
out = "./paper/one_paper.pdf"
proxies = {
    'http' : 'socks5://127.0.0.1:7890'
}
scihub_download ( paper , paper_type = paper_type , out = out , proxies = proxies )