vk url scraper
v0.3.30
用於從 vk.com URL 中抓取數據,尤其是影片和照片等媒體連結的 Python 庫。
您可以透過命令列使用它或將其作為 python 庫使用,請查看文件。
您可以透過pip install vk-url-scraper
從 pypi 安裝最新版本。
目前,您需要手動卸載並重新安裝一個依賴項(因為它是從 github 安裝的,而不是 pypi):
pip uninstall vk-api
pip install git+https://github.com/python273/vk_api.git@b99dac0ec2f832a6c4b20bde49869e7229ce4742
要使用該庫,您需要 vk.com 的有效使用者名稱/密碼組合。
# run this to learn more about the parameters
vk_url_scraper --help
# scrape a URL and get the JSON result in the console
vk_url_scraper --username " username here " --password " password here " --urls https://vk.com/wall12345_6789
# OR
vk_url_scraper -u " username here " -p " password here " --urls https://vk.com/wall12345_6789
# you can also have multiple urls
vk_url_scraper -u " username here " -p " password here " --urls https://vk.com/wall12345_6789 https://vk.com/photo-12345_6789 https://vk.com/video12345_6789
# you can pass a token as well to avoid always authenticating
# and possibly getting captcha prompts
# you can fetch the token from the vk_config.v2.json file generated under by searching for "access_token"
vk_url_scraper -u " username " -p " password " -t " vktoken goes here " --urls https://vk.com/wall12345_6789
# save the JSON output into a file
vk_url_scraper -u " username here " -p " password here " --urls https://vk.com/wall12345_6789 > output.json
# download any photos or videos found in these URLS
# this will use or create an output/ folder and dump the files there
vk_url_scraper -u " username here " -p " password here " --download --urls https://vk.com/wall12345_6789
# or
vk_url_scraper -u " username here " -p " password here " -d --urls https://vk.com/wall12345_6789
from vk_url_scraper import VkScraper
vks = VkScraper ( "username" , "password" )
# scrape any "photo" URL
res = vks . scrape ( "https://vk.com/photo1_278184324?rev=1" )
# scrape any "wall" URL
res = vks . scrape ( "https://vk.com/wall-1_398461" )
# scrape any "video" URL
res = vks . scrape ( "https://vk.com/video-6596301_145810025" )
print ( res [ 0 ][ "text" ]) # eg: -> to get the text from code
# Every scrape* function returns a list of dict like
{
"id" : "wall_id" ,
"text" : "text in this post" ,
"datetime" : utc datetime of post ,
"attachments" : {
# if photo, video, link exists
"photo" : [ list of urls with max quality ],
"video" : [ list of urls with max quality ],
"link" : [ list of urls with max quality ],
},
"payload" : " original JSON response converted to dict which you can parse for more data
}
有關所有可用功能,請參閱[文件]。
(更多資訊請參閱 CONTRIBUTING.md)。
pip install -r dev-requirements.txt
或pipenv install -r dev-requirements.txt
設定開發環境pip install -r requirements.txt
或pipenv install -r requirements.txt
設定環境make run-checks
(修復樣式)或單獨進行black .
和isort .
-> flake8 .
驗證 lintmypy .
pytest .
( pytest -v --color=yes --doctest-modules tests/ vk_url_scraper/
使用詳細、顏色和測試文件字串範例)make docs
產生 shpynx 文件 -> 如果需要,編輯 config.py要測試main .py 中可用的命令列介面,您需要將-m
選項傳遞給 python,如下所示: python -m vk_url_scraper -u "" -p "" --urls ...
pipenv run pip freeze > requirements.txt
vk-api==11.9.9
。./scripts/release.sh
建立標籤並推送,或者git tag vx.yz
來標記版本git push origin vx.yz
-> 這將觸發工作流程並將專案放在 pypi 上如果由於某種原因 GitHub Actions 發布工作流程失敗並出現需要修復的錯誤,您必須從 GitHub 中刪除標籤和相應的版本。推送修復後,使用以下命令從本地克隆中刪除標籤
git tag -l | xargs git tag -d && git fetch -t
然後重複上述步驟。