TUIS/Playwright_Scraper
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added everything to readme

Browse Source
This commit is contained in:
Kиро.Kрика 2024-08-14 21:05:04 +03:00 committed by GitHub
parent 635dcf20da
commit 0fa9f7003b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@ -3,6 +3,12 @@ Playwright scraper and crawler
# Versions and Differences # Versions and Differences
BFS version **BFS version**
The BFS version uses the Breadth-First Search Approach The BFS version uses the Breadth-First Search Approach
To ensure the crawler explores all pages more thoroughly the crawler processes all immediate links (siblings) at the current depth level before moving on to deeper levels. To ensure the crawler explores all pages more thoroughly the crawler processes all immediate links (siblings) at the current depth level before moving on to deeper levels.
**Scrape Everything**
This pretty much lets the crawler to go wild (can't recommend)
**Scrape Domain Scope only**
Scrapes within the domain scope (worse BFS version as this goes in a straight line and doesn't scan everything)