diff --git a/README.md b/README.md index 926c16b..3ea159c 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,12 @@ Playwright scraper and crawler # Versions and Differences -BFS version +**BFS version** The BFS version uses the Breadth-First Search Approach To ensure the crawler explores all pages more thoroughly the crawler processes all immediate links (siblings) at the current depth level before moving on to deeper levels. + +**Scrape Everything** +This pretty much lets the crawler to go wild (can't recommend) + +**Scrape Domain Scope only** +Scrapes within the domain scope (worse BFS version as this goes in a straight line and doesn't scan everything)