From 0fa9f7003b73aa04698f69efe159f3704a4d2b56 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?K=D0=B8=D1=80=D0=BE=2EK=D1=80=D0=B8=D0=BA=D0=B0?= <95271587+Goshko812@users.noreply.github.com> Date: Wed, 14 Aug 2024 21:05:04 +0300 Subject: [PATCH] Added everything to readme --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 926c16b..3ea159c 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,12 @@ Playwright scraper and crawler # Versions and Differences -BFS version +**BFS version** The BFS version uses the Breadth-First Search Approach To ensure the crawler explores all pages more thoroughly the crawler processes all immediate links (siblings) at the current depth level before moving on to deeper levels. + +**Scrape Everything** +This pretty much lets the crawler to go wild (can't recommend) + +**Scrape Domain Scope only** +Scrapes within the domain scope (worse BFS version as this goes in a straight line and doesn't scan everything)