Skip to content
Snippets Groups Projects
Commit 9eae988a authored by Simon Rebers's avatar Simon Rebers
Browse files

Update crawler.py

parent a9ab0c0c
No related branches found
No related tags found
No related merge requests found
......@@ -23,7 +23,7 @@ ix = indexing.get_index()
ALLOWED_DOMAINS = ('en.wikipedia.org',)
wikipedia = True
crawl_cooldown = timedelta(minutes=1440) # setting the crawl cooldown time
crawl_cooldown = timedelta(days=1) # setting the crawl cooldown time
depth_limit = 2 # making sure to limit the amount of websites we crawl
queue = Queue()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment