Commit Graph

8 Commits (f8f05ecba7885fe304448d5d22e7b9ab1aa1b6e5)

Author SHA1 Message Date
Michael Peter Christen 4a14122ba7 in case that a crawl profile has a collection assigned, use the
12 years ago
Michael Peter Christen c25d7bcb80 - added concurrency for robots.txt loading
12 years ago
Michael Peter Christen a33e2742cb - removed unnecessary synchronized and deadlock in crawler
12 years ago
Michael Peter Christen 43f3345c90 - removed dependencies from URIMetadataRow and made direct access to
12 years ago
Michael Peter Christen 21fe8339b4 - enhanced generation of url objects
12 years ago
Michael Peter Christen 5f0ab25382 removed the option to prevent removal of & parts inside of the
12 years ago
Michael Peter Christen 8219a445f3 refactoring
13 years ago
Michael Peter Christen 00c1c777fa refactoring
13 years ago