Commit Graph

8 Commits (02957d5982f0a74fe3877509c33482e6b6fec04f)

Author SHA1 Message Date
Michael Peter Christen 4a14122ba7 in case that a crawl profile has a collection assigned, use the
13 years ago
Michael Peter Christen c25d7bcb80 - added concurrency for robots.txt loading
13 years ago
Michael Peter Christen a33e2742cb - removed unnecessary synchronized and deadlock in crawler
13 years ago
Michael Peter Christen 43f3345c90 - removed dependencies from URIMetadataRow and made direct access to
13 years ago
Michael Peter Christen 21fe8339b4 - enhanced generation of url objects
13 years ago
Michael Peter Christen 5f0ab25382 removed the option to prevent removal of & parts inside of the
13 years ago
Michael Peter Christen 8219a445f3 refactoring
13 years ago
Michael Peter Christen 00c1c777fa refactoring
13 years ago