eca9380e3d
bugfix for crawler double-check: if an url is redirected, the
...
redirect-target was not double-checked. This is now done by replacing
the redirect-URL on the crawl queue again (where it is double-checked)
10 years ago
9ac0c93f17
fix for subpath crawl filter
10 years ago
9bc3e457dd
fix for termination of all crawls
10 years ago
66106bdaf0
fix for crawler attribute maxdompages
10 years ago
49d91b94c3
npe fix in crawler
10 years ago
8d650ca225
added hint to port forwarding videos
10 years ago
b7183a7321
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
ea2e627662
fix ConfigAccounts del user with uppercase letter in name
...
(usernames are case sensitive, userdb.delete used toLower)
10 years ago
c465b791af
typo
10 years ago
191ec8c82a
added concurrency to postprocess rewrite process
10 years ago
a1e8bdd5e9
log ppm instead of docs/second
10 years ago
cc0ded7abd
set process type of web graph according to fields as defined in the
...
schema
10 years ago
12fb9d7cd1
log postprocessing constraints in case that postprocessing is not
...
performed
10 years ago
3c23b89823
less logging
10 years ago
a0c53174c5
better solr query logging to detect unnecessary sort requests for more
...
performance profiling
10 years ago
338f574bdc
no sorting if http/www unique fields are not demanded (makes query
...
faster) and some code restrucuring
10 years ago
1609763be5
toString fix
10 years ago
b983e68254
more retries, less sleep
10 years ago
1503ba7794
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
8f77719091
fix "Ljava.lang.String" in crawl queue anchor name
...
(e.g. IndexCreateQueues_p.html?stack=LOCAL with images in queue)
10 years ago
0ceeceb35e
more logic on Solr queries; usage of the query terms in posprocessing,
...
saving one query for double document detection now per document
10 years ago
3963bca3b6
catch IndexControlRWIs_p error if RWI not connected
10 years ago
38864ae004
Merge branch 'master' of git@gitorious.org:yacy/rc1.git
10 years ago
4099296b45
added new classes which shall reduce call overhead to Solr (stub)
10 years ago
d0c02e1de7
adjust rss lat/lon to double
...
(common format across other classes)
10 years ago
3491ab4c38
removed unused images from webgraph edge computation
10 years ago
2371d6b8db
target linktexts must be string to enable search facets on these fields
10 years ago
001e05bb80
do not store failure of loading of robots.txt into the index as a fail
...
document
10 years ago
05d58e4df0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
98f45c9032
fix for image alt attachment to AnchorURLs in html parser.
10 years ago
22ce4fb4dd
better error handling for remote solr queries and exists-checks
10 years ago
b510b182d8
- update Maven pom
...
- add ppt parser test case
10 years ago
3dcfc717eb
This hopefully fixes http://mantis.tokeek.de/view.php?id=424
10 years ago
9df14fc126
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
477be17c51
Replaced old UPNP library with Weupnp. UPNP should
...
work now, at least it does on my network. UPNP code in YaCy can still
be improved though (see TODO comment: make port on gateway configurable
or find free one).
*) removed old code
*) added new lib
*) changed code to work with new lib
10 years ago
738989aab7
reverted commit f94c91315b because the
...
webgraph has not enough performance for that
10 years ago
e9163e7e10
fix for malformed hostpath names in crawl balancer
10 years ago
161a11070c
yacystats is gone :(
10 years ago
c115f3869c
enhanced snippet computation and test method in ViewFile
10 years ago
6c10b59f3e
move bootstrap peers test systems to its test class
...
var assignment not needed elsewhere.
10 years ago
7328c2883b
fix type in .init description
...
http://mantis.tokeek.de/view.php?id=430
10 years ago
94819f0797
set .ini default boost fields to same as assigned by button "reset to default"
...
(in RankingSolr_p)
- fix typo http://mantis.tokeek.de/view.php?id=430
10 years ago
b4b937a046
update to pdfbox 1.8.6
10 years ago
1027f3d04a
fix for the usage of ready-prepared solr queries, some queries are
...
formulated as edismax query but this was not set as query attribut. The
defType=edismax property needs a qf-field, so this was added as well. Do
not remove that field again! This fixes also a problem with title-unique
computation.
10 years ago
f94c91315b
if the webgraph is used, then use it also for reference computation to
...
avoid contradictions with references_i in the collection index.
10 years ago
6e1dc444c3
added a snippet test function in ViewFile: you can now search for a
...
specific word on the document; the servlet returns the snippet in the
same way as it would be shown in a search result.
10 years ago
c63e93df46
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
10 years ago
1bf605b6d1
toString() fix
10 years ago
4b06adb751
fix for file urls
10 years ago
08409ec680
no idea why the words max was an ordered one. This change increaes speed
...
dunring document processin a bit
10 years ago
Michael Peter Christen