2b563debbf
javadoc of new multiple-exist test
12 years ago
c03f75ebc3
fix DHT url receive see http://bugs.yacy.net/view.php?id=242
12 years ago
8fb1b1e290
*) simplified banner creation code
12 years ago
cd0b5f31b4
*) updated links to description of regex
12 years ago
8f2d3ce2f9
reduced locking situation in crawler: shifted synchronized location and
...
reduced time-out of robots.txt load limit
12 years ago
f93501e6e0
nice crawl name if crawl is started with file:// (was: null)
12 years ago
b4f0cac102
added the reindexing job servlet to the submenu structure
12 years ago
97ab5b90e8
- odt & ooxml (office document) parser correction to add content to fulltext index
...
- adjust Junit yacyVersionTest & ParserTest
- update yacyVersion.combined2prettyVersion to the default 4-digit minor ver.
12 years ago
b68fbe7d21
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/migration.java
12 years ago
06d3063dc9
- no downcase when using collection modifier
...
- removed warnings
12 years ago
8dbc80da70
redesign of index.exist-test: this shall now not be done using a single
...
id to be tested, but with a collection of ids. This will cause only a
single call to solr instead of many. The result is a much better
performace when testing the existence of many urls. The effect should
cause very much less IO during index transmission, both on sender and
receiver side.
12 years ago
7f63d3747d
more generic field selection for reindex option of documents with disabled fields
...
using Luke request to compare config with actual fields in index
12 years ago
c91c67c3cd
reject bad solr requests
12 years ago
44e363f37f
refactoring of WorkflowProcessor, added process counter, update of
...
process counter if an blocking thread dies. Added also a new column in
PerformanceConcurrency_p servlet to show the actual number of concurrent
processes.
12 years ago
4058369288
fixed query expressions for collection selection (added quotes)
12 years ago
f2e36fbd06
enhanced deletion process for very large number of documents
12 years ago
79401cb938
added reindex option for documents with disabled or obsolete fields to Solr Schema Editor page (IndexSchema_p.html)
...
this allows to remove obsolete fields from the index (according to current schema config)
by selecting all documents containig disabled fields.
12 years ago
cf36c1614f
prevent that concurrent deletion process causes wrong double-check in
...
crawl start
12 years ago
aeff31cd44
fix for workflow processor (cause: latest redesign for less threads)
12 years ago
77faeada4d
small memory leak patch
12 years ago
b24d1d18e4
removed synchronization and concurrency in Fulltext class, concurrent
...
deletions are now handled in ConcurrentUpdateSolrConnector
12 years ago
f965d04496
added new peer icons for Mentor peers and Mentee peers (not used yet)
12 years ago
b9b446bca6
- added ssl configuration sign (a lock) to network statistic/table
...
- fixed a bug in bitfield
12 years ago
7095446ad3
added checkbox (near port) to switch on ssl support (https access) to
...
the admin interface.
12 years ago
e6c8b545c2
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
a83c2fe833
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
4baa0d4a97
Added a default keystore for ssl encryption of the YaCy web interface.
...
This will enable https-access to YaCy, but this feature is disabled by
default using the new server.https=false attribute. This has two
purposes:
- make it easier for everyone to use https (just set server.https=true)
- provide the basis for secure yacy-to-yacy communication in the future
12 years ago
0aef60f66e
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
da191c839d
reduce SolrConnectorLogging setting (from default ALL to INFO)
12 years ago
aaddb4809c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
038f956821
fix for sitemap detection: the sitemap url was not visible if it
...
appeared after the declaration of robots allow/deny for the crawler
because the sitemap parser terminated after the allow/deny rules had
been found. Now the parser reads the robots.txt until the end to
discover also sitemap rules at the end of the file.
12 years ago
4fc6837690
- fix monitor url of crawl job in PerformanceQueues_p.html
...
- reduce logging of every index add (switch embeddedsolr.add from info to debug)
12 years ago
442ed50be0
removed some unnecessary synchronizations
12 years ago
9bd2aee180
migrated to solr 4.3.0
12 years ago
ad050ec88d
- upgraded httpclient, httpcore and httpmime
...
- removed httpclient 3.1 which has been used by solrj < 4.x.x and is now
not used any more
- fixed some parts in YaCy which used methods from httpclient 3.1
12 years ago
4b100f8b48
Merge branch 'master' of ssh://gitorious.org/yacy/rc1
12 years ago
3abf516ca7
merged classpath
...
Bitte geben Sie eine Versionsbeschreibung für Ihre Änderungen ein. Zeilen,
12 years ago
a1c989002b
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4652
...
generate dht data even if dht receive and dht transmission is switched
off
12 years ago
48e9a54e80
updated pdf parser
12 years ago
e26bdd4a52
fixes to deletion methods (removed unnecessary concurrency and added
...
removal of crawl queue entries)
12 years ago
f2c9b0b5f2
better robustness of Concurrent Solr Connector against update/deletion
...
thread failure
12 years ago
f7f3e28c5e
prevent that the size of the index is computed too many times.
...
Because the index size is now provided by solr, and the only way to do
that is a match for [* TO *], a size computation is quite complex and
time-consuming. Therefore this patch prevents that the method is called
at all and if necessary puts a DOS-preventing barrier in front of it.
12 years ago
cca19d94d4
re-declared some fields to be of type string rather than text which
...
makes them more efficient and less large
12 years ago
cc90f82dbb
increased default proxy client timeout to one minute
12 years ago
ed1d5bace6
draw the names of other peers which receive/send dht into the network
...
graphic
12 years ago
b528448332
enlarge network graph circle according to image height and reduce the
...
image height in the Network servlet. Overall, the image is now larger
but takes less space on the web page.
12 years ago
58d85b5b80
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
12 years ago
24d2b4baee
remove pre 1.0 migration statement which possibly overwrites user navigator setting
12 years ago
f1bb54943e
typo
12 years ago
d7fd346917
- added regular-expression based deletions
...
- on-demand collection-list generation for collection-based deletions
instead of a default collection-list presentation (this makes calling
the interface much faster since the computation of collections lists for
large indexes may take some seconds)
12 years ago
Michael Peter Christen