hermens
caff520988
Removed unnecessary and unused code.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4171 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
d732840f8a
Avoid ConcurrentModificationException when accessing the PerformanceQueues page while yacy is indexing.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4170 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
35303f9504
add real size values (KBytes) of the DHT-In/Out-RAM-Caches to the PerformanceQueues page. A lot of users seem to tweak this value and it might help in finding the best size in relation to the peer's memory ressources.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4169 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
a9aef8e5e0
remove duplicate entries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4168 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
38bbd4a4b3
no code changes. just touched yacyClient.java to trigger a rebuild of the file in an uncleaned tree.
...
NOTE: run "ant clean" before building SVN 4166/4167 in a tree that includes class files from a previous build to make sure, that every class file is rebuilt!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4167 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
f717beecb1
- Changed yFormatter handling to be more flexible and produce more readable code for server pages. There are serverObject.putNum() methods to allow adding of number type values in a formatted form, and put() methods for number types that add them without formatting. This reduces the need to transform them into Strings in server pages and removes the HTML encoding step which is unecessary for numbers.
...
- some minor code cleanups (mostly unnecessary casts, null checks)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4166 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ca83f5a8d9
Add external lib FontBox which is part of the PDFBox (they extracted the font handling code into this package in 0.7.3).
...
Add the packages to the eclipse .classpath.
Closes: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=453
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4165 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
3352474dd8
Remove grouping separator in Network.xml (yacystats will woork without it) and format a few more numbers.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4163 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
06e6a1ff62
Add a generalized Formatter class yFormatter inspired by http://forum.yacy-websuche.de/viewtopic.php?f=5&t=437
...
At the current state it allows formatting of numbers (integer + decimal types) for output according to the Locale derived from the language setting in yacy. Network.(html|xml) and Status.html have been changed to use it for now (TODO: should be integrated into other servlets as well to reduce duplicate formatting code).
NOTE: For now the output format for Network.xml simulates the old behaviour which is wrong (it uses '.' as decimal and grouping separator), to make sure external scripts like the yacystats.de one won't break with this update.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4162 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
e77aec8c9d
fix handling of encrypted PDF-Documents (with default user password "")
...
- update PDFBox package to current version 0.7.3
- use new security model in PDFBox to "guess" wether we can decrypt a document or not
NOTE: When upgrading to this version make sure the old PDFBox-0.7.2.jar is removed from libx/
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4161 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
b54fcd732b
*) fixed exceptions that occured when non-integer values were entered where integers were expected
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4160 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
52c68875bd
*) removed (hopefully only) surplus double encodings ( http://forum.yacy-websuche.de/viewtopic.php?t=368 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4159 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
b5f7df8d0a
Speed up remove operations in rowCollections.
...
- Array element shifting during remove is only done when it is necessary to keep the order of a row collection.
- This will speed up the most expensive operation "common word shrinking" by a factor of 500-1000 (in the worst cases we shifted > 60 GB of data during this operation)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4158 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
5c91359297
accidently commited personal testing values as defaults in last commit.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4157 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
e255888095
Add headless AWT, nice level and memory parameters to the init script. It should work like the startYACY.sh now.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4156 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ce0bb1dc8a
Increase defaults for the DHT Recieve Limits to prevent "busy" states.
...
see
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4155 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
fdb0b861f8
*) fixed wrong calculation of network words, network links, network PPM if peer is senior or principal peer
...
*) added network QPH
*) banner is cached for 1 second to avoid DOS
*) still no logo
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4154 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
3b8540198b
finally fix the init script. tested this time.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4153 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
508de558f7
sbStackCrawlThread is null during first cleanProfiles() run at startup.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4152 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
70614385ef
Attempt to fix the "lost profile handle" bug.
...
It seems improbable, but it might happen, that during a crawl all queues (indexing, crawling, ...) except the crawl URL stacker ran empty. This commit adds an additional check for an empty crawl stacker queue before executing the profile cleaner.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4151 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
905e7e60f5
change dir to the yacy base directory before starting it making sure relative path specifications work properly.
...
see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=405
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4150 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
507ecd8afa
*) added banner that can be displayed like this: http://localhost:8080/Banner.png
...
possible arguments: textcolor, bgcolor, bordercolor
example: http://localhost:8000/Banner.png?textcolor=ffffff&bgcolor=121212&bordercolor=ffffff
take care: YaCy uses CMY color model!
*) there are still some known bugs, but I can't continue coding right now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4149 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
70884da0eb
Add external package JARs to classpath. (copied from startYACY.sh)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4148 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ebfd1e0b42
remove left over '>' in description and replace ' ' by '+' in rss search where URL-encoded parameters are required.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4147 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4ce25b3661
- documentation update
...
- start of new development cycle.
in case your don't know: commits until 0.553 will not automatically be used be the auto-update funktion
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4146 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1a0f89d7e8
release 0.55
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4145 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ed20531e68
don't encode in channel element as well
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4144 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
9b0948cb4c
gnarf. mixed up the positions. finally fixed...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4143 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
c0f5fc51ef
bugfix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4142 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
33fb2f756d
added emergency fail case in remote crawls
...
in extreme situations this will cause that no remote crawls are send out any more
this is bad, but it protects the case where failing remote crawls fill up the local queue too much,
which is even worse
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4141 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
c5a8585ac6
fix more encooding problems in yacysearch.rss.
...
- URL encoding for search terms where required
- removed "ugly" CDATA escaping
- UTF-8 encoding for the XML
- no HTML style escaping for XML/RSS element values
Note: some unicode characters might still be encooded in a wrong way.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4140 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
6b00fe0c4e
fix ArrayIndexOutOfBoundsException
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4139 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
e2f3268c13
*) removed double encoding ( http://forum.yacy-websuche.de/viewtopic.php?t=368 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4138 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3e60ae93b9
modified remote search snippet fetch behavior: do not fetch snippets for more than 300 milliseconds, even if the snippets can be found locally without online fetch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4137 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
97f1ca52bd
fox for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=390
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4136 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
143fa40d77
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=394&p=2382#p2382
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4135 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
711641f167
extended client connection clean-up:
...
there are now two time-outs, one for the complete connection time, and one for an idle time
connections that are idle for more than 2 minutes are closed, and connections that are alive since more than one hour are also closed
if the complete number of connections exceeds 64, all connections more than 64 and have most idle time are also closed
During normal operation of peers these forced closings should never appear,
but the existence of the idle connection check ensures the availability of the peer and the usability of the host.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4134 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b19bb6e5b1
- reverted svn 4132; this did not solve the problem and removed the emergency mehtod which caused production failure for shure within some hours
...
- removed and added some debugging lines
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4133 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
1eba408d2f
Make sure that sockets which couldn't be opened aren't handled as active connections, in which case they wouldn't be closed.
...
Please test this and report any problems (connections that stay open for a very long time according to http://<your_yacy_peed>/Connections_p.html to http://forum.yacy-websuche.de/viewtopic.php?f=5&t=386
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4132 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
03c5b4ad68
more fixes to the yacysearch.rss, it's now 100% valid according to http://feedvalidator.org
...
- RFC-822 date time had to include the time instead of date only
- <opensearch:link> doesn't exist -> <atom:link>, see http://www.opensearch.org/Specifications/OpenSearch/1.1
- <link> elements are mandatory for <channel> and <item>
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4131 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
e3c6236eef
fixed the last opensearch/rss issue. The GUID-Tag in RSS is supposed to coontain a unique ID. By default, the ID is supposed to be a permanent link to the feed element (the permalink) in which case it's content _must_ match the syntax of a URL. The guid _can_ contain a non-URL ID, but it _must_ be specified as such with an additional isPermLink="false" attribute in this case.
...
see http://www.rssboard.org/rss-2-0#ltguidgtSubelementOfLtitemgt
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4130 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d69d386f7d
added additional forced client connection closing
...
if a specific number of simultanous connections is reached
the limit is currently set to 64 connections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4129 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dea7bee049
- increased minimum time before an active connection is interrupted from 1 minute to 10 minutes
...
- added sorting by connection time in client connection tabe of connectionTimeComparatorInstance
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4128 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
f8e69ce4dc
removed progress bar in Network list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4127 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c1440d2241
fixed problem with redirection: redirected URLs had not been tested with the double-check
...
see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=348
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4126 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
b183bf6f42
- fixed opensearch bugs
...
- added 'full domain' button to expert crawl start
- removed not-workin 'only one domain' button, the regex allowed crawling of other domains
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4125 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
7404f2c35c
Fix some of the issues with the RSS search interface, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=392
...
Note: the new DateFormatter822 in the plasmaSwitchboard is just a copy of the DateFormatter that always uses the US locale to allow formatting of a loocale independent date String.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4124 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
98abe0804d
another enhancement to crawl starts with link files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4123 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
fuchsi
ed2ca8fc4c
Add search type to top word suggestion searches.
...
Closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=391
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4122 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
aef1ab9526
#updated German translation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4121 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago