orbiter
26fafd85a5
- more refactoring
...
- fixed problem with parsers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6433 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3528b970d6
- refactoring
...
- added new experimental (not-yet-working) image parser
- added new test image
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6431 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a8ce192f63
- shifted main classes to new package net.yacy
...
- fixed some bugs in last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6427 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b79f4f062f
refactoring of yacy documents and parsers: they depend now only on the kelondro classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
0fd9540866
Configuration of HTTPDProxyHandler logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6425 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
cee7a05ff2
- de-serialized the pdf parser
...
- added fail callback for file indexer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6415 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
9db928ce53
replaced fontbox 0.7.3 with fontbox 0.8.0
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6414 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c2272785c7
- fix for xlsx and pptx parsing
...
- less exception logging for swf parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6413 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c864901087
- moved httpd.mime to defaults path
...
- some documentation fixes
- adopted a default setting for the search window: moves css setting to base.css
- some enhancements for the DocumentIndex class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6410 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
8829ec5f18
*) made sure that is replaced with a space and not just deleted in CharacterCoding.java
...
*) added annotations and made minor changes to serverObjects.java
*) set subversion properties for several files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6409 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6c347a37eb
more options for DocumentIndex
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6408 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6192205533
more final modifier
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6407 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0f6b011e1a
fix for new index location and better way to use own classes by reflection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6406 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
7a3bbd950f
:-(
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6405 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b953f04f90
one more reflection fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6404 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
77d6604856
fix for npe, see http://forum.yacy-websuche.de/viewtopic.php?p=17727#p17727
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6403 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2a7fe35f92
performance tuning using more final modifiers in the kelondro core
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6402 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
cb4de9ceee
fixed a bug in table iterator (did not recognize elements in write buffer)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6401 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e7f18ba24b
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6399 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ce8dc575ca
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6398 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bea3b99aff
moved table and util classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bd876eb4b7
moved io classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6396 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c0e0e1f422
moved blob classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6395 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1e4f8b56ed
accumulated classes from different packages into the new rwi package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6394 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
194da25a2f
moved kelondro index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6393 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4446acc8cd
moved kelondro order
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6392 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f677d534b1
start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root
...
- moved here the logging classes as part of the new net.yacy.kelondro package
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6391 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ea473e32b8
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6390 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
735e2737e3
* added index segments
...
This is a major change in the organization of indexes.
Please consider a back-up of your data before you run this update.
All existing index files will be moved and renamed to a new position.
With this change, it will be possible to maintain different indexes for different purposes and it will be possible to have a distinction between DHT-in and DHT-out specific indexes. Tenants may also have their own index, and it may be possible to have histories and back-ups of indexes. This is just the beginning, many servlets must be adopted after this change, but all functions that had been there should still work.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6389 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
09de5da74a
once again a performance hack
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6388 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2f6d88403e
uä
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6387 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d2615ea5a8
increased memory for scraper buffer to enhance parsing speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6386 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4bbbb74ec4
removed not necessary synchronization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6385 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
67e5464cc2
Fix for SVN6380: x[] Arrays are unsuitable Keys for Maps without using a proper Comparator.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6384 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
aeab8c7917
Prevent failed DHT attemps from overwriting newer peer info
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6382 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
9324b5b6c5
Enhancements to DHT
...
- speed up deletion of containers when selscted from whole index
- correctly eliminate all references to unavailable URLs, not just the first encountered
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6381 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
e49e2d75fe
Limit the time Transmission.Chunks stay in the transmissionCloud by using a Map that iterates entires in insertion order.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6380 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
92db7c5d07
increased timeout for index retrieval
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6379 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
386b9f35f6
activated resource observer for windows 7
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6378 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6e0dc39a7d
- some fixes to prevent blocking situations
...
- better logging for the crawler
- better default values for the crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6377 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
51f2bbf04b
possible fix for problem in http://forum.yacy-websuche.de/viewtopic.php?p=17655#p17655
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6376 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f8371707e5
- possibly better termination for SplitTable
...
- better abstraction in DidYouMean
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6375 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
87780f2562
produce did-you-mean also for queries with more than one word
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6374 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
04a548a1e3
- temporary integrated the transferURL servlet as static class instead as a class that is called using reflection to investigate the OOM problems in that class
...
- fixes for numerous other problems
- removed dead code
- resdesign of the strings-method, which produces now less memory overhead and may help to prevent OOMs
- another fix for the deadlock problem in SplitTable
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6373 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ea427df944
fixed a worst case situation of the condenser which may cause a temporary full CPU load because of a bad data structure usage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6372 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3e38035389
fix for interrupted thread during has() property check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6370 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5bd1c1d205
just added some comments that had been produced to learn about OAI-PMH
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6369 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6aa474f529
- better logging for web cache access and fail reasons
...
- better Exception handling for web cache access
- distinction between access of web cache for proxy and crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6367 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3671c37989
added experimental oai-pmh reader and integrated it with the existing dublin core parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6366 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
58a00205d5
re-activated the emergency close when too many server connections exist
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6364 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c57d2070e6
more logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6363 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a995b95367
tried a fix for the httpd access bug (too many unclosed sessions)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6362 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e1fba41cad
better logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6361 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2275f885a8
possible fix for concurrency problem
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6360 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
a6a3090c3d
*) blacklist cleaner supports usage of regular expressions now
...
*) refacored BlacklistCleaner_p.java for better readability
*) moved check of validity of patterns to the Balcklist implementation since patterns might be valid in one implementation, but not in another
*) added method to check validity to Blacklist interface
*) fixed some minor issues like typos or wrong whitespaces
*) set subversion properties for a whole bunch of files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6359 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5a93807781
improved web cache speed:
...
- removed one computation out of a synchronization
- removed one not necessary has() call
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6358 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2e8b2867ff
double performance of store method because it avoids one 'has'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6357 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
afda5b1adc
new join method for indexes (not yet used)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6356 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
65b66c2c18
better handling of array files of length 0
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6355 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1957b5797a
fix for seed generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6354 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
432154f725
new strategy for concurrent database index key retrieval
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6353 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a11cd9f80f
- removed reverse name lookup for http access logging (grr..)
...
- removed a synchronization in seed info string generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6351 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2e6bdce086
- added more logging to balancer
...
- changed balancer logic slightly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6350 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1171a72006
fix for deadlock as seen in http://forum.yacy-websuche.de/viewtopic.php?p=17521#p17521
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6343 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
031e6eefbd
some updates to dublin core, metadata browsing, file indexing and parser stability
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6342 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
62a7341c4d
Fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2204
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6341 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
f65bfaa9af
*) Removed base tag from errror page. This has been added by myself a long time ago as a workaround for some weird behavior of my router, but as it turns out, it does more bad than good in general: If HTTPS is used for communication with YaCy, entering a wrong passwort led to an errror page with a form which would send username and password unencrypted with the user possibly being unaware of this.
...
*) changed some comments, added some annotations, added SVN properties here and there
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6340 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e4797ebcde
fix for http://forum.yacy-websuche.de/viewtopic.php?p=17509#p17509
...
corrupted files are ignored
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6339 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
efa7fb34f0
better oom-awareness of miss-cache in cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6338 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3e9dcfc204
fix for http://forum.yacy-websuche.de/viewtopic.php?p=17504#p17504
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6337 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c3a4aee255
some redesign with a possible fix for the ReferenceContainerCache.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6336 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
aca8a78eb8
fix for shutdown of DocumentIndex objects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6333 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
23ab6fbca4
- navigation appear at correct position when opengeodb-results are also presented after a search
...
- show an about box if about.headline and about.body is set
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6332 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4db34eea73
fix for OOM problem in kelondro Cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6331 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8ea1d7ab59
fix for wrong assert condition in search abstract generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6330 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fbd77bd77c
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6328 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
54c7cbf1d9
- fast result for local search in case that less than 10 hits exists
...
- small change in display of RAM in profiling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6326 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
28d4b921b6
different approach for file search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6325 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f99f86c5c5
added concurrency to file indexing class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6324 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
902d16cf6c
fixes to parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6323 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4a1c852435
fix in usage of RAM copy for Table objects and some cosmetics in asserts.
...
This bug affected Tables in case that a removeOne() was called and a RAM copy of the table was active. It may happen for peer owners with a lot of RAM assigned to YaCy. The bug appeared especially during crawling when the balancer tried to get new entries from the crawl queue.
This bug may help to solve report at
http://forum.yacy-websuche.de/viewtopic.php?p=17417#p17417
and will be tracked there
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6322 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
dce450e2e0
possible fix for "hung" doc-documents
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6320 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e627f75415
one more fix to badwords and stopwords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6316 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
721b88efbd
- fixed a problem loading blacklists with new yacycore.jar
...
- fixed badwords and stopwords initialization
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6315 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
80d5005044
fixed seed upload methods - replaced reflection with direct instantiation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6314 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
68465c37af
added a convenience class to add files into a YaCy index
...
to make this possible, the yacyURL must be able to process file:// urls, which has also been implemented
testing of the new class resulted in some bugfixes in other classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6313 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2e41e10ffd
- updates to yacyVersion parser (remove old targets)
...
- added javadoc target to built script (does not work yet without errors)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6312 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
27d00285aa
- added a new file reader cache that may serve as full-file-copy of blob database files. This is not yet used
...
- removed class FileWriter and replaced all usage of that class with CachedFileWriter
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6309 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fd6b9cb7dc
refactoring of IO access classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6308 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d64569aa39
reuturn only recommendations of words that have a greater count than the original word
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6307 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
604c37927f
used comparator for did-you-mean that uses index sizes for comparisment, but:
...
- limit comparisment to only the first 10 elements that had been sorted before without IO
- added a size cache to index computation because the size is computed at least twice in set comparator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6306 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a58d9cae7d
- show location name in geolocalization search result
...
- added link from location icon to openstreetmap browser with coordinates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6305 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
573d03c7d7
added configuration to enable ram table copy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6304 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3be54e1891
fix to rule when to use a ram table copy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6302 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
700218846c
disabled or removed sleep calls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6301 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
342c5d0fd4
fixed city name detection: finds now also substrings of city names
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6300 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
18aa0609ca
fix for caching of word hash computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6299 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a10a6cce45
patch for http://forum.yacy-websuche.de/viewtopic.php?p=17289#p17289
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6298 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
53bbdfd19a
*) setting SVN keywords
...
*) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6297 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
25f6145934
*) preventing null pointer exception in case empty search word or only one character is enterd or all search words are removed by filters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6296 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
248f3fd9b5
*) cleaned up code for better readability
...
*) added a few copyright notices
*) removed redundancy in constructors of ListToken
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6295 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
eaddf2d464
- corrected layout of map preview
...
- added caption to maps containing latitude and longitude information
- prevented that maps occur on second search page
- added location names to did-you-mean
- some refactoring of did-you-mean
- added equal and compareTo test to Coordinates class to make that work in set
- fixed utf-8 support for library files
- fixed a bug in images search icon view caption
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6294 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
4b83875abd
Small fixes for the heapCacheIterator in ReferenceContainerCache:
...
- Start the iteration at startWordHash
- When used with rotation, let the iteration stop when the cache is empty
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6293 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fd668f531b
fixed map layout
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6292 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2740d9dd79
added integration of osm maps for search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6291 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
af3a696fc4
added a fast-fail concept in search processes. The search now has better control if all the remote searches may bring any result. If all processes are finished, then all search tasks fail fast.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6290 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ce972ff4ef
update to default ranking profile which has now some settings to deny some phpbb3 pages which are redundant in the index when crawling phpbb3.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6288 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
44579fa06d
- fixed a problem loading images through yacy's document loader,
...
this denied non-parseable documents which excluded all images
- fixed url of osm tile server
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6287 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
67eddaec4b
changed way to integrate dictionary files:
...
the must be downloaded manually by the user and placed in DATA/DICTIONARIES/source
for each externally imported dictionary file there will be a translator that converts the input file once
into a YaCy-internat data format.
Files that will be provided together with yacy releases may still be placed in <root>/dictionaries
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6286 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d656a94f55
fix for bad paths in dictionary processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6285 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3b9aaf9e9f
- inserted new library tests inside DidYouMean
...
- some redesign of DidYouMean that was necessary to follow
a special rule how a library should be used:
- the library provides words that start or end with a test
word which may be possibly also an empty set of words
- all words that the DidYouMean produced with the four
production rules are used to generate a set of
library-completed words
- if this process results in any words from the library,
only library-genrated words are taken
- if the is no library-generated word at all, take the
artifial generated word
- all words that result from these rules are tested against
the index
- the result is ordered using a lightweight comparator that
prefers short words
- a not-so-much-io test against the index is beeing prepared
next
- insered the library initialization into the switchboard
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6284 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8c35ffe34c
fixes to the dymlib
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6283 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bfa273bcc1
added a library provider which holds libraries in static objects,
...
which can be used by any other classes to support their functions.
libraries are designed in such a way that users can create and
insert their own library files, but can also be imported from
other sources. As an example the "Korpusbasierte Wortgrundformliste
DeReWo des Institut für Deutsche Sprache" from
http://www.ids-mannheim.de has been integrated. This dictionary
is licensed to be used for all non-profit purposes. In case that
YaCy is used for commercial uses, this library must be removed.
The new library provilder reads the original source and translates
it into a simple word list to be used for the did-you-mean library
provider. More libraries may be provided in the future using
a download-servlet which puts files from the internet into the
<application-root>/dictionaries/ path.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6282 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1762a7bcd6
- moved DidYouMean to the data package
...
- added a DidYouMeanLibrary class that shall support the did you mean function with additional word lists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6281 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bf8ed00e9e
removed debugging code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6280 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ead48c4b25
fix for preparation of search result pages with offset > 10:
...
- less pages are fetched in advance
- just-in-time fetch of next required pages
- fix for missing hand-over of offset to fetch threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6279 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
39a311d608
better care to do not loose the merge/dump thread
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6278 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
10d3e856b5
better concurrency, less blocking & performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6277 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1a9cfd8718
some performance hacks (CPU only, not IO)
...
this will cause better computation speed for single- and multi-core;
there are enhancements that will speed up old and slow machines as well
as multi-core CPUs. Indexing of surrogates has been speed up
from 4000 PPM to over 20000 PPM on a simple dual core office computer.
Since the enhancements are mostly in core routines, the hack should also
speed up search performance.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6276 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
92407009b2
cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6275 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0ba1beaf56
separated rwi constraint evaluation from rwi ranking and added concurrency
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6274 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ce7924d712
better concurrency for rwi entry parsing during search processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6273 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b0637600d5
enhanced url constraint computation: better position of constraint check during retrieval process
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6272 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
61748285c3
more refactoring of search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6270 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
323a8e733d
removed unused classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6269 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
72e5407115
refactoring of snippet cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6268 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0e471ba33b
- fixed a bug in fast digest computation
...
- added a open-on-demand hack to heap files: when a heap file is
opened the first time, it is first scanned to get a key index
and then it is closed again. This will free up file pointers
in cases where a really large number of blob files are opened
upon initialization of ArrayStack objects. This should solve
also a problem reported in
http://forum.yacy-websuche.de/viewtopic.php?p=17191#p17191
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6267 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
93b2622503
*) repaired and added IM online status indicators
...
*) added some missing SVN properties
*) removed unnecessary comment, added missing copyright notice
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6266 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e7736d9c8d
more refactoring: made all variables in SearchEvent private
...
to prepare splitting of the class into two parts: local and remote search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6265 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4b92d0b9b7
patch for possible problems with normalization of '/' in urls. This applies in rare cases when '/' appear in post-properties
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6264 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d8ca6e6bf1
more refactoring for search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6263 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fe4a4e3f6b
added missing class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6261 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
72ac5bd80f
refactoring of search process.
...
this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6260 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
c4d0e22a77
Further speed upof concurrent DHT-receive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6259 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
2fbc0696bf
Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2334
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6258 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
f1ori
d515bc11e2
added ooxmlparser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6256 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d9744b1b5d
replaced old caching strategy control class with lightweight simplearc
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6254 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8e56c2ace6
fix for fixes from this afternoon
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6253 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
cf739edc2e
fix for possible deadlock, see
...
http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6252 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6354b5e447
removed possible deadlock, see
...
http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6251 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5cc17ccf8a
a better caching with less overhead and more appropriate
...
synchronisation use in more than 10 different data objects
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6250 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
92edd24e70
fixed problem with switching of networks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6247 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0575f12838
fix for deadlock
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6246 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fbfdaf063d
- patch to omit IndexOutOfBoundsException when a b64-encoded key appears not to be well-formed. In that case the key is still accepted but rated higher than other regular keys to create a virtual ordering between well-formed and ill-formed keys
...
- check routine at the beginning of the import of table keys that check that all imported keys are well-formed. All records that have a ill-formed key are deleted. This is a hack and is not tested since I don't have bad data here to test with. If the effect is seen in the wild, please report in the forum.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6245 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c0e17de2fb
- fixes for some problems with the new crawling/caching strategies
...
- speed enhancements for the cache-only cache policy by using special no-delay rules in the balancer
- fixed some deadlock- and 100% CPU problems in the balancer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6243 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
634a01a9a4
replaced wget-requests with caching requests
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6242 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c6c97f23ad
- added cache usage properties to crawl start
...
- added special rule to balancer to omit forced delays if cache is used exclusively
- extended the htCache size by default to 32GB
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6241 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c4ae2cd03f
fixed bug that caused deletion of crawl profiles at every application startup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6240 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
161d2fd2ef
redesign of access to the HTCache (now http.client.Cache):
...
- better control to the cache by using combined request-header and content access methods
- refactoring of many classes to comply to this new access method
- make shure that the cache is always written if something was loaded
- some redesign of the process how http response results are feeded into the new indexing queue
- introduction of a cache read policy:
* never use the cache
* use the cache if entry exist
* use the cache if the proxy freshness rule confirmes
* use only the cache and go never online
- added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step.
- set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule'
- enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6239 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
ba2e6de538
fix empty version string again
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6236 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
51534df0cb
fix for possible synchronization problem
...
see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2292&hilit=&p=16787#p16787
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6234 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4da9042e8a
code simplification
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6233 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1d8d51075c
refactoring:
...
- removed the plasma package. The name of that package came from a very early pre-version of YaCy, even before YaCy was named AnomicHTTPProxy. The Proxy project introduced search for cache contents using class files that had been developed during the plasma project. Information from 2002 about plasma can be found here:
http://web.archive.org/web/20020802110827/http://anomic.de/AnomicPlasma/index.html
We stil have one class that comes mostly unchanged from the plasma project, the Condenser class. But this is now part of the document package and all other classes in the plasma package can be assigned to other packages.
- cleaned up the http package: better structure of that class and clean isolation of server and client classes. The old HTCache becomes part of the client sub-package of http.
- because the plasmaSwitchboard is now part of the search package all servlets had to be touched to declare a different package source.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6232 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
67da20647f
* add new odf parser based on sax-xml-parser
...
* remove odf_utils-jar
* test metadata in ParserTest
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6231 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
6d0e6d591b
* ops, fix compiler error :(
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6227 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
3e5beb1654
* fix for empty version in seedlist
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6226 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5bb8074150
removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency.
...
- The indexing queue was a historic data structure that was introduced at the very beginning at the project as a part of the switchboard organisation object structure. Without the indexing queue the switchboard queue becomes also superfluous. It has been removed as well.
- Removing the switchboard queue requires that all servlets are called without a opaque generic ('<?>'). That caused that all serlets had to be modified.
- Many servlets displayed the indexing queue or the size of that queue. In the past months the indexer was so fast that mostly the indexing queue appeared empty, so there was no use of it any more. Because the queue has been removed, the display in the servlets had also to be removed.
- The surrogate work task had been a part of the indexing queue control structure. Without the indexing queue the surrogates needed its own task management. That has been integrated here.
- Because the indexing queue had a special queue entry object and properties attached to this object, the propterties had to be moved to the queue entry object which is part of the new indexing queue withing the blocking queue, the Response Object. That object has now also the new properties of the removed indexing queue entry object.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6225 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
597393db3b
changed default visibility of classes/objects in upnp lib
...
(eclipse tells me that this would improve performance,
however, this removes compiler warnings)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6224 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
eea4c17ef2
removed rpm parser
...
- no-one used that thing
- loading huge rpm files bay be causes for crashes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6223 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b332dfad67
- inserted request object into response object which carries this now instead generating new objects
...
- fixed a problem with the crawler introduced in SVN 6216
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6222 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ca72ed7526
-removed superfluous crawl cache
...
-refactoring of crawler classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6221 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8103ccec4c
removed compiler warnings in imported classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6220 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
52e371b8f7
suppress warnings for upnplib code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6219 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
477807e0e6
* updated jxpath to latest v1.3
...
* added upnplib as source
without packages:
jmx
remote
samples
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6218 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
13c63f4082
a set of small fixes to crawling behaviour
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6216 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a564df3984
update to mime types in parsers and httpd.mime
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6215 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
43c8defd79
enhanced parser with more extension + mime attributes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6214 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
aee35bff6f
replaced StringBuffer with StringBuilder in tar lib
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6213 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
49bbb9bd45
replaced tar library with integrated apache ant tar lib
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6212 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f987fc6b4a
added tar classes from apache ant tools
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6211 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b2263bc720
enhanced document type recognition
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6209 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
aa38eb5a20
* maxfilesize -1 for infinite filesize
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6208 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
9cfe89c8fc
* process content-length as soon as it is received
...
* corrected indentation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6206 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
50cf80056f
removed jmimemagic library
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6203 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3f113f38a8
removed unused imports
...
removed unused libs from eclipse class path
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6201 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
9f083bb6b2
check filetype before loading (no more mp4 loading)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6200 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
076ae02c44
* added pl and py to extensions excepted by htmlParser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6198 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
d5e51cfd09
* workaround for non-working build property replacements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6197 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
f814e0fa81
enable warnings and fix most of it
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6196 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
8931c8d6b4
improvments to debianpackage:
...
* autoupdate completely disabled, display hint
* restart-button in interface works!
* moved all build-Variables to yacyBuildProperties
* fixed some warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6195 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
fc1dc38b55
*) added spaces to make sure that no words are concatinated by accident
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6194 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
f242e7d7bc
*) using Apache POI library to parse Word documents now
...
*) removed tm-extractors library (can be found at http://www.textmining.org/ if necessary again)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6193 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
caedd72400
- enhanced logging and exception details for parsers
...
- removed inconsistencies in mime type declaration (one mime type should only appear once in all parsers)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6192 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4b74ad0a46
fixed setting of parser configuration servlets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6191 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
57a88d435b
redesign of parser mime type detection and parser steering
...
There is now a mime-blacklist instead of a mime-whitelist
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6190 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
e15d27bc63
avoiding double/wrong parser errors
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6189 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
21b8704fb4
refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6188 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8ca1f5d400
- some work to integrate the html parser the same way as the other parsers are integrated (not finished)
...
- added migration of code of settings pages (hmm.. does not work correctly yet, sorry)
- more refactoring
- removed more unused code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6187 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
1ee109761f
*) added changes which were lost
...
*) additional annotations
*) additional svn properties
*) _no_ functional changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6186 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
499723891d
removed all non-http daemons; they had not been used and may be a potential security risk.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6185 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0e8647d62f
refactoring of search classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6184 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dafffd0153
refactoring of parsers and document processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6182 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
8041e91f56
*) Ooops!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6181 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
69551ff3d9
*) added several MIME types (derived from http://filext.com/ ), some of them might be rather uncommon
...
*) added an annotation forgotten in last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6180 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
11dfb2d54f
minor changes:
...
*) added annotations
*) set svn properties and added keywords to comments of parser classes
*) made a variable final to prevent (theoretical case of) change of object instance in synchronized block
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6179 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
77d2a3782c
removed strange debugging strings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6177 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
4320f69574
universal handling for crashed parsers
...
reverting r6090/1
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6176 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
024744245c
small refactoring to prepare for new queues
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6173 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
16efcd0366
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2252&hilit=&p=16389#p16389
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6172 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
0f3246e90a
* fix debian package
...
* add Class containing buildvariables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6171 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
8544cfd5a6
* remove seperate build-files for parsers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6170 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
24cb6d68bc
- renamed Stack to RecordStack to avoid name confusion with new classes
...
- added new Stack class that implements a stack on BLOB files
- added new Stacks class that can be used for a set of Stacks (a 'Stack Database')
- added methods to other classes to support the new stacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6169 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
995da28c73
all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path
...
There is no migration. All crawls must be restarted.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6167 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
aac89bf8ca
trying to avoid "exceeding limit" message of server
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6166 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
48d78166ed
* fix double copy of libraries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6164 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
7f868ca3c2
resource observer: support for yacyroot\DATA on an NTFS hardlink (Windows)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6162 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
409538e17a
code cleanup and code simplifcation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6161 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
160031758d
fix for problem with initializer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6160 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
302a02cec8
moved all libraries from libx to lib
...
removed libx directory
all libraries are now in lib, instead the test libraries in libt which are not part of releases
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6157 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1f1399e5c5
extending visibility of objects and methods to avoid synthetic accessor methods and increase performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6156 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
154bbc3364
code cleanup: call of static methods directly to the class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6155 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
222850414e
simplification of the code: removed unused classes, methods and variables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6154 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
93dfb51fd4
problems with code style
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6153 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
adf01c676e
reduce lookup time when merging a large number of BLOBs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6152 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9a674d8047
- After the removal of the Tree class some code simplifications are possible. This affects mostly the Records class, which can be refactored and the result of the refactoring results in a reduced number of classes.
...
- The EcoTable was renamed to Table.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6151 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c5122d6836
completed migration of BLOBTree to BLOBHeaps:
...
- removed migration code
- removed BLOBTree
after the removal of the BLOBTree, a lot of dead code appeared:
- removed dead code that was needed for BLOBTree
Some more classes may have not much use any more after the removal of BLOBTree, but still have some component that are needed elsewhere. Additional Refactoring steps are needed to clean up dependencies and then more code may appear that is unused and can be removed as well.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6150 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d1083a6913
maybe we have less problems with open connections to the server if we don't do BF forced sleeps (just a test)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6149 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ebe6c823ac
*) changed svn properties agains (hopefully doing it right this time)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6147 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
a80ac3a415
*) fixed wrong parser descriptions
...
*) changed svn properties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6146 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
457b6c0d6d
*) updated Apache POI library to be able to parse Visio files
...
*) updated PPT and XLS parsers to use new Apache POI library
*) added new Visio (VSD) parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6145 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a10c8022d1
DidYouMean:
...
- limit the number of consumer threads to available CPUs
- added some javadoc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6144 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
7eb3bff5b3
* workaround for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2220&hilit=#p16128
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6143 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
99fa265e1d
fix for search bug caused by tenant patch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6125 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
79875782af
be a bit more lazy when removing domain navigation entries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6120 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
57af311627
fix for wrong urls in navigator when a tenant is used
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6119 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
76b96337e2
just some chatty code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6118 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
91785d895c
*) minor changes in comments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6109 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bdda140c02
fix for json output (no doubleqotes any more, doublequote quoting did not work)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6105 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
2f84736120
ignore signature files that cannot be downloaded because of failed encoding
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6103 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
041d9c253e
some refactoring and more error-awareness in LogalizeHandler
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6102 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
6b307d6d59
more tolerance for corrupted index entries in exported row sets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6099 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
33aafa9b4b
better logging when writing merged dumps
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6098 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
db70badcf0
possibility to set remote host on upnp device
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6097 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4d29e90708
uaeh
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6096 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3c3e6499ae
added more logging for merge operation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6095 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15180fc95e
- patch for future computation in SplitTable
...
- added same concurrent process for has() from SPlitTable in ArrayStack
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6093 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9a5ec20b3c
avoid merge during startup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6092 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
bf6b92343c
try to avoid stuck pdf parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6091 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
c695c7f512
try to remove hung swf parser from queue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6090 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fc69a76197
update to web structure picture:
...
- allow bigger size
- better instructions for api usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6089 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ae015e8e98
refactoring of blob package classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6088 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8b8877c233
moved image collector
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6087 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
be1c7ddc64
refactoring of search classes -- moved Ranking Profile to search package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6086 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fd31a3616a
- more logging in server process
...
- fix for bas ascii in comment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6084 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5a7fd6b4c8
just some comment lines
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6081 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
31f60a3b3e
when doing searches, also apply a online caution to DHT transmission and stop transmissions while heavy load caused by searching. This omits the many requests to the URL database that are needed for DHT transfer and it avoids collisions with URL retrieval needed for search results.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6080 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
17dc6d4be5
small fix for new Logger
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6079 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ce1adf9955
serialized all logging using concurrency:
...
high-performance search query situations as seen in yacy-metager integration showed deadlock situation caused by synchronization effects inside of sun.java code. It appears that the logger is not completely safe against deadlock situations in concurrent calls of the logger. One possible solution would be a outside-synchronization with 'synchronized' statements, but that would further apply blocking on all high-efficient methods that call the logger. It is much better to do a non-blocking hand-over of logging lines and work off log entries with a concurrent log writer. This also disconnects IO operations from logging, which can also cause IO operation when a log is written to a file. This commit not only moves the logger from kelondro to yacy.logging, it also inserts the concurrency methods to realize non-blocking logging.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6078 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
aec3e7995a
autoconfig.pac can be used to browse .yacy-domains only
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6077 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bc6dd8194b
refactoring: moved search query class to new search package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6075 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a4805defdd
added stub for new search process
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6074 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b8e738a7be
a collection of
...
- small bug fixes
- better/more comments
- more asserts
- fixed synchronization
- test case enhancements
- code cleanup
- performance hacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6073 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
39779e4796
DidYouMean: as I moved to only 8 consumer and 4 producer threads, I removed poison pills as it does not make sense anymore - threads are interrupted directly. Having a consumer thread per test case just didn't make sense either (see svn 6070) due to the massive overhead.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6072 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
c3c4dd0933
DidYouMean - changed to much simpler LinkedBlockingQueue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6071 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
01ac1b5d7e
- blocking queue implementation of DidYouMean
...
- timeout ist set to 500ms
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6070 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b8bb1bb364
join with a timeout does not cause that the corresponding thread is stopped after the time-out. It does only cause that the waiting is stopped. Here we need additionally a signal to the thread to stop after we finished waiting.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6069 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b69f22e9ca
mistake in last commit: computation of loops in ReversingTwoConsecutiveLetters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6068 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3130334932
- start first with threads that run more loops
...
- join first with threads that run less loops
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6067 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
6cde7ebf16
DidYouMean
...
- without I/O intensive sorting by count
- but with multiple threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6066 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f348190566
tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6065 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
945777aa80
replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6064 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7c4d1d471c
hand-over of more specific object
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6062 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
09acfa66d1
- improved "did you mean"
...
- added &meanCount= to query string
- &meanCount=0 ==> no suggestion, no performance loss
- sorting suggestions by sb.indexSegment.termIndex().count()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6059 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
da6ce37f7b
- fixed encoding problem
...
- added limit to 10 suggestions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6058 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
54a48b4184
- added "did you mean" to search page
...
- currently works for single word queries only!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6057 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
550312ac85
added new command script to do a auto-Update from command line. this will make it easy to do mass-auto-updates in private yacy clusters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6052 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0fc1168554
- reduced time-out for socket-connection communication from 20 seconds to 5 seconds. This is a test to find out if the time-out was a cause for problems in metager environments
...
- turned a fine log entry in case of rejected connections on the server socket into a warning. (look for 'exceeding limit')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6051 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
28b86385cd
patch for bad behaving swf parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6050 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d58b395993
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15693#p15693
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6049 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
733385cdd7
enahnced database access times by removal of unnecessary synchronization.
...
added also more hacks that resulted from high-volum query testing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6047 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
398e210fef
removed synchronization in logging that causes deadlocks in high-performance environments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6044 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
db3a06dd81
removed cookie handling in httpc:
...
- no need to do cookie handling in proxy, this was switched off so far
- no need for cookies in crawler, this was switched on (by mistake)
This fix was needed for a case where a web server flooded the crawler with cookies and caused a complete blocking of the httpc.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6043 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c54ae4a63
some small changes in HandleMap Testing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6042 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
2c5554c912
small enhancements in search result computation speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6039 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e0b3984805
added navigation keys for site and author facets to remote search interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6038 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27fa6a66ad
- completed the author navigation
...
- removed some unused variables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6037 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a9a8b8d161
- added display of author navigation (usage of that navigator not yet implemented
...
- added a synchronization in pdf parser which should help to avoid deadlocks that occur when displaying several search results pointing to pdf sources
- fixed smaller bugs in navigation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6036 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c879783008
added steering of navigator computation:
...
- by default the navigator computation if off for servlet yacysearch.html, but:
- the servlet is called by default with a option to switch navigator results on
this will prevent that metasearch users will get slow results that are caused by unnecessary computations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6035 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c079b18ee7
- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.
...
- added a analysis method that counts bytes that could be saved in case the new HandleMap can be applied in the most efficient way. Look for the log messages beginning with "HeapReader saturation": in most cases we could save about 30% RAM!
- removed the old FlexTable database structure. It was not used any more.
- removed memory statistics in PerformanceMemory about flex tables and node caches (node caches were used by Tree Tables, which are also not used any more)
- add a stub for a steering of navigation functions. That should help to switch off naviagtion computation in cases where it is not demanded by a client
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6034 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bead0006da
replaced tmp file extensions by prt
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6033 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3189f9cd39
fixed problem with DCEntry initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6032 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a704d82280
patch for problem with digest
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6031 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3029ef6eb3
fixed a bug that was recently inserted which caused that no idx and gap files were written.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6030 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b6e274f211
omit most of forced crawl delays by using a separat delay table which flushes delayed URLs at the correct time
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6029 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d50be59088
- added a automatic re-construction of the domain stack after 10 minutes. this includes then urls to the domain stack that were left over in case of stack size limitations when the domain stack was created the last time
...
- changed the busy sleep time for the crawl thread to 30 millisecons. This is sufficient to crawl with 2000 PPM.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6028 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fdba0fa51
- fixed a not working selection rule in balancer
...
- more security about crawl-delay, be more fail-save
- better logging in case of long forced crawl-delays
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6027 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f5602404d5
another speed boost for the balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6026 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
95e8cbd1c3
new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6025 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c062385552
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15555#p15555
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6024 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
42ae40b9f6
some bugfixes to database close() methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6023 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a0c53abbe1
- wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521
...
- show only x+1 pages in page navigator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6022 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9bfd22f65d
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15523#p15523
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6020 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c77db670f
re-designed response format for navigation:
...
- changed json and rss response templates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6019 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15fad767c0
some refactoring of topic generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6018 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
cc49aedf12
- fixed problem with remote search NPE
...
- more abstraction for search requests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6015 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
9e18abc2ac
* fix charset detection, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2137
...
* why has this been uncommented???
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6014 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c38c852090
modified access method to get index entries out of a array of BLOBs:
...
iterate them, then merge; not collect them and merge then.
This should use less memory and may behave better in an environment with many queries.
To ensure that too many queries will not cause total blocking,
a time-out of one second was also added. After the time-out
the index data that was collected so far is returned.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6013 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ab06a6edd2
renamed topwords to topics and enhanced computation methods of topics
...
topics will now only be computed using the document title, not the document url,
because the host navigator is now responsible for statistical effects of urls.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6011 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a5d481eab1
enhanced navigation
...
- fixed too early computation of navigation
- moved navigation rendering to yacysearchtrailer
- added more asserts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6006 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7639ec2f38
- fixed letter case bug for dc record creation
...
- dc parser is now lazy against letter cases
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5998 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4522c13ee7
added option for a table prefix when importing phpbb3
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5996 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c69d9b8b6
more refactoring of the index classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5995 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d5f2ff544
- added new servlets to support search portal administrators for the integration of yacy search fields in their web pages
...
- moved some servlets from here to there..
- changed menu structure
- removed yacyui-portaltest.html which contained an example for the live search which is now integrated on all pages in yacy. The code snippet example from that page is integrated into the ConfigLiveSearch.html servlet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5994 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4d4315687f
fix for problem with concurrency in host navigator, bug reported by wsb
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5993 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
88426912ad
more refactoring to make the segment object easier to use and to be prepared to integrate author navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5992 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
d813fd26ed
reset sent/received counters on index delete
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5991 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
99bf0b8e41
refactoring of plasmaWordIndex:
...
divided that class into three parts:
- the peers object is now hosted by the plasmaSwitchboard
- the crawler elements are now in a new class, crawler.CrawlerSwitchboard
- the index elements are core of the new segment data structure, which is a bundle of different indexes for the full text and (in the future) navigation indexes and the metadata store. The new class is now in kelondro.text.Segment
The refactoring is inspired by the roadmap to create index segments, the option to host different indexes on one peer.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5990 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
876746602d
catch problems of file hash computation, see also:
...
http://forum.yacy-websuche.de/viewtopic.php?p=15245#p15245
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5989 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fec6f9054f
some refactoring of search methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5988 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d4b826ca5
migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically.
...
This removes the last very IO-intensive data structures which were still used for Wiki, Blog and Bookmarks. Old database files will still remain in the DATA subdirectory but can be deleted manually if no major bugs appear during migration. There is no need for any user action, all migration is done automatically.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5986 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4b4bddca00
added new submenu to crawler menu: import of phpbb3 forum postings from mysql
...
- yacy can import phpbb3 posts without crawling
- all data is written as surrogate
- indexed surrogate files can be re-used
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5985 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d8284046b0
enhanced speed of site navigation computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5980 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c72a5cf326
added stub for PHPBB3 extraction code using direct access to mySQL
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5979 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e735d3a69f
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15175#p15175
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5978 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
63a0255166
- refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index
...
- refactoring: migrated data objects for the new connector classes
- added a DAO interface class to specify an abstract interface for database retrieval connector methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f246928c20
first attempt to add 'real' Navigation to yacy search results: host navigation
...
- after a search is started, it is analysed how many hits are in each site
- this can be done really efficient, because the navigation information is hidden in the url hash and can be computed very fast
- the search result shows a column on the right with the hosts and the hits per host
- after a click on a host the search is modified using the efficient site: - operator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5976 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
54b9e99c01
- more information about peer tags
...
- peer tag is by default '*'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5975 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
26a46b5521
increased default maximum file size for database files to 2GB
...
Other file sizes can now be configured with the attributes
filesize.max.win and filesize.max.other
the default maximum file size for non-windows OS is now 32GB
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5974 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
addecdb18c
simplified code, removed one unused method in all implementing classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5972 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
47fce9020c
small change (Orbiter's wish)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5971 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
e07b14e5d7
finally a working fix for 5960
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5970 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
3ebb904d2c
fix for 5960, http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2119
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5969 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
734680dc70
initialize the ResourceObsever in own thread
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5968 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e005cfea37
fix for bug in -incell option of URLAnalysis
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5967 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a7e392f31b
The collection index will not be supported any more.
...
Existing indexes based on the old index collections must be migrated with YaCy 0.8
- removed index collection classes and all migration tools
- added a 'incell' reference collection feature in URL analysis
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5966 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a2f48863fc
- added prototype for navigation index
...
- refactoring of word index prototype
(no functional changes so far)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5965 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
47fd226bdb
proper parsing of sentences
...
does not affect tokens/words
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5964 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27eb8d62cb
- new development cycle
...
- removed temporary configuration with safe setting for indexer threads (=1) and replaced it with best value computed during performance tests (1/2 of number of processors)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5963 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b7457d3807
patch for http://forum.yacy-websuche.de/viewtopic.php?p=14720#p14720
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5960 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bffbe43e09
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14522#p14522
...
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5959 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f133d6065c
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5958 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
82af994041
added missing loglevel
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5956 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ad9762746d
no exception in case of uniq() time-out, see also
...
http://forum.yacy-websuche.de/viewtopic.php?p=13177#p13177
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5955 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1efe686e3f
fix for http://forum.yacy-websuche.de/viewtopic.php?p=13960#p13960
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5954 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
13fb84ab81
you can define your default number of search results displayed by search.items
...
this applies only to requests through the classic-style page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5953 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f2e4d156e8
removed debug messages
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5950 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
709bfc2cd4
added a memory check in http post protocol
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5949 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c01d6f43e1
- fixed problem with thread dump if no arguments are given
...
- rejecting peers that are older than 6 hours (not-seen during 6 hours)
- 0.78, targeting 0.8 at the end of the week
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5948 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a49edd9415
fix for bug in search with site: constraint
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5947 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c1e5fad9a7
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14767#p14767
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5944 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8ee3a94e82
fix for non-caching of sitehash, see http://forum.yacy-websuche.de/viewtopic.php?p=14440#p14440
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5942 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
21930d05ed
fix for [B@...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5941 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b6ba387e01
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14751#p14751
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5940 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4338dcf936
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2093&hilit=
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5937 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
bad7ce9286
experimental option trayIcon.force for unsupported platforms. java 1.6 needed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5936 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ea27853c59
*) some refactoring
...
*) added one assertion
*) no functional changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5935 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
d164b42604
*) cosmetics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5934 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
17150b2950
fixed bug in snippet computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5932 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
89aeb318d3
enhanced the wikimedia dump import process
...
enhanced the wiki parser and condenser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5931 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fb77116c6
added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see
...
http://localhost:8080/IndexImportWikimedia_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5930 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago