orbiter
2c5554c912
small enhancements in search result computation speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6039 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e0b3984805
added navigation keys for site and author facets to remote search interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6038 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27fa6a66ad
- completed the author navigation
...
- removed some unused variables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6037 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a9a8b8d161
- added display of author navigation (usage of that navigator not yet implemented
...
- added a synchronization in pdf parser which should help to avoid deadlocks that occur when displaying several search results pointing to pdf sources
- fixed smaller bugs in navigation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6036 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c879783008
added steering of navigator computation:
...
- by default the navigator computation if off for servlet yacysearch.html, but:
- the servlet is called by default with a option to switch navigator results on
this will prevent that metasearch users will get slow results that are caused by unnecessary computations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6035 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c079b18ee7
- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.
...
- added a analysis method that counts bytes that could be saved in case the new HandleMap can be applied in the most efficient way. Look for the log messages beginning with "HeapReader saturation": in most cases we could save about 30% RAM!
- removed the old FlexTable database structure. It was not used any more.
- removed memory statistics in PerformanceMemory about flex tables and node caches (node caches were used by Tree Tables, which are also not used any more)
- add a stub for a steering of navigation functions. That should help to switch off naviagtion computation in cases where it is not demanded by a client
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6034 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bead0006da
replaced tmp file extensions by prt
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6033 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3189f9cd39
fixed problem with DCEntry initialization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6032 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a704d82280
patch for problem with digest
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6031 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3029ef6eb3
fixed a bug that was recently inserted which caused that no idx and gap files were written.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6030 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b6e274f211
omit most of forced crawl delays by using a separat delay table which flushes delayed URLs at the correct time
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6029 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d50be59088
- added a automatic re-construction of the domain stack after 10 minutes. this includes then urls to the domain stack that were left over in case of stack size limitations when the domain stack was created the last time
...
- changed the busy sleep time for the crawl thread to 30 millisecons. This is sufficient to crawl with 2000 PPM.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6028 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fdba0fa51
- fixed a not working selection rule in balancer
...
- more security about crawl-delay, be more fail-save
- better logging in case of long forced crawl-delays
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6027 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f5602404d5
another speed boost for the balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6026 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
95e8cbd1c3
new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6025 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c062385552
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15555#p15555
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6024 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
42ae40b9f6
some bugfixes to database close() methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6023 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a0c53abbe1
- wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521
...
- show only x+1 pages in page navigator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6022 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9bfd22f65d
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15523#p15523
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6020 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c77db670f
re-designed response format for navigation:
...
- changed json and rss response templates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6019 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15fad767c0
some refactoring of topic generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6018 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
cc49aedf12
- fixed problem with remote search NPE
...
- more abstraction for search requests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6015 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
9e18abc2ac
* fix charset detection, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2137
...
* why has this been uncommented???
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6014 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c38c852090
modified access method to get index entries out of a array of BLOBs:
...
iterate them, then merge; not collect them and merge then.
This should use less memory and may behave better in an environment with many queries.
To ensure that too many queries will not cause total blocking,
a time-out of one second was also added. After the time-out
the index data that was collected so far is returned.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6013 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ab06a6edd2
renamed topwords to topics and enhanced computation methods of topics
...
topics will now only be computed using the document title, not the document url,
because the host navigator is now responsible for statistical effects of urls.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6011 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a5d481eab1
enhanced navigation
...
- fixed too early computation of navigation
- moved navigation rendering to yacysearchtrailer
- added more asserts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6006 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7639ec2f38
- fixed letter case bug for dc record creation
...
- dc parser is now lazy against letter cases
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5998 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4522c13ee7
added option for a table prefix when importing phpbb3
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5996 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c69d9b8b6
more refactoring of the index classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5995 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d5f2ff544
- added new servlets to support search portal administrators for the integration of yacy search fields in their web pages
...
- moved some servlets from here to there..
- changed menu structure
- removed yacyui-portaltest.html which contained an example for the live search which is now integrated on all pages in yacy. The code snippet example from that page is integrated into the ConfigLiveSearch.html servlet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5994 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4d4315687f
fix for problem with concurrency in host navigator, bug reported by wsb
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5993 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
88426912ad
more refactoring to make the segment object easier to use and to be prepared to integrate author navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5992 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
d813fd26ed
reset sent/received counters on index delete
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5991 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
99bf0b8e41
refactoring of plasmaWordIndex:
...
divided that class into three parts:
- the peers object is now hosted by the plasmaSwitchboard
- the crawler elements are now in a new class, crawler.CrawlerSwitchboard
- the index elements are core of the new segment data structure, which is a bundle of different indexes for the full text and (in the future) navigation indexes and the metadata store. The new class is now in kelondro.text.Segment
The refactoring is inspired by the roadmap to create index segments, the option to host different indexes on one peer.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5990 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
876746602d
catch problems of file hash computation, see also:
...
http://forum.yacy-websuche.de/viewtopic.php?p=15245#p15245
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5989 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fec6f9054f
some refactoring of search methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5988 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d4b826ca5
migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically.
...
This removes the last very IO-intensive data structures which were still used for Wiki, Blog and Bookmarks. Old database files will still remain in the DATA subdirectory but can be deleted manually if no major bugs appear during migration. There is no need for any user action, all migration is done automatically.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5986 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4b4bddca00
added new submenu to crawler menu: import of phpbb3 forum postings from mysql
...
- yacy can import phpbb3 posts without crawling
- all data is written as surrogate
- indexed surrogate files can be re-used
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5985 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d8284046b0
enhanced speed of site navigation computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5980 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c72a5cf326
added stub for PHPBB3 extraction code using direct access to mySQL
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5979 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e735d3a69f
fix for http://forum.yacy-websuche.de/viewtopic.php?p=15175#p15175
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5978 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
63a0255166
- refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index
...
- refactoring: migrated data objects for the new connector classes
- added a DAO interface class to specify an abstract interface for database retrieval connector methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f246928c20
first attempt to add 'real' Navigation to yacy search results: host navigation
...
- after a search is started, it is analysed how many hits are in each site
- this can be done really efficient, because the navigation information is hidden in the url hash and can be computed very fast
- the search result shows a column on the right with the hosts and the hits per host
- after a click on a host the search is modified using the efficient site: - operator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5976 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
54b9e99c01
- more information about peer tags
...
- peer tag is by default '*'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5975 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
26a46b5521
increased default maximum file size for database files to 2GB
...
Other file sizes can now be configured with the attributes
filesize.max.win and filesize.max.other
the default maximum file size for non-windows OS is now 32GB
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5974 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
addecdb18c
simplified code, removed one unused method in all implementing classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5972 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
47fce9020c
small change (Orbiter's wish)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5971 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
e07b14e5d7
finally a working fix for 5960
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5970 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
3ebb904d2c
fix for 5960, http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2119
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5969 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
734680dc70
initialize the ResourceObsever in own thread
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5968 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e005cfea37
fix for bug in -incell option of URLAnalysis
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5967 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a7e392f31b
The collection index will not be supported any more.
...
Existing indexes based on the old index collections must be migrated with YaCy 0.8
- removed index collection classes and all migration tools
- added a 'incell' reference collection feature in URL analysis
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5966 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a2f48863fc
- added prototype for navigation index
...
- refactoring of word index prototype
(no functional changes so far)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5965 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
47fd226bdb
proper parsing of sentences
...
does not affect tokens/words
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5964 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27eb8d62cb
- new development cycle
...
- removed temporary configuration with safe setting for indexer threads (=1) and replaced it with best value computed during performance tests (1/2 of number of processors)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5963 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b7457d3807
patch for http://forum.yacy-websuche.de/viewtopic.php?p=14720#p14720
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5960 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bffbe43e09
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14522#p14522
...
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5959 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f133d6065c
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5958 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
82af994041
added missing loglevel
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5956 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ad9762746d
no exception in case of uniq() time-out, see also
...
http://forum.yacy-websuche.de/viewtopic.php?p=13177#p13177
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5955 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1efe686e3f
fix for http://forum.yacy-websuche.de/viewtopic.php?p=13960#p13960
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5954 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
13fb84ab81
you can define your default number of search results displayed by search.items
...
this applies only to requests through the classic-style page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5953 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f2e4d156e8
removed debug messages
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5950 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
709bfc2cd4
added a memory check in http post protocol
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5949 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c01d6f43e1
- fixed problem with thread dump if no arguments are given
...
- rejecting peers that are older than 6 hours (not-seen during 6 hours)
- 0.78, targeting 0.8 at the end of the week
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5948 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a49edd9415
fix for bug in search with site: constraint
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5947 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c1e5fad9a7
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14767#p14767
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5944 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8ee3a94e82
fix for non-caching of sitehash, see http://forum.yacy-websuche.de/viewtopic.php?p=14440#p14440
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5942 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
21930d05ed
fix for [B@...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5941 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b6ba387e01
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14751#p14751
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5940 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4338dcf936
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2093&hilit=
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5937 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
bad7ce9286
experimental option trayIcon.force for unsupported platforms. java 1.6 needed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5936 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ea27853c59
*) some refactoring
...
*) added one assertion
*) no functional changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5935 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
d164b42604
*) cosmetics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5934 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
17150b2950
fixed bug in snippet computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5932 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
89aeb318d3
enhanced the wikimedia dump import process
...
enhanced the wiki parser and condenser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5931 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fb77116c6
added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see
...
http://localhost:8080/IndexImportWikimedia_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5930 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
hermens
df733af4fa
Try not to loose content from ram during IndexCell.delete by moving ram.delete after the dangerous operations on the array (array.get and array.delete)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5929 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
hermens
ac72005f2f
Let IndexCell.remove remove entries from the ram portion of the DB as well.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5928 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8ba7ff5353
a fix and another speed enhancement for the RWI cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5927 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
05f077e85f
added stack trace output to solve problem in
...
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2076&hilit=&p=14612#p14612
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5926 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
71a4cadf31
better and more performant synchronization in SimpleARC, the caching object for word hashes. Speeds up indexing.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5925 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e6773cbb33
better handling of RWI cache for concurrency and less overhead when writing new entries -> even more indexing speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5924 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c097531e3d
added a catch Exception to all thread to check if any of them silently dies without any other notification
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5922 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
083533e5ec
fix for bugs in IODispatcher
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5921 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
21fbca0410
better scaling of HEAP dump writer for small memory configurations;
...
should prevent OOMs during cache dumps
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5920 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
6e0b57284d
better care for states of the IODispatcher
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5919 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1db9cdd4e4
fixed bug in writing of robots.txt entries in case that host names exceeded 64 characters and some other problems
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5918 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
bde88b684a
* splitt off yacyRelease from yacyVersion
...
* added some gui infos about signatures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5916 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
057ce14c8e
more fixes (character encoding, parser exceptions, http client failure, blob writing)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5914 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d2ac0aa682
- fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling
...
- increased default memory size to 180MB
- fixed possible bug in http client reset (there was a deadlock)
- bug in BOBHeap marked, but not solved, cause is still unknown.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5912 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
1351d903a1
don't follow links like mailto:
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5909 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e88a66bcae
temporary disabling computation of all sublinks (check needed)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5908 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ff5f82d780
*) removed description of removed commands from wikiHelp ([= =])
...
*) used format function of Netbeans for wikiCode to make it more readable, no functional changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5907 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
eacf95213a
fix for crawling of mailto-links
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5906 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9c6ac43f66
fixes for wiki parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5905 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3a64c9d02f
- fix for problem with concurrency when computing word hashes
...
- fix for search in case that a urlfilter was used and zero results were returned
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5904 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d3f8aa5a2a
set of small fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5903 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
78ffb61297
*) got rid of unnecessary variable which might also fix IndexOutOfBoundsException
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5902 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d31e6f9c14
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14457#p14457
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5899 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago