theli
eee6322aaf
*) Adding redirection support to plasmaCrawlWorker.java
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@328 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
cd279907c0
*) Adding redirection support to plasmaCrawlWorker.java
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@327 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6697d5e52e
*) correcting fkt. mediaExtContains
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@326 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3addf58046
enhanced snippet-loading with threads
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@322 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
56d28a16f0
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@320 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d6c85228a6
enhanced snippet computation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@319 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
fafda068f9
*) allowing crawler to process resources with statuscode 203
...
- this is needed if yacy is behind a second proxy
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@316 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
aae9a433a6
*) correcting usage of supportedFileExt-List
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@315 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
1e7f062350
many bugfixes, memory leak fixes, performance enhancements; new kelondroHashtable; activated snippets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@313 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
68dc2b0c6b
added kelondroArray, the basis for upcoming kelondroHash and some bug fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@311 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a19541e563
code-enhancements after analysis with AppPerfect
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@307 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
85075269a6
extended fail-safe memory-managament. prevents too much allocation, too often GC and should help for the 100%CPU-bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@303 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e3c92818db
avoiding OutOfMemoryError routines
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@302 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3e8ee5a46d
enhanced caching in kelondroRecords and added better synchronization/finalizer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@301 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
db3ed75728
*) closing stream correctly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@293 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5d06ded005
enhanced html parser speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5a490aa065
fixed html parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@289 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a25b5b4986
fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
9e47ba5ad6
*) adding missing calls for function close() to avoid "too many open file" bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@282 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
9a98988c3c
*) Bugfix for SSL/NIO Bug
...
See: http://www.yacy-forum.de/viewtopic.php?t=516
- removing NIO from server/serverCore.java because of massive problems
with socket close issues
*) Adding support for remote port forwarding via sch
@Orbiter: Please take a look into
- hello.java
- server/serverCore.java.publicIP()
- yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a1ffc27041
preparations for image/movie/music indexing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@280 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a5b40923b6
added word migration to assortments (start with 'java -classpath classes yacy -migratewords')
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@278 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
890e3f4d4a
*) adding missing calls for function close() to avoid "too many open file" bug*) adding
...
*) bugfix in plasma/plasmaParser.java:
- parsers with missing dependencies wehre not ignored correctly
*) passing a logger instance to the parsers modules which can be used
for logging purposes by the parsers (not done yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@276 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6dd3ec0dc4
*) Adding debug="true" debuglevel="lines,vars,source" to ant build files
...
See: http://www.yacy-forum.de/viewtopic.php?p=4099
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@270 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
4f9c30ef49
using mime-type instead of file extension for doctype
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@269 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
ee9e110366
*) removing old logging configuration properties from yacy.init
...
*) serverLog.java logging functions now also accept exceptions als
additional parameters.
The Stacktrace of this ecceptions will then be appended to the
logging message and can e.g. be viewed on the gui logging page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@265 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c1a4e0dc28
*) changing reference to logger
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@252 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
d0083f845f
*) changing reference to logger
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@251 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
1b5ae054f8
*) changing reference to logger
...
*) parser will not be returned into pool if the parser was deactivated
via gui
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@250 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
68f30811fa
*) changing reference to logger
...
*) bugfix in function getCachePath
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@249 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
fbbea813c5
*) changing references to logger
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@248 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
4574fa4ce7
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@224 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
83b41ef2f7
*) Adding timeouts for shutdown
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@223 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
ef6851798b
*) changing thread priority while parsing a pdf file to avoid 100% CPU usage.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@222 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
33f9315e58
implemented multithreading of indexing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@221 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ca3b4ccaf4
added snippet-routines (not yet finished)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@218 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ee0758fe4d
bugfixes/empty-dir-deletion/snippet-test-activation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
594c591223
changes towards 0.38
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d8fdc2526e
added experimental snipplet-generation (to be disabled for 0.38)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3771b10b89
implemented automated migration indexCache 0.37 -> indexAssortmentCluster
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@205 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e89ded9e41
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@204 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
650ca3955a
added flush-thread for index cache and added language-name mapping in Language_p
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3d8a2ff937
enhanced parallelization of local/global/remote crawling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a05d738ea4
enhanced caching, removed bug causing outOfMemory
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@195 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
21110dcd5e
fixed bugs with open files and caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@175 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f8f8dd05db
fixed "Too many open files" - bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@174 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
74eb21f62e
*) adding image tag into rss template
...
*) adding a xslt stylesheet so that the rss document can be viewed in a normal webbrowser
*) adding pubDate tag to each search item
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@173 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5f90daa265
implemented localization environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@171 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
84f9d8f7f0
*) migrating ant build files to generate a single extension tar per default
...
This is buildconfigureable by changing the extensionTarget Property in the build.properties file
*) Trying to solve "yacy.java template replacement / ant build failed" bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@169 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
fdd606c8c8
fixed bugs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@168 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago