orbiter
ee0758fe4d
bugfixes/empty-dir-deletion/snippet-test-activation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
594c591223
changes towards 0.38
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d8fdc2526e
added experimental snipplet-generation (to be disabled for 0.38)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3771b10b89
implemented automated migration indexCache 0.37 -> indexAssortmentCluster
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@205 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e89ded9e41
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@204 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
650ca3955a
added flush-thread for index cache and added language-name mapping in Language_p
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
mkossin
3dd3431c58
Fixes a Problem with mySeed.txt. If this file is accessible but has no content it will be recreated: http://www.yacy-forum.de/viewtopic.php?p=3657#3657
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@202 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3d8a2ff937
enhanced parallelization of local/global/remote crawling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a05d738ea4
enhanced caching, removed bug causing outOfMemory
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@195 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
3b03d99797
Bugfix: Wrong Path of DATA/LOCALE Dir
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@182 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
8796987def
Copy the shipped Locales on Startup to DATA/LOCALES
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@180 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
f57d280afd
serverLog instead of System.out.println
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@179 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
a8b2dccc7a
New Translation System.
...
Use Wordlists like these:
de.lng
Network\ Overview=Netzwerk-Übersicht
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@176 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
21110dcd5e
fixed bugs with open files and caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@175 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f8f8dd05db
fixed "Too many open files" - bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@174 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
74eb21f62e
*) adding image tag into rss template
...
*) adding a xslt stylesheet so that the rss document can be viewed in a normal webbrowser
*) adding pubDate tag to each search item
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@173 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5f90daa265
implemented localization environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@171 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
84f9d8f7f0
*) migrating ant build files to generate a single extension tar per default
...
This is buildconfigureable by changing the extensionTarget Property in the build.properties file
*) Trying to solve "yacy.java template replacement / ant build failed" bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@169 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
fdd606c8c8
fixed bugs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@168 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
8bd49ba535
*) setting root dir for all tar files properly
...
*) automatically adding SVN Revision number to tar file name
*) introducing build.properties file that can be used to set the build version number and date
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@164 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
0c35171c85
assortment fine-tuning
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@163 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
76dc892017
refined assortment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@159 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
0e1d9e9722
*) shrinking httpc linebuffer when httpc is returned to pool. This is done to free memory
...
*) Making Seed-Upload configuration more verbose.
*) Some Changes in SOAP Search API (not finished yet).
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@158 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
0484c41a84
*) replacing system.xxx.println with logging statements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@156 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c2378a968f
*) setting thread name for search threads
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@155 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
7994c485f1
*) Trying to set the document title properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@154 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
285936d778
*) trying to set document title properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@153 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
573a8e8047
*) setting document title properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@152 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
4b01ff7548
activated assortments, removed write-queues
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@151 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
d63fae5bd4
SVN Templates Restored
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@150 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
e015d35b21
Updated version number and date (please don't forget in the future)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@149 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e26ac60c3e
modified assortment data structures
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@148 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
ced07e3aec
javadoc comment added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@146 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
79be6f003d
enhanced Assortment class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@141 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
9ee3e69021
*) Solving "Warning: You did not close the PDF Document" problem when an OutOfMemory Exception occured ...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@140 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5c6147a54c
introduced assortment structure (generalization of singletons)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@139 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
05ab7c4d68
*) Correcting Problems with "transparent proxy" mode.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@134 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
73e297f30f
*) adding proper default values for RealtimeParsableMimeTypes if something goes wrong with the configuration file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@132 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
0db0619204
*) Correcting wrong path
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@128 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
893a662329
*) Adding missing cast statement
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@127 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
361f05978d
Multiple updates regarding the yacy seedUpload facility,
...
optional content parsers, thread pool configuration ...
Please help me testing if everything works correct.
*) Migration of yacy seedUpload functionality
See: http://www.yacy-forum.de/viewtopic.php?t=256
- new uploaders can now be easily introduced because of a new modulare uploader system
- default uploaders are: none, file, ftp
- adding optional uploader for scp
- each uploader provides its own configuration file that will be
included into the settings page using the new template include feature
- Each uploader can define its libx dependencies. If not all needed libs are
available, the uploader is deactivated automatically.
*) Migration of optional parsers
See: http://www.yacy-forum.de/viewtopic.php?t=198
- Parsers can now also define there libx dependencies
- adding parser for bzip compressed content
- adding parser for gzip compressed content
- adding parser for zip files
- adding parser for tar files
- adding parser to detect the mime-type of a file
this is needed by the bzip/gzip Parser.java
- adding parser for rtf files
- removing extra configuration file yacy.parser
the list of enabled parsers is now stored in the main config file
*) Adding configuration option in the performance dialog to configure
See: http://www.yacy-forum.de/viewtopic.php?t=267
- maxActive / maxIdle / minIdle values for httpd-session-threadpool
- maxActive / maxIdle / minIdle values for crawler-threadpool
*) Changing Crawling Filter behaviour
See: http://www.yacy-forum.de/viewtopic.php?p=2631
*) Replacing some hardcoded strings with the proper constants of the httpHeader class
*) Adding new libs to libx directory. This libs are
- needed by new content parsers
- needed by new optional seed uploader
- needed by SOAP API (which will be committed later)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@126 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c57306f620
*) Correcting Problem with Template Include - Unresolved filename
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@125 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
07ca4083d3
right Prefix for Includes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@124 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
bcdc44c065
*) Adding more constants. Needed for next commit.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@123 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
ddc5675781
*) Correcting typo
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@120 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
d2c4e9a55e
*) Implementing yacy forum wishlist item: "Pause Crawling"
...
see: http://www.yacy-forum.de/viewtopic.php?t=48
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@118 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
2d751ba831
Fixed a spelling mistake
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@117 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
511f1c9b25
*) Bugfix of http://www.yacy-forum.de/viewtopic.php?p=2508&highlight=#2508
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@116 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
8559c6ed5f
*) removing obsolete class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@115 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
4dd387aae9
*) moving constants (see last commit) to proper httpHeader class
...
*) migrating fileHandler + proxyHandler to use constants instead of hardcoded values
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@114 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
aea3479a9c
*) introducing of additional constants
...
to improve maintainability of the sourcecode
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@113 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e5e46eccf8
fixed version/date
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@112 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
287d2e6f10
further enhanced caching (new cache flush methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@111 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
376b917c91
fixed shut-down by stopYACY.sh
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@108 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ea478f3975
enhanced indexing-caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@107 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
0cfe94bb66
fixed last commit + added missing files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@106 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
b4030e5023
implemented serverSwitchActions - action-hooks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@105 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
5a651284a2
*) Adding server shutdown hook
...
... to allow a graceful shutdown even if the java runtime was interrupted using
<ctrl>+<c>
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@104 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
77fd0fdf3f
Includes, which use a normale Template Key to find the sourcefile
...
Syntax: #%[key]%#
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@103 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
abb6750090
added Include Funktion. #%include.inc%# includes other Templates
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@102 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6f4d2e5272
*) fixing replace bug.
...
using
stringvar = stringvar.replace(xxx)
istead of
stringvar.replace()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@101 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
f678cda514
*) correcting bug in transparent proxy support
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@99 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
a5fec449c8
*) setting threadnames for kelondroMap:writequeue and publishSeed
...
so that a thread dump is more verbose
*) Moving code for transparent proxy support to a separate function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@98 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
dae6f46812
*) Forgotten to do transparent proxy support also for http head and post
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@97 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
74f12bb0f3
*) adding transparent proxy support
...
Now a firewall can transparently redirect all
http traffic through yacy.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@96 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
92a65874d6
*) fixed proxy authentication bug:
...
http://www.yacy-forum.de/viewtopic.php?t=174
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@95 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
10a4a2741d
fixed missing close
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@94 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
db1da3345d
introduced singleton-database
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@92 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
cbdc499ba6
*) adding many missing (File)?(Input|Output)Stream.close() calls to avoid "Too many open files bug".
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@90 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a9b22647dc
fixed bug in indexDump.stack - generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@88 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
1d7fed87dc
redesign of index caching - removed indexCache.db
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@86 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
3f85978519
Fixed one spelling mistake, limited input for ICQ numbers to 9 digits and made ICQ number in peer profiles clickable.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@85 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
1dad015b0b
*) Migration of Ant build files
...
- now the ant build file has the same functionality as the makerelease build file
- from now on the ant build files can be used instead of the makerelease build script
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@84 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
2aa5fe8f50
*) Import statements reorganized
...
Now it's easier to determine which class really uses which other class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@82 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
978c16edd3
Deleted startYACY_hidden.bat and fixed annoying spelling mistake.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@81 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
351c86d5d9
*) Migration of optional Content Parser integration
...
- each additional parser must be in a subpackage
of plasma.parser
- each parser must have its own ant build file (which will
be called automatically from the main build file)
- Calling the main build file results in building a separate
zip file for each optional parser. This zip file includes:
+ sources of the Parser.java
+ compiled classes of the Parser.java
+ needed additional libs (libx)
- To install an additional parser the user simply needs to
extract the zip file listed above into his/her yacy directory.
- The configuration (enabling/disabling) of a parser can be done
via the webinterface (currently the settings dialoge) and is
done "on-the-fly". The installation can not be done "on-the-fly"
at the moment because of classpath issues.
- The classpath of the linux startup/stop scripts is generated
automatically now (including all libraries from lib and libx).
*) Bugfix: File Extension was not calculated correctly by the crawler
e.g.: file extension was accidentally: .php?param=value
Corrected.
*) Adding additional parser for parsing of rss/atom feeds
- added needed libs to do this.
TODO:
- automatic building classpath for windows startup scripts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@78 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d0010ff0b0
last changes for release 0.37
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@76 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f99930c04b
fixed brute-force + peer-disconnect - Bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@75 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
c7c6aaf06e
many bug-fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@73 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
48650c082c
fixed 100%-CPU-Bug in plasmaCondenser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@72 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
995673d795
several bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@71 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
2de90020ed
fixed caching+synchronization+brute-force-denial
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@67 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
9156fd53bc
fixed bugs in last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@65 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e25f2354c2
removed synchronization and thread blockings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@63 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
3756e6d20f
*) "Httpc object was not returned to object pool." bug fixed.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@62 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
47e426ff7e
*) one possible deadlock (because of nested object locks) removed in class kelondroMap
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@61 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
58a65b60bd
*) synchronized keyword removed from function processLocalCrawling to avoid deadlocks.
...
This synchronized keyword is not needed anymore because of the crawler jobqueue which
is responsible for the synchronization now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@60 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
65fc650109
*) plasmaCrawlLoader shutdown problem fixed (hopefully)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@59 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ba16da72b4
fixed not-working kelondroRecords-Cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@56 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d03d60f8b5
separated yacy-core from yacy-libx; fixed makerelease
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@55 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
c09c54c652
staticIP Property, for people with dyndns aliases ;-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@54 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
d005d7484e
yacyDebugMode - allow Lan-IPs for testing
...
where was the Code from 0.25 lost?
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@53 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
7fb645b0ab
enhanced crawling performance, changed memory settings, new performace options
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@51 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
10078bb354
*) date string was accidentally replaced with the current value
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@50 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
fd584c113c
*) some minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@49 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
f44b219e44
*) Eclipse has accidentally copied in the wrong file header into the new files (because these headers were accidentally set as default for the whole workspace instead of the project)
...
Fixed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@48 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
081ebd5517
*) I've accidentally used Java 5.0 syntax for enumerations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@47 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
58b1a0ba40
*) adding an new package for extra content parsers
...
*) adding content parser for
- pdf (using the pdf-box library)
- doc (using the textmining.org library)
*) adding a Interface for content parsers
*) adding a configuration file which can be used to configure which parser is used for which mimeType
*) Sempahore class was moved and renamed to serverSemaphore
*) Changing yacy shutdown behaviour
Buzy waiting loop for shutdown was removed and replaced with a blocking call (using the semaphore class mentioned above) to the new switchboard.waitForShutdown method.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@46 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
8b31f9e202
enhanced shut-down behaviour & added experimental nio-wrapper for kelondroRA (not active yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@44 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
87a61a01c2
fixed bad-gzip-trailer behaviour (now cuts off trailer)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@42 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
00f223cfc1
fixed post-parsing (a case when the bluelist is empty)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@41 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c9c0a1f11c
*) Trying to speedup local crawling
...
- introduction of a threadpool for crawling
- introduction of a job queue to avoid buzy waiting for a free crawler slot
*) New classes added
- queue for receiving of crawler jobs
- semaphore class to do reader/writer synchronization (mutual exclusion)
- message object to hold all needed data about a crawler job
*) Trying to solve session-thread shutdown problem
- session thread stopped variable is now set from outside before interrupting the
session thread.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@39 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
942914ffd2
*) Adding additional functions to serverByteBuffer so that it
...
can be used instead of a ByteArrayOutputStream
*) Using a serverByteBuffer for lineBuffering in class httpc
instead of a ByteArrayOutputStream
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@35 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
432e01910b
*) Bugfix: Image falsification
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@34 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
97ec8d65e4
fixed makerelease & clean-up of dead code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@33 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
4a76ccc6d6
*) Some minor bugfixes
...
- httpc: wrong error-message on 404
- httpc: error message was accidentally shown when object
was released from pool
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@31 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
1fec00bc24
*) Bugfix to avoid Nullpointer-Exceptions
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@30 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
f39812da91
*) Some performance improvements
...
- many classes set to final
- implementation of a session-thread pool
- reusage of the server handler class (normally the httpd object)
within the session thread
- implementation of a httpc object pool
- introduction of a linebuffer in httpd which can be reused
- reusing the properties table in the httpc
- added to apache libs (commons-collections, commons-pool) which
are needed for the object/thread pool implementation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@26 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
b9203bdb50
bug fixes and code cleaning
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@22 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
c13411c198
Buildfile which inserts the Date.
...
The Version is set in the source; so it will be correct if you check old versions out.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@21 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
b7d4389e4b
*) support for Proxy Auto-Config File generation added.
...
File is accessible using:
http://proxy:8080/autoconfig.pac
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@20 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
c0807abd33
new crawl/proxy/cache design + fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@18 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e7d055b98e
very experimental integration of the new generic parser and optional disabling of bluelist filtering in proxy. Does not yet work properly. To disable the disable-feature, the presence of a non-empty bluelist is necessary
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@17 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
96516fc9d8
fixed bugs (search+kelondroException, dns)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@16 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a87a17a3c8
prepared generic text parser environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@15 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e374aca2cd
enhanced exception handling in kelondro
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@14 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
072052f150
fixed bugs (dns, seedDB)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@13 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
89eb9a2292
fixed bug with crawl profiles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@12 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
248077d3f0
initial load with yacy 0.36
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago