theli
933256bbfc
*) adding href's to allow direct access to the currently crawled website
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@317 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
83ceec3a63
*) Bugfix for wrong "Statusanzeige"
...
See: http://www.yacy-forum.de/viewtopic.php?p=4560
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@314 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
1e7f062350
many bugfixes, memory leak fixes, performance enhancements; new kelondroHashtable; activated snippets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@313 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
dc1d707eb2
*) Deactivating "Port Forwarding Feature" for a while because of too many problems at the moment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@312 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
low012
afe7cbe4de
just a typo...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@310 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
low012
10f6f19c08
Fixed one typo and added a few options to handle messages when messages get displayed.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@309 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
cf171ca35d
Some corrections to the webinterface...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@306 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
85075269a6
extended fail-safe memory-managament. prevents too much allocation, too often GC and should help for the 100%CPU-bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@303 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
10781f36b7
changed performance menu to accept milliseconds; 200 milliseconds may be appropriate for all crawling jobs; indexing should be set to 0
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@299 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
3227a9eba5
*) Adding retry function for seed uploading
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@298 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
aea355c03c
*) adding test for connection status of port forwarding feature
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@292 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5d06ded005
enhanced html parser speed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5a490aa065
fixed html parser
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@289 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a25b5b4986
fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
38747857c2
correction of correction to port-forwarding
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@287 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
dbda6e1e85
corrections to port forwarding
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@286 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
d8cb3324a9
*) property "mytime" was not set correctly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@285 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
b2c5305737
*) Bugfix: Uptime Uptime #minutes (kosmetisch)
...
See: http://www.yacy-forum.de/viewtopic.php?p=4294
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@283 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
9a98988c3c
*) Bugfix for SSL/NIO Bug
...
See: http://www.yacy-forum.de/viewtopic.php?t=516
- removing NIO from server/serverCore.java because of massive problems
with socket close issues
*) Adding support for remote port forwarding via sch
@Orbiter: Please take a look into
- hello.java
- server/serverCore.java.publicIP()
- yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a1ffc27041
preparations for image/movie/music indexing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@280 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f45dc29f35
maintenance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@279 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
a2e5018427
*) adding missing calls for function close() to avoid "too many open file" bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@273 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
3ee35b9ff9
*) Proxy-Error-Msg: align=center
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@272 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c7d294d8d4
*) Bugfix for:
...
- 302 redirection Problem on Amazon Server
- Wrong References in proxymsg/error.html
See: http://www.yacy-forum.de/viewtopic.php?t=515
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@271 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
878ff0ae7b
corrections
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@262 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
a566588e9b
*) adding configuration section for new http keep-alive support
...
*) moving transparent proxy configuration into new config section
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@238 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6bde37950f
*) adding xsl stylesheet that can be used by browsers to format the rss search result in a user friendly format
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@237 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
e302c50390
*) Changes needed because of logging migration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@236 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
f9a95b5cb8
*) Displaying more user friendly Memory Usage statistic
...
*) Displaying traffic consumed by yacy
- this is not finished yet
- at the moment only outgoing proxy traffic is counted
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@235 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
f8de2556f8
*) Changes needed to allow a move verbose error message
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@234 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
33f9315e58
implemented multithreading of indexing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@221 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
7c318f8886
Language files, not skins...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@220 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
52cf732fad
*) correcting "seed-ftp-upload/Nothing changed" bug:
...
See: http://www.yacy-forum.de/viewtopic.php?p=3986#3986
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@219 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ca3b4ccaf4
added snippet-routines (not yet finished)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@218 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a73e6de005
faq/doc - Fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@215 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
9af1bf4b38
*) displaying memory usage of yacy in Status.html
...
*) displaying more expressive uptime information on Status.html and Network.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@214 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ee0758fe4d
bugfixes/empty-dir-deletion/snippet-test-activation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
594c591223
changes towards 0.38
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d8fdc2526e
added experimental snipplet-generation (to be disabled for 0.38)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e89ded9e41
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@204 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
650ca3955a
added flush-thread for index cache and added language-name mapping in Language_p
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
ec77e6faa6
updated german language file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@201 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
4e63456dba
some corrections/enhancements to the webinterface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@198 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
3d8a2ff937
enhanced parallelization of local/global/remote crawling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
8ab021b39f
Your-Peer Stats added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@185 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
c6331cf088
"Comma separated Values File" for the Network Overview Page.
...
http://www.yacy-forum.de/viewtopic.php?t=4
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@184 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
7de98b1895
Deleted static locales because they conflict with the dynamic ones.
...
use Language_p.html and $rootdir/locales/*.lng to generate locales in htroot/locales
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@178 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
a8b2dccc7a
New Translation System.
...
Use Wordlists like these:
de.lng
Network\ Overview=Netzwerk-Übersicht
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@176 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
21110dcd5e
fixed bugs with open files and caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@175 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
74eb21f62e
*) adding image tag into rss template
...
*) adding a xslt stylesheet so that the rss document can be viewed in a normal webbrowser
*) adding pubDate tag to each search item
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@173 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
6f09251bbc
added peer-Name settings to Settings page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@172 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
5f90daa265
implemented localization environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@171 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6cc4297761
*) adding namespace for yacy topwords feature
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@170 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
73922f3501
experimental topword-intergartion in rss
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@167 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
6af3744a0b
http://www.yacy-forum.de/viewtopic.php?p=3368 fixed:
...
Newlines are now converted correctly.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@166 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
6ee564c482
a really simple Interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@165 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
76dc892017
refined assortment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@159 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
0e1d9e9722
*) shrinking httpc linebuffer when httpc is returned to pool. This is done to free memory
...
*) Making Seed-Upload configuration more verbose.
*) Some Changes in SOAP Search API (not finished yet).
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@158 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
b625aa91fd
*) Trying to solve Seed-Upload-Configuration - "Error with submitted information. Nothing changed." Bug:
...
see: http://www.yacy-forum.de/viewtopic.php?p=3233
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@157 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
4b01ff7548
activated assortments, removed write-queues
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@151 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e26ac60c3e
modified assortment data structures
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@148 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
377fb07250
Used YaCy logging class instead of stdout
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@147 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
85e7c31a57
Fixed a spelling mistake
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@145 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
76475f9f38
Fixed various spelling mistakes...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@144 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
bfff0a96a7
Fixed some spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@143 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
low012
71b9cb0c33
Just making it look like it is supposed to in Opera too. (Opera makes a difference between center and middle which most browsers don't.)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@138 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
01fb4e4eb9
Fixed problem described in http://www.yacy-forum.de/viewtopic.php?t=377
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@135 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
042f171022
*) Correcting problems with old seed configuration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@133 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
dfde28a244
changed kaskelix mascot
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@130 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
361f05978d
Multiple updates regarding the yacy seedUpload facility,
...
optional content parsers, thread pool configuration ...
Please help me testing if everything works correct.
*) Migration of yacy seedUpload functionality
See: http://www.yacy-forum.de/viewtopic.php?t=256
- new uploaders can now be easily introduced because of a new modulare uploader system
- default uploaders are: none, file, ftp
- adding optional uploader for scp
- each uploader provides its own configuration file that will be
included into the settings page using the new template include feature
- Each uploader can define its libx dependencies. If not all needed libs are
available, the uploader is deactivated automatically.
*) Migration of optional parsers
See: http://www.yacy-forum.de/viewtopic.php?t=198
- Parsers can now also define there libx dependencies
- adding parser for bzip compressed content
- adding parser for gzip compressed content
- adding parser for zip files
- adding parser for tar files
- adding parser to detect the mime-type of a file
this is needed by the bzip/gzip Parser.java
- adding parser for rtf files
- removing extra configuration file yacy.parser
the list of enabled parsers is now stored in the main config file
*) Adding configuration option in the performance dialog to configure
See: http://www.yacy-forum.de/viewtopic.php?t=267
- maxActive / maxIdle / minIdle values for httpd-session-threadpool
- maxActive / maxIdle / minIdle values for crawler-threadpool
*) Changing Crawling Filter behaviour
See: http://www.yacy-forum.de/viewtopic.php?p=2631
*) Replacing some hardcoded strings with the proper constants of the httpHeader class
*) Adding new libs to libx directory. This libs are
- needed by new content parsers
- needed by new optional seed uploader
- needed by SOAP API (which will be committed later)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@126 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
014b139c73
*) Bugfix of "There are xxx entries in the crawler queue. Showing 0 most recent entries" Bug.
...
see: http://www.yacy-forum.de/viewtopic.php?t=338
see: http://www.yacy-forum.de/viewtopic.php?p=2552
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@122 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
f9c7deca2e
Fixed some spelling mistakes and removed underline from accesskey chars
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@121 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
1d38599598
*) changing comment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@119 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
d2c4e9a55e
*) Implementing yacy forum wishlist item: "Pause Crawling"
...
see: http://www.yacy-forum.de/viewtopic.php?t=48
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@118 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
2d751ba831
Fixed a spelling mistake
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@117 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
287d2e6f10
further enhanced caching (new cache flush methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@111 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
85c2f3be8a
Fixed spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@110 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ea478f3975
enhanced indexing-caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@107 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
b4030e5023
implemented serverSwitchActions - action-hooks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@105 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
a5fec449c8
*) setting threadnames for kelondroMap:writequeue and publishSeed
...
so that a thread dump is more verbose
*) Moving code for transparent proxy support to a separate function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@98 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
74f12bb0f3
*) adding transparent proxy support
...
Now a firewall can transparently redirect all
http traffic through yacy.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@96 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
982c3ebeeb
Added some accesskeys to webinterface and updated help with some hints on how to use them.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@91 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
6c8e0fdbf5
Made Homepage and eMail clickable
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@87 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
1d7fed87dc
redesign of index caching - removed indexCache.db
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@86 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
3f85978519
Fixed one spelling mistake, limited input for ICQ numbers to 9 digits and made ICQ number in peer profiles clickable.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@85 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
e7f7aa0bb9
*) Import statements reorganized
...
Now it's easier to determine which class really uses which other class*) Reogranizing Import Statements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@83 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
351c86d5d9
*) Migration of optional Content Parser integration
...
- each additional parser must be in a subpackage
of plasma.parser
- each parser must have its own ant build file (which will
be called automatically from the main build file)
- Calling the main build file results in building a separate
zip file for each optional parser. This zip file includes:
+ sources of the Parser.java
+ compiled classes of the Parser.java
+ needed additional libs (libx)
- To install an additional parser the user simply needs to
extract the zip file listed above into his/her yacy directory.
- The configuration (enabling/disabling) of a parser can be done
via the webinterface (currently the settings dialoge) and is
done "on-the-fly". The installation can not be done "on-the-fly"
at the moment because of classpath issues.
- The classpath of the linux startup/stop scripts is generated
automatically now (including all libraries from lib and libx).
*) Bugfix: File Extension was not calculated correctly by the crawler
e.g.: file extension was accidentally: .php?param=value
Corrected.
*) Adding additional parser for parsing of rss/atom feeds
- added needed libs to do this.
TODO:
- automatic building classpath for windows startup scripts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@78 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f99930c04b
fixed brute-force + peer-disconnect - Bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@75 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
c7c6aaf06e
many bug-fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@73 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
52abc456fb
new Templates
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@70 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
56409402f0
Fixed some spelling mistakes...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@66 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
9cb8779208
Fixed some spelling mistakes...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@64 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
19e69f0efd
Changed <head> YACY into YaCy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@58 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
4c8cc101d6
Bugfix: Do not shot the first X lines, but the last X line of log
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@57 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
ba16da72b4
fixed not-working kelondroRecords-Cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@56 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
7fb645b0ab
enhanced crawling performance, changed memory settings, new performace options
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@51 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
58b1a0ba40
*) adding an new package for extra content parsers
...
*) adding content parser for
- pdf (using the pdf-box library)
- doc (using the textmining.org library)
*) adding a Interface for content parsers
*) adding a configuration file which can be used to configure which parser is used for which mimeType
*) Sempahore class was moved and renamed to serverSemaphore
*) Changing yacy shutdown behaviour
Buzy waiting loop for shutdown was removed and replaced with a blocking call (using the semaphore class mentioned above) to the new switchboard.waitForShutdown method.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@46 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
ff21586a27
Fixed some spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@43 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
00f223cfc1
fixed post-parsing (a case when the bluelist is empty)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@41 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c9c0a1f11c
*) Trying to speedup local crawling
...
- introduction of a threadpool for crawling
- introduction of a job queue to avoid buzy waiting for a free crawler slot
*) New classes added
- queue for receiving of crawler jobs
- semaphore class to do reader/writer synchronization (mutual exclusion)
- message object to hold all needed data about a crawler job
*) Trying to solve session-thread shutdown problem
- session thread stopped variable is now set from outside before interrupting the
session thread.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@39 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago