orbiter
7fb645b0ab
enhanced crawling performance, changed memory settings, new performace options
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@51 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
10078bb354
*) date string was accidentally replaced with the current value
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@50 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
fd584c113c
*) some minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@49 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
f44b219e44
*) Eclipse has accidentally copied in the wrong file header into the new files (because these headers were accidentally set as default for the whole workspace instead of the project)
...
Fixed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@48 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
081ebd5517
*) I've accidentally used Java 5.0 syntax for enumerations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@47 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
58b1a0ba40
*) adding an new package for extra content parsers
...
*) adding content parser for
- pdf (using the pdf-box library)
- doc (using the textmining.org library)
*) adding a Interface for content parsers
*) adding a configuration file which can be used to configure which parser is used for which mimeType
*) Sempahore class was moved and renamed to serverSemaphore
*) Changing yacy shutdown behaviour
Buzy waiting loop for shutdown was removed and replaced with a blocking call (using the semaphore class mentioned above) to the new switchboard.waitForShutdown method.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@46 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
17d993cfee
*) adding directory and classes needed by the new content parsers for pdf + doc
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@45 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
8b31f9e202
enhanced shut-down behaviour & added experimental nio-wrapper for kelondroRA (not active yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@44 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
ff21586a27
Fixed some spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@43 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
87a61a01c2
fixed bad-gzip-trailer behaviour (now cuts off trailer)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@42 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
00f223cfc1
fixed post-parsing (a case when the bluelist is empty)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@41 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
044b93412a
Copyright notice
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@40 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
c9c0a1f11c
*) Trying to speedup local crawling
...
- introduction of a threadpool for crawling
- introduction of a job queue to avoid buzy waiting for a free crawler slot
*) New classes added
- queue for receiving of crawler jobs
- semaphore class to do reader/writer synchronization (mutual exclusion)
- message object to hold all needed data about a crawler job
*) Trying to solve session-thread shutdown problem
- session thread stopped variable is now set from outside before interrupting the
session thread.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@39 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
4e429ae243
Fixed *.bat start-scripts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@38 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
ce7d8c4fe0
Fixed some spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@37 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
570de9c4f4
Fixed some spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@36 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
942914ffd2
*) Adding additional functions to serverByteBuffer so that it
...
can be used instead of a ByteArrayOutputStream
*) Using a serverByteBuffer for lineBuffering in class httpc
instead of a ByteArrayOutputStream
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@35 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
432e01910b
*) Bugfix: Image falsification
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@34 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
97ec8d65e4
fixed makerelease & clean-up of dead code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@33 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
b61567a39e
Fixed spelling mistake and inserted author as described in mailinglist
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@32 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
4a76ccc6d6
*) Some minor bugfixes
...
- httpc: wrong error-message on 404
- httpc: error message was accidentally shown when object
was released from pool
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@31 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
1fec00bc24
*) Bugfix to avoid Nullpointer-Exceptions
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@30 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
e2a884031c
*) added new lib dir to classpath.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@29 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
a1d3e1b896
*) Adding apache license file to lib dir
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@28 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
f8d82cee47
*) Merging didn't work correctly. Fixed.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@27 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
f39812da91
*) Some performance improvements
...
- many classes set to final
- implementation of a session-thread pool
- reusage of the server handler class (normally the httpd object)
within the session thread
- implementation of a httpc object pool
- introduction of a linebuffer in httpd which can be reused
- reusing the properties table in the httpc
- added to apache libs (commons-collections, commons-pool) which
are needed for the object/thread pool implementation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@26 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
d5ff81c636
*) Undoing last changes. Sorry.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@25 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
ea4849716f
*) Bugfix: plasmaCondenser.getWords() was not used correctly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@24 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
0a6cf3f5e7
*) Bugfix: Reference to plasmaHTCache.Entry.urlString was not set correctly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@23 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
b9203bdb50
bug fixes and code cleaning
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@22 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
c13411c198
Buildfile which inserts the Date.
...
The Version is set in the source; so it will be correct if you check old versions out.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@21 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
b7d4389e4b
*) support for Proxy Auto-Config File generation added.
...
File is accessible using:
http://proxy:8080/autoconfig.pac
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@20 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
(no author)
070758dbfa
Adding missing compile statements for
...
- ${htroot}/htdocsdefault
- ${htroot}/yacy
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@19 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
c0807abd33
new crawl/proxy/cache design + fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@18 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e7d055b98e
very experimental integration of the new generic parser and optional disabling of bluelist filtering in proxy. Does not yet work properly. To disable the disable-feature, the presence of a non-empty bluelist is necessary
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@17 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
96516fc9d8
fixed bugs (search+kelondroException, dns)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@16 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
a87a17a3c8
prepared generic text parser environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@15 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
e374aca2cd
enhanced exception handling in kelondro
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@14 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
072052f150
fixed bugs (dns, seedDB)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@13 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
89eb9a2292
fixed bug with crawl profiles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@12 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
97dad7eaf4
A Ant Buildfile for YaCY. It doesn't replace the REPL_VERSION and REPL_DATE Strings, yet.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@11 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
828b586c04
Added startYACY_hidden.bat
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@10 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
850c6ea729
Fixed annoying mistake made by me in earlier revision...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@9 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
a6154697ba
Fixed some grammar/spelling mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
1e33de27da
Log ist now skinnable
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
0fb1046530
Nullsoft Installscript to build the Windowsinstaller.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
rramthun
f824958184
Fixed some spelling/grammar mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
7eb3c81aad
name check on new peer names
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
allo
955d26b117
Bug: could not install from url and use the skin (because of a two names "use")
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
991d2420de
adopted makerelease.sh for non-existing classes folder
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago