orbiter
b21b9df2d0
added section headlines generation to html parser
...
can be viewed in cache control, but is not yet included to indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1320 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
c4487deba9
Minor changes collected over some time.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1319 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
6822dce57b
Using Orbiters function for auth
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1315 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
2028403670
- consolidated different orderings to kelondroNaturalOrder
...
- added another iteration method to rwihash-enumeration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1309 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9544c47684
added some UTF-8 handling.
...
hope this will help somehow.. for shure not THE solution to our UTF-8 problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1308 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
9d8dca750e
BUGFIX for my last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1306 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
5449193167
bugfix for http://www.yacy-forum.de/viewtopic.php?t=1706 (i hope)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1304 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
2a23f5d419
F..., Sorry, no time, later
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1303 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
3a2d13786e
bugfix for http://www.yacy-forum.de/viewtopic.php?t=1706
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1302 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
dc0999ec9c
adapted to new HTCache structure
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1290 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9086261476
refactoring of base64 encoding:
...
the kelondro database needs specific information about the order of
base64-encoded keys. Since no other package depends on base64
(only the httpd uses base64 for encryption, but does not need to encode these strings)
it is good to move base64 encoding to the new ordering classes in kelondro.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1284 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
b24fcc8ca4
oom
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1281 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
7da232b5b9
HTCache Reset if necessary
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1280 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
4f18f24d81
small change
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1278 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
c652527620
YaCy removes now the old HTCACHE data
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1277 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
69f65210e2
".yacy" has its own directory;
...
glad new year :)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1275 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
351fffc129
DATA/WORK for user-created content
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1274 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
a81cc9d969
no DATA/DATA to avoid confusion.
...
increasing version number
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1273 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
b95c5d5781
BUGFIX for URLs how "/../" ...;
...
new port handling;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1271 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
9cce3c5709
dates Table for bookmarksdb(needed for del.icio.us api)
...
Files in DATA/DATA
Migration: move bookmarks.db from SETTINGS in DATA
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1270 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
11fe95832e
avoid division by zero when index transfer is extremely fast
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1269 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
allo
4ac0fd328a
First Version of the Bookmarksmanager
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1248 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d7b6dcbe2e
*) Bugfix for MalformedURL problem if Location header is empty.
...
See: http://www.yacy-forum.de/viewtopic.php?p=14325#14325
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1247 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hermens
5b3e01bd3c
avoid division by zero when importing very small indexes (<100 entries)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1238 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
b7f9adc2c9
new filters added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1231 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
79667a172e
*) Bugfix for additional parser problem
...
See: http://www.yacy-forum.de/viewtopic.php?p=14146#14146
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1221 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8c594841a8
*) Bugfix for incorrectly indexing of URLs that were requested with Cookies in the
...
Request header
See: http://www.yacy-forum.de/viewtopic.php?p=14077
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1214 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b5d02d649a
fixed bug caused strange search result behaviour
...
(results from remote peers had not been saved propery after search)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1213 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4500506735
fixed some bugs concerning url entry retrieval and intexControl interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1212 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
83a34b838d
* added Object allocation monitor on performanceMemory page
...
* added some final statements
* changed shutdown sequence order
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1211 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4ff3d219e8
increased delay for cacheScan start and slowed down scan process
...
to provide more time to other tasks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1210 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3031903d50
re-design of RAM cache flush into assortment cluster
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1209 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0c762daf4b
better startup failure handling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1205 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f27f9ecf15
* activated write buffer for databases.
...
This should increase IO performance and reduce HD activity
* bugfixes for new exception-on-failure policy
* bugfixes for new IOChunks
* new Object pool for database write-buffer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1204 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
c59d1b2f5e
- Tests with write buffer (new class kelondroBufferedIOChunks, not yet active)
...
- minor bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1203 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bb79fb5d91
- changed handling of error cases retrieving urls from database
...
(no more NULL values are returned, instead, an IOException is thrown)
- removed ugly damagedURLS implementation from plasmaCrawlLURL.java
(this inserted a static value into the Object which is not really a good style)
- re-coded damagedURLS collection in yacy.java by catching an exception and evaluating the exception message
to do:
- the urldbcleanup feature must be re-tested
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1200 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e7d16ef831
*) Corrections in jMimeMagic MagicRule-file to detect some special rss feeds
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1196 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
386d9e45d8
*) Bugfix for code cleanup
...
- Code must be in finally block, otherwise it does not work if an error occurs!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1193 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
5a1d45715d
*) Bugfix for parser configuration bug
...
- it was not possible to disable all parsers
See: http://www.yacy-forum.de/viewtopic.php?t=1579
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1191 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
rramthun
a1061495d4
Fixed some spelling mistakes and added some text which (should) make it easier to understand the options.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1187 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0cdc58aaea
fixed indexing of local domains.
...
see http://www.yacy-forum.de/viewtopic.php?p=13680#13680
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1186 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e1c2d8ec5f
*) Speedup "removed from queue"
...
See: http://www.yacy-forum.de/viewtopic.php?p=13442#12188
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1183 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
hydrox
96930f0d2b
*)added function to removed malformed URLs from urlHash.db
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1182 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8862b6ba4b
*) Corrections for code cleanup 1175
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1179 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
13fdebc50d
added authentication for link deletion in search result
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1177 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
37f88b4017
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1176 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ec2b39c1ce
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1175 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
8f1f2daa5e
implemented interactive link deletion of search results.
...
next steps: attach voting and restrict to administrator
to see the deletion button, move the mouse pointer to the left of a search result
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1172 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
6d0f7e6988
*) Adding missing file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1171 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
44fa94ac52
*) Modifications for dbImport functionality
...
- dbImporter threads are now shutdown by the switchboard on server shutdown
- adding possibility to pause a importer thread via GUI
- Bugfix for abort function
See: http://www.yacy-forum.de/viewtopic.php?p=13363#13363
*) Modification of content parser configuration
- now it's possible to configure which parsers should be enabled for the proxy,
crawler, icap, etc. separately
-
*) htmlFilterContentScraper.java
- adding regular expression to normalize URLs containing /../ and /./ parts
*) httpc.java
- adding functionality to unzip gzipped content
- requested by roland: should be used later to allow gzipped seed lists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1170 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
dc778659fb
fixed problem with time-out during result joint which caused OR behavior instead of AND beahvior
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1167 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
3d8a5ae652
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1166 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
64478b1f02
*) Adding possibility to delete crawler queue entries using regular expressions
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1160 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a04930f025
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1158 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
low012
90b0eb144e
just a typo...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1155 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
129b15f3e1
*) Correcting logging output of db importer thread
...
See: http://www.yacy-forum.de/viewtopic.php?t=1555
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1154 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
420d56ce79
extended db-testing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1152 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
ecf765ec33
temporary fix to make jrpm extension compilable with my netbeans environment
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1151 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
8ed0aaae8d
*) Adding content Parser for RPM Files
...
- at the moment only the metadata is extracted
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1147 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
818d37ce44
*) Removing getSimpleName
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1143 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
b35c5a48bf
*) First version of urlRedirector.pl script
...
- with this script it's possible to pass URLs from squid
to yacy via the squid redirector interface
- this URLs are then used by YaCy to feed the crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1141 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bdf30117c1
*) Redesign of parser configuration
...
- restructuring of mimeTypes based on the parsers
- displaying parser usage count
- displaying human readably parser names
- displaying parser version information
*) httpdFileHandler.java
- adding possibility to support "streaming" servlets
which are special servlets that can communicate with
the client via the connection streams autonomous
- the name of these new servlet types must end with the
file extension .stream
- this feature will be needed by the yacy ScreenSaver
class to fetch statistic data from the peer without the
need to reconnect to the server all the time
*) Adding human readable names and version information for
all supported parsers
*) plasmaParser.java
- adding new structure to store parser statistic data
*) Adding openDocument parser
- can be used to parse odt files
*) jmimemagic
- adding rules to detect openDocument formats properly
*) serverLog.java
- adding functions that can be used to query if a given
logging level is enabled or not.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1140 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d4ac3e25b1
*) Bugfix for file system link bug during detection of invalid URLs
...
See: http://www.yacy-forum.de/viewtopic.php?p=13301
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1134 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
adf75bc9fa
better logging for invalid file path detection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1133 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
40621a5663
anhancements in ranking preparation and fixed problem with parser/mime recognition
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1132 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c650b112ea
*) Bugfix for relative URL Bug in Crawler
...
See: http://www.yacy-forum.de/viewtopic.php?p=13266#13266
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1130 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
4e73035aef
*) Bugfix for "too many open files" during index distribution
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1128 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
f57e2d67f5
shortened network overview (less columns fit easier on page)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1124 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
85282b1d98
enhanced YBR recognition and search result heuristics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1121 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
b9cc9029e3
added ybr selection for remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1119 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0e25020f51
added first generation and usage of YBR index-files. Enhanced overall ranking of search results.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1118 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
90d6c6223b
*) Adding color codes to network graphic legend
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1114 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
bfe51c7228
added generation of domain-list
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1112 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
0ec54d9c5f
enhanced CR-file handling and added first RCI-evaluation tests
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1110 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
c2fe3a1670
*) Updating jMimeMagic Ruleset
...
- to detect some special formated html documents correctly
- adding rule to detect vCards
*) plasmaParser now supports parsing of files that have a supported fileExtension
but a unsupported mimeType because the webserver has set it incorrectly to text/plain
*) Adding vCard new Parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1107 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
88e3234393
fine-tuning of rci-generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1105 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
a12759c1bf
first try to implement a rci-computation from cr-files
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1103 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
4a8e8f269e
refactoring of cr-processing; new kelondro class to handle the attribute file format
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1100 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
24dc0e0760
implemented cr-file processing and further transmission steps
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1099 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9d9a87f445
limited htcache storage length
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1096 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
d0dfccdb77
*) Making CrawlStacker pool configurable via GUI and config file
...
See: http://www.yacy-forum.de/viewtopic.php?t=1448
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1087 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3631cb1f6d
*) deleting empty entities during index selection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1086 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
ca26aab9b1
*) More debugging output for migrateWords
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1085 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
9b35ae9027
*) Correcting wrong % values on IndexTransfer_p page
...
See: http://www.yacy-forum.de/viewtopic.php?p=12646
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1084 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
e6bf9d90a5
*) Fixing Problems with MalformedURLs during Word Selection
...
- removing (lurl.toString() == null) comparison because toString() is never null
- adding (lurl.url() == null) condition because url() is null if we have selected a word entry with
a malformed URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1083 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
86a9210264
*) indexing queue slots are now configurable via config file
...
See: http://www.yacy-forum.de/viewtopic.php?t=1480
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1081 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
3c11d7b81c
*) Bugfix for minimizeUrlDB
...
- function didn't work correctly because of new url hash structure
See: http://www.yacy-forum.de/viewtopic.php?p=12753#12753
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1080 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
orbiter
9913049009
fixed outOfMemory bug caused by loops in kelondroTree during enumeration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1079 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
bbb936b9ea
*) Bugfix for not human readable content of PDFs while viewing the URL Content via GUI
...
- This Bug also affects the snippet generation on non html/text documents
See: http://www.yacy-forum.de/viewtopic.php?t=1472
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1075 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
445e3a620f
*) Avoid rejecting of html content by the crawler when the file extension is not set properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1074 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
theli
444a5a9368
*) Bugfix for Entries with null url in GlobalQueue
...
See: http://www.yacy-forum.de/viewtopic.php?p=12675#12675
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1069 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
ebac51df52
restore defaultRemoteProfile
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1063 6c8d7289-2bf4-0310-a012-ef5d649a1542
19 years ago
borg-0300
5778428455
move cutUrlText to nxTools,
...
max length from URLs(title) on searchpage now 120 chars
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1060 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
borg-0300
9158845c3b
bugfix for snippet text null bytes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1059 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
f763923e0a
added missing files for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1057 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
79818a320f
introduced citation-rank transmission protocol and activate transport for anonymisation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1055 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
7e0647f692
*) Bugfix for userDB usage during authentication
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1052 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
02f8013013
auto-delete of corrupted word files during word-migration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1047 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
orbiter
d2731418bf
added creation of global ranking files and changed url normal form usage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1046 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago
theli
6f9f8ed8f8
*) Automatic Reset of Stack Crawler DB on startup errors
...
See: http://www.yacy-forum.de/viewtopic.php?t=1432
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1045 6c8d7289-2bf4-0310-a012-ef5d649a1542
20 years ago