low012
19e7c56f7f
*) apply filter to dir list to only show .black files as blacklists
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5438 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c4c4c223b9
fixed a problem with attribute flags on RWI entries that prevented proper selection of index-of constraint
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5437 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
4bffe664ca
*) moved entry field for new expressions to top of the list as requested in forum ( http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1678 )
...
*) added some Javascript to disable list selection on bottom of list in cases it is not needed (edit, delete) and only enable it if needed (move), if JS is turned off everything will work as usual
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5435 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
9d5d30f877
*) http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1672
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5422 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
5448aad328
removed unused code
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5421 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
28d2d28573
added support for filetype search
...
(just use filetype:<type> in the search query)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5418 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
78c568331e
added test channel to /xml/feed.rss
...
can be obtained with
http://localhost:8080/xml/feed.rss?set=TEST
returns always a single feed entry with a fresh date
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5416 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e004da48d3
- added fast fingerprint computation for files (any). Will be used in new index dump method
...
- refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5415 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
eab72424df
*) Fixed small bug: When adding new elements to blacklist via import, the blacklist which the elements were added to was supposed to be displayed, which did not work correctly.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5414 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
0e56675596
*) cleaning up ;-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5413 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
cf69557ea2
*) blacklists can be exported as XML or plain text now
...
*) blacklist import via file upload works now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5412 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
1594a15be9
*) explicit mentioning of blacklist in blacklist cleaner
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5411 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
5a89266598
*) new parameters for future use (better blacklist handling for im- and export)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5403 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e34ac22fbd
- added new monitoring servlet at
...
http://localhost:8080/PerformanceConcurrency_p.html
- used the new monitoring to do some fine-tuning of the indexing queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5402 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d376d81fc4
replaced busy thread control of crawl stacker by blocking threads
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5400 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7535fd7447
- refactoring of CrawlEntry and CrawlStacker
...
- introduced blocking queues in CrawlStacker to make it ready for concurrency
- added a second busy thread for the CrawlStacker
The CrawlStacker is multithreaded. It shall be transformed into a BlockingThread in another step.
The concurrency of the stacker will hopefully solve some problems with cases where DNS blocks.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5395 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
6569cbbec1
npe fix: http://forum.yacy-websuche.de/viewtopic.php?t=1646
...
(break to avoid bad side effects)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5394 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2802138787
- refactoring of CrawlStacker (to prepare it for new multi-Threading to remove DNS lookup bottleneck)
...
- fix of shallBeOwnWord target computation heuristic
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5392 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b1e211b258
no error-alert: http://forum.yacy-websuche.de/viewtopic.php?t=1639
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5391 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
13cb0916ee
changes to statistics and content of thread dump servlet
...
(points now more directly to performance leaks without mentioning class calls inside of sun/java calls that cannot be changed anyway)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5390 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
e1acdb952c
fix for problem with userDB and bookmarksDB which was caused by changes in kelondroRA in SVN 5376
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5385 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
e918d64c23
show hand-cursor an labels
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5383 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
4a2dac659e
more speed hacks:
...
- modified and activated write buffer
- increased cache flush factor
- fixed a problem with deadlocking of indexing process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5382 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
1fb518a5b4
display <String> etc.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5380 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
47292e696a
more performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5379 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bd1dc9cd5d
thread dump with statistics, a little bit of profiling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5377 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d39d420b39
performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5376 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5280ad638d
added basic performance page
...
other performance settings can be found on advanced settings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5375 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
1a51d9fcfd
display proper values
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5374 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0b4808ba3d
added new interactive search feature:
...
- during the user types search queries, the local database is searched
- results are presented interactively
This was implemented using a new JSON result format for search results in YaCy
- added JSON as file format for servlets
- refactoring of current search servlets (xml and html)
- added JSON output format for search results
- added AJAX-based search page, that uses the yacysearch.json selrvlet to print results as a query is typed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5373 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fea82b54ef
more contrast on search snippets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5370 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
1951d30a62
addendum to last commit
...
handle words with length < 3 correctly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5369 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
325ba7bfb8
only query words with length > 2
...
this is not complete, yet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5368 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
489edb4473
improved pattern selection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5367 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
e423fa9846
*) added method to only get file names in directory listing which match a filter
...
*) only files which end with .black will be listed as blacklists
*) added a little bit of Javadoc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5366 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
577b53aee6
added more search engines
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5365 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
7f4d411c0d
npe-fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5364 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
1545e5440a
* index deletion: checkbox-confirmation
...
* watch crawler: less load on exhausted peers; wait for data before reloading again
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5359 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
10f5ec1040
reverted last commit (more testing needed)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5356 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
ba5b274b8c
#translation update:
...
-blacklist
-crawlstart
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5353 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0ca4bc7b79
- added reader and visualization for mediawiki-export files:
...
files exported from mediawiki using the xml schema according to
http://www.mediawiki.org/xml/export-0.3/
can be processed to be viewed in a YaCy servlet.
To acces such a file, place it into
DATA/HTCACHE/mediawiki/
i.e. the export from german wikipedia would be:
DATA/HTCACHE/mediawiki/wikipedia.de.xml
This file can then be accessed using the URL
http://localhost:8080/mediawiki_p.html?dump=wikipedia.de.xml&title=YaCy
if this is done the first time, an index file is created
(for this case: more than 4 million lines must be written, this takes about 15 minutes)
Then try the same url again.
- enhanced also the md5 computation speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5352 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
4f996a7651
fix for logparser pattern
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5349 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
867d0f2f56
removed some unnecessary pause delays
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5346 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fd83e59f8e
new remote search average
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5343 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dba7ef5144
extended crawling constraints:
...
- removed never-used secondary crawl depth
- added a must-not-match filter that can be used to exclude urls from a crawl
- added stub for crawl tags which will be used to identify search results that had been produced from specific crawls
please update the yacybar: replace property name 'crawlFilter' with 'mustmatch'.
Additionally, a new parameter named 'mustnotmatch' can be used, which should be by default the empty sring (match-never)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5342 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0ae84f4f8e
set some default values for a crawl start that should cause less confusion and mistakes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5334 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
4745e89451
auto-choose crawl type
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5331 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
421d056550
*) changed layout of blacklist adminstration (less cluttered)
...
*) it is possible to move/edit/delete more than one entry at a time now
*) it is easier to choose a target for blacklist import now
*) fixed several bugs
*) to be continued...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5330 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
674ad2d55b
different handling of error cases that occur during loading files with http or ftp:
...
methods throw exception instead of returning an error string
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5328 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
ae80f3e6a5
* extend opensearchdescription to support compare_yacy.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5324 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1b18d4bcf3
enhancement to crawling and remote crawling:
...
- for redirector and remote crawling place crawling url on notice queue instead of direct enqueueing in crawler queue
- when a request to a remote crawl provider fails, remove the peer from the network to prevent that the url fetcher gets stuck another time again
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5320 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3f746be5d4
- consolidation and refactoring of many DHT target - computing methods
...
- implemented vertical DHT acceptance ("my own DHT") to accept new targets
- added new target computation for global search: addresses vertical targets also
- enhanced remote crawling: collection of remote crawl urls if queue has less than 100 entries (was: 0 entries)
- better performance value computations for PPM selection in network configuration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5319 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d014b2728a
Design-check, Extension and Refactoring of DHT target position computation:
...
- two different computations (but mathematical equivalent) of the DHT distance had been consolidated
- moved from 0.0 .. 1.0 double-range position computation to 0 .. Long.Max range for DHT targets
- added fast Long - to - hash computation
- high-precision target computation of gaps for new peers
- added new target computation for horizontal and vertical DHT targets (not yet in use)
- old horizontal-only DHT targets will be upwards compatible to new horizontal and vertical DHT positions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5318 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
83967f8c77
*) servlet does not forget chosen blacklist anymore when editing, moving or delting an entry
...
*) move or edit will only be performed if new value actually differs from old one
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5309 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
04e41a392f
*) fixed bug where RegExes were not deleted and even added to the list a second time when the user tried to edit them
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5308 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
7bac4796d2
*) added servlet which returns all shared blacklists of a peer without information about which part of YaCy (crawler, proxy, ...) blacklist is activated for (to be used for better online import)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5306 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
baae3d91b1
*) fixed warning when compiling listManager
...
*) fixed display of values of information for which part of YaCy (crawler, proxy, ...) blacklist is activated for
*) replaced regular put() with putXML() in several cases
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5305 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
444575e33d
*) prevent XSS when importing blacklist
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5304 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
a99a629ed4
*) quick fix to prevent comments for blog entries which don't exist ( http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1554 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5302 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
00e27e5050
*) fixed bug which made it possible to write files outside of the DATA/LIST directory when creating a new blacklist
...
*) a blacklist will only be created if no blacklist with same name exists (some refactoring has been necessary for this)
*) further minor fixes
*) to be continued...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5301 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0edec2b760
FULL redesign of algorithms in htmlTools to encode/decode strings from/to unicode and html.
...
The old process used a not really efficient way to detect html encoding strings in texts.
All calling methods had been adoped to call the new class in an enhanced way with less parameters.
Many classes in interfaces used a XML encoding only (instead of full html conversion from unicode to html); this behavior was not changed with this commit but should be controlled again since it points out possible XSS leaks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5295 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
47f0c3b002
replaced the cacheAdmin with the ViewFile servlet, because the cacheAdmin was an interface to the old HTCACHE data structure which does not exist any more. Changed links to point to the ViewFile servlets.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5289 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1778fb420d
- added some performance tweaks to the new BLOB buffer
...
- removed the now superfluous HT storage thread
- reduced number of file decompression by shifting the compression moment to the future
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5286 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
77e41da7d2
*) further propagation of display value (see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1536 )
...
*) removed another depreciated parameter "time" which led to ugly -UNRESOLVED_PATTERN- in URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5285 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
ff46ce8520
*) fixed display=2 (see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1536 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5283 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
826ca79735
refactoring and new architecture to store the files of the web cache:
...
- files are not stored any more as individual files
- a new database structure using BLOBHeap files stores many cache entries in common files
- all file-writing procedures had been migrated to generate byte[] objects which are written with the new database methods
this is only an intermediate step to the final architecture, where cached files are written together with their metadata in one single database structure.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5276 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
2b18a9b2c4
*) removed depreciated parameter "time" which led to ugly -UNRESOLVED_PATTERN- in URL
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5275 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7860d5d632
fix for bug in seed list management (cause was bad class overloading, only visual effects!)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5265 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
603282bcf4
fix for out of bounds exception
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5264 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ffed5fc415
fixed problem with lost peers in database
...
migrated seedDB from BLOBTree to BLOBHeap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5263 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
736dd86193
- option enableSimpleConfig can disable hidden tables
...
- corrected some Xmx values
- friendlier welcome message format
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5259 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ff68f394dd
fix for problem with balancer and lost crawl profiles:
...
if crawl profile ist lost, no robots.txt is loaded any more
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5258 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
3717d2057a
YaCy-UI: fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1483
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5251 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d0bdcdd57c
small changes to attributes of DoS attack protection parameters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5246 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
9ac16f565b
- fixed several bugs in database management functions
...
- fixed a display bug for the performance graph
- fixed deadlock when initialization of awt happens simultanously
- removed some debugging output
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5245 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
7fdf65339d
system status dropped into next line if seed server was enabled. display needs about 230px, set fixed width again.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5243 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
7782a43060
fix if LANGUAGE: was not defined and the end of the query
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5242 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
2c23e6ad34
YaCy-UI: added some features for Admin Console
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5241 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
efcae14714
YaCy-UI:
...
- added 'Open' button to search result toolbar
- lets you open all selected search results in new window/tab
- added 'any language' filter as default
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5240 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
902a0d0f38
fieldset of system status was bigger than defined space. IE overlapped some text.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5238 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
820a03f9d6
- removed some warnings
...
- used fix in SVN 5233 for ysearch.java and search.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5237 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
69925a7e91
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1441
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5236 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
fe2792e9ce
use accept-language header instead of user agent for language detection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5235 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
e5904e6a21
removed color definition for input elements in default skin
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5234 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
93ddf206e6
opensearch fix if user agent had no language
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5233 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b8538fae04
search form like on result page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5232 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
3cce13d1b7
more compact search form
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5231 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
95fddf056c
- better support for narrow windows on searchpage
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5230 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c8bdd965ec
- larger update time for status page
...
- balancer writes cause of robots.txt in log file for crawl delay
- removed log output for forced GC
- smaller RAM flush for RWI cache, should cause more usage of cache and faster crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5228 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
3a919bf24e
better solution for search result layout
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5227 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
f95ec8b813
fix for non-accessible 2nd-line tabs on admin console
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5226 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c44e97d6dd
more lines in log on status page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5225 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
dc149df3b1
new status page layout:
...
- smaller kaskelix image to make room for more information
- added the memory graph, since this picture is widely used to monitor YaCys activities
- added border to log line iframe (looks better together with memory graph)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5224 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
daburna
298196e7a4
translation update
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5223 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
dda771db9d
- search result layout
...
- tray only for windows
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5222 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
7f88dadf87
YaCy-UI: added a notice to a bug in bookmarks folder recursion!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5221 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
2126315d40
layout changes for search result page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5220 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
f7003aa75f
*) updated whos.amung.us code in terminal page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5217 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
cc9f1e9dd9
YaCy-UI: enabled edit bookmarks in toolbar
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5215 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
00c843f7d9
YaCy-UI:
...
- added scrollbar to folder widget
- slightly improved Ajax handling for folder and tag widget
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5214 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
9d4f0325e1
- removed shutdown from search page (we have it in tray now!)
...
- fixed doubleclick action for tray
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5211 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
f293554216
XBEL: small bug fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5208 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
848dbbd280
YaCy-UI:
...
- added bookmark folder widget to sidebar (it uses the xbel.xml with XSLT!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5206 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
7462b5db7b
/xml/bookmarks/xbel/xbel.xml:
...
- added support for xsl-style sheets: style=/path-to-xsl/style.xsl
- removed fixed root folder "YaCy Bookmarks"
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5205 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
6e7d113eac
fix for wrong index initialization after network switch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5203 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7b35d54c6c
fixed some problems with network switching (was not completely 'clean')
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5200 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
8b6a0376db
YaCy-UI:
...
- another XSS fix (TagCloud)
- upload of missing icons for toolbar
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5198 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
2ff6f0360e
YaCy-UI: small fixes to accomodate Safari & Opera
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5196 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
a28af9d4c4
YaCy-UI: enabled language selection - special thanks to orbiter
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5195 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
1d9f0234c8
YaCy-UI:
...
- added basic functionallity to add bookmark(s) from search results
- re-design of sidebar search form
- replaced classic search with compare search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5194 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1198eeecc7
added language selection to search query:
...
- the language can be selected using a LANGUAGE:<language> element in the query line, i.e.:
java LANGUAGE:en
- the language can be selected with a post element in google-style syntax with the 'rl' element:
?lr=lang_en&query=java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5193 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
00c1535f84
added ranking and evaluation of language type in a search
...
the wanted language is taken from the browser user-agent string
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5192 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e201ad0e9f
YaCy-UI:
...
- moved search result to flexigrid
- added a toolbar similar to YaCyBar
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5191 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
06c07f2e0f
YaCy-UI: fixed broken style for add bookmark dialog
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5190 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
a5f61e8b1b
YaCy-UI:
...
- fixed xss vulnurability
- improved add bookmark dialog
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5189 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
a81cb78211
finally some putHTML on htroot/xml/
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5188 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bfcf9b7aa3
- added language detection using metadata from documents: html and odt documents provide this information
...
- metadata and results from statistical analysis are compared and result is printed out as debug lines
- added ranking profile for wanted language
- added class with ISO 639 table, a list of all valid country codes that will be used for the language identification
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5187 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
3768a1bd32
set encoding="UTF-8" for getpageinfo_p.xml
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5186 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
5e8bd0f29c
small fixes to getpageinfo_p.xml and htmlFilterContentScraper.java with respect to keyword extraction
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5185 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
029e16b653
replaced some put(String, String) by putHTML(String, String) on serverObjects respond
...
in htroot/ root
didn't touch htroot/xml/
this should solve potential xss issues
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5184 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
5b2a57bfd0
- /xml/util/getpageinfo_p.xml added <desc> and <lang> tags
...
- changed htmlFilterContentScraper.getKeywords() to split either space or comma charater not both
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5183 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ea5de7436d
added Sciencenet to the compare search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5178 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1bbf362cef
update to the crawl balancer: better organization and better crawl delay prediction
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5176 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ddcf285499
- fixed a bug in performance setting (did not work with german translation)
...
- reduced maximum number of error url references to save some memory (this was actually a small memory leak)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5174 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
0cd0fee546
fixed bug with wrong proxy result enqueueing. See:
...
http://forum.yacy-websuche.de/viewtopic.php?p=8130#p8130
- removed the online status property. This influenced the proxy behavior and created some complexity that was not needed because the online status was never used as it was ceated for (offline browsing)
- checked all proxy identification procedures during crawling and enhanced transparency and error checking
- fixed a proxy identification routine that caused the wrong selection of the proxy result queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5173 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e071f759d2
YaCy-UI: small optical changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5172 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
bbacf86fe8
- added /xml/bookmarks/posts/add_p.xml
...
- security fix to /xml/bookmarks/posts/delete_p.xml
- YaCy-UI: added 'add' and 'delete' bookmarks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5171 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
cd1ac5bb90
- fixed security issue with /xml/util/ynetSearch.xml
...
- hopefully fixed YaCy-UI local search with async=false for Ajax-request
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5170 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
c73cf05ddd
tried to fix local search in yacy-ui
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5169 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
99ff478d63
YaCy-UI: small optical fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5167 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
7e24c51fd5
- removed alternative search page in main menue in favor of rich client search page
...
- added necessary option to search request of yacy-ui to get snippets
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5165 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
7c5867a832
Major update to YaCy-UI
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5162 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
73f233bb11
* set resource observer to 1000MB
...
* transparent favicon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5160 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
1be24158a2
small fix / rendering option
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5159 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
105be67998
- some bugfixes to compare search
...
- redesigned input boxes: smaller, more space for result page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5158 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
693fa2a157
- renamed Comparison to compare_yacy
...
- added more search engines
- some refactoring and added a list that is used to present the search engine list in a specific order
- added simpleheader and no-header options
- added the compare search to the simple header
- added default compare search page selection storage - after re-start you get the same default search engines as you selected before
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5157 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
6216105ca5
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5150 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5e5178b5e8
please use putHTML to avoid XSS
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5149 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
b33a6cbb77
*) less disturbing elements in yacy frame
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5148 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
98d902b972
* remember last searchwords in Comparison_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5147 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bd45c5a2bc
integrated the comparison page into the main menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5146 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
5e0390a24c
*) Ooooooooops!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5145 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
dc56c35289
*) added page to compare results of 2 search engines
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5144 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
880d1a83e2
do not change memory and some non-defined tasks with performance profiles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5142 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b68d06a6e8
performance settings based on network's remote crawl speed
...
removed some _pro values from config
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5134 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
bb5c898441
enhancements to localsearch behavior
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5131 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
3c6e8d2015
set default ppm when network is switched
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5127 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
20c2d3c248
fix for bad formatting in CrawlResults
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5126 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
b92105c8b0
do not change auto recrawl scheduler with performance profiles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5123 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
d09ddabd09
corrected a design mistake (5-byte hashes not necessary)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5119 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago