ad0da5f246
added new web page snapshot infrastructure which will lead to the
...
ability to have web page previews in the search results.
(This is a stub, no function available with this yet...)
10 years ago
2beb6abeb6
disabled crazy sleep loop
10 years ago
421ee64f33
another fix to ordering of table indexes; fixes also network stats
...
graphics
10 years ago
72561926aa
do not overwrite yacy.conf in case of an exception
...
may be a fix for http://mantis.tokeek.de/view.php?id=180
10 years ago
ec6082c872
very bad language detection hack fix hack
10 years ago
0ceeceb35e
more logic on Solr queries; usage of the query terms in posprocessing,
...
saving one query for double document detection now per document
10 years ago
97983ba89f
fixed generics warnings for generic array instantiation that appeared
...
after migration to Java 7
11 years ago
88f4af90da
removed warnings
11 years ago
17e0956312
refactoring of SystemLoad calls (only one backend tool)
11 years ago
8b44fcf0f4
added missing @Override annotation
11 years ago
acc8d7faa7
fixed setting of shortMemoryStatus in MemoryControl
11 years ago
40a4030b55
configurable max-load values for YaCy-Threads:
...
try lower values on smal systems like a Pi
11 years ago
2c39b65409
fixes for searches containing stopwords. The fix was done using a
...
reconstruction of the search word set access method to protect that
words are deleted from the sets from the outside of the QueryGoal class.
11 years ago
1a4a69c226
set more logger to 'final static'
11 years ago
5e31bad711
- the webgraph shall store all links which appear on a web page and not
...
all unique links! This made it necessary, that a large portion of the
parser and link processing classes must be adopted to carry a different
type of link collection which carry a property attribute which are
attached to web anchors.
- introduction of a new URL class, AnchorURL
- the other url classes, DigestURI and MultiProtocolURI had been renamed
and refactored to fit into a new document package schema, document.id
- cleanup of net.yacy.cora.document package and refactoring
11 years ago
cb85b22725
redesign of the image search process (with much better results,
...
unfortunately the index schema has changed and p2p image search will not
be muchmuch better until many people update)
11 years ago
47b1c81d08
- refactoring
...
- generalized writing of url attributes to solr documents
- added more url attributes to error documents
11 years ago
841a28ae76
Added 'final' for all exception blocks as this helps the Java compiler
...
to optimize memory usage
Conflicts:
source/net/yacy/search/Switchboard.java
11 years ago
5878c1d599
- refactoring of log to ConcurrentLog:
...
jdk-based logger tend to block
at java.util.logging.Logger.log(Logger.java:476) in concurrent
environments. This makes logging a main performance issue. To overcome
this problem, this is a add-on to jdk logging to put log entries on a
concurrent message queue and log the messages one by one using a
separate process.
- FTPClient uses the concurrent logging instead of the log4j logger
12 years ago
7480e87386
- fix stopword handling for RWI see example http://bugs.yacy.net/view.php?id=247
...
- append language setting specific stopword list
- remove unused OVERHANG stack type
12 years ago
c1a2175fbc
added transparency to gif image animation and the integration to the
...
YaCy httpd for on-the-fly generated gifs (including animated gifs)
12 years ago
75f9568472
*) only install files from the RELEASE directory
...
*) minor changes
12 years ago
3bc5ee6e3d
*) added protection against CSRF in update download page
...
(http://localhost:8090/ConfigUpdate_p.html?releaseinstall=../../test.txt&deleteRelease=Delete+Release
does not work anymore)
12 years ago
5c0c56cfe1
Preparations to produce a click depth attribute in the search index.
...
This attribute can be used for ranking and for other purpose (demand by
customer)
The click depth is computed in two steps:
- during indexing the current fill-state of the reverse link index is
used to backtrack the current page to the root page. The length of that
backtrack is the clickdepth. But this does not discover the shortest
click depth. To get this, a second process to check again is needed
- added a process tag that can be used to do operations on the existing
index after a crawl; i.e. calculation the shortest clickpath. Added a
field to control this operation but not a method to operate on this.
- added a visualization of the clickpath length in the host browser
12 years ago
c5f67a5d6d
fixed a problem with local search from solr results: now all results
...
from solr are shown (again)
12 years ago
f8f05ecba7
- added a delete button in host browser to delete a complete subpath
...
- removed storage of default collection name - default is now "user"
- made stacking of crawl start points concurrently
12 years ago
f2d0418218
because the new PngEncoder had a problem with the PixelGrabber which is
...
caused by a JRE bug, the PixelGrabber had to be circumvented using an
own frame buffer which can be read without a PixelGrabber. This resulted
in ultra-fast and much less memory-consuming transformation. YaCy images
are now generated really fast!
12 years ago
b400fc7b4d
fix for file parser problem
12 years ago
6017691522
added an exception catch
12 years ago
613cf7da7f
enhancement to post argument parsing - possible fix to zero-filled
...
parameter values
12 years ago
a06930662c
replaced some more .getBytes() with UTF8/ASCII.getBytes()
12 years ago
a8167e6e5b
clean-up: removed unused methods in kelondro
12 years ago
8219a445f3
refactoring
12 years ago
00c1c777fa
refactoring
12 years ago
563d584420
removed more dependencies in cora from kelondro
12 years ago
4d29f59a27
removed warnings
12 years ago
2094df2e4e
- correct length computation for BStringObject (bugfix suggested by
...
apfelmaennchen)
- using ASCII for string conversion for Strings generated from Integer
12 years ago
1687737771
Abstraction of HandleMap and HandleSet
12 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
b0c408788b
made class methods static where possible
13 years ago
d3964253ae
- added @SuppressWarnings to unused servlet method parameters
...
- removed unnecessary casts
- removed unnecessary throw statements
13 years ago
ea10766bfd
cleaned unnecessary nested code
13 years ago
ce8d4b87d9
fixes for new eclipse 'Juno' warning 'Resource leak'.
13 years ago
b9d42fd9c8
using com.google.common.io.Files instead of homebrew methods
13 years ago
3b992e6b00
using utf8 String compression in Webstructure database
13 years ago
701b9a28a0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
htroot/PerformanceMemory_p.java
13 years ago
10c9c17d51
fixed handlemap spread factor and null iterator handling
13 years ago
461a0ce052
removed warnings
13 years ago
e0d8643226
- performance hacks
...
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
13 years ago
10da7335ea
performance hack: use a hash cache for all hashes that are computed by a
...
byte array. If this hash is used in a HashMap (which is very often the
case) then this hack eliminates a lot of re-computations of the same
hash.
13 years ago
Michael Peter Christen