to always get fresh lists of documents. This is necessary since the
postprocessing changes the same documents which the
postprocessing-collection query selects.
hold a date for each URL to record when a url was first seen. This is
then used to overwrite the modification date for urls upon recrawl in
case that the first-seen date is before the latest document date. This
behaviour is necessary due to the common behaviour of content management
systems which attach always the current date to all documents. Using the
firstSeen database it is possible to approximate a real first document
creation date in case that the crawler starts frequently for the same
domain. As a result the search results ordered by date have a much
better quality and the usage of YaCy as search agent for latest news has
a better quality.
attached to the seed info. API clients must be adopted. Documentation
will be fixed in
http://www.yacy-websuche.de/wiki/index.php/Dev:APIseedlist
Also added a new retrieval option for seeds, they can now be retrieved
by their name with the get parameter name=<name>
This is the first element of a new 'decoration' component which may hold
switches for different external appearance parameters.
The first switch in that context is decoration.audio (as usual in
yacy.init). This value is set to false by default, that means the audio
feedback element is switched off by default. To switch it on, set
decoration.audio = true (using /ConfigProperties_p.html). You will then
hear sounds for the following events:
- remote searches
- incoming dht transmissions
- new documents from the crawler
Sound clips are stored in htroot/env/soundclips/ which is done so
because a future implementation will read these files using the http
client and with configurable urls which will make it very easy for the
user to replace the given sounds with own sounds.
*) will try other ports if YaCy standard ports are not available
*) distinguish between internal and external port (not sure if this
works 100%)
Still to add: propery in config to enter own external port (in case of
manually configured NAT)
tested with IE11 and Firefox 32 (change worked for both to show 2nd line without cutting off height)
+fix charset parameter in metadataImageParser
+update start errMsgTxt to "java 1.7"
used parameter &cat=image is obsolete and returns no results
- remove &cat=image and &cat=href references
- remove &tenant= references (unused)
Use contentdom=image and inurl: parameter to make showPicture link display something (open in new window because of used inurl modifier changes original query)
removed preferred IPv4 in start options and added a new field IP6 in
peer seeds which will contain one or more IPv6 addresses. Now every peer
has one or more IP addresses assigned, even several IPv6 addresses are
possible. The peer-ping process must check all given and possible IP
addresses for a backping and return the one IP which was successful when
pinging the peer. The ping-ing peer must be able to recognize which of
the given IPs are available for outside access of the peer and store
this accordingly. If only one IPv6 address is available and no IPv4,
then the IPv6 is stored in the old IP field of the seed DNA.
Many methods in Seed.java are now marked as @deprecated because they had
been used for a single IP only. There is still a large construction site
left in YaCy now where all these deprecated methods must be replaced
with new method calls. The 'extra'-IPs, used by cluster assignment had
been removed since that can be replaced with IPv6 usage in p2p clusters.
All clusters must now use IPv6 if they want an intranet-routing.
it is now possible to get the results in two steps:
- first retrieve all IDs as given for a query
- then retieve each document individually
This was necessary for very large result sets where a query may run for
hours and is possibly terminated by a solr-internal timeout. This occurs
regulary during postprocessing and therefore this commit may fix
unwanted postprocessing terminations.
done. This makes it possible to search all documents on a single domain
even if no search word is given. This is in particular interesting when
searching for all images on a single domain.
Most links in Network.html are only available with transparent proxy = on, which is switched off by default,
to make the provided links useable in default setup a small forward servlet added (goto_p.java),
which takes the peer hash as parameter and forwards to current public ip (optional with path= parameter).
The servlet is protected ( _p ending) to assure forwarding works only for authorized YaCy users.
not point to youtube directly to prevent that the referer to the peer
address is given to youtube. Instead, a forwarder address at yacy.net is
used to redirect to the tutorial repository (and can be changed later).
- remove verify=cacheonly from admin screen search box to use the configured values
(otherwise definition overwrites configured behavior and is used for following searches what might give unexpected/confusing different results compared to using /yacysearch )
string which is visible in the browser. That makes it possible that the
browser instructs the user how to change a forgotten admin password
(during runtime).
advanced settings to a top-menu entry. Moved the advanced setting to a
less prominent place of the submenu. Removed the table administration
from target analysis submenu because it appeared double, the table
administration is now the default in the system administration.
Sorry for inconvenience if i constantly move menues around, but this
makes just more sense and YaCy is still not finished :)
Michael Peter Christen