Commit Graph

237 Commits (e37709219828ef321e5784ec967d856cc5758fb0)

Author SHA1 Message Date
Michael Peter Christen 99c74699de removed scroogle (scroogle is dead)
13 years ago
Michael Peter Christen 8bee1472c9 there is no noindex, only nofollow in links
13 years ago
Michael Peter Christen 4c5edab1ec added option to have exception search result windows
13 years ago
Michael Peter Christen 696ee5fc16 removed pdf from default parser deny list
13 years ago
Lotus c73af39e54 refactoring of tray icon class,
13 years ago
Michael Peter Christen 987b412491 updated solr scheme: generic declaration of solr schemes
13 years ago
Michael Peter Christen 0bcef2d156 added feature as requested in
13 years ago
Michael Christen 17f962fceb translator updates:
13 years ago
Michael Christen c715d19c09 fixes for dependency on svn
13 years ago
Michael Christen f62e6fb438 less frequent DHT distribution to reduce the load a bit on every peer
13 years ago
Michael Christen 9dbc93613e now that the whole world knows that we actually do p2p and not
13 years ago
orbiter f9216e388c - faster ping to clean up old peers faster
13 years ago
orbiter ac5bda205f - removed lower page navigation (it never looks nice)
13 years ago
orbiter c659310e89 - removed option to search for audio, video and applications. These things are still experimental and should not be shown to new users since this would cause them to argue that YaCy does not work. The functions are stil available, because:
13 years ago
orbiter 6cd27473f5 - better default values for caching and cache usage
13 years ago
orbiter 5866c73a09 fix for compare search: use scroogle instead of bing and get a default search if configured search engine is not available
13 years ago
orbiter e4a82ddd8b produce a bookmark entry from every crawl start. these bookmarks are always private.
13 years ago
orbiter f183d3822c added a default accept header in http requests since some http fraud detection functions check that this header field exist
13 years ago
orbiter 78ce3b13be typo
13 years ago
suessthomas 887f088dad The IP address of the YaCy-Demo portal added to Whitelist.
13 years ago
orbiter 1b45e33f04 added robots tag parser to solr scheme
13 years ago
orbiter cf4fd525ee added directDocByURL attribute in crawl profile
13 years ago
orbiter 5ad7f9612b added crawl settings for three new filters for each crawl:
13 years ago
orbiter 2c3161b4ac refactoring:
13 years ago
orbiter 6b22865dbc - removed some warinings
13 years ago
orbiter e48ce5d80e - style change for search box: larger font, selected by default
13 years ago
sixcooler ecb4986b38 refactored stuff from last commit to ReferenceContainer
13 years ago
orbiter 49e5ca579f added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled.
13 years ago
orbiter 9a8937f8b6 be more liberal when evaluating search results. This may cause that it is possible to fraud content on fresh peers, but that is better than looong waiting times for the evaluation of every link which causes that everybody rejects YaCy as 'too slow'. But this is only because of the high standards that YaCy sets to itself. If we are able to gain more users by lowering the standard, then that is useful. The option to set that flag to verify each link is still there.
13 years ago
orbiter 1c007188ad bugfixes in html parser
13 years ago
orbiter 5dd2efc9a2 - bugfixes in html parser
13 years ago
sixcooler 4fec99115b Implementation of strategies for controlling memory resources.
13 years ago
orbiter 77a9af99f1 same values for Xmx and Xms: memory extension may be difficult if the OS has not the remaining memory available and may kill the jvm. If the memory is reserved at the start but never used the OS may handle that as well and leave non-used space in swap area (and never swap)
13 years ago
orbiter 768c59740c - replaced solrj 3.1 with solrj 3.3
14 years ago
orbiter e7c7598923 docfix
14 years ago
orbiter b84089ff04 fix for solr scheme list definition
14 years ago
orbiter 2d4bb139d3 - added counting of links with noindex tag for solr index
14 years ago
lotus fa6f2c2b44 use proxy accounts by default for more security
14 years ago
orbiter bda3eec0ff added parsing of canonical link element to html parser
14 years ago
orbiter b6f09a475d - added an index profile editor in the /indexFederated_p.html servlet for solr indexes
14 years ago
orbiter 6deef60bc0 added keyword list for solr index attributes
14 years ago
f1ori fdc84d8319 small pi link on index page to administration pages
14 years ago
orbiter 84c9658644 added a file type navigator
14 years ago
suessthomas 66c477129e Creates a new network definition, yacy.networks.metager.unit.
14 years ago
f1ori 900dacbf97 * improve link rewriting in proxy-url
14 years ago
orbiter cc239b18cd fix for IPv6 localhost proxy client
14 years ago
orbiter 10e2f588f8 - enhanced ybr ranking computation
14 years ago
orbiter 3ed4a09368 small features, some bug fixes and performance hacks
14 years ago
orbiter d8e934c085 better abstraction of http client identification
14 years ago
orbiter b77b8cac0c - enhanced html parser: recognized much more details in the content
14 years ago