882d54067a
added dummy update servlet
13 years ago
1533bfd63b
refactoring
13 years ago
e49359cc95
removed tenant query attribute since it is not used any more and is
...
replaced by the site-operator in the GSA interface. This operator can
also be simulated in the Solr interface using the collections_sxt field.
13 years ago
872f83ebe0
refactoring
13 years ago
15ea053c3a
- added xml output in IndexControlURLs to get the storage page of index
...
dump commands
- adjusted the apicall.sh script to get the downloaded text as output to
stdout which is necessary to parse the content out of it
- added indexdump.sh script which creates a solr dump and prints out the
storage path for the index dump
- added synchronization to the Fulltext class to prevent that data is
stored to a non-existing solr index while this index is disabled during
the storage of the dump
13 years ago
1b474139dd
used the new zip writer/reader to add a solr dump process: the whole
...
solr index can be written to a zip dump and also restored during runtime
13 years ago
e57bf2ca39
simplified DHT classes
13 years ago
14897d4bfc
fixed mistake in wt-option which caused that the yacy json format
...
overlapped the solr built-in json format
13 years ago
8219a445f3
refactoring
13 years ago
fa7f6f0be8
added HostBrowser servlet (stub)
13 years ago
00c1c777fa
refactoring
13 years ago
563d584420
removed more dependencies in cora from kelondro
13 years ago
63762d8f89
removed kelondro dependencies from cora
13 years ago
089a03114e
full memory usage for debian and when changing the size: debian seems to
...
dislike the big difference between xmx and xms (I have crashes here
which stop if both values are same)
13 years ago
60b1e23f05
added new crawl options:
...
- indexUrlMustMatch and indexUrlMustNotMatch which can be used to select
loaded pages for indexing. Default patterns are in such a way that all
loaded pages are also indexed (as before) but when doing an expert crawl
start, then the user may select only specific urls to be indexed.
- crawlerNoDepthLimitMatch is a new pattern that can be used to remove
the crawl depth limitation. This filter a never-match by default (which
causes that the depth is used) but the user can select paths which will
be loaded completely even if a crawl depth is reached.
13 years ago
6ec02deec6
added new crawl attributes in crawl profile (not active yet)
13 years ago
a13e5153ac
- added the possibility to have not one but a list of crawl start urls
...
- the list of urls is entered in the expert crawl start in a textfield;
the one-line input field was replaced with a text box
- start urls can also be given in one single line where the urls are
separated by a '|'-character
- as an effect, the crawl profile cannot carry a single start url for
identificaton because it is possible to have more. Therefore the url was
removed from the crawl profile
- this affect all servlets which display a crawl profile: removed the
url field from all there servlets
- to work consistently with several start urls and the other crawl
starts which computed crawl start url lists from sitelists or sitemaps,
the crawl start servlet was restructured completely
- new rules for must-match patterns were created to make it possible
that site crawl starts also work with several crawl starts at once
13 years ago
975bc95ddf
added default facet fields for json response format (stub)
13 years ago
2f218df55d
added missing license headers
13 years ago
a30653a864
added a regular expression test servlet which is linked within the
...
parser/crawler error page whenever a problem with regular expression
occurs.
This makes it easy to correct and enhance the must-match and
must-not-match patterns just by trying out which pattern could be
correct.
13 years ago
0504b01bdc
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
a55e77a115
added twitter search heuristic
13 years ago
e54ac38095
- some corrections in usage of getFile() and getFileName()
...
- added more attributes in json response writer according to yacy
servlet
13 years ago
9644c186a4
added search functionality to ViewFile.html servlet
13 years ago
b69ed96f0b
- added collections to yacydoc
...
- changed yacydoc.htm to yacydoc.json
- added query logging in solr and gsa search result
13 years ago
5df553c152
- added a json writer for solr (yes there was one using xslt but this
...
one writes the same way as yacysearch.json)
- using the new json solr result to change the ajax search in
IndexControlURLs to the new solr search
13 years ago
4d29f59a27
removed warnings
13 years ago
8c099d2106
Merge remote-tracking branch 'origin/master'
...
Conflicts:
htroot/api/ymarks/import_ymark.java
source/de/anomic/data/ymark/YMarkEntry.java
source/de/anomic/data/ymark/YMarkTables.java
13 years ago
59bd478ed1
Added more sophisticated RDF output for YMarks, including the folder
...
structure (b:Topic) and support for multiple tags (dc:subject) and
folders (b:hasTopic) via rdf:Bag container.
13 years ago
d31a632951
- added dmoz RDF dump importer
...
- added indexing to Tables columns to support larger bookmark
collections
- added RDF output (HTTP) for public bookmarks at /YMarks.rdf
- YMarkRDF also provides a Jena RDF Model as "internal" API
- various other changes/fixes for YMarks (mainly backend)
13 years ago
66ac4076c2
added disjunction '|' option to site parameter in GSA API
13 years ago
9ee2e09983
statistics for solr-cache
13 years ago
d8425e6809
added collections to crawl monitor
13 years ago
4b36a2c3b4
small style changes
13 years ago
8ca842b137
added new button design to more buttons
13 years ago
b2b516cc3e
added a collection attribute to crawls and searches:
...
- a solr field collection_sxt can be used to store a set of crawl tags
- when this field is activated, a crawl tag can be assigned when crawls
are started
- the content of the collection field can be comma-separated, all of
them are assigned to the documents when they are indexed as result of
such a crawl start
- a search result can be drilled down to a specific collection; this is
currently only available in the solr interface and also in the gsa
interface using the 'site' option
- this adds a mandatory field for gsa queries (the google api demands
that field all the time)
13 years ago
174530a9e0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
43f3a932fd
removed jquery.slider as it is already included as part of jquery-ui
...
package
13 years ago
a01eb1b7fe
removed unused jquery plugin slider as it is part of jquery-ui package
13 years ago
f75b3f8a47
added more patches to work without RWI data structure
13 years ago
a427a68bac
removed many warnings
13 years ago
c72c435517
- moved the gsa search interface from /gsa/searchresult? to /gsa/search?
...
- fixed the NB field data
13 years ago
31d4d38804
- extended the solr interface by a references-by-word-count method
...
- reduced danger that a non-existing RWI database causes NPEs
- added Solr queries to did-you-mean: this makes it possible that our
did-you-mean algorithm works together with only Solr and without RWIs
13 years ago
528d6763fa
- added new solr fields:
...
title_count_i, title_chars_val, title_words_val
description_count_i, description_chars_val, description_words_val
- added many asserts to ensure data type correctness from YaCy to Solr
and vice versa
- made many fixes according to new findings from these asserts (!)
13 years ago
3142e675e8
fixed problems with GSA api:
...
- better FS attribute
- highlightning of searched words in title
13 years ago
3b19fe7b52
- fixed num parameter in GSA api
...
- changed FS attribute in GSA api
13 years ago
75d5e3475d
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
dc468dad01
add content control features for custom filter lists
13 years ago
316b5fe116
- added a solr type definition verifier
...
- fixed type definition found by the verifier
- added multivalue-string fields for solr with extension 'sxt'
- added multivalue-integer fields for solr with extension 'val'
- renamed some solr attributes from txt to sxt
- changed solr query line to an explicit AND/OR structure
- added a country code second level domain list to Domains class; with
parser
- added a host string parser to get domain class name, country-code
second-level domain and subdomain out of it
- removed old coordinate attributes
13 years ago
2d2be546fe
fix path to env/grafics to display api icon on meta data page
13 years ago
7ac259477f
added a direct access to solr search api to enhance the visibility if
...
the embedded solr
13 years ago
67f2866cd0
small fixes
13 years ago
479bfca571
refctoring
13 years ago
48a82bc705
log queries anonymous from gsa+solr requests
13 years ago
ab6ec4ec52
added snippet computation to solr/rss and gsa result writer
13 years ago
4716546ef5
- reduced memory usage in index transmission using a transformation of
...
Node to Row objects
- removed peerDeparture in solr remote search in case that peer does not
answer (this may be normal because it is allowed to switch this off)
13 years ago
0ad52ac4c3
gsa bugfix for date parser
13 years ago
3ce4c2f937
fixes for gsa result format
13 years ago
2d5fdfeb65
added authorization-based maximum results limitation to solr and gsa
...
search
13 years ago
6fc5400f91
added a tooltip for search navigation to mention that search pages can
...
be navigated using the TAB key
13 years ago
a06123aec6
more abstraction and less parameter overhead for remote search
13 years ago
f00733186b
code simplifications
13 years ago
780f8974e7
added ramaining iteration methods for solr in fulltext class
13 years ago
6f01542aaa
explicit double-check in transferURL
13 years ago
d54b80327a
refactoring
13 years ago
0cab06c47c
refactoring
13 years ago
40c0856489
refactoring
13 years ago
e651d3e320
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
06a78eecb7
code simplification
13 years ago
8a91f4fa42
local robots.txt: disallow external crawlers to follow the URL proxy
13 years ago
18f989dfb1
- refactoring (load -> getMetadata)
...
- added getDocument to retrieve Solr documents which shall replace
getMetadata
13 years ago
6197caf698
added clear-text search words in query params
13 years ago
23226676c6
FOR THE BRAVE.. this is a forced migration to solr which is now ready
...
for production as a replacement of the metadata-db.
This intermediate release 1.041 will switch on the previously optional
solr index and the old metadata-db will still work as it did before.
Solr+metadata are accessed in mixed mode, no migration is done yet.
If this causes not a catastrophe until the end of the weekend, we will
do a YaCy 1.1 main release containing this as default.
13 years ago
7c31be1c80
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
6456a1656a
changed local robots.txt to prevent external crawlers to submit random
...
search queries
13 years ago
703f427303
fixed some peer-ping connection details
...
- larger time-out
- removed too old seedlist
- fixed a bug in connection test
13 years ago
597bb76e4f
get the peer location more quickly
13 years ago
156d457aec
fix for Index out of bounds exception in Network servlet
13 years ago
ae9cd7a118
fix xss bug #204
13 years ago
d988ba50cf
added a very rudimentary, incomplete, non-verified GSA response writer
...
for solr. Try this:
http://localhost:8090/gsa/searchresult?q=pdf&site=col1&num=10
13 years ago
aab0b680c3
- added xslt support for solr result formats.
...
try i.e.
http://localhost:8090/solr/select?q=*:*&start=0&rows=10&wt=xslt&tr=json.xsl
- added servlet-side mime-type configuration for streamed servlets. this
is used for the result formatters in solr result formats
13 years ago
ad62609ec7
added a possibility to define a custom network definition URL for remote
...
management
13 years ago
fb0f430685
Merge remote-tracking branch 'original yacy/master'
13 years ago
b51df6c7e8
- added coordinate storage in solr schema
...
- fixed shutdown process
- fixed some solr-to-metadata reading
- added a large number of metadata attributes in ViewFile.html
13 years ago
9b88433f45
patch from hint in
...
http://forum.yacy-websuche.de/viewtopic.php?p=26858#p26858
from gaston
13 years ago
e816b88b55
changed behaviour of metadata storage: in case that any solr is
...
attached, the metadata is not written to the metadata-db, even if it is
enabled but instead to solr. This prevents that metadata is written in
two store systems at the same time. It is also the next step to migrate
the current metadata-db to solr.
13 years ago
f9c0e6e950
- Implemented and integrated the URIMetadataNode object which is a
...
metadata representation from the solr index. This shall replace metadata
from the built-in database in the future.
- added the Solr-driven metadata into the search index of YaCy which
makes it now possible to run YaCy without the old metadata index. This
is a major stept forward to a full migration to Solr.
13 years ago
b2b480fff2
more abstraction of the YaCySchema -> Opensearch matching process
13 years ago
73f6d69d03
more abstraction for solr query params parsing
13 years ago
24462e9baa
set the title every time, it is possible that it has changed
13 years ago
136fcb1ad9
refactoring
13 years ago
a12f693ec9
added two response writer for embedded solr interface:
...
a rss/opensearch writer and an enhanced solr xml writer.
The enhanced solr writer has less configuration overhead than the
original writer and should by slightly faster. The rss/opensearch writer
is at this time slightly incomplete compared with the already existing
rss search result form YaCy and also snippets are missing at this time.
To test the new interface, open for example:
http://localhost:8090/solr/select?wt=rss&q=olympia
The wt-code for the new result writers are=
wt=rss for opensearch
wt=exml for the enhanced solr xml writer.
Additionally, the SRU search parameters had been added to the solr
interface which can now also be used for a normal solr/xml search.
13 years ago
67edfd991c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
d9173ba7ed
added more solr fields to integrate values from URIMetadataRow. All
...
writings to the Metadata-DB are now also done to solr. This includes
metadata transfer during search and rwi transfer.
The new/added solr fields are:
## time when resource was loaded
load_date_dt
## date until resource shall be considered as fresh
fresh_date_dt
## id of the host, a 6-byte hash that is part of the document id
host_id_s
## ids of referrer to this document
referrer_id_ss
## the md5 of the raw source
md5_s
## the name of the publisher of the document
publisher_t
## the language used in the document; starts with primary language
language_ss
## an external ranking value
ranking_i
## the size of the raw source
size_i
## number of links to audio resources
audiolinkscount_i
## number of links to video resources
videolinkscount_i
## number of links to application resources
applinkscount_i
13 years ago
70b10e8316
added the JSON response writer to solr interface, add &wt=json to the
...
servlet GET properties to use this format
13 years ago
8d944f6517
nowrap from gaston in forum
...
http://forum.yacy-websuche.de/viewtopic.php?p=26815#p26815
13 years ago
24d9db1613
snippet retrieval loading processes may use a smaller minimum load time
...
value than crawling processes. This speeds up the search result
preparation dramatically.
13 years ago
1687737771
Abstraction of HandleMap and HandleSet
13 years ago
3bcd9d622b
cleaned up classes and methods which are either superfluous at this time
...
or will be superfluous or subject of complete redesign after the
migration to solr. Removing these things now will make the transition to
solr more simple.
13 years ago
6f1ddb2519
Moved solr index-add method to the same method where the YaCy index is
...
written. Also done some code-cleanup.
13 years ago
315d83cfa0
cleanup
13 years ago
76202f068e
extended abstraction of local and remote solr index using one front-end
...
for index administration and querying.
13 years ago
7ec7341f60
added user-authentication protection to solr search (same as implemented
...
for yacysearch)
13 years ago
e2a97ef8f6
better explain how to access the embedded solr
13 years ago
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
13 years ago
cba4ab862e
fix for http://bugs.yacy.net/view.php?id=202
13 years ago
36c9875b6e
removed localized number formatting from num-results_totalcount response (this is only used in xml and json where localized format is not valid)
13 years ago
69e743d9e3
- more abstraction for the RWI index as preparation for solr integration
...
- added options in search index to switch parts of the index on or off
13 years ago
6cc5d1094e
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
05a3ffd03a
patches to ensure that solr connectors are active ony if they have a
...
solr object assigned and vice versa
13 years ago
5a3c829872
embedded solr is only initiated if it is activated with
...
IndexFederated_p.html
13 years ago
3a350a2f83
partial html fix for
...
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4454
13 years ago
97b7bcf2a6
added a solr search index
...
- by default, a (empty) solr storage instance is created at
SEGMENTS/solr_36
- the index is written if in /IndexFederated_p.html the flag "embedded
solr search index" is switched on
- a standard solr query interface is available now with a new servlet at
http://127.0.0.1:8090/solr/select
To test this, do the following:
- switch to webportal mode
- switch on the feature as described
- do a crawl. this fills the solr index. The normal YaCy search will NOT
work now!
- do a solr query, like:
http://127.0.0.1:8090/solr/select?q= *:*
http://127.0.0.1:8090/solr/select?q=text_t:Help
play with different search fields as you can see in
/IndexFederated_p.html
You can use the standard solr query attributes as described in
http://wiki.apache.org/solr/SearchHandler
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
c00a3cf74d
less usage of generic logger to avoid logger generation overhead
13 years ago
e76159040b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bbfa497a3c
replaced more size() > 0 by !isEmpty()
13 years ago
e3aa05b9dd
added creation of subpath pattern when crawl start is 'from file'
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
aef9dd0350
- removed cleaning of blacklist cache on startup
...
- added cleaning of blacklist cache if cache is modified in interface
- extended cache saving to all cache types
- moved cache location to DATA/LISTS
- fixed static file path which was relative to the application path but
should be relative to data path - which is different in debian and mac
implementations
13 years ago
c7afa8bc48
using SwitchboardConstants for solr attributes
13 years ago
62202e2d71
refactoring of query attribute variable names for better consistency
...
with (next) stored query words
13 years ago
91f14ea38e
fix to solr configuration (case where the external solr was not online)
13 years ago
2c5b68d932
more abstraction of error message
13 years ago
9758c521ab
abstraction of error message
13 years ago
9b6e4e46ca
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4430
13 years ago
b0c408788b
made class methods static where possible
13 years ago
5bd3c90907
- removed unnecessary semicolons
...
- added default case for switch
13 years ago
7c1ba99755
removed more unused method parameters
13 years ago
0301aba1e9
removed unused method parameters
13 years ago
241dd8410a
removed snippet pattern filter - it was not used
13 years ago
d3964253ae
- added @SuppressWarnings to unused servlet method parameters
...
- removed unnecessary casts
- removed unnecessary throw statements
13 years ago
ea10766bfd
cleaned unnecessary nested code
13 years ago
78fc3cf8f8
refactoring and new usage of SentenceReader: this class appeared as one
...
of the major CPU users during snippet verification. The class was not
efficient for two reasons:
- it used a too complex input stream; generated from sources and UTF8
byte-conversions. The BufferedReader applied a strong overhead.
- to feed data into the SentenceReader, multiple toString/getBytes had
been applied until a buffered Reader from an input stream was possible.
These superfluous conversions had been removed.
- the best source for the Sentence Reader is a String. Therefore the
production of Strings had been forced inside the Document class.
13 years ago
276a66a793
Adding a limit of 1000 links that a parser shall store during indexing.
...
A limit was necessary because some web pages have such huge numbers of
links that it can easily cause a OOM just by the number of links.
The quesion if the number of 1000 links is sufficient or too weak must
be answered with the result of testing this feature.
13 years ago
1825f165b8
better integration of blacklist according to use case
13 years ago
c18fa9fa75
Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
13 years ago
ce8d4b87d9
fixes for new eclipse 'Juno' warning 'Resource leak'.
13 years ago
067728bccc
add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages)
13 years ago
03280fb161
removed segments-concept and the Segments class:
...
the segments had been there to create a tenant-infrastructure but were
never be used since that was all much too complex. There will be a
replacement using a solr navigation using a segment field in the search
index.
13 years ago
9116013c64
- allow lazy initialization of solr value (if using 'lazy', then no
...
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
13 years ago
e6792ed37d
Merge remote-tracking branch 'original yacy/master'
13 years ago
96aeb127e3
generalized localhost naming.
...
this is also a preparation for a better IPv6 implementation.
13 years ago
77f795756c
fixing redirects and status codes: storing of status code in
...
ResponseHeader to make it available for late evaluations, like storage
in solr.
13 years ago
8dd469b9dd
added option to configure the autocommit delay time of solr on-the-fly
13 years ago
b9dfca4b0a
- fixed IndexFederated Servlet / a embedded Solr can now be selected
...
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
13 years ago
fad3b14813
added jetty libraries, needed for future use as web server and as
...
application server for the solr search interface
13 years ago
b9d42fd9c8
using com.google.common.io.Files instead of homebrew methods
13 years ago
a5eb91fa60
refactoring
13 years ago
c1ba58ae51
Augmented browsing: Small CSS fix
13 years ago
b2b205aa38
Augmented browsing: small js fix
13 years ago
dc9ee0cdb3
Augmented browsing: CSS fix
13 years ago
74fcc6f8c5
Augmented browsing: small UI modifications
13 years ago
c63c3a4495
Show additional interaction elements in footer section on each page, if
...
activated in ConfigPortal.html.
This footer is also visible in augmented browsing proxy mode.
13 years ago
fa98657bb3
Augmented Browsing: changed the settings page
13 years ago
751eeade0d
Merge remote-tracking branch 'original yacy/master'
13 years ago
84a11ec48c
Corrected loading of default page settings on ConfigPortal.html
13 years ago
bea002dc15
correct table in new look of Crawler_p
13 years ago
8738336408
set Xms lower than Xmx
13 years ago
6b4545d6b0
Only load tag information if necessary
13 years ago
011f8a5818
Auto Tagging: Add hyperlinks to tags (provisional)
13 years ago
1d4e206b2b
bugfix in vocabulary generation
13 years ago
2c89975378
Merge remote-tracking branch 'original yacy/master'
13 years ago
71047fe63a
Augmented browsing: CSS fix
13 years ago
52f5d40043
better abstraction of document model generation
13 years ago
8b7c4d3144
produce a rdf output containing the triplestore with yacydoc; ie:
...
http://localhost:8090/api/yacydoc.rdf?urlhash=yOiCM7Fh1hyQ
13 years ago
f7160dae5c
Merge remote-tracking branch 'original yacy/master'
13 years ago
e4555cbee3
Augmented browsing: Pass on additional action parameter
13 years ago
24bbe359ca
integrate also geonames library files for less cities. these are more
...
useful for tagging since less normal words are false-identified as
location
13 years ago
5a41e739b4
better apilink description
13 years ago
e16e4bd2ba
added ontology extraction in xml as api call for vocabularies
13 years ago
8cf47a8335
Merge remote-tracking branch 'original yacy/master'
13 years ago
b85f01a14e
Augmented browsing: small UI fix
13 years ago
26cb1c65c2
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/document/importer/OAIPMHLoader.java
13 years ago
963f92ed9a
- merged files
...
- changed behaviour of delete button in vocabulary edit
- fixed size numbe in vocabulary listing
13 years ago
d8815db877
Merge remote-tracking branch 'original yacy/master'
13 years ago
e4dab19045
Augmented Browsing: added template for document info bar
13 years ago
743b0ec89f
- added size of vocabulary to vocabulary view
...
- fixed bad terms in vocabulary-from-titles autogeneration
13 years ago
22d5e33c5e
added more methods to vocabulary generation: scrape document title and
...
document author to vocabulary
13 years ago
b2d1c25ebb
removed warnings/unused entities
13 years ago
f1aa4c4390
- accept only location names wit a minimum length
...
- remove comma from synonym terms
13 years ago
cc9ad7198a
- use only names which consists of at least two parts
...
- remove word from derewo from locations
13 years ago
9264d8b4af
removed old navigation practice using subject tags in favor of
...
triplestore-tags
13 years ago
eeb4fd8b8c
refactoring (geolocalzation -> geolocation)
13 years ago
64c0268b2b
show triplestore metadata in yacydoc and viewfile
13 years ago
c2f0d16d2c
fixed vocabulary initialization
13 years ago
fbded1f466
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
df3531f8d5
added the generation of virtual vocabularies using the pnd
13 years ago
e806106b10
jquery bugfix
13 years ago
a0f1decd82
- added loading of the dbpedia pnd triplestore in the dictionary loader
...
- renamed the dictionary loader to knowledge loader
- some refactoring in the library provider method names
13 years ago
6d17686258
made triplestore persistent by default
...
added a size display in triplestore servlet
13 years ago
8d6e77ad0c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
2ac7a5c1f2
Augmented browsing: Add overlay bar which shows the vocabulary tags
13 years ago
777d22e145
renamed "augmented proxy" to "augmented browsing"
13 years ago
bddac2839e
add missing files for tag display
13 years ago
441430f507
Merge remote-tracking branch 'original yacy/master'
13 years ago
3c255c025b
Show tags in search results (if activated in ConfigPortal_p.html)
13 years ago
1f9120d189
create new vocabularies also without an objectspace. this creates an
...
empty vocabulary
13 years ago
a5cdfb91de
- fixed Cache link (below snippet)
...
- added 'Augmented Proxy' link below snippet
- added configuration options for augmented proxy
13 years ago
492b3e09f2
added api icon to triplestore
13 years ago
16d8f33795
added objectlink generation to vocabulary generation and editor
13 years ago
f1f97b7c95
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
b3eaaf5ebc
check also delete triplestore by default
13 years ago
f2f07a11f1
hotfix for unresolved pattern
13 years ago
5fd1a15fcf
hotfix until we have updated query routine for tags
13 years ago
f49d92d8da
Cleanup of interaction class and helper routines
13 years ago
56b0115054
Triplestore: modify routines to access per user store
13 years ago
d45718251e
refactoring (Localization -> Location)
13 years ago
b8b3c87ba7
- renamed localization to location (that was confusing)
...
- renamed 'Locale' navigator to 'Location'
- produce Location navigation only if geolocation libraries are loaded
13 years ago
f64e78497a
fix for reload-feature in Crawler_p
13 years ago
e89747bb67
- added automated generation of vocabularies from url stubs
...
- added clear of all terms for vocabularies
- added deletion of vocabularies
13 years ago
79464189a4
The 'Locale' vocabulary, which is generated by geo data, has now the
...
objectspace "http://dbpedia.org/resource/ "
13 years ago
eca38c53e7
added a vocabulary editor
13 years ago
80e8aaabc8
moved new servlets into one submenu "Content Semantic"
13 years ago
2bbb6c52cf
added option to clean the triplestore when deleting the index
13 years ago
8b53771db2
changed behavior of navigation processing:
...
- vocabulary annotation is not done any more into the metadata of urldb
- vocabularies are written into the jena triplestore using a rdf
vocabulary
- vocabularies for rdf tripel must be updated; refactoring done
- with the new navigation tags in the triplestore a faster
pre-urldb-lookup is possible: navigation is processed now within the RWI
during pre-ranking retrieval
- added also a Owl vocabulary stub to add the plain-text url to the
triplestore using the owl:sameas predicate
13 years ago
5fc6524ca8
- moved triple store to net.yacy.cora.lod (should be generalized there
...
later
- added abstract add, delete, get methods in the triplestore
- added generation of triples after auto-annotation
- migrated all MultiProtocolURI objects to DigestURI in the parser since
the url hash is needed as subject value in the triples in the triple
store
13 years ago
c90f174799
preparation and generalization of augmented browsing methods
13 years ago
edaa09b9b1
Rewrote all String blacklist types to enum 'BlacklistType', closes bug
...
#143
Conflicts:
htroot/Supporter.java
htroot/yacy/crawlReceipt.java
htroot/yacy/transferRWI.java
htroot/yacy/transferURL.java
source/de/anomic/crawler/CrawlStacker.java
source/de/anomic/data/ListManager.java
source/net/yacy/peers/Protocol.java
source/net/yacy/repository/Blacklist.java
source/net/yacy/repository/LoaderDispatcher.java
source/net/yacy/search/Switchboard.java
source/net/yacy/search/index/MetadataRepository.java
source/net/yacy/search/index/Segment.java
source/net/yacy/search/query/RWIProcess.java
source/net/yacy/search/snippet/MediaSnippet.java
13 years ago
213f006bf1
One is okay ...
...
Conflicts:
htroot/Trails.html
13 years ago
af5a597e47
Scroogle is not comming back, remove dead code
...
Conflicts:
source/net/yacy/search/Switchboard.java
13 years ago
7a4dab6d1d
- removed unused variables
...
- do not replace malformed or invalid URLs in urlproxy
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7835
6c8d7289-2bf4-0310-a012-ef5d649a1542
Conflicts:
source/de/anomic/http/server/HTTPDFileHandler.java
13 years ago
90c6fc4b63
load all - but not the persistent local.rdf - triples from
...
DATA/TRIPLESTORE at startup time. The local.rdf is loaded only if the
persistent switch is on (as before).
13 years ago
a9eb40c160
fix for autocomplete in index.html
13 years ago
dd020a1a8a
removed autocrawler and feedback servlet link since that was not
...
cherry-picked
13 years ago
aa0295917c
augmentation
...
Conflicts:
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
87a3fbb3c2
interaction javascript
13 years ago
ed2ea0f08e
augmented browsing modification
...
Conflicts:
htroot/interaction/OverlayInteraction.html
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
d4802dc8d5
small change
13 years ago
a120ef660b
RDF demo servlet
13 years ago
09a34cfe1b
prepare RDF dump routines
13 years ago
300b235ce8
Updated Demo Servlet
...
Conflicts:
htroot/About.html
htroot/DemoServlet.html
htroot/DemoServlet.java
htroot/interaction/interaction.js
source/net/yacy/interaction/Interaction.java
13 years ago
90512640bf
Added config switches for custom parser
...
Conflicts:
source/net/yacy/document/TextParser.java
13 years ago
a12cbcba36
Add a global value store
13 years ago
e14f2881ae
interaction: add special table interaction
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
4e4e7a99f8
interaction: add global variable store
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
bde07ed7a8
Add tagging overlay element
...
Conflicts:
htroot/env/templates/jqueryheader.template
htroot/yacysearchitem.java
source/net/yacy/interaction/Interaction.java
13 years ago
bee3bee8f3
Small fix - return value of JSON should be empty
13 years ago
ff4ba3ee05
Small fix
...
Conflicts:
htroot/yacysearchitem.java
13 years ago
f05e3968f7
Quick fix
13 years ago
e859481889
Add Triplestore settings functionality
...
Conflicts:
htroot/env/templates/header.template
13 years ago
b0bc0b4572
Add new demonstration module for client-side key-value store (backend:
...
triplestore): /DemoServletInteraction.html
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
c9dc6cda02
Demonstration: include value from interaction in search results
...
Conflicts:
htroot/interaction/OverlayInteraction.html
htroot/yacysearchitem.java
13 years ago
ae8adb0e58
Small changes
13 years ago
bcbd8eee33
Add several parsers, for RDFa and rdf files.
...
Conflicts:
source/net/yacy/document/TextParser.java
13 years ago
9ef5a80f4e
add interaction for triples and selector for augmented browsing
...
Conflicts:
htroot/interaction/interaction.js
source/net/yacy/interaction/Interaction.java
13 years ago
5d20cd324a
Add Triplestore and RDF query interface
...
Conflicts:
build.xml
defaults/yacy.init
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
bc9a618e0a
augmented browsing: ignore js and css, integrate more user interaction
...
Conflicts:
htroot/interaction/Footer.html
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
9cbfc1a1c0
augmentedProxy, which forwards every proxy request to a
...
rewrite engine to customize existing webpages. originally implemented by
Florian Richter.
Conflicts:
source/de/anomic/http/server/HTTPDProxyHandler.java
13 years ago
1626be7916
Add menu entries for urlproxy / augmented browsing
13 years ago
5b25272f40
added location search to main menu
13 years ago
ea0dceb55d
bugfix: do not switch off standard memory strategy when performing a
...
forced GC
PLEASE CHECK if your peer has standard memory switched on!
13 years ago
dd14b19c26
lazy initialization of block rank table ... only normal web search uses
...
this. When interactive search or location search is used, the block rank
is switched off
13 years ago
701b9a28a0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
htroot/PerformanceMemory_p.java
13 years ago
ab7107b34b
fixed RWIProcess queue limits: now discovering hidden results for mass
...
result retrieval
13 years ago
10c9c17d51
fixed handlemap spread factor and null iterator handling
13 years ago
a61f44f9e4
lazy initialization of block rank table.
...
this causes that the table is not initialized when there is no search is
done. the effect is most strong if YaCy is started headless which causes
no browser pop-up which otherwise would load the search page and
therefore trigger the initialization of the table.
13 years ago
c8bbd180e4
enhanced hint for debian package automatic update
13 years ago
9ad84c5e9f
fix for NPE in PerformanceMemory
13 years ago
96e9d77270
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
13 years ago
d7eb18cdf2
accept also file names beginning with "file://" for crawl start from
...
file.
13 years ago
3dd8376825
added automatic cleaning of cache if metadata and file database size is
...
not equal. It might happen that these data is different because one of
that caches is cleaned after a while or when it is too big. The metadata
is then not cleaned, but now wiped after a checkup process at every
application start. This should cause a bit less memory usage.
13 years ago
d0ec8018f5
fixes for bad long computation
13 years ago
96c8119b50
added GeoLocation / GeoPoint classes which uses less memory than
...
Location/Coordinates and has initializers with correct order of lat,lon
coordinates
13 years ago
461a0ce052
removed warnings
13 years ago
62ae9bbfda
allow more POIs, get more at once
13 years ago
a1fe65b115
performance hacks
13 years ago
2fe207f813
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
5aee19daa4
added show from cache in search results (not yet finished)
13 years ago
e0d8643226
- performance hacks
...
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
13 years ago
cf79b6cee3
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
6e83b02b83
- bugfix for surrogate file reader
...
- bugfix for location search: suppress empty search
13 years ago
9b4c699526
ehanced location search:
...
- search request are now made using a map boundary
- search results are only computed for the map boundary
- the number of results is adopted to the results in the visible range
- added a double-buffering for the search result markers
- added a search query option for the search results:
/radius/<lat>/<lon>/<radius>
13 years ago
434af404c1
- added double-buffering for search layers
...
- added automatic zooming to search result
to location search
13 years ago
4d9b2dc487
automatically zoom to result layer bounds
13 years ago
6b40803adf
- show number of results in map search interface
...
- transfer view radius within query
13 years ago
a8778e9c47
npe fix
13 years ago
1a6fab60e0
added node state to xml
13 years ago
20e0cc0822
fix for bad location evaluation
13 years ago
1ab3de0885
fixes to location search
13 years ago
f167a1c69f
removed osmarender from yacysearch_location because that caused a
...
javascript error
13 years ago
71c3163f3d
- fixes to node identification
...
- added link to node in network list
- added marking of portal search node peers
13 years ago
d1e9fe3db5
enhanced RootState icon
13 years ago
ad222be7f8
added node state icon in network list
13 years ago
638390930d
another patch to fix the Crawler_p layout
13 years ago
c846e9ca14
redesign of the crawler monitor page: show crawled pages instead of
...
queue of urls that shall be crawled
13 years ago
8b974905ee
changed log-in text for all servlets with authentication:
...
- added hint how to set the password using a shell script
- added a shell script to change the password
13 years ago
16b21f7a5b
Added more steering in Crawler_p.html interface
13 years ago
c15fcde1c8
add-on to latest commit
13 years ago
cf47d94888
performance hack to parse numbers inside of substrings without actually
...
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
13 years ago
7bf421b9dd
- fixed image search page navigation
...
- removed some deadlocks and ConcurrentModificationExceptions during
DidYouMean collection
13 years ago
6696cb1313
bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer
...
SeedDB.lookupByName searche for lowercase peerNames, while MapColumnIndex.getIndex uses peername as is in the keyset.
Changed the index init to insert lowercase peer names as key
13 years ago
4298f00d2d
fixed bad usage of given words
13 years ago
0d32a766ed
relax verify attribute for search widget to make it faster:
...
set to "cacheonly"
13 years ago
ae335a4190
bugfix Tables_p for edit and delete selected row (correction to use "pk_" html prefex)
13 years ago
f294f2e295
bugfix to http://bugs.yacy.net/view.php?id=181
...
tried to make a bit less 'noise' to dns server
also included: less processes in snippet fetch to reduce load during
search on small computers
13 years ago
1473e2258e
fix for http://bugs.yacy.net/view.php?id=154
13 years ago
3e1bc9477f
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
fbb946f913
Made a method static (Eclipse suggested it), removed unused import, pk=null check does now output a warning in logfile
13 years ago
5f983faef9
No & in JavaScript-embeded URLs, added ability to stop focus in
...
ConfigPortal.html preview (is this not secured with _p????)
Conflicts:
htroot/yacyinteractive.java
htroot/yacysearch.java
13 years ago
5b3acc12cd
Pattern.quote() replaces \\Q and \\E according to publication in
...
http://www.cs.washington.edu/homes/mernst/pubs/regex-types-ftfjp2012.pdf
13 years ago
89142d1e8d
removed (not all) warnings
13 years ago
ffa4553229
typo
13 years ago
5deebd02ea
added serialization
13 years ago
b2175ea4ef
Add possibility to set custom Solr field names for the YaCy default Solr attributes.
...
- Changing the format of YaCy's solr.key.list while maintainig backward compatibility
Federated index config screens adjusted accordingly
- modified the Solr update request to use a 3 min Solr autocommit intervall
13 years ago
0d58fea210
made multiple connector default
13 years ago
8864141872
more abstraction in solr connection classes
13 years ago
c00efc2717
made the solr connection more generic
13 years ago
f130ab39e8
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
a691023d04
*) better formatting for network QPM
...
*) refactoring
13 years ago
dcccbe0be8
removed superfluous column
13 years ago
77f8e9fb9b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
ba6aaabc51
refactoring + parser bugfixes
13 years ago
a18b6dee04
Merge remote branch 'bbyacy-rc1/master'
13 years ago
ea932f841c
changed link to opensearchdescription document to a absolute uri (in yacysearch.html and yacysearch.rss)
...
see http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_5#The_.22Description.22_element
13 years ago
453010bd68
- solved problems with backpath normalization
...
- redesigned in/outbound link handover
- removed iframe links from inbound/outbound in solr scheme
13 years ago
5f5ed33ed8
patch for media search (audio, video apps)
13 years ago
0e13022147
- enhanced solr field documentation
...
- added xml api button to IndexFederated_p - the solr schema.xml file
can be generated by YaCy
13 years ago
08dcf3e5d1
hack to get all results if the actual number is between 10 and 64
13 years ago
19efbf1b0f
- apply directDocByURL to NOLOAD Queue
...
- choose pushing to NOLOAD as default for site crawl
13 years ago
5c66880be2
fix for search result selection in case that contentdom is not set
13 years ago
3bea25c513
increased image preview size
13 years ago
a3badd3205
changed search process for images: no more media snippet load process,
...
show only links from index which had been on the text search page
before. This creates a superfast search process for images!
13 years ago
4aa0eedead
one more scroogle...
13 years ago
347612ddd4
removed scroogle parser
13 years ago
f8cd57c92f
new indexing strategy: ALL links that appear anywhere are indexed, not
...
only links where the content can be parsed. All non-parseable links are
placed into the noload queue. The search process must therefore be able
to filter out non-text search results.
- This fixes the problem that image search results appeared in the text
search.
- The interactive search can retrieve now ALL types of links
- The p2p interface is now extended to retrieve only certain types of
links (text, image, video, apps)
- The search process has an extension to filter the right document type
according to the search query
13 years ago
14f67f217c
refactoring of ContentDomain: now subclass of Classification
13 years ago
a5d7da68a0
refactoring: removed dependency from switchboard in Balancer/CrawlQueues
13 years ago
33d1062c79
refactoring: the cache belongs to the crawler
13 years ago
8429967ea7
no more SVN
13 years ago
0466bb0ddf
no more SVN..
13 years ago
4844e124b1
one more warning in case that crawling is paused because of low disk
...
space
13 years ago
0ec2713af8
'download'
13 years ago
f30c577fdb
add hint to speed up search results
13 years ago
6b133de3e9
add hint for consulting support
13 years ago
eb2c8ffa62
display is not used any more
13 years ago
91a86f0b06
fixed to network graph testing
13 years ago
f31ad84d98
automatic generation of blacklist pattern, see
...
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2685&p=25305#p25305
13 years ago
7b5b9baee0
added citation rank to ranking profile
13 years ago
06951ef751
remove heuristic scroogle from search option help text in index.html
13 years ago
e377092198
fix to xml output format
13 years ago
41be98dc9d
extended webstructure api to show together with incoming links also
...
outgoing links
13 years ago
8f89c8ef07
added information about inbound, outbound and citation links into
...
yacydoc api servlet
13 years ago
71649a1296
added an api to retrieve the new citation.index with the
...
webstructure.xml api. This api will respond with details about a single
URL if requested with 'webstructure.xml?about=[url|urlhash|host]'.
13 years ago
3e61287326
some better feedback on properties change
13 years ago
96ac95cff9
added hint how to change integration options
13 years ago
4f61b8fd82
Fixes for compare-search
13 years ago
e0680de7b3
Remove Scroogle from compare-search, Scroogle is dead
13 years ago
78f0d8f046
no focus on preview frames for search integration
...
fixes bug http://bugs.yacy.net/view.php?id=161
13 years ago
7792ac6406
fix links & bug #163
13 years ago
532c7cf827
added physics experiment to the graph plotter. not active by default
13 years ago
aba9b1bfa0
better names for elements of a linked graph
13 years ago
2fc8ecee36
ConcurrentLinkedQueue has a VERY long return time on the .size() method.
...
See
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html
and the following test programm:
public class QueueLengthTimeTest {
public static long countTest(Queue<Integer> q, int c) {
long t = System.currentTimeMillis();
for (int i = 0; i < c; i++) {
q.add(q.size());
}
return System.currentTimeMillis() - t;
}
public static void main(String[] args) {
int c = 1;
for (int i = 0; i < 100; i++) {
Runtime.getRuntime().gc();
long t1 = countTest(new ArrayBlockingQueue<Integer>(c), c);
Runtime.getRuntime().gc();
long t2 = countTest(new LinkedBlockingQueue<Integer>(), c);
Runtime.getRuntime().gc();
long t3 = countTest(new ConcurrentLinkedQueue<Integer>(),
c);
System.out.println("count = " + c + ": ArrayBlockingQueue =
" + t1 + ", LinkedBlockingQueue = " + t2 + ", ConcurrentLinkedQueue = "
+ t3);
c = c * 2;
}
}
}
13 years ago
8aba045ba1
if a new pop-up page is set in config portal, then this page applies
...
also to the default page configuration for the httpd if no path is
given.
13 years ago
fa7b3481b3
better navigation in file search: less results by first try, but much
...
faster. after the first search is done, buttons appear to get more
results for the same search
13 years ago
8c06925984
animation of the web structure picture
13 years ago
99c74699de
removed scroogle (scroogle is dead)
13 years ago
6e51a00a2f
Revert "fix for page navigation: show only as much pages as are available for given navigation constraints, not as given by total results size"
...
This reverts commit 73f5a9e8b3
13 years ago
73f5a9e8b3
fix for page navigation: show only as much pages as are available for
...
given navigation constraints, not as given by total results size
13 years ago
9c51dc0f13
fixed a bug with navigation: if a navigation was applied to file type or
...
protocol, then it was not possible to remove that again. This is the fix
for that.
13 years ago
8bfc987374
enhanced hint how to enter file:// urls
13 years ago
c6c61be3f0
fix for http://bugs.yacy.net/view.php?id=148
13 years ago
edaa8ac94c
Merge commit 'e15e633a0128b8d31011283a65b4ef26a6dddcd8'
13 years ago
e15e633a01
Bugfix for IE9 (doesn't accept html form within form)
...
changes of API schedule row data changed form input form to unique field names
using row pk.
Fix for issue 96 http://bugs.yacy.net/view.php?id=96
IE9-64bit doesn't interprete iframe with align parameter as desired
misaligns following content (in CrawlProfileEditor_p.html)
13 years ago
a9b4d49b75
removed debug output
13 years ago
8d63a5887c
bugfixes
13 years ago
9ad1d8dde2
complete redesign of crawl queue monitoring: do not look at a
...
ready-prepared crawl list but at the stacks of the domains that are
stored for balanced crawling. This affects also the balancer since that
does not need to prepare the pre-selected crawl list for monitoring. As
a effect:
- it is no more possible to see the correct order of next to-be-crawled
links, since that depends on the actual state of the balancer stack the
next time another url is requested for loading
- the balancer works better since the next url can be selected according
to the current situation and not according to a pre-selected order.
13 years ago
e2f8f263e8
changed storage of search words: keep order
13 years ago
c166eb68b6
fixes in solr schema file
13 years ago
2e5cd6a1b2
fixed parser extension deny list generation and usage
13 years ago
f077b11d38
Merge branch 'master' of git://git.gitorious.org/yacy/rc1.git
13 years ago
29675d9766
more label on search options (usability)
13 years ago
b4bc1e2875
remote search does not do snippet generation
13 years ago
335a776351
xss hardening on Status.html
13 years ago
4c5edab1ec
added option to have exception search result windows
13 years ago
32adad7dd5
show less navigation by default
13 years ago
88b86afc89
no DoS protection for intranet mode
13 years ago
0f443ac755
automatic switching off of navigation that is not useful
13 years ago
852ce43d99
better rules for default open/close of navigation objetcs
13 years ago
e8d24fd802
author navigator can be switched off
13 years ago
558ab7bd4e
made the protocol navigator reversible
13 years ago
96cb75f1d4
made the filetype navigator be able to deselect the search constraint
13 years ago
30891d026f
added a remove-navigation for vocabularies
13 years ago
d5ead5314d
changed navigation links: now using checkboxes.
...
This looks better and allows that negative checkboxes (such that remove
the navigation) are possible. These are not yet implemented (comming
next)
13 years ago
ef5192f8c9
using the generic document parser for crawl starts instead of the html
...
parser. This makes it possible that every type of document can be a
crawl start point, not only text documents or html documents. Testet
this with a pdf document.
13 years ago
ce620be783
for for crawl start with smb url
13 years ago
18227790ca
variable search works at least on Firefox again
13 years ago
c73af39e54
refactoring of tray icon class,
...
now uses Java 6 methods natively
13 years ago
5afec4af39
fix for http://bugs.yacy.net/view.php?id=131
13 years ago
dc165275ad
bugfix for usage of multiple vocabulary navigators
13 years ago
190b77c55e
added Ukrainian translation
13 years ago
7f9b6b7a0c
added switches to ConfigParser to accept/deny documents by their
...
extension
13 years ago
83009d86f7
added the vocabulary navigator. It can be very simply tested by
...
switching on the locale dictionaries.
13 years ago
37d43e5589
jump back to where we want to modify the scheduler in api steering after
...
submit
13 years ago
26e227f17d
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
7053f8ab46
added automatic generation of a solr schema.xml file
13 years ago
c7f88f3fd1
fix for http://bugs.yacy.net/view.php?id=101 - the default crawl
...
depth for bookmarks is now editable.
13 years ago
c602eaaf46
enhanced search process
13 years ago
a71f751cc8
Merge branch 'master' of git://gitorious.org/~apfelmaennchen/yacy/apfelmaennchens-rc1
13 years ago
a94ced7cb2
Solved ( http://bugs.yacy.net/view.php?id=124 ) accessibility issue with
...
page 'Ranking_p.html'
13 years ago
eff966f396
fix for search process (it was aborted too early during remote search)
13 years ago
6d8faa8066
small fix
...
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
1dc5d9f0f3
make ConnectionInfo comparable and sort list of connections in Connections_p
...
ConnectionInfo compare by initTime
Connections_p implement wish to sort connections, descending
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
c1973c78a3
Blog + BlogComments method propose() make similar + fix anonymous poster's IP resolve in Blog
...
this method share very much common code in both classes,
would be nice to unify
+fix some ugly code
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
7189a70a62
BlogComments dateString() - use localized date/time format
...
use same method from Blog class, reduce code duplication
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
f727c35722
Blog dateString() print in localized date/time format
...
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
8318103f25
Blacklist addBlacklistEntry() - dont add an empty entry
...
Signed-off-by: Marek Otahal <markotahal@gmail.com>
13 years ago
972d411d99
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bd40a10230
added autotaggig stub .. only reading and parsing of vocabularies at
...
this time
13 years ago
f214f6ebb4
added no-load queues to the crawler monitor
13 years ago
2ee8cbeb2c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/search/Switchboard.java
13 years ago
992dbdf4bb
added noload statistic to servlets
13 years ago
216a287a85
Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r
...
Conflicts:
source/de/anomic/crawler/CrawlQueues.java
13 years ago
d18095dc48
Patch fuer Issue 0000102
...
and fixes to Patch (private peer status is a property of a peer, not a
status)
13 years ago
ce289eb64f
remove heuristics if index receive is switched off
13 years ago
fa08ed5ae5
Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check
13 years ago
1d66496362
added more properties to opensearchdescription, see also:
...
http://bugs.yacy.net/view.php?id=114
13 years ago
1cf0f35621
the link to the path shall be the path
13 years ago
87c16fb67f
show server list as server side include
13 years ago
5bfb287753
make a bad fix even worse
13 years ago
4656bb0365
allow html in greeting line since that is set by administrator only.
...
html in that line may be wanted
13 years ago
85bd4cc8bc
better lookup for peer names
13 years ago
20e3084bd4
redesign of fining of peers by ip: more leightweight method to read the
...
seed databases
13 years ago
b09988c9f8
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
c67c30db34
added language selector for russian and chinese in ConfigBasic
13 years ago
9e5894c784
Removed handling of components objects for URIMetadataRows.
...
This is a preparation to replace this rows with nodes from the node
store.
13 years ago
c04bfaa51b
refactoring
13 years ago
e9dc99fe15
added rules to set specific RWIs as private RWIs which are not
...
transmitted to remote peers. This will be used for private index copies
and phonetic indexes.
13 years ago
0bcef2d156
added feature as requested in
...
http://forum.yacy-websuche.de/viewtopic.php?f=18&t=3461
The search can now be configured with a non-display host list.
the search will always exlude the given list of host unless they are
requested directly using the host navigation
13 years ago
66e4832989
work around for overflow in YMarks.html import tab
13 years ago
d6e6f7715b
added "about" box configuration
13 years ago
204c29f010
small bugfixes for search result display and cache display
13 years ago
9aa73a13a8
stop words are on by default in site crawl. This causes normally nothing
...
since the stopwords are empty by default.
13 years ago
17f962fceb
translator updates:
...
- config string for chinese
- do not copy the language file to DATA/LOCALE any more (and do not use
them there, this is really confusing for new translators)
13 years ago
ac5b1e891d
new
13 years ago
391fc9bd57
Merge branch 'master' of gitorious.org:yacy/rc1
13 years ago
23afee58fe
Merge branch 'master' of git://github.com/f1ori/yacy
13 years ago
ff19fcdb28
bugfix for YMarks XBEL import and export; thanks to Dominic
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8138 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
c715d19c09
fixes for dependency on svn
13 years ago
0d1042363c
showing /16 and /24 subnet option instead of 'bigrange' in network
...
scanner
13 years ago
584f853c4b
added YBR computation process as servlet
13 years ago
864c8d5f05
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
f14faf503b
better ranking because we wait a very little time during the search
...
process more to get better remote sear results into the ranking priority
stack
13 years ago
89dc04115f
fix for stretched network image
13 years ago
2548a9cb04
Fix for http://bugs.yacy.net/view.php?id=93 , Added a link to Bugtracker on YaCy Webinterface.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8137 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
402e9d71ef
changed ording on release files: main criteria is not the svn any more; releases are now ordered by
...
- release number
- date
- svn number
additionally there is a new option to remove the svn number completely
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8135 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
11729061f2
added an option in the bookmark import process to put everything into the crawler
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8134 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
77317a88e0
Added nice jquery tagsinput to bookmarks dialog - similar to delicious.com ;-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8133 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
8f30d288e9
small change to mouse over text for crawl starts within bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8131 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
4b8ff84705
- search bugfixes (page counter and number of results per page; recognition of new search)
...
- experiments to speed-up the network image production (commented out)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8130 6c8d7289-2bf4-0310-a012-ef5d649a1542
13 years ago
Michael Peter Christen