e432bb9cd9
better calculation of possible saving in HeapReader index data structure
12 years ago
9549984c65
documentation/comments
12 years ago
3bcd9d622b
cleaned up classes and methods which are either superfluous at this time
...
or will be superfluous or subject of complete redesign after the
migration to solr. Removing these things now will make the transition to
solr more simple.
12 years ago
6f1ddb2519
Moved solr index-add method to the same method where the YaCy index is
...
written. Also done some code-cleanup.
12 years ago
315d83cfa0
cleanup
12 years ago
1f41d9c6f5
bugfix for a NPE
12 years ago
76202f068e
extended abstraction of local and remote solr index using one front-end
...
for index administration and querying.
12 years ago
d3f243e2e1
fixed node type calculation for principal peers
12 years ago
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
12 years ago
cba4ab862e
fix for http://bugs.yacy.net/view.php?id=202
12 years ago
69e743d9e3
- more abstraction for the RWI index as preparation for solr integration
...
- added options in search index to switch parts of the index on or off
12 years ago
05a3ffd03a
patches to ensure that solr connectors are active ony if they have a
...
solr object assigned and vice versa
13 years ago
5a3c829872
embedded solr is only initiated if it is activated with
...
IndexFederated_p.html
13 years ago
97b7bcf2a6
added a solr search index
...
- by default, a (empty) solr storage instance is created at
SEGMENTS/solr_36
- the index is written if in /IndexFederated_p.html the flag "embedded
solr search index" is switched on
- a standard solr query interface is available now with a new servlet at
http://127.0.0.1:8090/solr/select
To test this, do the following:
- switch to webportal mode
- switch on the feature as described
- do a crawl. this fills the solr index. The normal YaCy search will NOT
work now!
- do a solr query, like:
http://127.0.0.1:8090/solr/select?q= *:*
http://127.0.0.1:8090/solr/select?q=text_t:Help
play with different search fields as you can see in
/IndexFederated_p.html
You can use the standard solr query attributes as described in
http://wiki.apache.org/solr/SearchHandler
13 years ago
f0a079ac9f
allow larger log entries
13 years ago
784a4abb18
enhancement in internal data organization which should generate less
...
synchronizations in database access
13 years ago
f78ce93a80
collection of speed and memory saving hacks
13 years ago
c00a3cf74d
less usage of generic logger to avoid logger generation overhead
13 years ago
a196f24f60
prevent enqueueing of non-loggeable logging entries
13 years ago
482afed07c
reduced logging overhead (a bit)
13 years ago
e76159040b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
bbfa497a3c
replaced more size() > 0 by !isEmpty()
13 years ago
58e7d1952f
reduction of logging to prevent too much IO caused be logging
13 years ago
83da68c4c1
fixed a memory leak inside the logger which appeared if the log was
...
writter faster that the logger is able to print this out to its out
stream. A very large collection of unwritten log outputs had been seen
during strong crawling. The new ArrayBlockingQueue is limited to prevent
this case.
13 years ago
0cbda0b2b8
- replaced all length() == 0 and size() == 0 with isEmpty()
...
- replaced some length() > 0 and size() > 0 with !isEmpty() - cannot be
done automatically
- implemented some isEmpty() methods
13 years ago
28b30231c3
fix for url matcher of multiple amp& in an url, see:
...
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=4439&p=26650#p26650
13 years ago
aef9dd0350
- removed cleaning of blacklist cache on startup
...
- added cleaning of blacklist cache if cache is modified in interface
- extended cache saving to all cache types
- moved cache location to DATA/LISTS
- fixed static file path which was relative to the application path but
should be relative to data path - which is different in debian and mac
implementations
13 years ago
c7afa8bc48
using SwitchboardConstants for solr attributes
13 years ago
c6d8950651
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
5f3b8dc040
fix for RSS reader
13 years ago
62202e2d71
refactoring of query attribute variable names for better consistency
...
with (next) stored query words
13 years ago
1addbc792c
use less memory for md5 cache
13 years ago
f32de94723
more logging
13 years ago
d09d9f2364
filter old peers from bootstrap (now stronger: 60 minutes instead of
...
240).
13 years ago
434ee90c59
added classification for control file types which shall not be loaded
...
but placed onto the noload-queue
13 years ago
a90bcb48f6
added webm
13 years ago
801972fe6f
fix for url camel case parser and sentence reader
13 years ago
fbc1a2030d
fix for sitemap importer: can now also import very large sitemaps within
...
small memory configurations
13 years ago
92731e5287
fix for sevenzip parser
13 years ago
45641b0c23
catch and log a warning in RasterPlotter
13 years ago
8efc1c1078
- fixed a memory leak (or bad usage) during parsing/snippet fetch
...
- more logging for errors
13 years ago
c3db015410
prevent loading of content from the cache when retrieval with IFFRESH is
...
used and cache is stale. Should speed up snippet generation when cache
strategy is IFFRESH.
13 years ago
b1e7c11fba
fix for pattern matcher in html parser
13 years ago
8a6edc0031
fix for solr shutdown
13 years ago
b8bcc06283
fix for urls beginning with "//"
13 years ago
b0c408788b
made class methods static where possible
13 years ago
5bd3c90907
- removed unnecessary semicolons
...
- added default case for switch
13 years ago
132afaf687
removed unaccessible code
13 years ago
7c1ba99755
removed more unused method parameters
13 years ago
83701a1b4c
removed unused ImageReference package
13 years ago
0301aba1e9
removed unused method parameters
13 years ago
241dd8410a
removed snippet pattern filter - it was not used
13 years ago
d3964253ae
- added @SuppressWarnings to unused servlet method parameters
...
- removed unnecessary casts
- removed unnecessary throw statements
13 years ago
ea10766bfd
cleaned unnecessary nested code
13 years ago
1481037820
replaced non-generic array with collection
13 years ago
fc0f9543fe
More SentenceReader cleanup
13 years ago
586bb0eb6a
Simplified SentenceReader (no more Reader inside..)
13 years ago
7f851d62a7
replaced HashARC with SizeLimited Objects which are less costly
13 years ago
d4291ac1f3
more tolerance when creating solar document
13 years ago
78fc3cf8f8
refactoring and new usage of SentenceReader: this class appeared as one
...
of the major CPU users during snippet verification. The class was not
efficient for two reasons:
- it used a too complex input stream; generated from sources and UTF8
byte-conversions. The BufferedReader applied a strong overhead.
- to feed data into the SentenceReader, multiple toString/getBytes had
been applied until a buffered Reader from an input stream was possible.
These superfluous conversions had been removed.
- the best source for the Sentence Reader is a String. Therefore the
production of Strings had been forced inside the Document class.
13 years ago
bb8dcb4911
automatically adopt size of word cache to available memory
13 years ago
ad09b786bf
clean up parser data
13 years ago
276a66a793
Adding a limit of 1000 links that a parser shall store during indexing.
...
A limit was necessary because some web pages have such huge numbers of
links that it can easily cause a OOM just by the number of links.
The quesion if the number of 1000 links is sufficient or too weak must
be answered with the result of testing this feature.
13 years ago
613b45f604
- better data structures in secondary search
...
- fixed a big memory leak in secondary search
13 years ago
de903a53a0
parser refactoring & hacks
13 years ago
8a82609360
- smaller caches to save memory
...
- close cloneable iterators to free memory
13 years ago
7249d9c9de
bugfix for concurrent seed loader
13 years ago
c72d3b12cd
concurrently initialize the seed list during p2p network bootstrap
13 years ago
1825f165b8
better integration of blacklist according to use case
13 years ago
c18fa9fa75
Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
13 years ago
ce8d4b87d9
fixes for new eclipse 'Juno' warning 'Resource leak'.
13 years ago
0c345d1559
giving threads name so its easier to see whats happening during
...
debugging and within a thread dump
13 years ago
067728bccc
add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages)
13 years ago
03280fb161
removed segments-concept and the Segments class:
...
the segments had been there to create a tenant-infrastructure but were
never be used since that was all much too complex. There will be a
replacement using a solr navigation using a segment field in the search
index.
13 years ago
508a81b86c
added solr field 'refresh_s' which stores the refresh url contained in
...
the meta-refresh html header field.
13 years ago
f3167def64
do not fill the keywords with title content if keywords do not exist.
13 years ago
9116013c64
- allow lazy initialization of solr value (if using 'lazy', then no
...
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
13 years ago
97f60010d8
fix crawl start from file
13 years ago
0294a53459
- add canonical field only if requested by solr schema
...
- remove canonical url from in/outbound urls if present
13 years ago
3fd4a01286
added option to record urls that are forwarded to the solr index
13 years ago
d763e4d94b
fixed bad referer computation in SSIs which causes a NPE during host
...
computation. This error was there before the latest IPv6 hack but did
not cause a NPE. The IPv6 hack was not the cause for this bug, but it
discovered the misconfiguration of the 'referer' referrer.
13 years ago
358b04885e
more IPv6 hacks
13 years ago
96aeb127e3
generalized localhost naming.
...
this is also a preparation for a better IPv6 implementation.
13 years ago
77f795756c
fixing redirects and status codes: storing of status code in
...
ResponseHeader to make it available for late evaluations, like storage
in solr.
13 years ago
8dd469b9dd
added option to configure the autocommit delay time of solr on-the-fly
13 years ago
b9dfca4b0a
- fixed IndexFederated Servlet / a embedded Solr can now be selected
...
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
13 years ago
fad3b14813
added jetty libraries, needed for future use as web server and as
...
application server for the solr search interface
13 years ago
a38b0a2c46
extended embedded solr tests to ensure that it will be usable within a
...
jetty instance
13 years ago
b9d42fd9c8
using com.google.common.io.Files instead of homebrew methods
13 years ago
a5eb91fa60
refactoring
13 years ago
1be0025a9c
- added test for EmbeddedSolrConnector
...
- added needed libraries for this test
this includes most (all) files needed for an embedded solr
13 years ago
dbdd697f4d
moved RDFaParser.xsl configuration file to defaults
13 years ago
90b82ce994
using guava for host resolution (non-blocking for ips) and time-out
13 years ago
e12bb254b4
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
3f55dc7c1e
- added solr core and libraries that solr needs (lucene is missing, will
...
follow later)
- added embedded solr connector which can connect to solr
programmatically (without using a server in between)
13 years ago
c337190a00
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
c63c3a4495
Show additional interaction elements in footer section on each page, if
...
activated in ConfigPortal.html.
This footer is also visible in augmented browsing proxy mode.
13 years ago
786be7d175
better integration of RDFaParser
13 years ago
de3ef8ad73
removed unimportant warnings
13 years ago
82a682b31d
fixed problem with seed when switching network
13 years ago
8c544edee4
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
7dc59979bc
fix for npe, possibly for http://bugs.yacy.net/view.php?id=195
13 years ago
1d4e206b2b
bugfix in vocabulary generation
13 years ago
2c89975378
Merge remote-tracking branch 'original yacy/master'
13 years ago
52f5d40043
better abstraction of document model generation
13 years ago
8b7c4d3144
produce a rdf output containing the triplestore with yacydoc; ie:
...
http://localhost:8090/api/yacydoc.rdf?urlhash=yOiCM7Fh1hyQ
13 years ago
f7160dae5c
Merge remote-tracking branch 'original yacy/master'
13 years ago
e4555cbee3
Augmented browsing: Pass on additional action parameter
13 years ago
24bbe359ca
integrate also geonames library files for less cities. these are more
...
useful for tagging since less normal words are false-identified as
location
13 years ago
223a5440ab
preventing that an empty pnd is inserted into the vocabularies
13 years ago
8e97ada7c9
IPv6 bugfix
13 years ago
963f92ed9a
- merged files
...
- changed behaviour of delete button in vocabulary edit
- fixed size numbe in vocabulary listing
13 years ago
dd88d0ace2
more logging
13 years ago
94d54e2d91
added recognition of multi-word terms in vocabulary matching
...
this makes the PND usable: it is now possible to recognize persons and
navigate with a 'Persons' facet.
13 years ago
64c0268b2b
show triplestore metadata in yacydoc and viewfile
13 years ago
0fbd749207
ipv6 update
13 years ago
c2f0d16d2c
fixed vocabulary initialization
13 years ago
df3531f8d5
added the generation of virtual vocabularies using the pnd
13 years ago
a0f1decd82
- added loading of the dbpedia pnd triplestore in the dictionary loader
...
- renamed the dictionary loader to knowledge loader
- some refactoring in the library provider method names
13 years ago
2ac7a5c1f2
Augmented browsing: Add overlay bar which shows the vocabulary tags
13 years ago
3c255c025b
Show tags in search results (if activated in ConfigPortal_p.html)
13 years ago
16d8f33795
added objectlink generation to vocabulary generation and editor
13 years ago
f49d92d8da
Cleanup of interaction class and helper routines
13 years ago
56b0115054
Triplestore: modify routines to access per user store
13 years ago
a95127c9af
Triplestore: initalize per-user triplestores
13 years ago
d45718251e
refactoring (Localization -> Location)
13 years ago
b8b3c87ba7
- renamed localization to location (that was confusing)
...
- renamed 'Locale' navigator to 'Location'
- produce Location navigation only if geolocation libraries are loaded
13 years ago
e89747bb67
- added automated generation of vocabularies from url stubs
...
- added clear of all terms for vocabularies
- added deletion of vocabularies
13 years ago
79464189a4
The 'Locale' vocabulary, which is generated by geo data, has now the
...
objectspace "http://dbpedia.org/resource/ "
13 years ago
eca38c53e7
added a vocabulary editor
13 years ago
61bb52d55c
- using http://purl.org/dc/terms/references to refer from an
...
auto-annotated document to a 'pseudo-linked' document which has an url
created with an object-prefix as defined in the vocabulary file
13 years ago
2bbb6c52cf
added option to clean the triplestore when deleting the index
13 years ago
50c576599b
allow multiple parser options instead of printing an error
13 years ago
c02d742e53
proper namespaces in triplestore dump
13 years ago
8b53771db2
changed behavior of navigation processing:
...
- vocabulary annotation is not done any more into the metadata of urldb
- vocabularies are written into the jena triplestore using a rdf
vocabulary
- vocabularies for rdf tripel must be updated; refactoring done
- with the new navigation tags in the triplestore a faster
pre-urldb-lookup is possible: navigation is processed now within the RWI
during pre-ranking retrieval
- added also a Owl vocabulary stub to add the plain-text url to the
triplestore using the owl:sameas predicate
13 years ago
5fc6524ca8
- moved triple store to net.yacy.cora.lod (should be generalized there
...
later
- added abstract add, delete, get methods in the triplestore
- added generation of triples after auto-annotation
- migrated all MultiProtocolURI objects to DigestURI in the parser since
the url hash is needed as subject value in the triples in the triple
store
13 years ago
8d2e6355f8
augmented browsing: remove non-existing external snippet file
13 years ago
c90f174799
preparation and generalization of augmented browsing methods
13 years ago
bef823c247
close the reader if finished
13 years ago
4ee6fb1de9
added missing blacklist dht cache storage (maybe due to mistakes in
...
cherry picking)
13 years ago
e4d36fa5eb
Fix to make all values lower-case (this should make all existing blacklists compatible with the new enum)
13 years ago
edaa09b9b1
Rewrote all String blacklist types to enum 'BlacklistType', closes bug
...
#143
Conflicts:
htroot/Supporter.java
htroot/yacy/crawlReceipt.java
htroot/yacy/transferRWI.java
htroot/yacy/transferURL.java
source/de/anomic/crawler/CrawlStacker.java
source/de/anomic/data/ListManager.java
source/net/yacy/peers/Protocol.java
source/net/yacy/repository/Blacklist.java
source/net/yacy/repository/LoaderDispatcher.java
source/net/yacy/search/Switchboard.java
source/net/yacy/search/index/MetadataRepository.java
source/net/yacy/search/index/Segment.java
source/net/yacy/search/query/RWIProcess.java
source/net/yacy/search/snippet/MediaSnippet.java
13 years ago
af5a597e47
Scroogle is not comming back, remove dead code
...
Conflicts:
source/net/yacy/search/Switchboard.java
13 years ago
ca93835713
removed usage of deprecated methods
13 years ago
23e38bd918
do not load the "_triplestore.rdf" files which are of special use in
...
sciencenet
13 years ago
90c6fc4b63
load all - but not the persistent local.rdf - triples from
...
DATA/TRIPLESTORE at startup time. The local.rdf is loaded only if the
persistent switch is on (as before).
13 years ago
bbfc53b663
bugfix
13 years ago
65c5826d93
bugfix
...
Conflicts:
source/net/yacy/document/parser/augment/AugmentParser.java
13 years ago
aa0295917c
augmentation
...
Conflicts:
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
ed2ea0f08e
augmented browsing modification
...
Conflicts:
htroot/interaction/OverlayInteraction.html
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
6b32f7c1f6
re-enable augmented proxy
13 years ago
3b08edec2e
bugfix
...
Conflicts:
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
5f8ba7f4f2
small changes
...
Conflicts:
source/net/yacy/document/parser/augment/AugmentParser.java
source/net/yacy/interaction/Interaction.java
13 years ago
300b235ce8
Updated Demo Servlet
...
Conflicts:
htroot/About.html
htroot/DemoServlet.html
htroot/DemoServlet.java
htroot/interaction/interaction.js
source/net/yacy/interaction/Interaction.java
13 years ago
90512640bf
Added config switches for custom parser
...
Conflicts:
source/net/yacy/document/TextParser.java
13 years ago
df47f31235
interaction: add special table interaction
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
e14f2881ae
interaction: add special table interaction
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
d7326079a8
interaction: add global variable store
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
4e4e7a99f8
interaction: add global variable store
...
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
bde07ed7a8
Add tagging overlay element
...
Conflicts:
htroot/env/templates/jqueryheader.template
htroot/yacysearchitem.java
source/net/yacy/interaction/Interaction.java
13 years ago
b0bc0b4572
Add new demonstration module for client-side key-value store (backend:
...
triplestore): /DemoServletInteraction.html
Conflicts:
source/net/yacy/interaction/Interaction.java
13 years ago
c9dc6cda02
Demonstration: include value from interaction in search results
...
Conflicts:
htroot/interaction/OverlayInteraction.html
htroot/yacysearchitem.java
13 years ago
bcbd8eee33
Add several parsers, for RDFa and rdf files.
...
Conflicts:
source/net/yacy/document/TextParser.java
13 years ago
9ef5a80f4e
add interaction for triples and selector for augmented browsing
...
Conflicts:
htroot/interaction/interaction.js
source/net/yacy/interaction/Interaction.java
13 years ago
282c1620d6
Allow TripleStore to be persistent after reboot
13 years ago
5d20cd324a
Add Triplestore and RDF query interface
...
Conflicts:
build.xml
defaults/yacy.init
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
bc9a618e0a
augmented browsing: ignore js and css, integrate more user interaction
...
Conflicts:
htroot/interaction/Footer.html
source/net/yacy/interaction/AugmentHtmlStream.java
13 years ago
b21048892b
augmentedParser add features and integrate external html parser to
...
modify existing web pages
Conflicts:
addon/YaCy.app/Contents/Info.plist
build.xml
13 years ago
9cbfc1a1c0
augmentedProxy, which forwards every proxy request to a
...
rewrite engine to customize existing webpages. originally implemented by
Florian Richter.
Conflicts:
source/de/anomic/http/server/HTTPDProxyHandler.java
13 years ago
3b992e6b00
using utf8 String compression in Webstructure database
13 years ago
26301a538d
bugfix in Domains - dns-lookup
13 years ago
cde20911bb
saved a bit more ram using UTF8 String compression for OpenGeoDB and
...
Geonames data files.
13 years ago
225ee42879
made the GeoLocation into an interface with the current
...
integer implementation as accuracy implementation of 1.863cm
13 years ago
2280a7b276
- changed initialization order to prefer allocation of memory for table
...
files first
- bugfixes in memory amount calculation
13 years ago
0746308bc2
only the metadata tables shall be able to use the tail cache
13 years ago
7ec9bef0c3
fix for OOM
13 years ago
41c02cb10e
- less restrictions for usage of Table RAM copy
...
- new limit to use the table copy (instead of flag): 400MB available. If
less is available, then a copy is never used. If more is available, then
it can be used if there is a remaining space of at least 200MB
- flush caches more often: flush the Digest cache
13 years ago
b8f56a9803
npe bugfix
13 years ago
dd14b19c26
lazy initialization of block rank table ... only normal web search uses
...
this. When interactive search or location search is used, the block rank
is switched off
13 years ago
ba10caf89a
lazy initialization of database tables
13 years ago
701b9a28a0
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
htroot/PerformanceMemory_p.java
13 years ago
ab7107b34b
fixed RWIProcess queue limits: now discovering hidden results for mass
...
result retrieval
13 years ago
10c9c17d51
fixed handlemap spread factor and null iterator handling
13 years ago
b0095c8d3c
flush the compressor cache when a cleanup is done
13 years ago
a61f44f9e4
lazy initialization of block rank table.
...
this causes that the table is not initialized when there is no search is
done. the effect is most strong if YaCy is started headless which causes
no browser pop-up which otherwise would load the search page and
therefore trigger the initialization of the table.
13 years ago
96e9d77270
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
...
Conflicts:
source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
13 years ago
00f2df1120
a variety of possible memory leak fixes
13 years ago
3dd8376825
added automatic cleaning of cache if metadata and file database size is
...
not equal. It might happen that these data is different because one of
that caches is cleaned after a while or when it is too big. The metadata
is then not cleaned, but now wiped after a checkup process at every
application start. This should cause a bit less memory usage.
13 years ago
d0ec8018f5
fixes for bad long computation
13 years ago
6bb07afcc3
accept also files with other file prefix; used to read 'foreign' cache
...
files
13 years ago
96c8119b50
added GeoLocation / GeoPoint classes which uses less memory than
...
Location/Coordinates and has initializers with correct order of lat,lon
coordinates
13 years ago
461a0ce052
removed warnings
13 years ago
62ae9bbfda
allow more POIs, get more at once
13 years ago
407fdf6968
more bug fixes and performance hacks for search process
13 years ago
a1fe65b115
performance hacks
13 years ago
2fe207f813
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
5aee19daa4
added show from cache in search results (not yet finished)
13 years ago
5e562dcdb7
adopted vocabulary usage within anotation/naviagtion feature of search
...
to new SimpleVocabulary class
13 years ago
514700291a
moved Vocabulary to cora package (added in git
...
964406ad17
13 years ago
0284a4d88f
more fixes for double precision of coordinates
13 years ago
964406ad17
added concurrency enhancement to xml parser
13 years ago
240045cf7c
fix for bad distance computation
13 years ago
e0d8643226
- performance hacks
...
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
13 years ago
7a329465b3
using pre-compile pattern in blacklist; should enhance search speed
13 years ago
6e83b02b83
- bugfix for surrogate file reader
...
- bugfix for location search: suppress empty search
13 years ago
9b4c699526
ehanced location search:
...
- search request are now made using a map boundary
- search results are only computed for the map boundary
- the number of results is adopted to the results in the visible range
- added a double-buffering for the search result markers
- added a search query option for the search results:
/radius/<lat>/<lon>/<radius>
13 years ago
834dc6b263
store more data from interface access
13 years ago
1f48d1528b
performance hacks
13 years ago
c70aaccdc9
better location to generate a guid for rss messages
13 years ago
10da7335ea
performance hack: use a hash cache for all hashes that are computed by a
...
byte array. If this hash is used in a HashMap (which is very often the
case) then this hack eliminates a lot of re-computations of the same
hash.
13 years ago
f8a0cf6d7c
RSSMessages do not need a concurrent hash map -> removed overhead
13 years ago
07ca7e4dd1
enhanced RSS parsing by ensuring that it is parsed with a buffered input
...
stream
13 years ago
7c1feefb28
introduced a default 10 second time-out in rwi normalization time
...
uring search process to prevent endless deadlocks after a very long
running search
13 years ago
8d997d55b6
better logging
13 years ago
65d37e6a20
only ASCII needed in seed bitflags
13 years ago
0f82fb3628
using double instead float for a better release ordering
13 years ago
43c2c6e588
better logging
13 years ago
56087c1f23
bump to httpclient- httpcore-, httpmime- 4.2
13 years ago
71c3163f3d
- fixes to node identification
...
- added link to node in network list
- added marking of portal search node peers
13 years ago
4d3cc02168
replaced old bzip2 library against better documented commons-compress
...
package from http://commons.apache.org/compress/
13 years ago
ad222be7f8
added node state icon in network list
13 years ago
3c2bec681f
added a root node flag: identifies peers with short ping time
13 years ago
c846e9ca14
redesign of the crawler monitor page: show crawled pages instead of
...
queue of urls that shall be crawled
13 years ago
c15fcde1c8
add-on to latest commit
13 years ago
cf47d94888
performance hack to parse numbers inside of substrings without actually
...
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
13 years ago
7e0ddbd275
added a "fromCache" flag in Response object to omit one cache.has()
...
check during snippet generation. This should cause less blockings
13 years ago
81737dcb18
removed stack trace from swf parser since we cant do anything there
13 years ago
7bf421b9dd
- fixed image search page navigation
...
- removed some deadlocks and ConcurrentModificationExceptions during
DidYouMean collection
13 years ago
c6a09eab0b
synchronization needed
13 years ago
fb94b47b1a
changed queue sizes to have less memory occupied during indexing
13 years ago
76157dc2c3
bugfix for http://bugs.yacy.net/view.php?id=173
13 years ago
6696cb1313
bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer
...
SeedDB.lookupByName searche for lowercase peerNames, while MapColumnIndex.getIndex uses peername as is in the keyset.
Changed the index init to insert lowercase peer names as key
13 years ago
c6558cba08
more classification bugs
13 years ago
082831b9d6
search contentdom was checked in wrong way - fixed
13 years ago
ee553d971e
correct typo in scripts_txt comment
13 years ago
f294f2e295
bugfix to http://bugs.yacy.net/view.php?id=181
...
tried to make a bit less 'noise' to dns server
also included: less processes in snippet fetch to reduce load during
search on small computers
13 years ago
acf8d521a2
fix for http://bugs.yacy.net/view.php?id=126
13 years ago
bb88878b4d
the last commit was incomplete..
13 years ago
d320a31ae1
bugfix for http://bugs.yacy.net/view.php?id=186
13 years ago
fa735f4f04
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
3e1bc9477f
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
13 years ago
6f8a2fef1f
small speed enhancement using a column factory
13 years ago
d10627d591
More sync in close() methods
...
Conflicts:
source/net/yacy/kelondro/logging/GuiHandler.java
source/net/yacy/kelondro/workflow/InstantBusyThread.java
13 years ago
b3ae2aa41f
With or without 'final'? At least please try it in other methods
...
Conflicts:
source/de/anomic/tools/tarTools.java
13 years ago
fbb946f913
Made a method static (Eclipse suggested it), removed unused import, pk=null check does now output a warning in logfile
13 years ago
52d307c735
prevent that the snippet fectch process removes catchall entries
13 years ago
7eece0256f
moved yacy.logging to defaults according to request in
...
http://bugs.yacy.net/view.php?id=55
13 years ago
89142d1e8d
removed (not all) warnings
13 years ago
5deebd02ea
added serialization
13 years ago
b2175ea4ef
Add possibility to set custom Solr field names for the YaCy default Solr attributes.
...
- Changing the format of YaCy's solr.key.list while maintainig backward compatibility
Federated index config screens adjusted accordingly
- modified the Solr update request to use a 3 min Solr autocommit intervall
13 years ago
Michael Peter Christen