orbiter
5efc0dce0b
fix for domain options in search box
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6848 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fc5efcc05a
enhanced and fixed OAI-PMH import
...
- now importing OAI-PMH server list fron two sources
- simultanous import from several servers (even > 2000)
- check buttons on OAI-PMH server list to select multiple servers for import start
- it is possible to select all servers at once for import
- imported XML data is gzipped after import from surrogate reader
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6847 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
90c3e5d6f6
- cleanup, removed unused imports
...
- added crawling queue sizes to /api/status_p.xml, syntax same as in queues_p.html
- fixed a bug in queue enumeration that caused a out of bounds exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6842 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b18a7606a0
some performance hacks and fixed after reading dump in
...
http://forum.yacy-websuche.de/viewtopic.php?p=19920#p19920
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6837 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b7efbe55cf
i guess these documents are here by mistake
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6833 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
7417425e6a
Added (b)log notice in header of table in Network.html
...
Added some missing translations for German de.lng in Network.html page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6818 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1a8a134e0c
continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790
...
The result should be a less usage of new String() and less memory usage (since a String-encapsulated byte[] has 40 bytes overhead)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6815 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0d04ab1422
new access tracking data type strategy; previous data types may have caused deadlocks of httpd when performing statistic cleanups
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6809 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
55d8e686ea
performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6807 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5d8238680f
better thread dump statistic evaluation in threaddump servlet when creating multiple dumps for profiling: no distinguishing between thread states
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6803 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
2bc459252e
*) changes for better code readability
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6797 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
hermens
4ec0092677
more null == proxy fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6794 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
25aef069a6
continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6790 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
aad0a3045a
Fixed CSS file yacyui-portalsearch.cs
...
border-width: small; --> border-width: thin;
margin-botton --> margin-bottom
Forum Thread: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2767
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6789 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
6ef4900ec5
Changed function name status() -> statuscheck() in yacyui-portalsearch.js fixing the "status is not a function" bug visible in Firefox error console:
...
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2625
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6788 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
7a3c19846f
Updated German translation de.lng: added new Table_RobotsTxt_p.html and some other changes
...
Changed 'Sprache' -> 'Language' in yacydoc.html and added translation in de.lng
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6783 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
000dad034b
*) better usage of JS timers (intervals)
...
*) added messages when status is checked
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6781 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1e8e79b9ef
redesign of reference hash (URL-hash) parameter hand-over:
...
pass value as byte[], not as String. This should cause that less
byte[] <-> String conversions are made during time-critical tasks.
This redesign is not yet complete, more to come ..
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6775 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
9ddb8e4a43
set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6772 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0018163c07
moved table row/column matching method from front-end to back-end
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6770 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
21fcbcc35f
added sorting function in network table, reverting SVN 6736 (not removing new sorttable)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6768 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fad3abb524
Tables_p.html servlet can now show tables with selected rows using a search field
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6765 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
4c6dc396d8
*) more beautyful (IMO) code, no functional changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6764 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6c093d6aed
- enhanced domain navigator computation
...
- fixed domain navigator content in case that a mustmatch constraint was given
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6763 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bb63c5d075
using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6762 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
5233177a7f
A small typo fixed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6761 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
ea69300857
fix bad floating navigators on little results
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6758 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
bfb518cd47
some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6755 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0465f28f7f
applied 'null in rss2.js' fix from Quix0r, see
...
http://forum.yacy-websuche.de/viewtopic.php?p=19612#p19612
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6752 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
748abfcffa
added patches to prevent yacy-protocol DoS settings
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6751 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e820ed061a
avoiding excessive DNS lookups to determine localhost
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6750 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0f8004f9da
enhanced html parser to recognize a href tags inside header tags
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6743 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3300930fc5
- (almost) fixed FTP crawler
...
- integrated/fixed SMB crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6742 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
61493a9a9f
added more information about metadata in ViewFile.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6740 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
1198b9989d
bugfixes, more sorttable
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6739 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
27b2998eb4
added searchtable function to more tables in interface
...
you can now sort by any column in most tables in YaCy just by clicking on the headline column of the table
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6738 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
9623d9e6d2
added a smb loader component for the YaCy crawler
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6737 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c77fbd0390
added sorttable ( http://www.kryogenix.org/code/browser/sorttable/ )
...
javascript library to make tables sortable
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6736 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3014e5f6f9
- integrated live search in the IndexControlURLs input window for URLs:
...
this searchs for occurrences of the given word in URLs and presents them
in a pop-up list below the input line
- some bugfixes for the new robots table viewer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6735 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0769517129
added a robots.txt monitor in the crawler monitor submenu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6733 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
72f00dee59
removed never-used server access account function
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6731 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
474bb4de82
ups
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6730 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8c88abf685
added follow-me link for twitter in status hints
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6729 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
58d75a6bde
allow more results for a single query at the same time if the client is not authorized. This is necessary for the search widget where the default number of results is now set to 20 instead of 10 to cause that a scroll bar is shown which is necessary to get a trigger for new searches for more results.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6728 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4144927d94
show less errors
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6725 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
c4bdb1e7f2
added one more option in ViewFile to show an iframe like for the orginal web page content but using the cache than the direct link to the content in the web. Upgraded the very old and previously not any more used CacheResource_p servlet to a new and working version.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6719 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
884b262130
- added a new Wiki Namespace Navigator
...
- some redesign of Navigator data structures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6716 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b0c6d0108b
fix for select-all toggle in tables servlet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6715 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
270fb38674
- fixed some bugs in Table viewer
...
- added 'select all' feature in Tables_p
- enhanced ViewFile.html: has now an input field to load arbitrary resources from the web and analyze them (!!!)
- included the ViewFile servlet into the Index Administration menu
- show in ViewFile if ressource is in url-db and/or in Web cache
- bugfixes to BEncodedHeap and Tables management
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6713 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
38d7a28cd2
fix in viewfile needed when ViewFile is called only with 'url' parameter
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6712 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
599c3766c4
added authentication to automated API call
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6711 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
727dd9b193
- fixed a bug in robots.txt parser
...
- moved storage of robots.txt entries to WorkTables, so it is now possible to browse the robots entries with the table browser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6710 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
f336ed568d
fir for parameters on reload
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6708 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
ddc21e7b73
adapted banner colors to current standard template
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6707 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
e76f1e6cc0
Update property name bookmarks to display_bookmarks (changed in SVN 5554) to fix the broken Bookmark RSS feed
...
- Fixes RSS part of http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1824
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6706 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
8a19be24de
tell non-windows users about the tray-icon
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6703 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6b89f681c5
added bing, dbpedia and wolfram alpha to the compare-search options
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6701 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4b6efe3b48
more ergonomic default values for crawl start
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6698 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f175f9a2d3
changed way how number of search requests are counted:
...
so far only search requests at the remote search interface had been counted.
This was done to protect the privacy of searchers, because counting was not done and published at the own search interface.
This caused that no search requests of robinson peers had been counted, becuase they cannot be counted at remote peer.
This change introduces a distinction of locally done search requests at the local search interface from search requests that are on the local interface but had been submitted from a remote IP without authentication.
Now 3 counters are maintained:
- partial count of remote searches
- total count of local searches on robinson peers from non-authenticated clients
- total count of local searches on robinson peers from localhost or authenticated clients
In the global statistic of search requests now the first two counters of the three cases are added
Because we habe a large number of robinson peers with a large number of remote non-authenticated requests the statistic should show at least three times of the number of search requests.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6696 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
23bf81d225
new buttons to select/deselect all entries
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6681 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
564927ce72
redesign of CrawlResult data structures because of OOM occurrences during URL deletion processes.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6675 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d378ca4604
better handling of concurrency in seed
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6669 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
6538043d89
fix for http://forum.yacy-websuche.de/viewtopic.php?p=19189#p19189
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6668 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
047f8718a7
added kiosk-mode button on standard search page and interactive search page
...
see also:
http://forum.yacy-websuche.de/viewtopic.php?p=19178#p19178
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6667 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ac492fa2a5
added a close button image
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6666 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
1a4cb51744
ups, unclosed HTML tag fixed....
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6664 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
9e14958115
minor corrections and bug fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6663 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
945e0ba5a5
allow global search if res. observer disabled index transmission
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6658 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
063b29060c
Minor Changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6656 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
11188cd7eb
resource observer now uses the Java 6 method to check for free space. thus, disk observing now needs Java 6 installed.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6652 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
be18b5d8cd
fix for 'cannot switch back to default language'-bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6649 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fe78edac32
- view API calls in correct date-order
...
- execute recorded API calls in date-order
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6646 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
308a973503
refactoring of tables data organisation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6644 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8a76f38d26
Added a new steering servlet that can be used to repeat actions that had been made on the yacy interface. This can be used to:
...
- start again a previously started crawl
- submit settings (again). This option will be used to transmit
all settings of one peer to another peer if the remote-peer
steering function is ready
This steering framework will also be used for a 'schedule-everything'
which will also include a new scheduler for crawling.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6642 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
530b77d05a
adapted loglevel for accepted urls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6641 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
5f661d9f5d
fix receive of crawl receipt in private peer mode
...
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2655
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6640 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
840527689b
more simplification of bookmark class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6639 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
ada0ce9de3
refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks
...
will loose its leading role for the re-crawl funtion when the new api tables will work. To be prepared for a replacement
of such functions the bookmark class is re-organised.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6637 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
24060885b6
- added Tables abstraction in data.Tables.java
...
fix for
http://forum.yacy-websuche.de/viewtopic.php?p=18910#p18910
http://forum.yacy-websuche.de/viewtopic.php?p=18894#p18894
http://forum.yacy-websuche.de/viewtopic.php?p=18814#p18814
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6631 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
7fdf59a77f
misc NPE check
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
a29b17a2fd
Minor HTML Changes, Images recompress.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6627 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
67b8d5149e
Small Changes on Network.html - Specifying image dimension
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6625 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
e87f0e83a2
Updated German translation:
...
- Fixed a typo in ConfigPHPBB3Search.html
- Added section for ConfigPHPBB3Search.html and translation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6620 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
d915deaa2b
Fixed type in CrawlStart_p.html
...
Changed to German language file:
- Updated Crawl Start Page
- Added section for indexing MediaWikis
- Fixed some more start and end tags so that syntax highlighting works correctly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6617 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
3889438db6
fix for bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6615 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
597a410f58
ups
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6609 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8ce936bcdd
added an api recording function: it shall be possible to record
...
all operations on YaCy in a database that should make it possible
1) to re-create a setting on fresh peers
2) to transmit a setting from one peer to another
3) to re-create crawl starts after a complete deletion of the index
This functionality will also support
4) scheduled re-crawls (new implementation)
To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use)
- added a b-encoder
- enhanced the b-decoder
- added a b-encoded map heap data structure
- added a table organisation based on b-encoded heaps
- added a servlet to maintain such tables (see Tables_p.html)
- integrated the servlet into the Advanced Settings menu
- added an api recording based on the new tables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8df1694288
- added options to switch on/off search domains (text, image, audio, video, app)
...
- more memory by default
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6605 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
eac3b696ab
Fixed typos in WikiHelp.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6599 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
03fb4a1980
Fixed typos CrawlProfileEditor_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6598 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
78daea95c3
show more navigation pages if there are too less locally
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6595 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
mikeworks
25c545c4b0
first commit fixing two typos
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6594 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
234f733a3d
- relocation of seed db is better for network switch than re-initialization because of the embedding of the peers object in other objects
...
- small refactoring of blacklist interface code to remove PMD warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6593 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
763e0d447c
removed unnecessary cast
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6586 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
34354cf9b2
added a servlet that has been removed in SVN 4881; this servlet is now splitted and will be used for a simple crawl start and a remote crawl monitor (not yet integrated into the interface)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6582 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
81035e7080
moved a sub-menu entry
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6578 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
5f1565cc95
Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2540&sid=507e4dd9e234af001ba60c0d8adf45be , and small Layout Changes.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6577 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
82f57f79e5
more PMD enhancements
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5d930c96f0
more fixes to search result page navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6575 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a06f7ddb33
more PMD recommendations
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
9bbd546e64
in live search, show at least 20 entries instead of only 10
...
this is a work-around for the problem that the search widget
does not load a second page if the first page did not fill up
the window with enough lines such that a scrollbar ist visible.
Because the scrollbar triggers loading of following pages, this
must be enforced with the trick that more result lines must be
shown after the first search.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6570 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
18172451a0
better search computation:
...
- increased sort limit, now 3000 entries, before: 1000
this should cause that more results can be shown in case
of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d126d6c1b5
renamed the servlet WatchCrawler_p to Crawler_p
...
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
909a4f91c7
added a logging output for crawl starts that shows the URL that can be used to start the crawl again
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6566 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2113fcd7e5
- fixed usage of isEmpty() which is not available in java 1.5
...
- increased visibility of some methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
dd459281c8
applied code changes that are recommended by PMD
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
eac2daf2e8
* reenable DHT if yet enough memory is available
...
* reset treshold on reconfiguratoin
(thanks to sixcooler)
* display status message in web interface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
d77a8f3b3e
added some modifications recommended by PMD for better performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a3b8b7b5c5
some redesign of the main menu structure:
...
- moved all index generation servlets to it's own main menu item, including proxy indexing
- removed external index import because this operation is not recommended any more. Joining an index can simply be done by moving the index files from one peer to the other peer; they will be merged automatically
- fix to prevent endless loops when disconnecting http sessions
- fix to prevent application of bad blacklist entries that can cause a 'Dangling meta character' exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6558 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
32972139af
added nice configuration for the resource observer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
low012
04d91c67d6
*) added message if unrestricted access from localhost is possible
...
*) added link to admin access config page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6543 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
suessthomas
d933763b12
Small Changes on the de. language File and small correction to the ConfigAppearance_p.html page. Please test and report bugs to me (th-suess@gmx.de).
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6534 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
57d729e377
fix for negative numbers in network statistic
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6532 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
lotus
f8dc1dfabe
include query in document title
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6531 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2126dffea2
fix for rss/xml output for image search from AndreasB
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6523 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
362b7a929b
added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6521 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
8281e29963
- more configuration for profiling graph (number of events)
...
- more logging for a shutdown: print reason and accessing IP into log
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6520 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a44112b562
- moved index cleaner to blacklist submenu, because the index cleaner cleans the index with the blacklist
...
- version switch to 0.93 to reflect advancements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6516 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4782d2c438
fix for search bug that appeared when looking at page 3 of results or further
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6515 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
66923ebc6c
- modified method in RequestHeader that delivers the host name of requester: no more reverse domain lookup (may have killed interface performance in some cases)
...
- added logging output for shutdown servlet: show ip of requester of the shutdown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6512 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e34e63a039
preset of proper HashMap dimensions: should prevent re-hashing and increase performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6511 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4a5100789f
replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6510 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f4946eaf27
- better thread dump
...
- suppressed one server exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6509 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
491ba6a1ba
- some refactoring in workflow
...
- some refactoring in search process
- fixed image search for json and rss output
- search navigation on bottom of search result page in cases where there are more than 6 results on page
- fixes for number of displayed documents
- disabled pseudostemming
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6504 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
969123385b
added json and rss output for image search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6503 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
016fdfa310
stub for json image search (not yet ready)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6502 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
23aef43786
- better synchronization in SortStack
...
- better ThreadGroup organization
- less worker threads for media search (64 was too much...)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6497 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
fe41a84330
some enhancements in web caching: avoid double loading of response metadata and/or content
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6491 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4c6312d103
enhanced image search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6489 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2d8f3ee301
some performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6488 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
36fbfdcb21
more performance for remote search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6487 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5c7b32a4fa
better performance for list api (blacklist transfer)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6486 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
013f337d3f
- avoid unnecessary host name lookups for localhost
...
- avoid unnecessary reverse domain name lookups for remote access
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6481 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
29fe436e36
- fixed post-ranking including prefer mask
...
- enhanced a core database access method / less wasted ram
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6473 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
5399d1e2bc
refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6471 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4c99d4683d
possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6465 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
18b21eaffe
small fixes to search default values and server logging
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6460 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
4431b9767e
added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6458 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
e3025ee691
- new icon for OAI-PMH loading action
...
- added many stack trace outputs for exceptions in crawl profile handler to find the 'missing profile handle' bug
- catched one more timeout exception in httpd file loader
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6457 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
f0b8db93f0
- more abstraction of serverCore thread access
...
- no more keep-alive when number of connections exceeds 1/2 of the allowed number of connection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6456 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
11f7da06ed
- fixes to csv parser
...
- automatic OAI-PMH import by just clicking on one link from the provided resource list
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6449 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
9b6762ec2e
- added a csv "comma separated values" parser to parse OAI-PMH sources from
...
http://roar.eprints.org/index.php?action=csv
- integrated the csv parser into the crawlers parser list
- added an extension to the OAI-PMH import function to download and show the roar csv file using the csv parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6448 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
0f63de8236
- it is now possible to start several OAI imports concurrently
...
(still not possible to start them with one single request, that will be next)
- added a monitor for all running and finished OAI imports (with a little bit of animation..)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6447 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
176e334aa4
fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6446 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
2fa6bf440b
workflow update to OAI-PMH importer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6445 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
b0b7a4f9a5
- added function to OAI-PMH reader that can pull all records from a server using an evaluation of the resumption token to get URL to retrieve remaining records
...
- added monitoring for retrieved records
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6444 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
350d13e153
very first working version of oai-pmh importer: if given the right url, the importer can read and index listRecord xml files and calculate the right resumptionURL which is then given as next default start point for the importer url input.
...
no automatic harvesting by now, this will be done later
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6443 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
a0e891c63d
- some redesign in UI menu structure to make room for new 'Content Integration' main menu containing import servlets for Wikimedia Dumps, phpbb3 forum imports and OAI-PMH imports
...
- extended the OAI-PMH test applet and integrated it into the menu. Does still not import OAI-PMH records, but shows that it is able to read and parse this data
- some redesign in ZURL storage: refactoring of access methods, better concurrency, less synchronization
- added a limitation to the LURL metadata database table cache to 20 million entries: this cache was until now not limited and only limited by the available RAM which may have caused a memory-leak-like behavior.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6440 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
30f108f97d
added stub of oai-pmh importer (not working yet)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6437 6c8d7289-2bf4-0310-a012-ef5d649a1542
15 years ago
orbiter
52470d0de4
- fix for xls parser
...
- fix for image parser
- temporary integration of images as document types in the crawler and indexer for testing of the image parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6435 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5e8038ac4d
- refactoring of blacklists
...
- refactoring of event origin encoding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6434 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
26fafd85a5
- more refactoring
...
- fixed problem with parsers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6433 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3528b970d6
- refactoring
...
- added new experimental (not-yet-working) image parser
- added new test image
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6431 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b79f4f062f
refactoring of yacy documents and parsers: they depend now only on the kelondro classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
519c3619ff
*) minor changes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6424 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
f5656b2ae1
*) Made sure that only files with appropriate file endings are listed as skin or language files.
...
*) Introduced protection against directory traversal attacks in configuration servlets for skin and language configuration. Files can only be deleted if they are contained in a list of files which has been read by the servlet first.
Until now it was possible to delete any data on a system YaCy is running on and which can be deleted by the user who's account has been used to start YaCy. Most of the times a user of YaCy is also the owner of the machine the peer is running on, but this might not always be the case and not even the owner of the machine should be able to use YaCy as a replacement for "rm" or "del".
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6423 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
3434ca381f
*) grrr
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6422 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ae42c51cf7
*) Skin names and language names are displayed in alphabetical order in dropdown menu now.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6421 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
suessthomas
56a5bd090d
Small fixes to header.template for more XHTML compatibility.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6420 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
76bca8cffd
show interactive search without menu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6417 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d5eeb842a
new default skin 'pdblue'
...
The old default skin named 'default' is renamed to 'classic-blue'.
All users will keep their current default skin named default, but YaCy will copy the classic-blue also to the skin folder.
For all new peers, the new skin pdblue is used.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6416 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c864901087
- moved httpd.mime to defaults path
...
- some documentation fixes
- adopted a default setting for the search window: moves css setting to base.css
- some enhancements for the DocumentIndex class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6410 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5841ee83d3
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6400 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ce8dc575ca
refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6398 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bea3b99aff
moved table and util classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1e4f8b56ed
accumulated classes from different packages into the new rwi package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6394 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
194da25a2f
moved kelondro index
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6393 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4446acc8cd
moved kelondro order
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6392 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f677d534b1
start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root
...
- moved here the logging classes as part of the new net.yacy.kelondro package
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6391 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
735e2737e3
* added index segments
...
This is a major change in the organization of indexes.
Please consider a back-up of your data before you run this update.
All existing index files will be moved and renamed to a new position.
With this change, it will be possible to maintain different indexes for different purposes and it will be possible to have a distinction between DHT-in and DHT-out specific indexes. Tenants may also have their own index, and it may be possible to have histories and back-ups of indexes. This is just the beginning, many servlets must be adopted after this change, but all functions that had been there should still work.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6389 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
04a548a1e3
- temporary integrated the transferURL servlet as static class instead as a class that is called using reflection to investigate the OOM problems in that class
...
- fixes for numerous other problems
- removed dead code
- resdesign of the strings-method, which produces now less memory overhead and may help to prevent OOMs
- another fix for the deadlock problem in SplitTable
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6373 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
6aa474f529
- better logging for web cache access and fail reasons
...
- better Exception handling for web cache access
- distinction between access of web cache for proxy and crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6367 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0c17b600c6
remote search by default off
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6365 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a995b95367
tried a fix for the httpd access bug (too many unclosed sessions)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6362 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
a6a3090c3d
*) blacklist cleaner supports usage of regular expressions now
...
*) refacored BlacklistCleaner_p.java for better readability
*) moved check of validity of patterns to the Balcklist implementation since patterns might be valid in one implementation, but not in another
*) added method to check validity to Blacklist interface
*) fixed some minor issues like typos or wrong whitespaces
*) set subversion properties for a whole bunch of files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6359 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
3c4064932c
*) added width and hight to prevent the page from "jumping" when the image is reloaded automatically in Opera 10
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6349 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
5e4f267a36
*) added subversion properties and edited a few comments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6348 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
2d01411bbc
*) tested string will appear in input box now (when testing a similar URL it does need to be copied anymore, it can be edited it right away)
...
*) https:// and ftp:// can be used as start of the string to be tested now too
*) better error handling (no text in Java anymore)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6347 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3faa011e3d
added another search integration help page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6346 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
26b81bd1f1
added another search integration help page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6345 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
031e6eefbd
some updates to dublin core, metadata browsing, file indexing and parser stability
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6342 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
f65bfaa9af
*) Removed base tag from errror page. This has been added by myself a long time ago as a workaround for some weird behavior of my router, but as it turns out, it does more bad than good in general: If HTTPS is used for communication with YaCy, entering a wrong passwort led to an errror page with a form which would send username and password unencrypted with the user possibly being unaware of this.
...
*) changed some comments, added some annotations, added SVN properties here and there
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6340 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c3a4aee255
some redesign with a possible fix for the ReferenceContainerCache.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6336 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
feece4bfcb
slightly changed default skins:
...
- search result headline font is slightly larger and has an underline (like g**gle)
- no dashed line between results in grey style
- no search results with menu on left side by default (but is still possible)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6334 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
23ab6fbca4
- navigation appear at correct position when opengeodb-results are also presented after a search
...
- show an about box if about.headline and about.body is set
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6332 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
54c7cbf1d9
- fast result for local search in case that less than 10 hits exists
...
- small change in display of RAM in profiling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6326 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
033554dc12
more logging in tranferURL to find OOM bug. See also
...
http://forum.yacy-websuche.de/viewtopic.php?p=17361#p17361
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6317 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5658044388
- when there are no topics, the topics window is not shown
...
- othervise: close topics window by default
There will be more navigation in the future (i.e. update-time selection)
and the topics will be less important than the others, therefore
it is now hidden by default
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6310 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fd6b9cb7dc
refactoring of IO access classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6308 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
604c37927f
used comparator for did-you-mean that uses index sizes for comparisment, but:
...
- limit comparisment to only the first 10 elements that had been sorted before without IO
- added a size cache to index computation because the size is computed at least twice in set comparator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6306 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a58d9cae7d
- show location name in geolocalization search result
...
- added link from location icon to openstreetmap browser with coordinates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6305 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
8274251467
*) stopped page content from jumping in Opera 10 when loading an image took too long
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6303 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
eaddf2d464
- corrected layout of map preview
...
- added caption to maps containing latitude and longitude information
- prevented that maps occur on second search page
- added location names to did-you-mean
- some refactoring of did-you-mean
- added equal and compareTo test to Coordinates class to make that work in set
- fixed utf-8 support for library files
- fixed a bug in images search icon view caption
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6294 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fd668f531b
fixed map layout
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6292 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
2740d9dd79
added integration of osm maps for search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6291 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
af3a696fc4
added a fast-fail concept in search processes. The search now has better control if all the remote searches may bring any result. If all processes are finished, then all search tasks fail fast.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6290 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
847c3027ff
added
...
- a time-out of 10 seconds
- and a clear-on-error
in yacyui-portalsearch, to make the loading bar disappear when nothing is found.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6289 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ce972ff4ef
update to default ranking profile which has now some settings to deny some phpbb3 pages which are redundant in the index when crawling phpbb3.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6288 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1762a7bcd6
- moved DidYouMean to the data package
...
- added a DidYouMeanLibrary class that shall support the did you mean function with additional word lists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6281 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ead48c4b25
fix for preparation of search result pages with offset > 10:
...
- less pages are fetched in advance
- just-in-time fetch of next required pages
- fix for missing hand-over of offset to fetch threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6279 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1a9cfd8718
some performance hacks (CPU only, not IO)
...
this will cause better computation speed for single- and multi-core;
there are enhancements that will speed up old and slow machines as well
as multi-core CPUs. Indexing of surrogates has been speed up
from 4000 PPM to over 20000 PPM on a simple dual core office computer.
Since the enhancements are mostly in core routines, the hack should also
speed up search performance.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6276 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3ebb228ea1
added smaller icon for widget
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6271 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
61748285c3
more refactoring of search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6270 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
72e5407115
refactoring of snippet cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6268 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
93b2622503
*) repaired and added IM online status indicators
...
*) added some missing SVN properties
*) removed unnecessary comment, added missing copyright notice
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6266 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e7736d9c8d
more refactoring: made all variables in SearchEvent private
...
to prepare splitting of the class into two parts: local and remote search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6265 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d8ca6e6bf1
more refactoring for search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6263 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
72ac5bd80f
refactoring of search process.
...
this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6260 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
2e01bd955d
wrong display of hints / hints wrong / incomplete
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6249 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
39ae96450b
draw more peers in network picture
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6248 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c0e17de2fb
- fixes for some problems with the new crawling/caching strategies
...
- speed enhancements for the cache-only cache policy by using special no-delay rules in the balancer
- fixed some deadlock- and 100% CPU problems in the balancer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6243 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
634a01a9a4
replaced wget-requests with caching requests
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6242 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c6c97f23ad
- added cache usage properties to crawl start
...
- added special rule to balancer to omit forced delays if cache is used exclusively
- extended the htCache size by default to 32GB
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6241 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
161d2fd2ef
redesign of access to the HTCache (now http.client.Cache):
...
- better control to the cache by using combined request-header and content access methods
- refactoring of many classes to comply to this new access method
- make shure that the cache is always written if something was loaded
- some redesign of the process how http response results are feeded into the new indexing queue
- introduction of a cache read policy:
* never use the cache
* use the cache if entry exist
* use the cache if the proxy freshness rule confirmes
* use only the cache and go never online
- added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step.
- set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule'
- enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6239 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lulabad
da43164dd6
fix for UNRESOLVED_PATTERN see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2300
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6238 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1d8d51075c
refactoring:
...
- removed the plasma package. The name of that package came from a very early pre-version of YaCy, even before YaCy was named AnomicHTTPProxy. The Proxy project introduced search for cache contents using class files that had been developed during the plasma project. Information from 2002 about plasma can be found here:
http://web.archive.org/web/20020802110827/http://anomic.de/AnomicPlasma/index.html
We stil have one class that comes mostly unchanged from the plasma project, the Condenser class. But this is now part of the document package and all other classes in the plasma package can be assigned to other packages.
- cleaned up the http package: better structure of that class and clean isolation of server and client classes. The old HTCache becomes part of the client sub-package of http.
- because the plasmaSwitchboard is now part of the search package all servlets had to be touched to declare a different package source.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6232 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5bb8074150
removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency.
...
- The indexing queue was a historic data structure that was introduced at the very beginning at the project as a part of the switchboard organisation object structure. Without the indexing queue the switchboard queue becomes also superfluous. It has been removed as well.
- Removing the switchboard queue requires that all servlets are called without a opaque generic ('<?>'). That caused that all serlets had to be modified.
- Many servlets displayed the indexing queue or the size of that queue. In the past months the indexer was so fast that mostly the indexing queue appeared empty, so there was no use of it any more. Because the queue has been removed, the display in the servlets had also to be removed.
- The surrogate work task had been a part of the indexing queue control structure. Without the indexing queue the surrogates needed its own task management. That has been integrated here.
- Because the indexing queue had a special queue entry object and properties attached to this object, the propterties had to be moved to the queue entry object which is part of the new indexing queue withing the blocking queue, the Response Object. That object has now also the new properties of the removed indexing queue entry object.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6225 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b332dfad67
- inserted request object into response object which carries this now instead generating new objects
...
- fixed a problem with the crawler introduced in SVN 6216
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6222 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ca72ed7526
-removed superfluous crawl cache
...
-refactoring of crawler classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6221 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
13c63f4082
a set of small fixes to crawling behaviour
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6216 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b2263bc720
enhanced document type recognition
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6209 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7d493cf8cc
moved parser configuration in separate servelet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6207 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3f113f38a8
removed unused imports
...
removed unused libs from eclipse class path
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6201 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
9f083bb6b2
check filetype before loading (no more mp4 loading)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6200 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
f814e0fa81
enable warnings and fix most of it
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6196 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
8931c8d6b4
improvments to debianpackage:
...
* autoupdate completely disabled, display hint
* restart-button in interface works!
* moved all build-Variables to yacyBuildProperties
* fixed some warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6195 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4b74ad0a46
fixed setting of parser configuration servlets
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6191 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
57a88d435b
redesign of parser mime type detection and parser steering
...
There is now a mime-blacklist instead of a mime-whitelist
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6190 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
21b8704fb4
refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6188 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8ca1f5d400
- some work to integrate the html parser the same way as the other parsers are integrated (not finished)
...
- added migration of code of settings pages (hmm.. does not work correctly yet, sorry)
- more refactoring
- removed more unused code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6187 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
499723891d
removed all non-http daemons; they had not been used and may be a potential security risk.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6185 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
0e8647d62f
refactoring of search classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6184 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dafffd0153
refactoring of parsers and document processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6182 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
ba7905aaa8
PortalSearch:
...
- removed buttons
- auto load on scroll
- some small fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6178 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
ec2970cc76
higher dht distribution speed by default
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6168 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
995da28c73
all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path
...
There is no migration. All crawls must be restarted.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6167 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
409538e17a
code cleanup and code simplifcation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6161 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
154bbc3364
code cleanup: call of static methods directly to the class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6155 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
222850414e
simplification of the code: removed unused classes, methods and variables
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6154 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
9a674d8047
- After the removal of the Tree class some code simplifications are possible. This affects mostly the Records class, which can be refactored and the result of the refactoring results in a reduced number of classes.
...
- The EcoTable was renamed to Table.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6151 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
7eb3bff5b3
* workaround for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2220&hilit=#p16128
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6143 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
d69d89caed
portalsearch: better navigator call href='javascript:...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6142 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
477a4e14f9
updated styles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6141 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
0a38dd41b3
replacing styles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6140 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
565230d2a0
another atempt to fix the widget on linuxtag.de
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6139 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
f92daf9cc5
fixed the "1 to 10 of 10" bug in portalsearch for less than 10 results
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6138 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
e51ba46b8a
css fix for forum search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6137 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
0923f20993
font-size fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6136 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
edef81b41d
again a css fix for portalsearch :-(
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6135 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
32d0ee964f
more css fixes ;-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6134 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
cfd3d672a7
portalsearch css fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6133 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a71bb7178d
small fix for portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6132 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
69b573a8ac
- next release cycle
...
- doku update for phpBB3
- added more files for search test scripts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6130 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
121728643b
- css fix for navigators
...
- maybe this fixes the display problems on linuxtag.de
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6129 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
93c69fa1cb
- added hints to integrate a yacy search in phpBB3
...
- added also a phpBB3 crawl start with optimized crawl attributes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6126 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
99fa265e1d
fix for search bug caused by tenant patch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6125 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
3164d9a205
small fix for favicons
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6124 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
edcb78dcb6
updated documentation / removed faviconize
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6123 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b6058a7db1
yacyui-portalsearch:
...
- more bug fixes
- moved from faviconize to YaCy's favicons
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6122 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
96d4e9383a
added new jquer version in wiki search widget snippet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6121 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
57af311627
fix for wrong urls in navigator when a tenant is used
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6119 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
01853ce849
removed "illegal character"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6117 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
187ee4d06e
another IE fix (also same names in html and js)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6116 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
48051fef4b
another fix for IE
...
http://forum.yacy-websuche.de/viewtopic.php?p=16030#p16030
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6115 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
37c162053d
fix for IE & WatchCrawler (were same names in js and html)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6114 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f2db0d248b
fix for multi-word search (makes a replace-all)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6113 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
7fcffdcb3c
- yacyui-portalsearch: fix for multiple search terms
...
- http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2209#p16015
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6112 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b07e518bcc
added faviconcode to json search output
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6111 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
f091ad3e23
- hopefully a fix for the 'koblenz'-bug
...
- http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2209#p16005
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6110 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
d268bb5108
yacyui-portalsearch: auto open navigator sidebar
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6107 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a22103560c
yacyui-portalsearch waits with execution until all svripts have been loaded. It checks the load status every 200ms!
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6106 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
49955cc7b9
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2209&p=15984#p15984
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6104 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
2bb020a7a5
yacyui-portalsearch: small improvement to navigators
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6101 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
fee88d5fa3
maybe this fixes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2209#p15967
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6100 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a119860b82
moved IndexImportWikimedia into different menu position
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6094 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fc69a76197
update to web structure picture:
...
- allow bigger size
- better instructions for api usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6089 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ae015e8e98
refactoring of blob package classes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6088 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
be1c7ddc64
refactoring of search classes -- moved Ranking Profile to search package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6086 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b5bc399cea
added necessary synchronization for logging statistics (causes deadlock)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6083 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lulabad
e377a1e9a1
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1969
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6082 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ce1adf9955
serialized all logging using concurrency:
...
high-performance search query situations as seen in yacy-metager integration showed deadlock situation caused by synchronization effects inside of sun.java code. It appears that the logger is not completely safe against deadlock situations in concurrent calls of the logger. One possible solution would be a outside-synchronization with 'synchronized' statements, but that would further apply blocking on all high-efficient methods that call the logger. It is much better to do a non-blocking hand-over of logging lines and work off log entries with a concurrent log writer. This also disconnects IO operations from logging, which can also cause IO operation when a log is written to a file. This commit not only moves the logger from kelondro to yacy.logging, it also inserts the concurrency methods to realize non-blocking logging.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6078 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
aec3e7995a
autoconfig.pac can be used to browse .yacy-domains only
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6077 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4e825852d2
added stub for phpBB3 search integration guide
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6076 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bc6dd8194b
refactoring: moved search query class to new search package
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6075 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f348190566
tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6065 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
945777aa80
replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6064 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
303ccda69f
small fix for "did you mean"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6063 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7c4d1d471c
hand-over of more specific object
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6062 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9150bc0f7d
- don't show empty "did you mean"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6061 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
6c116be536
- set default &meanCount=5
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6060 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
09acfa66d1
- improved "did you mean"
...
- added &meanCount= to query string
- &meanCount=0 ==> no suggestion, no performance loss
- sorting suggestions by sb.indexSegment.termIndex().count()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6059 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
54a48b4184
- added "did you mean" to search page
...
- currently works for single word queries only!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6057 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
31360ba40c
- Updated ConfigLiveSearch.html
...
- added documentation for load_js and load_css
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6056 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
ab09d8ebb3
- small noscript fix
...
- noscript is now functionall but ugly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6055 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
55ef9ae12a
small fix for last post
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6054 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
36dc9b09ac
- partial update to jquery-1.3.2
...
- partial update to jquery-ui-1.7.2
- yacyportalsearch fixed sidebar for navigators
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6053 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
cffef67dc5
added a short info line about the latency monitor
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6048 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
733385cdd7
enahnced database access times by removal of unnecessary synchronization.
...
added also more hacks that resulted from high-volum query testing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6047 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
5a7dec880e
- some improvements for: http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15668
...
- portalsearch: introduced yconf.load_js and yconf.load_css
- yacysearch.html still having problems with focus after sidebar is loaded
- yacysearchtrailer.json seems not to be valid json for ?nav=all
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6046 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b21e9149f5
another fix for navigation results, the json result format and searches with yacyinteractive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6041 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15c5406b9c
fixed yacyinteractive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6040 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e0b3984805
added navigation keys for site and author facets to remote search interface
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6038 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
27fa6a66ad
- completed the author navigation
...
- removed some unused variables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6037 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a9a8b8d161
- added display of author navigation (usage of that navigator not yet implemented
...
- added a synchronization in pdf parser which should help to avoid deadlocks that occur when displaying several search results pointing to pdf sources
- fixed smaller bugs in navigation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6036 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c879783008
added steering of navigator computation:
...
- by default the navigator computation if off for servlet yacysearch.html, but:
- the servlet is called by default with a option to switch navigator results on
this will prevent that metasearch users will get slow results that are caused by unnecessary computations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6035 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c079b18ee7
- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.
...
- added a analysis method that counts bytes that could be saved in case the new HandleMap can be applied in the most efficient way. Look for the log messages beginning with "HeapReader saturation": in most cases we could save about 30% RAM!
- removed the old FlexTable database structure. It was not used any more.
- removed memory statistics in PerformanceMemory about flex tables and node caches (node caches were used by Tree Tables, which are also not used any more)
- add a stub for a steering of navigation functions. That should help to switch off naviagtion computation in cases where it is not demanded by a client
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6034 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a0c53abbe1
- wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521
...
- show only x+1 pages in page navigator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6022 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
94f3d90af2
added a hint about regular expressions in crawl start
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6021 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1c77db670f
re-designed response format for navigation:
...
- changed json and rss response templates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6019 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
15fad767c0
some refactoring of topic generation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6018 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f28f62fb21
added servlet for easy wiki content and search window integration
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6017 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
efe97f446a
better proxy configuration in case of remote proxy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6016 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
cc49aedf12
- fixed problem with remote search NPE
...
- more abstraction for search requests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6015 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
55ff919b5d
- yacysearchtrailer.html ... just an idea for a timeline
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6012 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ab06a6edd2
renamed topwords to topics and enhanced computation methods of topics
...
topics will now only be computed using the document title, not the document url,
because the host navigator is now responsible for statistical effects of urls.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6011 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
61d9e131b4
better/new proxy auto config
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6010 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
0d44a6d503
- yacy portalsearch experiments with navigation in sidebar (topwords & domains)
...
- not yet functional ...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6009 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9f9a1b4ad8
- yacysearchtrailer.html small temporary work around for jquery-css display bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6008 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b0e2d854e0
- fixed sidebar for yacysearch.html & yacysearchtrailer.html
...
- @orbiter: please do not use <h2> or <h1> tags in the context of the sidebar!!!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6007 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a5d481eab1
enhanced navigation
...
- fixed too early computation of navigation
- moved navigation rendering to yacysearchtrailer
- added more asserts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6006 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3ca1f109c4
added more jquery themes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6005 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3ea399ec91
fix for absolute paths for repository path
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6004 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
6b92155eb6
corrected spelling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6003 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5eac607166
fixed configuration of repository path
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6002 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
daee735ad7
- fix for yacysearch.html
...
- navigation/sidebar JavaScript is now also triggerd by #(navigation)# instead of display=3
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6001 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
8fe69da2bb
- some fixes for prev. post
...
- better resizing and dragging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6000 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
0eb3bffe97
- added 'drawer' (sidebar) for future navigational items to yacyui-portalsearch.js
...
- http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15311
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5999 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
34af8b4877
- yacysearch.html compromise for positioning the sidebar
...
- position is now fixed on the right top side
- should scale down to window width 800px, smaller windows will cause distortions
- see http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15293
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5997 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4522c13ee7
added option for a table prefix when importing phpbb3
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5996 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
3d5f2ff544
- added new servlets to support search portal administrators for the integration of yacy search fields in their web pages
...
- moved some servlets from here to there..
- changed menu structure
- removed yacyui-portaltest.html which contained an example for the live search which is now integrated on all pages in yacy. The code snippet example from that page is integrated into the ConfigLiveSearch.html servlet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5994 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
88426912ad
more refactoring to make the segment object easier to use and to be prepared to integrate author navigation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5992 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
99bf0b8e41
refactoring of plasmaWordIndex:
...
divided that class into three parts:
- the peers object is now hosted by the plasmaSwitchboard
- the crawler elements are now in a new class, crawler.CrawlerSwitchboard
- the index elements are core of the new segment data structure, which is a bundle of different indexes for the full text and (in the future) navigation indexes and the metadata store. The new class is now in kelondro.text.Segment
The refactoring is inspired by the roadmap to create index segments, the option to host different indexes on one peer.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5990 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fec6f9054f
some refactoring of search methods
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5988 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
4b4bddca00
added new submenu to crawler menu: import of phpbb3 forum postings from mysql
...
- yacy can import phpbb3 posts without crawling
- all data is written as surrogate
- indexed surrogate files can be re-used
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5985 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
f1a9253baa
see: http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15198
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5984 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
369bb78a7a
- added "visibility: hidden;" for yacysearch.html sidebar
...
- this only works with javascript....sorry
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5983 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
16a226b016
removed debug-alert() from last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5982 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
01334686bd
small fixes for yacysearch.html sidebar (display=3)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5981 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
63a0255166
- refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index
...
- refactoring: migrated data objects for the new connector classes
- added a DAO interface class to specify an abstract interface for database retrieval connector methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f246928c20
first attempt to add 'real' Navigation to yacy search results: host navigation
...
- after a search is started, it is analysed how many hits are in each site
- this can be done really efficient, because the navigation information is hidden in the url hash and can be computed very fast
- the search result shows a column on the right with the hosts and the hits per host
- after a click on a host the search is modified using the efficient site: - operator
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5976 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
54b9e99c01
- more information about peer tags
...
- peer tag is by default '*'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5975 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
41dd31cad2
replaced new navigation icons with same images but smaller resolution (16x16 instead of 128x128)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5961 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bffbe43e09
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14522#p14522
...
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14955#p14955
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5959 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
16d28496ac
*) YaCy will load status page once it is up again after restart or update if Javascript is enabled.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5957 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1efe686e3f
fix for http://forum.yacy-websuche.de/viewtopic.php?p=13960#p13960
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5954 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
13fb84ab81
you can define your default number of search results displayed by search.items
...
this applies only to requests through the classic-style page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5953 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a642d6a7b5
- added navigation icons for search result pages
...
- modified result page rendering to use new icons instead of numbers
- set different default values in yacy.init for higher indexing performance; removed pro-values
- modified WatchCrawler to accept 30000 PPM instead of only a maximum of 6000 PPM
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5952 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
012157c87f
*) fixed "select all" and "deselect all"-button
...
This bug was introduced when search boxes were added to the header of each page and there may be more pages which are affected.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5951 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c01d6f43e1
- fixed problem with thread dump if no arguments are given
...
- rejecting peers that are older than 6 hours (not-seen during 6 hours)
- 0.78, targeting 0.8 at the end of the week
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5948 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d3ca038979
fixed domain graph applet (visible in Terminal)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5946 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
04ec42e1d0
removed index Transfer, because it does not make sense any more to do a complete index transfer using the DHT methods. It is far easier and faster to just copy the whole index files from the source to the target peer. To do this, copy all files from DATA/INDEX/<network>/TEXT/RICACHE and DATA/INDEX/<network>/TEXT/METADATA from the source to the destination peer.
...
Future versions of YaCy will have different file structures than given here, because of the new index segment concept, where each peer may have several indexes (called segments) for each network definition. A copy from one peer to another will then create a new index segment.
To track changes for this feature, see forum discussion at
http://forum.yacy-websuche.de/viewtopic.php?p=14728#p14728
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5945 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
29f93b2abf
Kaskelix looks forward to see you again :-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5943 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
ea5729ce21
hint for search parameters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5938 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
55d51e4b8c
fixed bug in indexcleaner
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5933 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
89aeb318d3
enhanced the wikimedia dump import process
...
enhanced the wiki parser and condenser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5931 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
5fb77116c6
added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see
...
http://localhost:8080/IndexImportWikimedia_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5930 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
492165338d
small addition for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5917 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
bde88b684a
* splitt off yacyRelease from yacyVersion
...
* added some gui infos about signatures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5916 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
ac3139a1fd
fix for live-search positioning in header.template
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5915 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lulabad
c3cd70b461
better human readable date format
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5913 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d2ac0aa682
- fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling
...
- increased default memory size to 180MB
- fixed possible bug in http client reset (there was a deadlock)
- bug in BOBHeap marked, but not solved, cause is still unknown.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5912 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
acfd79ab4a
yacyui-portalsearch: added keyboard controlls
...
ESC: close window
PageDown: next search result page
PageUp: prev. page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5911 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
f490eab6ba
yacyui-portalsearch: clear input on window close
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5910 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
ff5f82d780
*) removed description of removed commands from wikiHelp ([= =])
...
*) used format function of Netbeans for wikiCode to make it more readable, no functional changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5907 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d3f8aa5a2a
set of small fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5903 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
84518710d0
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5901 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
63097513ae
maybe this smoothens the timing problems yacyui-portalsearch.js a little bit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5900 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d31e6f9c14
fix for http://forum.yacy-websuche.de/viewtopic.php?p=14457#p14457
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5899 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
0e01e846ef
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5898 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
d8fca85c11
fixed search: allow dots in operators
...
added new operator "tld:" which was the former "site:"
"site:" uses fast site operator introduced in r5770
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5897 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d079d6dfdb
small changes in surrogate reader, wiki code and portal test
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5894 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
07f09742bb
set of small fixes and comments
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5893 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
d9e62508e5
fixed a NPE bug (from SVN 5888)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5892 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
50e96ee894
- moved live search window in yacy to different location
...
- changed div id for live seach from 'yacy' to 'yacylivesearch'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5889 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
dd489fcce0
fixed some json encoding problems in yacysearchitem
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5888 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
0b4e4bbddd
fixed css interferance for previous posts
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5887 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
ed4fb3bf75
small fix to portal search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5886 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
753affca4b
- request: http://forum.yacy-websuche.de/viewtopic.php?f=9&t=2041#p14336
...
- integrated portal search in yacy web interface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5885 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
4cf8b08eec
Portal Search:
...
- request: http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1762&start=50#p14350
- window closes for empty query
- example for fancy input field
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5884 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
42d936288e
small update for RichClient LogViewer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5883 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
2714ff034b
avoid undefined in rssTerminal
...
thanks to freq.9! http://forum.yacy-websuche.de/viewtopic.php?p=14288
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5881 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
6f5ea7b1a8
small fix for previous post
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5879 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
2eabd989ce
- added a log viewer to RichClient (alpha version, very slow)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5878 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
557c2a32a3
small fix for yacyui-portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5876 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b4539a61dd
some more documentation for yacyui-portaltest.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5875 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
64a63306b8
added portal-test explanation page to the customization submenu
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5874 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
64ce9da60f
- new yconf parameter global
...
- see http://forum.yacy-websuche.de/posting.php?mode=quote&f=9&p=14207#pr14207
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5873 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
5ca306da9a
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2054#p14306
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5872 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9325198c42
hopefully a fix for http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1762#p14305
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5871 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
675f350d18
YaCy Portal Search Widget
...
- see http://localhost:8080/yacy/ui/yacyui-portaltest.html
- two new parameters (logo and link) for yconf as requested at http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1762#p14101
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5869 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
09987e93fd
fixed some more bad handling of byte[]
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5865 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
fbca4f8354
more stability on watchcrawler
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5861 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dec495ac78
added dummy class for help page
...
see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2033&hilit=&p=14107#p14107
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5854 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
daea87d436
do not accept dht from bad versions
...
delete bad hashes on receive
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5852 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dfb96ecb72
more fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5844 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
1b8d346b4c
fixes in connection with transiton to byte[] hashes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5843 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e7559f3234
fix for http://forum.yacy-websuche.de/viewtopic.php?p=13977#p13977
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5838 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
fa3adbbfc6
added domain checks to surrogate reader and RWI transfer receiver to prevent spaming using surrogates
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5837 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
31c6934df2
*) fix for r5832
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5835 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
616a4d724f
high-end favicon with 2 versions:
...
* true color + alpha channel for modern browsers
* 256 colors and non-transparent background for others
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5833 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
d1116c049f
*) added new method "contains()" to Blacklist interface
...
*) implemented contains() in class AbstractBlacklist
*) used new method in Blacklist_p to prevent double entries in blacklists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5832 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
94a6c83256
* rewrite code without using java 1.6 features
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5829 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
29e96c1a60
bugfixes and performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5824 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
44daec7936
* introduce signatures to autoupdate
...
as long as there aren't publickeys for the updatelocations set,
no signatures are checked
* wiki-article follows...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5822 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
e16c25ddf7
(peak-) performance hacks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5819 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
63cd152969
fixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5818 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
870066ab35
another fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5814 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b15b059c0d
fix for latest commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5813 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c8624903c6
full redesign of index access data model:
...
terms (words) are not any more retrieved by their word hash string, but by a byte[] containing the word hash.
this has strong advantages when RWIs are sorted in the ReferenceContainer Cache and compared with the sun.java TreeMap method, which needed getBytes() and new String() transformations before.
Many thousands of such conversions are now omitted every second, which increases the indexing speed by a factor of two.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5812 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
dd6b5005ff
* fix missing charset handling in getpageinfo_p
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5811 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bd5f4c78d8
- added default profile for surrogate indexing
...
- integrated surrogate indexing into indexing queue process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5810 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
ad78e3a59f
- less lines in rssTerminal
...
- crawl more documents: if remote crawling is enabled, a remote crawl list is also loaded if a local crawl is running in case that the indexer is idle
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5809 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
12d81e98eb
- fixed bad search results when searching for empty string
...
- simplified result handling and page composition in case that nothing was searched
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5807 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
8a24350036
- fix for join method with new generalized RWI data structure (caused by latest commit)
...
- added more functions to mediawiki parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5806 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
89ec3acb3e
- full abstraction of index content type: the kelondro full text index may now also contain indexes about other content than text, i.e. navigation indexes or reverse linking indexes.
...
- during index joins all word positions are maintained: better ranking for word distance possible; exact phrase match can be implemented soundly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5804 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
dc2af61bc9
allow up to 50 results from remote peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5802 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
6504b21cea
*) fix for http://forum.yacy-websuche.de/viewtopic.php?t=1976
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5800 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
601d63ef48
removed comment tag (no use at this point)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5797 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
low012
0c8fd811dc
*) first and very limited version of XML import, does not use benefits provided by XML yet
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5795 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c08f9b36a4
refactoring of wiki parser.
...
This was done to prepare the wiki parser as parser for wikipedia dumps, which will be used for performance test (to omit crawling)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5785 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
faeff21012
- fix for display of automatic ReCrawls in surftips
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5784 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
44e01afa5b
- refactoring
...
- a little bit more abstraction
- new interfaces for index abstraction
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5783 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
c2359f20dd
refactoring: better abstraction of reference and metadata prototypes.
...
This is a preparation to introduce other index tables as used now only for reverse text indexes. Next application of the reverse index is a citation index.
Moved to version 0.74
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5777 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
a29a11e526
added evaluation of incoming links in webstructure api
...
the api hash changed, new XML schema.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5774 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
f21a8c9e9c
a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5771 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7ba078daa1
- added fast site-operator
...
- refactoring merge into BLOBArray
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5770 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bd409fb7ba
added web structure analysis for a special domain that can be requested from the api.
...
Example:
http://localhost:8080/api/webstructure.xml?about=www.yacy.net
returns a xml with the following content:
<?xml version="1.0"?>
<webstructure>
<domains reference="reverse" count="1" maxref="300">
<domain host="www.yacy.net" id="FXg39Q" date="20090401">
<citation host="java.sun.com" id="o-R3yY" count="1" />
<citation host="yacy-suche.de" id="-KCLaB" count="1" />
<citation host="suma-ev.de" id="VRAHIA" count="1" />
<citation host="www.kit.edu" id="EMaLDQ" count="1" />
<citation host="yacy.net" id="Fh1hyQ" count="1" />
<citation host="www.fzk.de" id="V2Kl-A" count="1" />
<citation host="en.wikipedia.org" id="rwtdfR" count="3" />
<citation host="vimeo.com" id="MmdQDY" count="3" />
<citation host="liebel.fzk.de" id="sX4ozA" count="6" />
</domain>
</domains>
</webstructure>
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5766 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
587838bd09
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5758 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
96eaecda3e
- added migration class to go from index collections to the index cell data structure.
...
- added better control over file deletion, because this sometimes fails, especially on windows
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5756 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9ab009b16b
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1890#p13476
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5755 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
eb65990f85
small fix for opera in yacyui-portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5751 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
695c420bcd
small fix for yacyui-portalsearch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5750 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
95885a263a
- added default properies to yacyui-portalsearch
...
- see http://localhost.8080/yacy/ui/yacyui-portaltest.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5749 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
c001a020a9
- small modifications to yacyui-portalsearch
...
- see http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1762&p=13459#p13459
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5748 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
34a825f90d
small fix for yacyui-portaltest.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5746 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
9f9d7f875d
small fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5745 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
453f3aaa94
RichClient: further clean-up
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5744 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
e888c9a934
RichClient:
...
- renamed base theme to start theme
- reoved all but start theme
- additional themes can be downloaded from http://jquery-ui.googlecode.com/files/jquery-ui-themes-1.7.zip
- or a custom theme can be generated at http://jqueryui.com/themeroller/
- themes are installed into DATA/LOCALE/htroot/yacy/ui/css/themes
- update for RichClient theme selection will follow soon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5743 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
42c5f930c8
reverted an accidental commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5742 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b5e6232f8d
small correction of font-size for portal search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5741 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
7425c6c3ca
added an ajax loading graph to portal search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5740 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a975ae4a7e
Added YaCy portal search: http://localhost:8080/yacy/ui/yacyui-portaltest.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5739 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b57a1820bd
small fix for jquery-faviconize-1.0.js to handle https properly
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5738 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
075b58a0a9
minor fixes for RichClient search
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5737 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
8c494afcfe
svn attributes added
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5734 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
67aaffc0a2
- added Latency control to the crawler:
...
because of the strongly enhanced indexing speed when using the new IndexCell RWI data structures (> 2000PPM on my notebook), it is now necessary to control the crawling speed depending on the response time of the target server (which is also YaCy in case of some intranet indexing use cases).
The latency factor in crawl delay times is derived from the time that a target hosts takes to answer on http requests. For internet domains, the crawl delay is a minimum of twice the response time, in intranet cases the delay time is now a halve of the response time.
- added API to monitor the latency times of the crawler:
a new api at /api/latency_p.xml returns the current response times of domains, the time when the domain was accessed by the crawler the last time and many more attributes.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5733 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
61f9dbf0cc
- fixed a display problem in watch crawler
...
- another small enhancement in balancer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5729 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
b3f75e48fa
- enhanced balancer: auto-solving of waiting-deadlocks
...
- removed deprecated cache-init size value
- more debug lines for IndexCell cache dump merge
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5728 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
83792d9233
more refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5722 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
474aac65af
more refactoring
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5719 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago