lotus
480497f7c9
changed recrawl
...
use a specific date to define old documents
this solves an unwanted recrawl-loop during a running crawl
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5081 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
da1b0b2fc6
added two new classes that will be used for the new htcache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5080 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
536e77e8b7
modifications towards a single database operation to read/write http header and cached file at once:
...
- removed distinction between header file types for http and ftp; ftp is simulated by using http properties
- removed all old resourceInfo classes that handled this distinction
- introduced a new distinction between http request and http response objects
- unified new response objects with two other object types that had been introduced elsewhere
- changed all servlet call methods to use the new http request header object type
- divided static object keys for http header properties into request and response types
- refactoring here and there (a large number of type changes and many methods merged/moved)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5079 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
04310a7255
* added long options and help-option to linux startscript
...
* redirect all error messages to /dev/null
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5078 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
borg-0300
08cdf6db8a
fix for wrong "VegaYacyB" peers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5077 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
296fa2265b
YaCy-UI: removed unused Servlet ymarks.java
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5076 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
a9cf3b42c4
YaCy-UI: removed unused JavaScript
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5075 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
08bf3fd235
YaCy-UI: updated to jQuery-ui 1.6b
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5074 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
4d937f6b21
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1396
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5073 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
lotus
5192081283
workaround for BindException at restart on Vista
...
therefore IPv6 support is lost (Windows only)
see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6286189 and http://forum.yacy-websuche.de/viewtopic.php?t=1385
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5072 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
58d7e6f1a6
- some small, rather optical changes to bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5071 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
bd931a82f7
- added dynamic filters to autoReCrawl.conf
...
- Restrict to sub-path: sub
- Restrict to start-domain: dom
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5070 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
8551e1d106
- added Info link (/xml/util/getpageinfo_p.xml) to bookmarks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5069 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
8d1bedfc3a
- added bookmarkTitle to CrawlStart_p.html
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5068 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
b3fc5e96a3
- removed unused import from bookmarksDB
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5067 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
apfelmaennchen
bc048db7b6
- bugfix for bookmarksDB's rebuildDates()
...
- dates are now saved as String.valueOf(TimeStamp)
- it might be a good idea to delete (backup) bookmarkDates.db and restart YaCy to rebuild it
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5066 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
3c68905540
remove redundant null checks
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5065 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
f9a715dc33
simplified Linux start-script
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5064 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
753a1ae430
- changed default browser from netscape to firefox
...
- fixed "Inefficient use of keySet iterator instead of entrySet iterator" [WMI_WRONG_MAP_ITERATOR, FindBugs]
- fixed some possible null pointer accesses
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5063 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
7989335ed6
Preparations to replace the HTCache with a new storage data structure:
...
- refactoring of the HTCache (separation of cache entry)
- added new storage class for BLOBs. (not used yet, this is half-way to a new structure)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5062 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
c05edba6b8
* added text/html-URL to opensearchdescription, so one-click-installation in firefox is possible
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5060 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
be28af50f5
- fixed "yacy2yacy no proxy"-problem
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5058 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
6450a51473
* fixed quoting problem in initscript
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5057 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
f99c307eff
* correct debian build dependencies
...
* add huge mem page detection in general initscript
* disable logging completely in jmimemagic-library
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5056 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter
bdae051d9a
- extended new performance graph (better timing)
...
- added paths for new libraries in classpath for eclipse
- refactoring to remove compiler warnings (static access to finals variables)
- removed some unused import
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5055 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
d9cea5ff23
removed annotations which broke the build with java 1.5
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5054 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
f1ori
7d8f332bbd
* build yacy for debianpackage only once
...
* do not try to sign .changes file
* add some java arguments from ./startYACY.sh to addon/yacyInit.m4
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5053 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
danielr
2547700de8
relaxed dependencies for use with debian 4.0 etch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5052 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
c2d49cc01e
* add build target "deb" to create debian packages from svn
...
still needs testing...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5051 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
6a550c64f1
* clean up svn ignorelist, added yacy.pid and logfiles
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5050 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
59c4e35aaf
corrected LargePages-detection
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5049 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
low012
f825e88a09
*) large memory pages will only be used if start script can confirm that Linux supports them or that the OS is Solaris, this should eliminate all sorts of weird behavior including not working shutdown
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5048 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
a087090bbb
fixed starting crawl results in "No parser available to parse mimetype 'application/octet-stream'"
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5047 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
7e7e6a099a
undo 5044
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5046 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
f2d0bd7790
fix for NPE in JakartaHttpClient.setProxy
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5045 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
bb6a6fc233
fixed 'FileUploadException Stream ended unexpectedly'
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5044 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
8422ee5ec4
- fixed UnsupportedEncoding (in proxy) using defaultCharset if no characterEncoding can be determined
...
- serverFileUtils.copy* use now Charset instead of String
- added some warnings for ignored exceptions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5043 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
3ac1988059
Add some sanity checks for invalid seeds
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5042 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
hermens
cff4393f0c
Fix HTCache so oldest Files get deleted first
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5041 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
31d97f2b9f
replaced httpd.parseMultipart() by a 'right' implementation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5040 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
621b473b18
* removed some warnings of findbugs ( http://findbugs.sf.net )
...
- removed unnecessary code (unused variables, String.toString)
- corrected some calculations (cast int to double or long ;)
- improved little performance (using Integer.valueOf() instead of new Integer)
- log if some File-actions fail (mkdir(), delete(), ...) and some ignored exceptions
- finalized some (more) fields
- finally close some streams
- made inner classes static if not using environment
- generalized some equals (from specificClass to Object)
- fixed some potential nullpointer accesses
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5039 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
lotus
5627ca1c60
* rename yacy.nsi to build.nsi
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5038 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
f1ori
d72846f898
* moved nsisant library in seperate libbuild-directory to keep root directory clean
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5037 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
3f871b57de
- again an error in defaults/autoReCrawl.conf
...
- you have to delte or edit DATA/SETTINGS/autoReCrawl.conf to fix this problem, sorry!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5036 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
0500b1179e
added a 2 min start up delay to serverBusyThread autoReCrawl to avoid a Null Pointer Exception...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5035 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
3b6229d2f3
small fix - parseBoolean doesn't recognize 1 and 0 as true and false in autoReCrawl.conf
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5034 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
apfelmaennchen
e1574fe02e
- added autoReCrawl folders to bookmarks (DATA/SETTINGS/autoReCrawl.conf)
...
- the serverBusyThread checks folders every 60 min. (==> autoReCrawl_idlesleep in yacy.conf)
- added option to create bookmarks from CrawlStart URL
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5033 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
orbiter
ebb40d324b
enhanced memory chart: shows now also the size of the word cache as third vector.
...
The PPM is now shown without a scale, but with a new anotation at the chart entry.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5032 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
17b7845eb5
* refactoring
...
- moved constants from plasmaSwitchboard to own class (all 232 ;)
- moved remoteProxy-Methods to httpRemoteProxyConfig, better names
- removed some unnecessary code (else-statements)
* formatting (correct indentation)
* minor bugfixes (due to findbugs.sf.net)
* hopefully fixed "missing quote" (announcing StringParts as UTF-8)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5031 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago
danielr
3bb870bfcd
added final where possible
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5030 6c8d7289-2bf4-0310-a012-ef5d649a1542
17 years ago