boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity
5758 Commits
1 Branch
20 Tags
851 MiB
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'fd31a3616a'
${ noResults }
Commit Graph

3802 Commits (fd31a3616a094d8a4356da183f724f662230ecbc)

Author SHA1 Message Date
f1ori fcb77c3140 * added .im (Isle of Man) to TLD-list
16 years ago
orbiter b81c7467d8 protection against too many files in RICELL in case of massive emergency dumps caused by low memory
16 years ago
orbiter d4d87d90c4 - extended experimental wikipedia dump parser
16 years ago
orbiter c3aff2521e fix for NPE
16 years ago
orbiter 57c00dd8c9 fix for bad filtering of common http error
16 years ago
orbiter 14361f1ca4 added log message for index generation in HeapReader
16 years ago
orbiter c08f9b36a4 refactoring of wiki parser.
16 years ago
orbiter 44e01afa5b - refactoring
16 years ago
orbiter 82fb60a720 increased memory limit for emergency cache flush
16 years ago
low012 9180617dd9 *) Classes to handle import of lists (especially blacklists) from XML files, not used yet, but will be used soon.
16 years ago
lotus 596e6215dc fix in case of white space in path name
16 years ago
orbiter b887f4a116 keep more free mem
16 years ago
orbiter c2359f20dd refactoring: better abstraction of reference and metadata prototypes.
16 years ago
orbiter ab656687d7 more strict BLOB initialization .. may also help to save some ram
16 years ago
orbiter 5b138ada16 fixes to web structure reference collection and url construction
16 years ago
orbiter a29a11e526 added evaluation of incoming links in webstructure api
16 years ago
orbiter f6691411b5 - migration of files from SplitTable (which are used for the URL-DB) to a different file name format.
16 years ago
shostakovich 1f37cc6107 Robots.txt is now reused after one day. See forum-topic:
16 years ago
orbiter f21a8c9e9c a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.
16 years ago
orbiter 7ba078daa1 - added fast site-operator
16 years ago
orbiter b4126432bc hardening of index dump write process
16 years ago
orbiter 9bfb2641db - removed deprecated threads
16 years ago
orbiter 293290c317 fix for bad assert in last commit
16 years ago
orbiter bd409fb7ba added web structure analysis for a special domain that can be requested from the api.
16 years ago
orbiter b6c2167143 - patch for bad web structure dumps
16 years ago
orbiter 0139988c04 - added writing of temporary file names and renaming to final file name when index dump/merge are done. Interrupted merges can be cleaned up.
16 years ago
orbiter 3621aa96ab - added a memory protection for the IndexCell migration
16 years ago
orbiter 568e8f1741 fix in unmountBLOB
16 years ago
orbiter 9da69d6b68 - better selection of files to be merged
16 years ago
orbiter d39a5b42ca more care about open file handles. Now files also close on windows and can be deleted afterwards.
16 years ago
orbiter 029495e64d fixed bug introduced in SVN 5756 in EcoTable.put()
16 years ago
orbiter 587838bd09 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5758 6c8d7289-2bf4-0310-a012-ef5d649a1542
16 years ago
orbiter d2e2420a68 - added another file selection method for index cell merge
16 years ago
orbiter 96eaecda3e - added migration class to go from index collections to the index cell data structure.
16 years ago
orbiter 0f0b4aec75 better index cell merge logic
16 years ago
orbiter 832fef670f migration of urls-files into subdirectory METADATA
16 years ago
orbiter fa07234d4e fix for clear method: now deletes files
16 years ago
lulabad df87e4dbf6 missing count of send Index and URLs
16 years ago
borg-0300 c450e3746b svn attributes added
16 years ago
orbiter 37f892b988 added new concurrent merger class for IndexCell RWI data
16 years ago
borg-0300 8c494afcfe svn attributes added
16 years ago
orbiter 67aaffc0a2 - added Latency control to the crawler:
16 years ago
orbiter 0926310461 another performance hack
16 years ago
orbiter ebe5d69d14 performance hacks
16 years ago
orbiter 61f9dbf0cc - fixed a display problem in watch crawler
16 years ago
orbiter b3f75e48fa - enhanced balancer: auto-solving of waiting-deadlocks
16 years ago
orbiter 9a90ea05e0 added a merge operation for IndexCell data structures
16 years ago
orbiter d99ff745aa fix for http://forum.yacy-websuche.de/viewtopic.php?p=13378#p13378
16 years ago
orbiter 0c3ab291c4 fix for http://forum.yacy-websuche.de/viewtopic.php?p=13354#p13354
16 years ago
orbiter a9cea419ef Integration of the new index data structure IndexCell
16 years ago
borg-0300 fd0976c0a7 refactoring
16 years ago
orbiter 83792d9233 more refactoring
16 years ago
borg-0300 ce79239322 "typo"
16 years ago
borg-0300 cdbdc731c5 small updates: unescape, isCGI
16 years ago
orbiter 474aac65af more refactoring
16 years ago
orbiter 209f25f5f5 refactoring to integrate indexCell data structures
16 years ago
borg-0300 359a238acf faster isCGI()
16 years ago
borg-0300 f75628e53b some corrections
16 years ago
orbiter b7138e5fcb even more efficient comparator calls (less System.arraycopy for primary keys)
16 years ago
orbiter 65784eb656 - more efficient comparator calls
16 years ago
orbiter 44874cb550 added a deleteOnExit for blob file deletion in case that a deletion is not successful.
16 years ago
orbiter 66f78d67e0 bad idea. Concurrency in index management will be done differently
16 years ago
orbiter 7dff1cba62 removed option to use different primary keys in kelondro tables
16 years ago
orbiter 7f67238f8b refactoring of plasmaWordIndex: less methods in the class, separated the index to CachedIndexCollection
16 years ago
orbiter 14a1c33823 refactoring of wordIndex class
16 years ago
orbiter d49238a637 more performance hacks: better default values for scaling, less memory usage
16 years ago
orbiter 39644dc14e performance hacks to compare methods in database core
16 years ago
orbiter e2e7949feb replaced old PPM computation with a better one that simply sums up events that had been stored in the profiling table.
16 years ago
orbiter f6d989aa04 added new class RowSetArray which arranges RowSet objects like Elements in a hashtable, but still provides the functionality of sorted enumeration. The new class is now integrated into the ObjectIndexCache, which is the core class to provide index functions to all database files. The new index access is about twice as fast as before. This has strong speed enhancement effects on all parts of YaCy.
16 years ago
borg-0300 0a2fabeef3 static TMPDIR
16 years ago
lotus 9f7e62e900 refactoring
16 years ago
lotus f35dc11dc4 allow crawl start from pages with script tags
16 years ago
orbiter 6958eff196 removed unnecessary exceptions, extended testing in IntegerHandleIndex
16 years ago
orbiter 13c666adef performance hack to ObjectIndex put() method:
16 years ago
orbiter 1f1be1518c added stub for another performance hack: concurrent indexes
16 years ago
orbiter 3e4c28e188 enhanced count feature for kelondroRowSet. This is about twice as fast as before. Should speed up the collection analysis (half time!)
16 years ago
orbiter 84e37387a2 fix for last commit and more testing stub
16 years ago
orbiter ca006c506d stub for performance enhancements for RowSet (no functional change yet)
16 years ago
orbiter d988204875 better shutdown of tools
16 years ago
orbiter 100247bdda added also an export and delete-feature to the URLAnalysis. This completes the clean-up feature for URLs. To do a complete clean-up of the url database, start the following:
16 years ago
hermens 8c60d6d117 In DHT selection delete only those references that were actually selected
16 years ago
orbiter 60078cf322 added next tool for url analysis: check for references, that occur in the URL-DB but not in the RICOLLECTIONS
16 years ago
orbiter b1ddc4a83f do not merge collections if ram == false
16 years ago
orbiter dbdd10da84 better logging and startup behaviour for referenceHash computation
16 years ago
orbiter d64836c34f added statistical analysis of URL reference
16 years ago
orbiter 3b28daab40 code-beautification (to be consistent with external documentation paper)
16 years ago
orbiter 485c9406e5 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1915&hilit=&p=13249#p13249
16 years ago
orbiter 858f800a07 more logging in httpd to detect shutdown cause. See also:
16 years ago
orbiter b80db04667 - refactoring of IntegerHandleIndex and LongHandleIndex (better method names)
16 years ago
lotus 8ee946bf1d show upnp status
16 years ago
orbiter 16f5c6a85e fixed merge method initialization in ReferenceContainer
16 years ago
orbiter d7a493b4f5 added experimental timeline api
16 years ago
orbiter efcd95dc37 simplification of (internal) query process / refactoring
16 years ago
orbiter f1b712c29a small corrections to image loading methods in result presentation
16 years ago
orbiter d4b56d5819 added more asserts to BLOBHeap.flushBuffer() to fix the problem described in
16 years ago
f1ori c545fcb9fa * add class to handle keys and signatures
16 years ago
orbiter aa44d9bad9 more refactoring of kelondro.text / deleted de.anomic.index
16 years ago
orbiter 6ffc6e3389 more refactoring of indexer and kelondro classes;
16 years ago
orbiter 404bc21da9 simplification of (internal) query process / refactoring
16 years ago
orbiter 76ef5f0f14 refactoring of index package: better names for the classes (to be continued)
16 years ago
orbiter 2df57b1fd1 refactoring of index collection class
16 years ago
lotus 39a177649b * added upnp listener for devices that do not respond to discovery but advertise themselves
16 years ago
orbiter d1d9fbae5c enabling the URLAnalysis to operate on multime input files, just use a wild card when calling the class from the command line
16 years ago
orbiter c728879ab8 fixes to yacyURL - more exceptions in case that urls are strange
16 years ago
orbiter 7542336ae5 performance enhancement to yacyURL: omit second processing of resolveBackpath. This method is already applied during initialization of the object and was called a second time when the url was exportet.
16 years ago
orbiter 7ea53fe47b added another url list transformation option:
16 years ago
orbiter e521e81148 bugfix in yacyURL (for latest performance hack)
16 years ago
orbiter 54625360f7 performance update
16 years ago
orbiter d884c4718a added gzip support for URLAnalysis:
16 years ago
orbiter 46632f4385 performance update to yacyURL
16 years ago
orbiter cf9b74e6e3 added another method to process url lists: extract hosts only
16 years ago
orbiter 89d8e824ed memory protection for URLAnalysis
16 years ago
orbiter 0f6fa804ff performance update to URLAnalysis
16 years ago
orbiter 8444357291 added new row interator in kelondro tables files that enumerates rows
16 years ago
orbiter e8f5f2f612 added tool to analyse url strings
16 years ago
lotus 6117e083e5 option to customize tray label (tooltip) with tray.label
16 years ago
orbiter b8c3803bfc don't panic when canceling server sessions
16 years ago
orbiter de714783b1 - added host, path, filename to search result
16 years ago
lotus 9519d84372 changed "dooble" variable to "browserintegration" to be less specific
16 years ago
lotus 8429083972 adjusted tray for dooble:
16 years ago
orbiter ef62ec635e removed overwriting of logging config
16 years ago
orbiter c852d2d70e - reject too old seeds
16 years ago
orbiter aca973e2d9 catch more exceptions
16 years ago
orbiter 9559bc23fd automatic clean-up of dead connections
16 years ago
hermens 02dfd6183b Fix logging in serverCore
16 years ago
hermens d30456e2c8 Fix logging in serverCore
16 years ago
orbiter 4f9dae2571 remove reference in crawl entries
16 years ago
orbiter 1ba4301920 automated interruption of dead incoming connections, if they are there for more than one minute
16 years ago
orbiter c12bb8a6d0 - refactoring of the http client
16 years ago
orbiter 5d3983faae the soLinger parameter was wrong.
16 years ago
orbiter 62505bb3cb more bugfixes as recommendet by findbugs
16 years ago
orbiter 6b450d09ca some fixes recommended by findbugs
16 years ago
orbiter 4db80065ac select more
16 years ago
orbiter 94c42691d8 - reject less transmissions as transmission receiver
16 years ago
orbiter f887fc159f try to reduce the large number of unclosed incoming connections
16 years ago
orbiter e04a0e05c3 fix for last commit
16 years ago
orbiter a9ad863686 second part of 'doubles' fix - better handling of doubles in RAMIndex. More logging.
16 years ago
orbiter 59427064fb first part of 'doubles' fix (not fully ready yet)
16 years ago
orbiter 26978b2a25 - better memory protection in kelondro caches: computation of needed memory for cache grow
16 years ago
lotus e9e2fff47a better scaling on performance graph
16 years ago
lotus 4aad461100 added UPnP support
16 years ago
orbiter 99b9788e54 fix for possible 100% CPU caused by concurrent access of HashMap
16 years ago
orbiter be0c492ae5 fix for memory leak bug in new dht transmissions
16 years ago
hermens 2173865f92 Prevent race condition when switching timezones.
16 years ago
orbiter 40d9849aa4 - better control of chunk size in dht selection
16 years ago
orbiter 30a1de41b3 disabled the BufferedIOChunks, because I consider it as broken.
16 years ago
orbiter 411f2212f2 more memory leak fixing hacks
16 years ago
orbiter 985d421f91 found and fixed some memory leaks
16 years ago
orbiter 333489420b - fix for NPE when loading the cytag image
16 years ago
orbiter 6a32193916 - refactoring of cache naming in web index cache (no more dht semantics there)
16 years ago
orbiter 6c627dbdff update to the server core
16 years ago
orbiter 5393f356aa fix for termination problem
16 years ago
orbiter 6a876ecb88 first fixes to the DHT transmission process
16 years ago
orbiter c25c334b75 replaced old DHT transmission method with new method. Many things have changed! some of them:
16 years ago
orbiter e9a4182e6a using a concurrent hash map for the template cache
16 years ago
lotus e8ae2599fd * some refactoring/moves to consoleInterface
16 years ago
orbiter 01b97ef3f8 added new cybertag-tracking feature that was inspired by itgrl
16 years ago
borg-0300 b19bc611b0 gc: better logging
16 years ago
orbiter b1f9c00118 fix for bug in merge operator initialization
16 years ago
orbiter b57c9da1f8 - fixes to doc, ppt, xls parser: better title
16 years ago
f1ori 7936e58fe7 * sorry,previous version didn't compile
16 years ago
f1ori 76cdc59789 * added some convertions to and from UTF-8
16 years ago
orbiter 65a1de6c05 longer timeout for remote crawl queries
16 years ago
orbiter 072dd01ac8 more logging for RSS parser (to fix the remote crawl problem)
16 years ago
orbiter 9d282d2c16 - renamed interactivesearch to yacyinteractive
16 years ago
orbiter 180fe81ef7 quick hack to copy new log configuration over old one
16 years ago
orbiter d3e33fd6c1 removed strange retry logic from DHT transfer
16 years ago
orbiter db510b5d52 more exception logging
16 years ago
orbiter ef82cced01 removed default line 'P2P WEB SEARCH' if no line is given
16 years ago
orbiter 78b7361937 fixed problem with logging
16 years ago
low012 f136ddcfd4 *) this change is supposed to prevent the creation of temporary files by Apache Commons Fileupload library in cases where it is not necessary (as proposed by thq in http://forum.yacy-websuche.de/viewtopic.php?f=8&t=1806)
16 years ago
orbiter 94110df85a moved logging partially to kelondro
16 years ago
orbiter 024da2916b refactoring of logging
16 years ago
orbiter 83ce65707a (almost) completed partition of classes in kelondro
16 years ago
orbiter 7ee494fde5 more refactoring of kelondro:
16 years ago
lotus d4281b78da dynamic memory scale
16 years ago
orbiter bf93767ec6 refactoring of kelondro database classes
16 years ago
orbiter fc27bf8c4c refactoring of kelondro classes:
16 years ago
orbiter fe77fc3d62 - added new property setting 'repositoryPath'
16 years ago
orbiter 6cbca1e508 extended last fix, preventing more sorts
16 years ago
orbiter f9672d3f97 applied fix for inefficient put method as recommended by celle, see
16 years ago
apfelmaennchen 3484e55be4 - small fix for bookmarksDB
16 years ago
apfelmaennchen 6dd52422ea - added two dialogs to manage bookmark tags in YaCy-UI
16 years ago
orbiter 3154926311 some better memory protection and OOM prevention in EcoFS
16 years ago
f1ori aaafe05c02 * revert debug change
16 years ago
f1ori 5570fa817b * remove & from openBrowser command (see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1728&hilit=#p12321)
16 years ago
orbiter 419469ac27 added more methods to control the vertical DHT (not yet active .. )
16 years ago
lotus 4ef6b15eb8 limit -Xmx setting to 1999m on win32. bigger values would never work.
16 years ago
orbiter dedfc7df7f removed distinction between DHT-in and DHT-out. This is necessary to make room for the new cell data structure, which cannot use this this distinction in the first place, but will enable the same meaning with different mechanisms (segments, later)
16 years ago
orbiter b74159feb8 preparations to integrate the new 'cell' index data structure
16 years ago
orbiter d399444e49 added debug information to class loader
16 years ago
orbiter 5080fc33bf fix for http://forum.yacy-websuche.de/viewtopic.php?p=12247#p12247
16 years ago
orbiter 335d6ce8fc fix for class loading problem
16 years ago
low012 78778df464 *) this should adjust the Dev/Main detection of the updater to the new version numbers (0.7x is Dev, if x != 0)
16 years ago
orbiter b423d0a036 moved all servlets from htroot/xml to htroot/api
16 years ago
orbiter 91af105373 last changes before release
16 years ago
f1ori 7eade3f181 * fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1728
16 years ago
orbiter d1bace5e4d enhanced cleanup function
16 years ago
orbiter cb76d9e0e4 more synchronized in BLOBHeap (will not fix problem with Runtime-Error as reported in forum)
16 years ago
orbiter ff41da613e removed exception printout during load of snippets
16 years ago
orbiter 814a28775f removed thread dump writing in case of invocation target exception in httpd (looked bad, not serious)
16 years ago
orbiter bed38a5f8c fix for uncaught exception in RSSReader
16 years ago
orbiter 05c235de32 fix for npe
16 years ago
low012 7608944081 *) bugfix for REMOTE_HOST environment variable in CGI code (shows hostname of client instead of hostname of YaCy peer now)
16 years ago
orbiter a6b29cf72c reverted change of search event processing in SVN 5460. The new code did not work properly,
16 years ago
orbiter 9ef77d57f5 added an access control to the search interface using white/blacklists:
16 years ago
orbiter efe801173c better dht-in cache flush. see also:
16 years ago
orbiter 941ab78d9b better termination for blocking threads
16 years ago
apfelmaennchen 3dc208fad0 bugfix: bookmarks can now handle folder names like /news and /newspaper without getting confused...
16 years ago
orbiter e948df68ac longer timeout for queues during shutdown
16 years ago
orbiter 2b32248079 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1516&p=10545#p10545
16 years ago
low012 c1330f5743 *) added environment variable DOCUMENT_ROOT
16 years ago
low012 f26b8fcb1b *) comment mode is 'moderated' instead of 'activated' by default now (to avoid spam being visible)
16 years ago
orbiter b2a8c653ee small fixes
16 years ago
orbiter f675d47f86 better protection against database failures
16 years ago
orbiter 4f45605f04 small update for timing in search result processing
16 years ago
orbiter 9d119c6b61 migration of auto-update rules to new release strategy:
16 years ago
orbiter 4d5b401f00 try to fix some performance problems with the internal index management:
16 years ago
borg-0300 a0605325bb fixed a NullPointer Exception
16 years ago
orbiter b2b7edae18 fixed interactive search
16 years ago
lotus 2be119f0df adjusted big peer to 28M links
16 years ago
orbiter c6880ce28b removed the permanent cache flush and replaced it with a periodic cache flush
16 years ago
orbiter ef7fe537c5 fixed a cache-bug in cachedFileRA
16 years ago
orbiter 6c7e83909b - refactoring of data access methods to be prepared for new cell data structure
16 years ago
lotus c8451614f3 fix for overflow
16 years ago
orbiter c4c4c223b9 fixed a problem with attribute flags on RWI entries that prevented proper selection of index-of constraint
16 years ago
orbiter 6072831235 no cr transmission for robinson peers
16 years ago
low012 afe98bc11c *) added changes as proposed by Halborinda in http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1674
16 years ago
orbiter 07fc115e90 removed active profiling in kelondroRowSet
16 years ago
orbiter be4c458951 refactoring (implemented Iterable in kelondroRowCollection)
16 years ago
low012 bb5c2cd12e *) ISINDEX parameters will not be put on commandline anymore to prevent possible security hazards (better safe than sorry). Parmeters will have to be read from QUERY_STRING in ISINDEX case too which does not seem to be uncommon behaviour for web servers: http://vms.pdv-systeme.de/users/martinv/cgi_basics/cgi_basics.html#Datenuebergabe
16 years ago
orbiter b6bba18c37 replaced the storing procedure for the index ram cache with a method that generates BLOBHeap-compatible dumps
16 years ago
low012 db1cfae3e7 *) cleaning up after myself
16 years ago
low012 f547f9a78c *) added CGI capabilities (run Perl scripts and other software via HTTP GET and POST)
16 years ago
f1ori bdc380cd84 * add lastModified to templateCache
16 years ago
f1ori 025094675f * remove empty directory
16 years ago
f1ori c5691180cb * skip style-tags in HTML-files
16 years ago
orbiter 3567c58b18 added another filed information for BLOBHeap dumps: the gaps
16 years ago
orbiter abdd4aa414 added a index dump for blob heaps:
16 years ago
orbiter 8c3205b62e fix for OOB Exception
16 years ago
orbiter 78c568331e added test channel to /xml/feed.rss
16 years ago
orbiter e004da48d3 - added fast fingerprint computation for files (any). Will be used in new index dump method
16 years ago
f1ori 2d2ce24011 * remove all encoding-stuff from proxy
16 years ago
f1ori 73c8a0839c * abort download, when proxy connection is closed
16 years ago
orbiter bb935fdbb0 less organization overhead for DNS caching and prefetching
16 years ago
f1ori 4907697cfa * make fileuploads through proxy bigger than 65500 bytes possible
16 years ago
orbiter fc8189f3fb better self-healing of corrupted databases
16 years ago
f1ori 963da8c3f9 * updated tm-extractors to new version 1.0
16 years ago
orbiter e34ac22fbd - added new monitoring servlet at
16 years ago
lotus 449e697436 fix for null-seed in seedfile
16 years ago