YaCy '#[clientname]#': Performance Settings for Memory

#%env/templates/metas.template%# #%env/templates/header.template%# #%env/templates/submenuPerformance.template%#

Performance Settings for Memory

Memory Usage:

Type	After Startup	After Initializations before GC	After Initializations after GC	Now #(gc)#before GC::after GC#(/gc)#	Description
Max	#[memoryMax]# MByte				maximum memory that the JVM will attempt to use
Available	#[memoryAvailAfterStartup]# MByte	#[memoryAvailAfterInitBGC]# MByte	#[memoryAvailAfterInitAGC]# MByte	#[memoryAvailNow]# MByte	total available memory including free for the JVM within maximum
Total	#[memoryTotalAfterStartup]# KByte	#[memoryTotalAfterInitBGC]# KByte	#[memoryTotalAfterInitAGC]# KByte	#[memoryTotalNow]# MByte	total memory taken from the OS
Free	#[memoryFreeAfterStartup]# KByte	#[memoryFreeAfterInitBGC]# KByte	#[memoryFreeAfterInitAGC]# KByte	#[memoryFreeNow]# MByte	free memory in the JVM within total amount
Used	#[memoryUsedAfterStartup]# KByte	#[memoryUsedAfterInitBGC]# KByte	#[memoryUsedAfterInitAGC]# KByte	#[memoryUsedNow]# MByte	used memory in the JVM within total amount

RAM Cache for Database Files:

#(useRWICache)#:: #(/useRWICache)# #(usePreNURLCache)#:: #(/usePreNURLCache)# #(useNURLCache)#:: #(/useNURLCache)# #(useEURLCache)#:: #(/useEURLCache)#

Database

DB Size

Node Cache

Object Cache

Memory Occupation (KBytes)

Description

Chunk
Size
(bytes)

Empty
(avail.)
Slots

Used
(filled)
Slots

Node-Cache
Hit:Miss
Uniq:Doub
Del:Flush

Chunk
Size
(bytes)

Max
Size
Hit/Miss

Hit-Size
Miss-Size

Hit-Cache
Hit:Miss
Uniq:Doub
Del:Flush

Miss-Cache
Hit:Miss
Uniq:Doub
Del:Flush

Used Now

Assigned Max

Default Max

Best Max

RWI Assortment Cluster

#[slreqRWI]#

#[nodszRWI]#

#[slempRWI]#

#[slfilRWI]#

#[slhittmissRWI]#
#[sluniqdoubRWI]#
#[slflushRWI]#

#[ochunksizRWI]#

#[ochunkmaxRWI]#
#[omisscmaxRWI]#

#[ochunkcurRWI]#

#[ohittmissRWI]#
#[ouniqdoubRWI]#
#[oflushRWI]#

#[nhittmissRWI]#
#[nuniqdoubRWI]#
#[nflushRWI]#

#[usedRWI]#

#[dfltRWI]#

#[bestRWI]#

The Assortment Cluster stores most of the page indexes. Flushing speed of the temporary RWI cache depends on the size of this file cache. Increasing the space of this cache will speed up crawls with a depth > 3.

HTTP Response Header

#[slreqHTTP]#

#[nodszHTTP]#

#[slempHTTP]#

#[slfilHTTP]#

#[slhittmissHTTP]#
#[sluniqdoubHTTP]#
#[slflushHTTP]#

#[ochunksizHTTP]#

#[ochunkmaxHTTP]#
#[omisscmaxHTTP]#

#[ochunkcurHTTP]#

#[ohittmissHTTP]#
#[ouniqdoubHTTP]#
#[oflushHTTP]#

#[nhittmissHTTP]#
#[nuniqdoubHTTP]#
#[nflushHTTP]#

#[usedHTTP]#

#[dfltHTTP]#

#[bestHTTP]#

The Response Header database stores the HTTP heades that other servers send when YaCy retrieves web pages during proxy mode, when performing crawls or if it fetches pages for snippet generation. Increasing this cache will be most important for a fast proxy mode.

'loaded' URLs

#[slreqLURL]#

#[nodszLURL]#

#[slempLURL]#

#[slfilLURL]#

#[slhittmissLURL]#
#[sluniqdoubLURL]#
#[slflushLURL]#

#[ochunksizLURL]#

#[ochunkmaxLURL]#
#[omisscmaxLURL]#

#[ochunkcurLURL]#

#[ohittmissLURL]#
#[ouniqdoubLURL]#
#[oflushLURL]#

#[nhittmissLURL]#
#[nuniqdoubLURL]#
#[nflushLURL]#

#[usedLURL]#

#[dfltLURL]#

#[bestLURL]#

This is the database that holds the hash/url - relation and properties regarding the url like load date and server date. This cache is very important for a fast search process. Increasing the cache size will result in more search results and less IO during DHT transfer.

'pre-noticed' URLs

#[slreqPreNURL]#

#[nodszPreNURL]#

#[slempPreNURL]#

#[slfilPreNURL]#

#[slhittmissPreNURL]#
#[sluniqdoubPreNURL]#
#[slflushPreNURL]#

#[ochunksizPreNURL]#

#[ochunkmaxPreNURL]#
#[omisscmaxPreNURL]#

#[ochunkcurPreNURL]#

#[ohittmissPreNURL]#
#[ouniqdoubPreNURL]#
#[oflushPreNURL]#

#[nhittmissPreNURL]#
#[nuniqdoubPreNURL]#
#[nflushPreNURL]#

#[usedPreNURL]#

#[dfltPreNURL]#

#[bestPreNURL]#

'noticed' URLs

#[slreqNURL]#

#[nodszNURL]#

#[slempNURL]#

#[slfilNURL]#

#[slhittmissNURL]#
#[sluniqdoubNURL]#
#[slflushNURL]#

#[ochunksizNURL]#

#[ochunkmaxNURL]#
#[omisscmaxNURL]#

#[ochunkcurNURL]#

#[ohittmissNURL]#
#[ouniqdoubNURL]#
#[oflushNURL]#

#[nhittmissNURL]#
#[nuniqdoubNURL]#
#[nflushNURL]#

#[usedNURL]#

#[dfltNURL]#

#[bestNURL]#

A noticed URL is one that was discovered during crawling but was not loaded yet. Increasing the cache size will result in faster double-check during URL recognition when doing crawls.

'error' URLs

#[slreqEURL]#

#[nodszEURL]#

#[slempEURL]#

#[slfilEURL]#

#[slhittmissEURL]#
#[sluniqdoubEURL]#
#[slflushEURL]#

#[ochunksizEURL]#

#[ochunkmaxEURL]#
#[omisscmaxEURL]#

#[ochunkcurEURL]#

#[ohittmissEURL]#
#[ouniqdoubEURL]#
#[oflushEURL]#

#[nhittmissEURL]#
#[nuniqdoubEURL]#
#[nflushEURL]#

#[usedEURL]#

#[dfltEURL]#

#[bestEURL]#

URLs that cannot be loaded are stored in this database. It is also used for double-checked during crawling. Increasing the cache size will most probably speed up crawling slightly, but not significantly.

DHT Control

#[slreqDHT]#

#[nodszDHT]#

#[slempDHT]#

#[slfilDHT]#

#[slhittmissDHT]#
#[sluniqdoubDHT]#
#[slflushDHT]#

#[ochunksizDHT]#

#[ochunkmaxDHT]#
#[omisscmaxDHT]#

#[ochunkcurDHT]#

#[ohittmissDHT]#
#[ouniqdoubDHT]#
#[oflushDHT]#

#[nhittmissDHT]#
#[nuniqdoubDHT]#
#[nflushDHT]#

#[usedDHT]#

#[dfltDHT]#

#[bestDHT]#

This is simply the cache for the seed-dbs (active, passive, potential). This cache is divided into three equal parts. Increasing this cache may speed up many functions, but we need to test this to see the effects.

Messages

#[slreqMessage]#

#[nodszMessage]#

#[slempMessage]#

#[slfilMessage]#

#[slhittmissMessage]#
#[sluniqdoubMessage]#
#[slflushMessage]#

#[ochunksizMessage]#

#[ochunkmaxMessage]#
#[omisscmaxMessage]#

#[ochunkcurMessage]#

#[ohittmissMessage]#
#[ouniqdoubMessage]#
#[oflushMessage]#

#[nhittmissMessage]#
#[nuniqdoubMessage]#
#[nflushMessage]#

#[usedMessage]#

#[dfltMessage]#

#[bestMessage]#

The Message cache for peer-to-peer messages. Less important.

Wiki

#[slreqWiki]#

#[nodszWiki]#

#[slempWiki]#

#[slfilWiki]#

#[slhittmissWiki]#
#[sluniqdoubWiki]#
#[slflushWiki]#

#[ochunksizWiki]#

#[ochunkmaxWiki]#
#[omisscmaxWiki]#

#[ochunkcurWiki]#

#[ohittmissWiki]#
#[ouniqdoubWiki]#
#[oflushWiki]#

#[nhittmissWiki]#
#[nuniqdoubWiki]#
#[nflushWiki]#

#[usedWiki]#

#[dfltWiki]#

#[bestWiki]#

The YaCy-Wiki uses a database to store its pages. This cache is divided in two parts, one for the wiki database and one for its backup. Increasing this cache may speed up access to the wiki pages.

Blog

#[slreqBlog]#

#[nodszBlog]#

#[slempBlog]#

#[slfilBlog]#

#[slhittmissBlog]#
#[sluniqdoubBlog]#
#[slflushBlog]#

#[ochunksizBlog]#

#[ochunkmaxBlog]#

#[ochunkcurBlog]#

#[ohittmissBlog]#
#[ouniqdoubBlog]#
#[oflushBlog]#

#[nhittmissBlog]#
#[nuniqdoubBlog]#
#[nflushBlog]#

#[usedBlog]#

#[dfltBlog]#

#[bestBlog]#

The YaCy-Blog uses a database to store its entries. Increasing this cache may speed up access to the Blog.

News

#[slreqNews]#

#[nodszNews]#

#[slempNews]#

#[slfilNews]#

#[slhittmissNews]#
#[sluniqdoubNews]#
#[slflushNews]#

#[ochunksizNews]#

#[ochunkmaxNews]#

#[ochunkcurNews]#

#[ohittmissNews]#
#[ouniqdoubNews]#
#[oflushNews]#

#[nhittmissNews]#
#[nuniqdoubNews]#
#[nflushNews]#

#[usedNews]#

#[dfltNews]#

#[bestNews]#

The News-DB stores property-lists for news that are included in seeds. Increasing this cache may speed up the peer-ping.

robots.txt DB

#[slreqRobots]#

#[nodszRobots]#

#[slempRobots]#

#[slfilRobots]#

#[slhittmissRobots]#
#[sluniqdoubRobots]#
#[slflushRobots]#

#[ochunksizRobots]#

#[ochunkmaxRobots]#

#[ochunkcurRobots]#

#[ohittmissRobots]#
#[ouniqdoubRobots]#
#[oflushRobots]#

#[nhittmissRobots]#
#[nuniqdoubRobots]#
#[nflushRobots]#

#[usedRobots]#

#[dfltRobots]#

#[bestRobots]#

The robots.txt DB stores downloaded records from robots.txt files. Increasing this cache may speed up validation if crawling of the URL is allowed.

Crawl Profiles

#[slreqProfiles]#

#[nodszProfiles]#

#[slempProfiles]#

#[slfilProfiles]#

#[slhittmissProfiles]#
#[sluniqdoubProfiles]#
#[slflushProfiles]#

#[ochunksizProfiles]#

#[ochunkmaxProfiles]#

#[ochunkcurProfiles]#

#[ohittmissProfiles]#
#[ouniqdoubProfiles]#
#[oflushProfiles]#

#[nhittmissProfiles]#
#[nuniqdoubProfiles]#
#[nflushProfiles]#

#[usedProfiles]#

#[dfltProfiles]#

#[bestProfiles]#

The profile database stores properties for each crawl that is started on the local peer. Increasing this cache may speed up crawling, but not much space is needed, so the effect may be low.

Totals

#[usedTotal]# MB

#[currTotal]# MB

#[dfltTotal]# MB

#[bestTotal]# MB

Sum of memory amounts

Re-Configuration:

these
custom
values

all
default
values

all
optimum
values

Changes take effect after re-start of YaCy

Other caching structures:

Type	Amount
DNSCache	#[namecache.hit]#
DNSNoCache	#[namecache.noCache]#
HashBlacklistedCache	#[blacklistcache.size]#

#%env/templates/footer.template%#