boxtec
/
yacy_search_server
mirror of https://github.com/yacy/yacy_search_server
4
1
Fork 0
Code Issues Projects Releases Wiki Activity

fixed doku link

Browse Source
pull/419/head
Michael Peter Christen 4 years ago
parent c4659f0fb0
commit 3959d43a5c
6 changed files with 191 additions and 187 deletions
Show Stats Download Patch File Download Diff File

4
htroot/CrawlStartExpert.html
Unescape View File

@ -217,7 +217,7 @@
#%env/templates/submenuIndexCreate.template%# #%env/templates/submenuIndexCreate.template%#
<div id="api"> <div id="api">
<a href="http://www.yacy-websearch.net/wiki/index.php/Dev:APICrawler" id="apilink" target="_blank"><img src="env/grafics/api.png" width="60" height="40" alt="API"/></a> <a href="https://wiki.yacy.net/index.php/Dev:APICrawler" id="apilink" target="_blank"><img src="env/grafics/api.png" width="60" height="40" alt="API"/></a>
<span>Click on this API button to see a documentation of the POST request parameter for crawl starts.</span> <span>Click on this API button to see a documentation of the POST request parameter for crawl starts.</span>
</div> </div>
@ -228,7 +228,7 @@
You can define URLs as start points for Web page crawling and start crawling here. You can define URLs as start points for Web page crawling and start crawling here.
"Crawling" means that YaCy will download the given website, extract all links in it and then download the content behind these links. "Crawling" means that YaCy will download the given website, extract all links in it and then download the content behind these links.
This is repeated as long as specified under "Crawling Depth". This is repeated as long as specified under "Crawling Depth".
A crawl can also be started using wget and the <a href="http://www.yacy-websearch.net/wiki/index.php/Dev:APICrawler" target="_blank">post arguments</a> for this web page. A crawl can also be started using wget and the <a href="https://wiki.yacy.net/index.php/Dev:APICrawler" target="_blank">post arguments</a> for this web page.
</p> </p>
<form id="Crawler" action="Crawler_p.html" method="post" enctype="multipart/form-data" accept-charset="UTF-8"> <form id="Crawler" action="Crawler_p.html" method="post" enctype="multipart/form-data" accept-charset="UTF-8">

2
htroot/IndexFederated_p.html
Unescape View File

@ -34,7 +34,7 @@
If you switch off this index, a remote Solr must be activated.</dd> If you switch off this index, a remote Solr must be activated.</dd>
<dt>Use remote Solr server(s)&nbsp;<input type="checkbox" name="solr.indexing.solrremote" id="solr_indexing_solrremote" #(solr.indexing.solrremote.checked)#:: checked="checked"#(/solr.indexing.solrremote.checked)# onclick="if(!document.getElementById('config').solr_indexing_solrremote.checked) {document.getElementById('config').core_service_fulltext.checked = true;}"/></dt> <dt>Use remote Solr server(s)&nbsp;<input type="checkbox" name="solr.indexing.solrremote" id="solr_indexing_solrremote" #(solr.indexing.solrremote.checked)#:: checked="checked"#(/solr.indexing.solrremote.checked)# onclick="if(!document.getElementById('config').solr_indexing_solrremote.checked) {document.getElementById('config').core_service_fulltext.checked = true;}"/></dt>
<dd>It's easy to <a href="http://www.yacy-websearch.net/wiki/index.php/Dev:Solr" target="_blank">attach an external Solr to YaCy</a>. <dd>It's easy to <a href="https://wiki.yacy.net/index.php/Dev:Solr" target="_blank">attach an external Solr to YaCy</a>.
This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored. This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.
</dd> </dd>

2
locales/master.lng.xlf
Unescape View File

@ -4935,7 +4935,7 @@
<source>Index Size</source> <source>Index Size</source>
</trans-unit> </trans-unit>
<trans-unit id="e587b435" xml:space="preserve" approved="no" translate="yes"> <trans-unit id="e587b435" xml:space="preserve" approved="no" translate="yes">
<source>It's easy to &lt;a href="http://www.yacy-websearch.net/wiki/index.php/Dev:Solr" target="_blank"&gt;attach an external Solr to YaCy&lt;/a&gt;.</source> <source>It's easy to &lt;a href="https://wiki.yacy.net/index.php/Dev:Solr" target="_blank"&gt;attach an external Solr to YaCy&lt;/a&gt;.</source>
</trans-unit> </trans-unit>
<trans-unit id="35e7810" xml:space="preserve" approved="no" translate="yes"> <trans-unit id="35e7810" xml:space="preserve" approved="no" translate="yes">
<source>This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.</source> <source>This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.</source>

2
locales/ru.lng
Unescape View File

@ -578,7 +578,7 @@ Use remote Solr server(s)==Использовать удалённую базу
Solr Hosts==Хосты Solr Solr Hosts==Хосты Solr
Solr Host Administration Interface==Интерфейс управления Solr Solr Host Administration Interface==Интерфейс управления Solr
Index Size==Документов в индексе Index Size==Документов в индексе
It's easy to <a href="http://www.yacy-websearch.net/wiki/index.php/Dev:Solr" target="_blank">attach an external Solr to YaCy</a>.==Присоединить внешнюю базу Solr <a href="http://www.yacy-websearch.net/wiki/index.php/Dev:Solr" target="_blank">просто</a>. It's easy to <a href="https://wiki.yacy.net/index.php/Dev:Solr" target="_blank">attach an external Solr to YaCy</a>.==Присоединить внешнюю базу Solr <a href="https://wiki.yacy.net/index.php/Dev:Solr" target="_blank">просто</a>.
This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.==Внешняя база данных Solr будет использоваться вместо встроенной. Вы также можете использовать дополнительно встроенную базу, но тогда индексы будут сохраняться в обе базы. This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.==Внешняя база данных Solr будет использоваться вместо встроенной. Вы также можете использовать дополнительно встроенную базу, но тогда индексы будут сохраняться в обе базы.
Solr URL(s)==Ссылки на базу Solr Solr URL(s)==Ссылки на базу Solr
You can set one or more Solr targets here which are accessed as a shard. For several targets, list them using a ',' (comma) as separator.==Вы можете установить одну или более баз Solr, которые будут доступны распределённо. Адреса нескольких баз указывайте через запятую. You can set one or more Solr targets here which are accessed as a shard. For several targets, list them using a ',' (comma) as separator.==Вы можете установить одну или более баз Solr, которые будут доступны распределённо. Адреса нескольких баз указывайте через запятую.

14
startYACY.sh
Unescape View File

@ -184,16 +184,20 @@ fi
#turn on MMap for Solr if OS is a 64bit OS #turn on MMap for Solr if OS is a 64bit OS
if [ -n "`uname -m | grep 64`" ]; then JAVA_ARGS="$JAVA_ARGS -Dsolr.directoryFactory=solr.MMapDirectoryFactory"; fi if [ -n "`uname -m | grep 64`" ]; then JAVA_ARGS="$JAVA_ARGS -Dsolr.directoryFactory=solr.MMapDirectoryFactory"; fi
if [ ! -f $CONFIGFILE -a -f "$YACY_PARENT_DATA_PATH/DATA/SETTINGS/httpProxy.conf" ]
then
# old config if new does not exist
CONFIGFILE="$YACY_PARENT_DATA_PATH/DATA/SETTINGS/httpProxy.conf"
fi
if [ -f $CONFIGFILE ] if [ -f $CONFIGFILE ]
then then
# startup memory # startup memory
if [ -z "$YACY_JAVASTART_XMX" ]
then
# When YACY_JAVASTART_XMX is not set or empty:
# Read from $CONFIGFILE
j="`grep javastart_Xmx $CONFIGFILE | sed 's/^[^=]*=//'`"; j="`grep javastart_Xmx $CONFIGFILE | sed 's/^[^=]*=//'`";
if [ -n "$j" ]; then JAVA_ARGS="-$j $JAVA_ARGS"; fi; if [ -n "$j" ]; then JAVA_ARGS="-$j $JAVA_ARGS"; fi;
else
# use the YACY_JAVASTART_XMX variable
JAVA_ARGS="-$YACY_JAVASTART_XMX $JAVA_ARGS"
fi
# Priority # Priority
j="`grep javastart_priority $CONFIGFILE | sed 's/^[^=]*=//'`"; j="`grep javastart_priority $CONFIGFILE | sed 's/^[^=]*=//'`";

42
test/java/net/yacy/document/parser/GenericXMLParserTest.java
Unescape View File

@ -23,9 +23,9 @@
package net.yacy.document.parser; package net.yacy.document.parser;
import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue; import static org.junit.Assert.assertTrue;
import static org.junit.Assert.assertFalse;
import java.io.ByteArrayInputStream; import java.io.ByteArrayInputStream;
import java.io.File; import java.io.File;
@ -58,7 +58,7 @@ public class GenericXMLParserTest {
@Before @Before
public void setUp() { public void setUp() {
this.parser = new GenericXMLParser(); parser = new GenericXMLParser();
} }
/** /**
@ -77,7 +77,7 @@ public class GenericXMLParserTest {
FileInputStream inStream = new FileInputStream(new File(folder, fileName)); FileInputStream inStream = new FileInputStream(new File(folder, fileName));
DigestURL location = new DigestURL("http://localhost/" + fileName); DigestURL location = new DigestURL("http://localhost/" + fileName);
try { try {
Document[] documents = this.parser.parse(location, "text/xml", null, new VocabularyScraper(), 0, Document[] documents = parser.parse(location, "text/xml", null, new VocabularyScraper(), 0,
inStream); inStream);
assertNotNull("Parser result must not be null for file " + fileName, documents); assertNotNull("Parser result must not be null for file " + fileName, documents);
assertNotNull("Parsed text must not be empty for file " + fileName, documents[0].getTextString()); assertNotNull("Parsed text must not be empty for file " + fileName, documents[0].getTextString());
@ -138,7 +138,7 @@ public class GenericXMLParserTest {
*/ */
byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-8\"?>" + UMLAUT_TEXT_TAG) byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-8\"?>" + UMLAUT_TEXT_TAG)
.getBytes(StandardCharsets.UTF_8); .getBytes(StandardCharsets.UTF_8);
testCharsetDetection(this.parser, encodedXML, "application/xml; charset=utf-8", StandardCharsets.UTF_8.name(), testCharsetDetection(parser, encodedXML, "application/xml; charset=utf-8", StandardCharsets.UTF_8.name(),
"Maßkrügen"); "Maßkrügen");
/* /*
@ -146,7 +146,7 @@ public class GenericXMLParserTest {
* declaration * declaration
*/ */
encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_8); encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_8);
testCharsetDetection(this.parser, encodedXML, "application/xml; charset=utf-8", StandardCharsets.UTF_8.name(), testCharsetDetection(parser, encodedXML, "application/xml; charset=utf-8", StandardCharsets.UTF_8.name(),
"Maßkrügen"); "Maßkrügen");
} }
@ -169,7 +169,7 @@ public class GenericXMLParserTest {
*/ */
byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-16\"?>" + UMLAUT_TEXT_TAG) byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-16\"?>" + UMLAUT_TEXT_TAG)
.getBytes(StandardCharsets.UTF_16); .getBytes(StandardCharsets.UTF_16);
testCharsetDetection(this.parser, encodedXML, "application/xml; charset=utf-16", StandardCharsets.UTF_16.name(), testCharsetDetection(parser, encodedXML, "application/xml; charset=utf-16", StandardCharsets.UTF_16.name(),
"Maßkrügen"); "Maßkrügen");
/* /*
@ -177,7 +177,7 @@ public class GenericXMLParserTest {
* XML declaration having only BOM (Byte Order Mark) * XML declaration having only BOM (Byte Order Mark)
*/ */
encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_16); encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_16);
testCharsetDetection(this.parser, encodedXML, "application/xml; charset=utf-16", testCharsetDetection(parser, encodedXML, "application/xml; charset=utf-16",
StandardCharsets.UTF_16BE.name(), "Maßkrügen"); StandardCharsets.UTF_16BE.name(), "Maßkrügen");
/* /*
@ -186,14 +186,14 @@ public class GenericXMLParserTest {
*/ */
encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-16\"?>" + UMLAUT_TEXT_TAG) encodedXML = ("<?xml version=\"1.0\" encoding=\"utf-16\"?>" + UMLAUT_TEXT_TAG)
.getBytes(StandardCharsets.UTF_16); .getBytes(StandardCharsets.UTF_16);
testCharsetDetection(this.parser, encodedXML, "application/xml", StandardCharsets.UTF_16.name(), "Maßkrügen"); testCharsetDetection(parser, encodedXML, "application/xml", StandardCharsets.UTF_16.name(), "Maßkrügen");
/* /*
* Charset is omitted in both Content-Type HTTP header and XML * Charset is omitted in both Content-Type HTTP header and XML
* declaration with BOM (Byte Order Mark) * declaration with BOM (Byte Order Mark)
*/ */
encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_16); encodedXML = ("<?xml version=\"1.0\"?>" + UMLAUT_TEXT_TAG).getBytes(StandardCharsets.UTF_16);
testCharsetDetection(this.parser, encodedXML, "application/xml", StandardCharsets.UTF_16BE.name(), "Maßkrügen"); testCharsetDetection(parser, encodedXML, "application/xml", StandardCharsets.UTF_16BE.name(), "Maßkrügen");
} }
/** /**
@ -213,7 +213,7 @@ public class GenericXMLParserTest {
*/ */
byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>" + UMLAUT_TEXT_TAG) byte[] encodedXML = ("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>" + UMLAUT_TEXT_TAG)
.getBytes(StandardCharsets.ISO_8859_1); .getBytes(StandardCharsets.ISO_8859_1);
testCharsetDetection(this.parser, encodedXML, "application/xml", StandardCharsets.ISO_8859_1.name(), testCharsetDetection(parser, encodedXML, "application/xml", StandardCharsets.ISO_8859_1.name(),
"Maßkrügen"); "Maßkrügen");
} }
@ -242,7 +242,7 @@ public class GenericXMLParserTest {
encodedXML = ("<?xml version=\"1.0\"?>" encodedXML = ("<?xml version=\"1.0\"?>"
+ "<text>In M&#x000FC;nchen steht ein Hofbr&#x000E4;uhaus, dort gibt es Bier in Ma&#x000DF;kr&#x000FC;gen</text>") + "<text>In M&#x000FC;nchen steht ein Hofbr&#x000E4;uhaus, dort gibt es Bier in Ma&#x000DF;kr&#x000FC;gen</text>")
.getBytes(StandardCharsets.US_ASCII); .getBytes(StandardCharsets.US_ASCII);
testCharsetDetection(this.parser, encodedXML, "application/xml", StandardCharsets.UTF_8.name(), "Maßkrügen"); testCharsetDetection(parser, encodedXML, "application/xml", StandardCharsets.UTF_8.name(), "Maßkrügen");
} }
/** /**
@ -262,7 +262,7 @@ public class GenericXMLParserTest {
*/ */
byte[] encodedXML = ("<?xml version='1.0' encoding='utf-16be'?>" + UMLAUT_TEXT_TAG) byte[] encodedXML = ("<?xml version='1.0' encoding='utf-16be'?>" + UMLAUT_TEXT_TAG)
.getBytes(StandardCharsets.UTF_16BE); .getBytes(StandardCharsets.UTF_16BE);
testCharsetDetection(this.parser, encodedXML, "application/xml; charset=utf-16be", testCharsetDetection(parser, encodedXML, "application/xml; charset=utf-16be",
StandardCharsets.UTF_16BE.name(), "Maßkrügen"); StandardCharsets.UTF_16BE.name(), "Maßkrügen");
} }
@ -288,7 +288,7 @@ public class GenericXMLParserTest {
String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader); String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader);
DigestURL location = new DigestURL("http://localhost/testfile.xml"); DigestURL location = new DigestURL("http://localhost/testfile.xml");
try { try {
Document[] documents = this.parser.parse(location, contentTypeHeader, charsetFromHttpHeader, Document[] documents = parser.parse(location, contentTypeHeader, charsetFromHttpHeader,
new VocabularyScraper(), 0, inStream); new VocabularyScraper(), 0, inStream);
assertEquals(1, documents.length); assertEquals(1, documents.length);
Collection<AnchorURL> detectedAnchors = documents[0].getAnchors(); Collection<AnchorURL> detectedAnchors = documents[0].getAnchors();
@ -324,7 +324,7 @@ public class GenericXMLParserTest {
String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader); String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader);
DigestURL location = new DigestURL("http://localhost/testfile.xml"); DigestURL location = new DigestURL("http://localhost/testfile.xml");
try { try {
Document[] documents = this.parser.parse(location, contentTypeHeader, charsetFromHttpHeader, Document[] documents = parser.parse(location, contentTypeHeader, charsetFromHttpHeader,
new VocabularyScraper(), 0, inStream); new VocabularyScraper(), 0, inStream);
assertEquals(1, documents.length); assertEquals(1, documents.length);
Collection<AnchorURL> detectedAnchors = documents[0].getAnchors(); Collection<AnchorURL> detectedAnchors = documents[0].getAnchors();
@ -351,7 +351,7 @@ public class GenericXMLParserTest {
String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader); String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader);
DigestURL location = new DigestURL("http://localhost/testfile.xml"); DigestURL location = new DigestURL("http://localhost/testfile.xml");
try { try {
Document[] documents = this.parser.parse(location, contentTypeHeader, charsetFromHttpHeader, Document[] documents = parser.parse(location, contentTypeHeader, charsetFromHttpHeader,
new VocabularyScraper(), 0, inStream); new VocabularyScraper(), 0, inStream);
assertEquals(1, documents.length); assertEquals(1, documents.length);
assertEquals("Node content1 Node content2", documents[0].getTextString()); assertEquals("Node content1 Node content2", documents[0].getTextString());
@ -376,7 +376,7 @@ public class GenericXMLParserTest {
+ "Home page : http://yacy.net - International Forum : " + "Home page : http://yacy.net - International Forum : "
+ "https://searchlab.eu " + "https://searchlab.eu "
+ "and this is a mention to a relative URL : /document.html</p>" + "and this is a mention to a relative URL : /document.html</p>"
+ "<p>Here are YaCy<a href=\"http://mantis.tokeek.de\">bug tracker</a> and <a href=\"http://www.yacy-websearch.net/wiki/\">Wiki</a>." + "<p>Here are YaCy<a href=\"http://mantis.tokeek.de\">bug tracker</a> and <a href=\"https://wiki.yacy.net/index.php/\">Wiki</a>."
+ "And this is a relative link to another <a href=\"/document2.html\">sub document</a></p>" + "And this is a relative link to another <a href=\"/document2.html\">sub document</a></p>"
+ "</body>" + "</html>"; + "</body>" + "</html>";
@ -386,7 +386,7 @@ public class GenericXMLParserTest {
String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader); String charsetFromHttpHeader = HeaderFramework.getCharacterEncoding(contentTypeHeader);
DigestURL location = new DigestURL("http://localhost/testfile.xml"); DigestURL location = new DigestURL("http://localhost/testfile.xml");
try { try {
Document[] documents = this.parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader, new VocabularyScraper(), 0, inStream, Integer.MAX_VALUE, Long.MAX_VALUE); Document[] documents = parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader, new VocabularyScraper(), 0, inStream, Integer.MAX_VALUE, Long.MAX_VALUE);
assertEquals(1, documents.length); assertEquals(1, documents.length);
assertFalse(documents[0].isPartiallyParsed()); assertFalse(documents[0].isPartiallyParsed());
@ -399,7 +399,7 @@ public class GenericXMLParserTest {
assertTrue(detectedAnchors.contains(new AnchorURL("http://yacy.net"))); assertTrue(detectedAnchors.contains(new AnchorURL("http://yacy.net")));
assertTrue(detectedAnchors.contains(new AnchorURL("https://searchlab.eu"))); assertTrue(detectedAnchors.contains(new AnchorURL("https://searchlab.eu")));
assertTrue(detectedAnchors.contains(new AnchorURL("http://mantis.tokeek.de"))); assertTrue(detectedAnchors.contains(new AnchorURL("http://mantis.tokeek.de")));
assertTrue(detectedAnchors.contains(new AnchorURL("http://www.yacy-websearch.net/wiki/"))); assertTrue(detectedAnchors.contains(new AnchorURL("https://wiki.yacy.net/index.php/")));
} finally { } finally {
inStream.close(); inStream.close();
} }
@ -407,7 +407,7 @@ public class GenericXMLParserTest {
/* Links limit exceeded */ /* Links limit exceeded */
inStream = new ByteArrayInputStream(xhtml.getBytes(StandardCharsets.UTF_8.name())); inStream = new ByteArrayInputStream(xhtml.getBytes(StandardCharsets.UTF_8.name()));
try { try {
Document[] documents = this.parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader, Document[] documents = parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader,
new VocabularyScraper(), 0, inStream, 2, Long.MAX_VALUE); new VocabularyScraper(), 0, inStream, 2, Long.MAX_VALUE);
assertEquals(1, documents.length); assertEquals(1, documents.length);
assertTrue(documents[0].isPartiallyParsed()); assertTrue(documents[0].isPartiallyParsed());
@ -443,12 +443,12 @@ public class GenericXMLParserTest {
} }
int firstBytes = xhtmlBuilder.toString().getBytes(StandardCharsets.UTF_8.name()).length; int firstBytes = xhtmlBuilder.toString().getBytes(StandardCharsets.UTF_8.name()).length;
xhtmlBuilder.append("<p>Here are YaCy<a href=\"http://mantis.tokeek.de\">bug tracker</a> and <a href=\"http://www.yacy-websearch.net/wiki/\">Wiki</a>.") xhtmlBuilder.append("<p>Here are YaCy<a href=\"http://mantis.tokeek.de\">bug tracker</a> and <a href=\"https://wiki.yacy.net/index.php/\">Wiki</a>.")
.append("And this is a relative link to another <a href=\"/document2.html\">sub document</a></p>") .append("And this is a relative link to another <a href=\"/document2.html\">sub document</a></p>")
.append("</body></html>"); .append("</body></html>");
inStream = new ByteArrayInputStream(xhtmlBuilder.toString().getBytes(StandardCharsets.UTF_8.name())); inStream = new ByteArrayInputStream(xhtmlBuilder.toString().getBytes(StandardCharsets.UTF_8.name()));
try { try {
Document[] documents = this.parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader, new VocabularyScraper(), 0, inStream, Integer.MAX_VALUE, firstBytes); Document[] documents = parser.parseWithLimits(location, contentTypeHeader, charsetFromHttpHeader, new VocabularyScraper(), 0, inStream, Integer.MAX_VALUE, firstBytes);
assertEquals(1, documents.length); assertEquals(1, documents.length);
assertTrue(documents[0].isPartiallyParsed()); assertTrue(documents[0].isPartiallyParsed());

Write Preview
Loading…
Cancel
Save