Some Hints for Nutch
作者:Fenng
出处:http://www.dbanotes.net
日期:February 19, 2005
« 盛大收购新浪股权:变数横生? | Blog首页 | Tom 的经验谈 »
好久没关注Nutch了，看邮件列表，学到了几招关于 Nutch 的小技巧．
- 如何索引动态 URL 站点？
调整 regex-urlfilter.txt 或是 crawl-urlfilter.txt 文件． 参见行"# skip URLs containing certain characters as probable queries,后面的内容．
- 编译 Nutch 需要用到的 Ant 版本至少要 1.6 以上．
- 验证regex-urlfilter是否正常(by Michael Nebel)：
If you want to know, if your regex-urlfilter works as expectet, you can check it with the command: cat FILE-WITH-URLS | nutch net/nutch/net/RegexURLFilter or by calling "nutch net/nutch/net/RegexURLFilter" and entering the URL by hand. Everyline line beginning with a "+" ist accepted - a line with a "-" is accepted. For example: $ echo "http://www.nutch.org" | nutch net/nutch/net/RegexURLFilter run with heapsize 256 -Xmx256m 050202 173520 loadingfile:/home/nutch/nutch-0.7/conf/nutch-default.xml 050202 173520 loading file:/home/nutch/nutch-0.7/conf/nutch-site.xml 050202 173520 found resource regex-urlfilter.txt at file:/home/nutch/nutch-0.7/conf/regex-urlfilter.txt
Posted by Fenng at February 19, 2005 02:49 PM
Last updated at 06:17 PM on August 20, 2005
相关文章：
- 关于 Nutch 的一个问题: 中文乱码 - Sep 17, 2005
- Nutch 开始被 Apache "孵化" - Jan 28, 2005
- Nutch 0.6 is out ! - Jan 17, 2005
Trackback Pings
TrackBack URL for this entry:
http://www.dbanotes.net/mt/mt-tb.cgi/133