在Ruby 1.9.1/Windows 7上用mechanize遭遇segfault

之前用Mechanize写了点小脚本，不过有些网页跳转啊什么的明明可以并行做，Ruby 1.9之前却会因为green thread不是原生线程而无法真的并行，比较郁闷。想换到Ruby 1.9.1用NeverBlock来看看情况会不会好一些，不过却遇到了问题。

从Ruby-mswin32抓得Ruby 1.9.1的stable release。运行机器是我那装着Windows 7 Beta的老本。

刚开始运行gem的时候总是会抱怨找不到zlib.dll，只好从前面装的Ruby 1.8.6的bin目录把zlib1.dll拷贝过去，重命名为zlib.dll。然后gem install基本上都顺利。

但是require 'mechanize'的时候却得到了segfault……

E:\ruby191\bin>gem install mechanize
Successfully installed nokogiri-1.1.1-x86-mswin32-60
Successfully installed mechanize-0.9.0
2 gems installed
Installing ri documentation for nokogiri-1.1.1-x86-mswin32-60...
Installing ri documentation for mechanize-0.9.0...
Updating class cache with 1261 classes...
Installing RDoc documentation for nokogiri-1.1.1-x86-mswin32-60...
Installing RDoc documentation for mechanize-0.9.0...

E:\ruby191\bin>gem install rubyscript2exe
Successfully installed rubyscript2exe-0.5.3
1 gem installed
Updating class cache with 1361 classes...

E:\ruby191\bin>gem install hpricot
Successfully installed hpricot-0.6.164-x86-mswin32
1 gem installed
Installing ri documentation for hpricot-0.6.164-x86-mswin32...
Installing RDoc documentation for hpricot-0.6.164-x86-mswin32...

E:\ruby191\bin>irb
irb(main):001:0> require 'uri'
=> true
irb(main):002:0> require 'mechanize'
E:/ruby191/lib/ruby/gems/1.9.1/gems/nokogiri-1.1.1-x86-mswin32-60/ext/nokogiri/native.so: [BUG] Segmentation fault
ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-mswin32]

-- control frame ----------
c:0034 p:-8950286 s:0112 b:0112 l:000111 d:000111 TOP
c:0033 p:---- s:0110 b:0110 l:000109 d:000109 CFUNC  :require
c:0032 p:0222 s:0106 b:0106 l:000105 d:000105 TOP    E:/ruby191/lib/ruby/gems/1.9.1/gems/nokogiri-1.1.1-x86-mswin32-60/lib/nokogiri.rb:15
c:0031 p:---- s:0104 b:0104 l:000103 d:000103 FINISH
c:0030 p:---- s:0102 b:0102 l:000101 d:000101 CFUNC  :require
c:0029 p:0107 s:0098 b:0098 l:000097 d:000097 TOP    E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/www/mechanize.rb:9
c:0028 p:---- s:0096 b:0096 l:000095 d:000095 FINISH
c:0027 p:---- s:0094 b:0094 l:000093 d:000093 CFUNC  :require
c:0026 p:0011 s:0090 b:0090 l:000089 d:000089 TOP    E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/mechanize.rb:7
c:0025 p:---- s:0088 b:0088 l:000087 d:000087 FINISH
c:0024 p:---- s:0086 b:0086 l:000085 d:000085 CFUNC  :require
c:0023 p:0011 s:0082 b:0082 l:0020ec d:000d3c EVAL   (irb):2
c:0022 p:---- s:0080 b:0080 l:000079 d:000079 FINISH
c:0021 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC  :eval
c:0020 p:0027 s:0071 b:0071 l:000070 d:000070 METHOD E:/ruby191/lib/ruby/1.9.1/irb/workspace.rb:80
c:0019 p:0031 s:0064 b:0063 l:000062 d:000062 METHOD E:/ruby191/lib/ruby/1.9.1/irb/context.rb:218
c:0018 p:0030 s:0058 b:0058 l:0018b4 d:000057 BLOCK  E:/ruby191/lib/ruby/1.9.1/irb.rb:149
c:0017 p:0037 s:0050 b:0050 l:000049 d:000049 METHOD E:/ruby191/lib/ruby/1.9.1/irb.rb:263
c:0016 p:0011 s:0045 b:0045 l:0018b4 d:000044 BLOCK  E:/ruby191/lib/ruby/1.9.1/irb.rb:146
c:0015 p:0132 s:0041 b:0041 l:000024 d:000040 BLOCK  E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:244
c:0014 p:---- s:0038 b:0038 l:000037 d:000037 FINISH
c:0013 p:---- s:0036 b:0036 l:000035 d:000035 CFUNC  :loop
c:0012 p:0009 s:0033 b:0033 l:000024 d:000032 BLOCK  E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:230
c:0011 p:---- s:0031 b:0031 l:000030 d:000030 FINISH
c:0010 p:---- s:0029 b:0029 l:000028 d:000028 CFUNC  :catch
c:0009 p:0023 s:0025 b:0025 l:000024 d:000024 METHOD E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:229
c:0008 p:0042 s:0022 b:0022 l:0018b4 d:0018b4 METHOD E:/ruby191/lib/ruby/1.9.1/irb.rb:145
c:0007 p:0011 s:0019 b:0019 l:001824 d:000018 BLOCK  E:/ruby191/lib/ruby/1.9.1/irb.rb:69
c:0006 p:---- s:0017 b:0017 l:000016 d:000016 FINISH
c:0005 p:---- s:0015 b:0015 l:000014 d:000014 CFUNC  :catch
c:0004 p:0172 s:0011 b:0011 l:001824 d:001824 METHOD E:/ruby191/lib/ruby/1.9.1/irb.rb:68
c:0003 p:0039 s:0006 b:0006 l:0000f4 d:00248c EVAL   E:/ruby191/bin/irb.bat:20
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:0000f4 d:0000f4 TOP
----------------------------- Ruby level backtrace information-----------------------------------------
E:/ruby191/lib/ruby/gems/1.9.1/gems/nokogiri-1.1.1-x86-mswin32-60/lib/nokogiri.rb:15:in `require'
E:/ruby191/lib/ruby/gems/1.9.1/gems/nokogiri-1.1.1-x86-mswin32-60/lib/nokogiri.rb:15:in `<top (required)>'
E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/www/mechanize.rb:9:in `require'
E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/www/mechanize.rb:9:in `<top (required)>'
E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/mechanize.rb:7:in `require'
E:/ruby191/lib/ruby/gems/1.9.1/gems/mechanize-0.9.0/lib/mechanize.rb:7:in `<top(required)>'
(irb):2:in `require'
(irb):2:in `irb_binding'
E:/ruby191/lib/ruby/1.9.1/irb/workspace.rb:80:in `eval'
E:/ruby191/lib/ruby/1.9.1/irb/workspace.rb:80:in `evaluate'
E:/ruby191/lib/ruby/1.9.1/irb/context.rb:218:in `evaluate'
E:/ruby191/lib/ruby/1.9.1/irb.rb:149:in `block (2 levels) in eval_input'
E:/ruby191/lib/ruby/1.9.1/irb.rb:263:in `signal_status'
E:/ruby191/lib/ruby/1.9.1/irb.rb:146:in `block in eval_input'
E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:244:in `block (2 levels) in each_top_level_statement'
E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:230:in `loop'
E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:230:in `block in each_top_level_statement'
E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `catch'
E:/ruby191/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `each_top_level_statement'
E:/ruby191/lib/ruby/1.9.1/irb.rb:145:in `eval_input'
E:/ruby191/lib/ruby/1.9.1/irb.rb:69:in `block in start'
E:/ruby191/lib/ruby/1.9.1/irb.rb:68:in `catch'
E:/ruby191/lib/ruby/1.9.1/irb.rb:68:in `start'
E:/ruby191/bin/irb.bat:20:in `<main>'

[NOTE]
You may encounter a bug of Ruby interpreter. Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html


This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

然后再试了试，发现require 'hpricot'也会在一个.so文件发生segfault。或许是Ruby 1.9.1与Windows 7 Beta的兼容性问题……

不知道这个问题跟之前别人遇到的在Windows 7 Beta上运行git遇到segfault的问题是不是一样的。是的话或许用Vista的msvcrt放在Ruby 1.9.1的bin目录里就能解决问题。回头试试看……

================================================================

啊，在Windows 7 Beta上用JRuby 1.1.6/JRE 1.6.0u11来运行mechanize也是不行。看起来是JRuby 1.1.6与Nokogiri相性不好……

irb(main):017:0> page = agent.get 'http://www.google.com/'
NoMethodError: undefined method `read_memory' for Nokogiri::HTML::Document:Class
        from E:/jruby-1.1.6/lib/ruby/gems/1.8/gems/nokogiri-1.1.1-java/lib/nokogiri/html.rb:36:in `parse'
        from E:/jruby-1.1.6/lib/ruby/gems/1.8/gems/mechanize-0.9.0/lib/www/mechanize/page.rb:46:in `parser'
        from E:/jruby-1.1.6/lib/ruby/gems/1.8/gems/mechanize-0.9.0/lib/www/mechanize/page.rb:105:in `meta'
        from E:/jruby-1.1.6/lib/ruby/gems/1.8/gems/mechanize-0.9.0/lib/www/mechanize.rb:499:in `fetch_page'
        from E:/jruby-1.1.6/lib/ruby/gems/1.8/gems/mechanize-0.9.0/lib/www/mechanize.rb:224:in `get'
        from (irb):18

评论

10 楼 RednaxelaFX 2009-05-20  

嗯……所以我才一直懒得自己编译nokogiri的 TvT
而且我也不太喜欢进Cygwin……好吧我回头也用Cygwin试试

9 楼 night_stalker 2009-05-20  

RednaxelaFX 写道

night_stalker 写道刚才 1.9 上试了 mechanize，用上 libxml2 和 libxslt 的头文件和 lib，编译过去鸟。
我之前遇到segfault的是nokogiri而不是mechanize吧？你有顺带编译nokogiri么？有的话拜托传我.so吧 T T

嗯，我是在 cygwin 编译的 ，两个 lib 安装很舒服。后来也爬了些网页，没出问题。

但是 cygwin 外面的话…… libxml2 还好说，至少给了 msvc 的编译方式，但 libxslt 只看见 rpm 包的下载……

8 楼 RednaxelaFX 2009-05-20  

night_stalker 写道

刚才 1.9 上试了 mechanize，用上 libxml2 和 libxslt 的头文件和 lib，编译过去鸟。

我之前遇到segfault的是nokogiri而不是mechanize吧？你有顺带编译nokogiri么？有的话拜托传我.so吧 T T

7 楼 night_stalker 2009-05-20  

刚才 1.9 上试了 mechanize，用上 libxml2 和 libxslt 的头文件和 lib，编译过去鸟。

6 楼 RednaxelaFX 2009-04-17  

话说刚才在同学那边做实验，用了个折中的办法来在JRuby上用Mechanize。
老的Mechanize还是用Hpricot的，而0.6.x系列的Hpricot有Java版，所以两者都换回老一些的版本就能凑合用。

刚才在同学那边用的是：
Mechanize 0.8.5
Hpricot 0.6.161-Java
在JRuby 1.2.0里稍微试了下，都能行 T T

5 楼 oCameLo 2009-04-09  

检查一下native.so等文件链的是哪个版本的Ruby？在1.9里require了1.8的so，就会出Segmentation fault。

4 楼 RednaxelaFX 2009-03-04  

呃……看来只有等了。在Ruby的Issue tracking system上的这个bug被分类为third party issue了 T T

3 楼 idol 2009-03-02  

Ruby 1.9.1跟Mechanize的兼容性问题

2 楼 RednaxelaFX 2009-02-27  

idol 写道

碰到同样的问题，有解决方案么

是哪部分一样？Ruby 1.9.1还是JRuby 1.1.6还是Win7？Win7的话请试试看把Vista下的Windows\System32里的C运行时文件复制到Win7这边的Ruby的bin目录里试试。如果是Ruby 1.9.1跟Mechanize的兼容性问题的话我还不清楚……

我在Win7上遇到的问题暂时没找到解决的办法。最近在忙些别的事，Ruby这边有点放下了……呃，尽快看看能不能找到办法解决吧 T T

1 楼 idol 2009-02-27  

碰到同样的问题，有解决方案么