ruby / spec Goto Github PK
View Code? Open in Web Editor NEWThe Ruby Spec Suite aka ruby/spec
License: MIT License
The Ruby Spec Suite aka ruby/spec
License: MIT License
The number of errors+failures on Windows is now only 24! https://ci.appveyor.com/project/eregon/spec/build/140
However, these ones seem non-trivial to fix. Some of them might be just not applicable on Windows, but most of these seem to be inconsistent with the behavior on UNIX-like platforms. Some of them are also worth to write a MRI bug.
Examples can be skipped on Windows using platform_is_not :windows
,
but then the behavior on Windows should also be specified in a platform_is :windows
. If the behavior is strange, it's worth a MRI bug/issue to ask if this is intentional/Windows limitation/actual bug.
@vais @nobu Could I ask for your help to identify which failures/errors fall in which category and to get down to 0 failures/errors but still report bad behavior on MRI's bug tracker?
Hi, @eregon, what's the "right" way to skip the entire https://github.com/ruby/spec/blob/master/core/argf/read_nonblock_spec.rb on Windows?
ARGF.read_nonblock
is not supported on Windows (https://bugs.ruby-lang.org/issues/5954#note-5), and it does not raise a nice exception like, e.g. fork
does on Windows.
fork
spec basically wraps most of the file in a "not on windows" block, but there's also a small "on windows" block there. Since the entire ARGF.read_nonblock
spec can be skipped on Windows, is there a better way than to wrap the entire spec in a "not on windows" block, indenting the whole thing 2 spaces?
There are some weird conflicts between tests that define methods explicitly.
it
are evaluated on the context of MSpec's @env
b
.b
to be undefined.rescue_spec
runs before return_spec
everything is fine.return_spec
runs before rescue_spec
this test fails.Should we replace all occurrences of explicitly defined methods to evaluate("def ...", &block)
?
RubySpec never really died, it just split into two worlds: the ruby/rubyspec world, managed by ruby-core and JRuby committers; and the rubinius world. Both worlds have continued to add specs. Only the ruby/rubyspec world has committed to staying compatible (or rather, only that world has the keys to Ruby's future).
I'm wondering what if anything we should do with specs the Rubinius folks continue to add to their repo.
There's a third option...we just ignore their specs. I think it would be a shame to lose them, though.
It might be possible to convince some of the Rubinius committers to submit their specs to ruby/rubyspec.
The 3 failures about Math.atan2 are observed on Solaris 10 on Sparc architecture.
3)
Math.atan2 returns positive zero when passed -0.0, 0.0 FAILED
Expected -0.0 to be 0.0
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2_spec.rb:44:in `block (3 levels) in <top (required)>'
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2_spec.rb:4:in `<top (required)>'
4)
Math.atan2 returns positive zero when passed 0.0, -0.0 FAILED
Expected 3.141592653589793 to be 0.0
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2_spec.rb:56:in `block (3 levels) in <top (required)>'
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2_spec.rb:4:in `<top (required)>'
5)
Math.atan2
Expected -3.141592653589793 to be 0.0
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2
/XXXXX-trunk-51812/spec/rubyspec/core/math/atan2_spec.rb:4:in `<top (required)>'
In core/math/atan2_spec.rb, the above 3 specs are guarded by "platform_is :solaris", introduced by the commit ac7ca51.
However, the behavior of Math.atan2 is completely the same as that of other platforms, and these are apparently inappropriate and unnecessary.
The code and the behavior of Math.atan2 in CRuby have been changed when releasing Ruby 1.9.3 and Ruby 2.2, and I guess that the original author of ac7ca51 misunderstood the difference between Ruby versions as platform specific.
My build environment is a bit strange because I use "/mnt/sdb1/ruby" for another disk and I put link to "/home/ko1/ruby" (ln -s /mnt/sdb1/ruby /home/ko1/ruby).
Recent changes, there are two failed on such an environment.
./miniruby -I../../gitruby/lib -I. -I.ext/common ../../gitruby/tool/runruby.rb --extout=.ext -- --disable-gems -r./x86_64-linux-fake ../../gitruby/spec/mspec/bin/mspec run -B ../../gitruby/spec/default.mspec
ruby 2.3.0dev (2015-06-17 remove_cf_klass 50935) [x86_64-linux]
last_commit=fix Proc#inspect and so on
1)
The -C command line option changes the PWD when using a file FAILED
Expected "/home/ko1/src/ruby/gitruby/spec/rubyspec/command_line/fixtures"
to equal "/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/fixtures"
/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/dash_upper_c_spec.rb:12:in `block (2 levels) in <top (required)>'
/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/dash_upper_c_spec.rb:3:in `<top (required)>'
2)
The -C command line option changes the PWD when using -e FAILED
Expected "/home/ko1/src/ruby/gitruby/spec/rubyspec/command_line/fixtures"
to equal "/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/fixtures"
/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/dash_upper_c_spec.rb:18:in `block (2 levels) in <top (required)>'
/mnt/sdb1/ruby/gitruby/spec/rubyspec/command_line/dash_upper_c_spec.rb:3:in `<top (required)>'
...
Finished in 210.856470 seconds
3453 files, 24539 examples, 174262 expectations, 2 failures, 0 errors, 0 tagged
Thanks,
Koichi
These two expectations are incorrect: https://github.com/ruby/spec/blob/master/core/argf/binmode_spec.rb#L15-L29
Basically, there is a file with Unix-style line breaks, and it is expecting that setting binmode
will convert these Unix-style line endings into Windows-style line endings. That does not make any sense, because it's just not what binmode
does. Per http://ruby-doc.org/core-2.0.0/ARGF.html#method-i-binmode
Puts ARGF into binary mode. Once a stream is in binary mode, it cannot be reset to non-binary mode. This option has the following effects:
- Newline conversion is disabled.
- Encoding conversion is disabled.
- Content is treated as ASCII-8BIT.
The proof is in the pudding:
C:\Users\Administrator\GitHub\spec>C:\Ruby\ruby-2.2.4-i386-mingw32\bin\ruby.exe -ve "ARGF.binmode; p ARGF.gets" C:\Users\Administrator\GitHub\spec\core\argf\fixtures\bin_file.txt
ruby 2.2.4p230 (2015-12-16 revision 53155) [i386-mingw32]
"test\n"
And here is the same result using ruby 1.8.7 (cause I wondered where these expectations came from, perhaps this is how it used to work on 1.8? But no.)
C:\Users\Administrator\GitHub\spec>C:\Ruby\ruby-1.8.7-p374-i386-mingw32\bin\ruby.exe -ve "ARGF.binmode; p ARGF.gets" C:\Users\Administrator\GitHub\spec\core\argf\fixtures\bin_file.txt
ruby 1.8.7 (2013-06-27 patchlevel 374) [i386-mingw32]
"test\n"
Ok, so to fix this and test ARGF.binmode
properly, we need to add another fixture - a text file (named win_file.txt
) containing Windows-style line breaks, then reading it in binmode
should give us Windows-style line breaks, and reading it without binmode
should give us Unix-style line endings:
C:\Users\Administrator\GitHub\spec>C:\Ruby\ruby-2.2.4-i386-mingw32\bin\ruby.exe -ve "ARGF.binmode; p ARGF.gets" C:\Users\Administrator\GitHub\spec\core\argf\fixtures\win_file.txt
ruby 2.2.4p230 (2015-12-16 revision 53155) [i386-mingw32]
"test\r\n"
C:\Users\Administrator\GitHub\spec>C:\Ruby\ruby-2.2.4-i386-mingw32\bin\ruby.exe -ve "p ARGF.gets" C:\Users\Administrator\GitHub\spec\core\argf\fixtures\win_file
.txt
ruby 2.2.4p230 (2015-12-16 revision 53155) [i386-mingw32]
"test\n"
@eregon unless there are any objections, I will be sending a PR for this, LMK...
I attempted to make a spec for seeking with offsets > 2**32 without creating a large file.
The attempt here worked ok on OS X, but not on Linux.
I modified it to only run on OS X, but obviously we'd like something that runs on Linux at least.
Suggestions?
RubyCI on Debian wheezy fails:
I guess reverting f0d5c14 will fix the problem.
This isn't necessarily a bug, but I think the behavior is incorrect. (If I should open this as a bug against MRI, let me know.)
On systems that do not support the posix_fadvise
function, a call to IO#advise (with suitable args) returns nil/success. In other words, it silently fails without actually executing that OS function. However, I believe it should raise NotImplementedError
on those platforms. That behavior seems more correct rather than silently failing.
I ran across this while running the kernel/core/io specs against the Rubinius ffi-io branch (I'm the principle author of that branch). A few months ago I modified the Rubinius specs with a feature guard with_feature :posix_fadvise
so it would skip those specs on unsupported platforms.
What do you think?
core/dir/read_spec.rb
just tests only one entry.
I think it should be tested that all results are included in the Dir#entries
result, and it returns nil
after all entries are read.
How can I write tests for this kind of sequence behavior in mspec?
One year ago the canonical RubySpec project was retired. It was suggested to @matz by me in a private email discussion that the name be changed from RubySpec to something else. I thought this was a good idea because the name RubySpec carried a bit of baggage, and changing the name would also respect the original author's wishes for its retirement.
That plea was never addressed, and I never posted to an issue tracker or mailing list. Mea culpa. One year later, the issue of the project's name has come up again. There have been conversations on Twitter about the possibility in changing the name. I'm not going to link to them, but suffice it to say there is support for opening this issue as well as changing the name.
This is a formal request to change the name. I recommend Spec
. I think it lives nicely under the namespace ruby/spec
and is close enough to the original w/o being too close. I've heard RbSpec
thrown out there, which isn't bad. Other options are welcome, but I think it's important at this point to seriously consider a departure from the name RubySpec
and let that project be laid to rest in name.
Rubyspec does not seem to contain any tests for conditional regular expressions like the following:
conditional with positional capture group
/\A(foo)?(?(1)(T)|(F))\z/
conditional with named capture group
/\A(?<word>foo)?(?(<word>)(T)|(F))\z/
conditional with only a true branch
# positional
/\A(foo)?(?(1)(T))\z/
# named
/\A(?<word>foo)?(?(<word>)(T))\z/
Does rubyspec try to cover all regular expression syntax? If so I would be happy to PR the necessary tests
Hi!
Currently ruby/rubyspec
doesn't show up in github searches because it's a forked repo. Instead, rubyspec/rubyspec
shows up so some people might get the idea that the project is actually dead when it's not (specially twitter-less people like me :) ). Maybe it makes sense to detach the repo from rubyspec/rubyspec
(and credit the original repo somewhere in the README instead, for example).
I did this with pry-byebug
a while ago, because a user raised the issue (deivid-rodriguez/pry-byebug#29). You have to request it to github support, as specified here, and maybe it's also a good idea to ask/inform the old mantainers.
Thanks for the hard work!
These methods could have the majority of specs shared between them as they do basically the same thing.
The quantity and organisation of the specs here is also very limited. Many of the examples I think come from other test setups and are just added with descriptions such as 'passes kstephens's tests'.
We could do with copying the example in the specs for Array#pack and String#unpack, where tests are heavily parameterised with a separate file for each format character and several levels of sharing and a larger number of much smaller tests.
Current rubyspec doesn't have its own decision process. I worry that rubyspec becomes patch works. As far as I understand rubyspec should decide coding style (how to write specific specs, what helpers should be added, and so on).
I think, rubyspec should also adopt the honest dictator model. People who want to change rubyspec will talk with the dictator.
How do you think?
These encoding failures are only happening in Ruby 2.2. Anyone else see this?
mspec core/dir/element_reference_spec.rb core/dir/glob_spec.rb
ruby 2.2.2p86 (2015-03-03 revision 49825) [x86_64-darwin14]
[/ | ==================100%================== | 00:00:00] 2F 0E
1)
Dir.[] returns Strings in the encoding of the pattern FAILED
Expected #<Encoding:UTF-8>
to be identical to #<Encoding:IBM437>
/Users/anthonycrumley/Projects/ruby/spec/rubyspec/core/dir/element_reference_spec.rb:58:in `block (3 levels) in <top (required)>'
/Users/anthonycrumley/Projects/ruby/spec/rubyspec/core/dir/element_reference_spec.rb:39:in `block in <top (required)>'
/Users/anthonycrumley/Projects/ruby/spec/rubyspec/core/dir/element_reference_spec.rb:38:in `<top (required)>'
2)
Dir.glob with encoding returns Strings in the encoding of the pattern FAILED
Expected #<Encoding:UTF-8>
to be identical to #<Encoding:IBM437>
/Users/anthonycrumley/Projects/ruby/spec/rubyspec/core/dir/glob_spec.rb:50:in `block (4 levels) in <top (required)>'
/Users/anthonycrumley/Projects/ruby/spec/rubyspec/core/dir/glob_spec.rb:29:in `<top (required)>'
Finished in 0.133423 seconds
2 files, 113 examples, 126 expectations, 2 failures, 0 errors
Should we prefer MSpec tags to ruby_bug
guards?
All other implementations use tags, that is files listing excluded specs to ensure all good specs are still good and not run known failing specs.
MRI situation is a bit different as it is the reference implementation. But I am wondering if ruby_bug
is really worth it.
Using tags instead would mean we would have tag files in some repository (MRI repo under spec/tags
, in respective branches?), probably one set of tags per released version to allow different versions to fail different specs (either by using branches or an external tag repo with different directories for 2.0, 2.1, 2.2 and trunk).
Of course, in an ideal world, there would be no failing spec. But this is reality, and backporting a fix to an already released version might not always be possible or a good decision. So, some old released versions must at least exclude some specs.
For trunk, there really should be close to zero excluded spec. But some bugs are not easy to fix and might need some temporary exclusion until they eventually get fixed.
I know somes pros and cons, but I would like to hear your opinion, particularly @nurse, @hsbt and @headius.
Pros of ruby_bug
:
comment
, but it is less visible). Value of the issue number to other implementations is difficult to say, but normally the spec itself should be enough.Pros of tags:
ruby_bug
, while MRI fails them. It is quite rare people depend on the specified behavior if even MRI does not conform to it. A good ruby_version_is
guard is also sometimes a good replacement for ruby_bug since it means the behavior is different or unspecified before some given version (for instance see 23095b7 which now uses a version guard).mspec tag
)ruby_bug
in that it does not modify the spec source.SizedQueue behaves very much like Queue. Shared behavior like that seems to be generally specified in a shared/queue folder where the individual objects are specified in library/queue and library/sizedqueue respectively. The individual object specifications then use it_behaves_like for the shared behavior.
Is this the way to specify two very similar objects like this or is there another way?
This is how I plan to proceed with specifying SizedQueue unless a better way is shown.
I am thinking to have a separate branch for ruby 2.0.0 as it behaves quite differently from newer versions.
Getting the build stable is almost impossible as it fails with EBADF or what looks like encoding confusion.
Maintaining proper guards for 2.0 is also not trivial.
The branch would not be updated with new specs but only receive occasional fixes if it can help the CI.
For the current head of the branch I am thinking to e0769b6 as it seems rather stable on Linux+OS X on Travis at that time and for a few commits after (and backport caf25d4).
@hsbt @nurse what do you think?
P.S.: It seems even the bundled test-all tends to fail sporadically on http://rubyci.org/.
On Solaris 10 running on SPARC architecture CPU, the following spec fails.
Rational#** returns Infinity for Rational(0, 1) passed a negative Float FAILED
Expected -1
to equal 1
Source code of the spec is:
it "returns Infinity for Rational(0, 1) passed a negative Float" do
[-1.0, -3.0, -3.14].each do |exponent|
(Rational(0, 1) ** exponent).infinite?.should == 1
end
end
When running the spec, Rational(0,1) is implicitly converted to Float value 0.0, and the libm function pow(0.0, -1.0) is called.
On Solaris, the return value of pow(0.0, -1.0) varies depending on compile-time options.
http://docs.oracle.com/cd/E37069_01/html/E39019/z4000ac610479.html
See "pow(0,x<0)" line in the Table E-1 about the return value of pow(0.0, -1.0).
By default (==X/Open, C90), pow(0.0, -1.0) returns -HUGE_VAL (== -Infinity in Ruby).
When "-xc99=all" or some equivalent options are given to CFLAGS, pow(0.0, -1.0) returns +Infinity, which is under C99 standards, and the spec passes without failure.
How do we treat the spec in such case?
Right now, MRI's sprintf uses IEEE 754 round-nearest-to-even mode (https://en.wikipedia.org/wiki/IEEE_floating_point#Roundings_to_nearest). We're trying to get Float#round to match that behavior (https://bugs.ruby-lang.org/issues/12548). It breaks RubySpec in a few places.
Looking at tests, some are clearly incompatible with that mode. For instance, from core/float/round_spec.rb around line 70:
it "returns rounded values for big values" do
+2.5e20.round(-20).should eql( +3 * 10 ** 20 )
This should round to (+2 * 10 ** 20) in that mode, not (+3 * 10 ** 20).
All four test failures below are similar -- they seem to depending on the rounding behavior of various types not following IEEE 754 round-nearest-to-even.
Here's Nobuyoshi Nakada's test output when using a local implementation of that mode:
1)
Float#round returns rounded values for big values FAILED
Expected 200000000000000000000
to have same value and type as 300000000000000000000
spec/rubyspec/core/float/round_spec.rb:70:in `block (2 levels) in <top (required)>'
spec/rubyspec/core/float/round_spec.rb:3:in `<top (required)>'
2)
Integer#round returns itself rounded if passed a negative value FAILED
Expected 200
to have same value and type as 300
spec/rubyspec/core/integer/round_spec.rb:22:in `block (2 levels) in <top (required)>'
spec/rubyspec/core/integer/round_spec.rb:4:in `<top (required)>'
3)
Rational#round with no arguments (precision = 0) returns the truncated value toward the nearest integer FAILED
Expected 0
to equal 1
spec/rubyspec/shared/rational/round.rb:17:in `block (3 levels) in <top (required)>'
spec/rubyspec/core/rational/round_spec.rb:3:in `<top (required)>'
4)
Time#strftime rounds an offset to the nearest second when formatting with %z FAILED
Expected "+01:01:04"
to equal "+01:01:05"
spec/rubyspec/core/time/strftime_spec.rb:50:in `block (2 levels) in <top (required)>'
spec/rubyspec/core/time/strftime_spec.rb:7:in `<top (required)>'
see http://rubyci.blob.core.windows.net/centos7/ruby-trunk/log/20150225T050002Z.fail.html.gz
I investigate this. I found some broken specs.
$ ruby mspec/bin/mspec -fd rubyspec/core/argf/codepoints_spec.rb
I got this fail.
% ruby mspec/bin/mspec -fd rubyspec/core/argf/codepoints_spec.rb
ruby 2.1.5p299 (2015-02-19 revision 49652) [x86_64-darwin14.0]
E....
1)
An exception occurred during: after :each
ARGF.codepoints is a public method ERROR
Errno::ENOENT: No such file or directory @ rb_sysopen - -fd
/Users/(snip)/rubyspec/core/argf/shared/each_codepoint.rb:12:in `closed?'
Is there an expectation that the following should work?
begin
require 'uri'
require 'net/http'
p ::File.exist?('/tmp/tomcat-8/v8.0.39/bin/apache-tomcat-8.0.39.tar.gz.md5')
uri = URI( 'file:/tmp/tomcat-8/v8.0.39/bin/apache-tomcat-8.0.39.tar.gz.md5' )
req= Net::HTTP.new(uri.host, uri.port)
req.get( uri )
rescue => e
p e.inspect
p e.backtrace
end
Currently on ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
it results in:
true
"#<TypeError: can't dup NilClass>"
["/opt/chef/embedded/lib/ruby/2.3.0/net/http/generic_request.rb:18:in `dup'",
"/opt/chef/embedded/lib/ruby/2.3.0/net/http/generic_request.rb:18:in `initialize'",
"/opt/chef/embedded/lib/ruby/2.3.0/net/http/request.rb:15:in `initialize'",
"/opt/chef/embedded/lib/ruby/2.3.0/net/http.rb:1156:in `new'",
"/opt/chef/embedded/lib/ruby/2.3.0/net/http.rb:1156:in `get'",
"/tmp/issue.rb:7:in `<main>'"]
Maintaining RubySpec for ruby 2.0.0 releases tends be a much bigger challenge than >= 2.1 versions.
Failures often happen on Travis for hard-to-track reasons (https://travis-ci.org/ruby/rubyspec/jobs/54859872 for instance). I am not sure how we can fix these. @nobu, any idea? ๐
Locally I have 11 failures + 2 errors with 2.0.0p643 which also seem non obvious to fix (badly-encoded strings, EBADF, etc).
Does RubyCI runs the new RubySpec for 2.0 as well?
We should aim to write specs for all new features, as usually the coverage is rather limited and it would greatly help other implementations to get the behavior right and faster.
From https://github.com/ruby/ruby/blob/v2_3_0/NEWS:
Only relevant items are kept for brevity.
object&.foo
, method #foo is called onobject
if it is not nil.try!
in Active Support, except:
obj.try! {}
# validobj&. {}
# syntax errorobj.try!(:foo, bar()) # bar() is always evaluated
obj&.foo(bar()) # bar() is conditionally evaluated
obj&.attr += 1
did_you_mean
gem:
"Yuki".starts_with?("Y")
# => NoMethodError: undefined method `starts_with?' for "Yuki":String
# Did you mean? start_with?
<<~
.exception: false
like IO#read_nonblock.flags
is added.encoding
.exception: false
.exception: false
to returnThis became possible thanks to @Azolo's valiant effort to finally push out a new RubyInstaller with 2.3.1 support, but it's still currently blocked on AppVeyor adding Ruby 2.3.1 support (appveyor/ci#925).
At #263, I removed some specs for Ruby 2.4 and up. Eventually, there should be some new, somewhat different, tests for Ruby 2.4. When should these be provided? How can I write a spec so that we can check it as soon as possible, but avoid that some of the specs fail on trunk before the new features go live, or on older versions of trunk before a given version/commit?
Hello!
What is wrong?
$ ruby -v /home/alex/Projects/ruby/mspec/bin/mspec-run core/time
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]
1)
Time#zone returns the time zone used for time FAILED
Expected "+03"
to equal "AST"
/home/alex/Projects/ruby/spec/core/time/zone_spec.rb:8:in `block (4 levels) in <top (required)>'
/home/alex/Projects/ruby/spec/core/time/zone_spec.rb:7:in `block (3 levels) in <top (required)>'
/home/alex/Projects/ruby/spec/core/time/zone_spec.rb:3:in `<top (required)>'
[/ | ==================100%================== | 00:00:00] 1F 0E
Finished in 0.077151 seconds
62 files, 503 examples, 9514 expectations, 1 failure, 0 errors, 0 tagged
Arch Linux:
$ uname -a
Linux Alex-ArchMainPC 4.9.11-1-ARCH #1 SMP PREEMPT Sun Feb 19 13:45:52 UTC 2017 x86_64 GNU/Linux
Europe/Moscow:
$ timedatectl
Local time: Tue 2017-03-07 00:06:43 MSK
Universal time: Mon 2017-03-06 21:06:43 UTC
RTC time: Mon 2017-03-06 21:06:44
Time zone: Europe/Moscow (MSK, +0300)
Network time on: no
NTP synchronized: no
RTC in local TZ: no
Ruby 2.4.0:
$ ruby -v
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]
Currently a few tests are really slow and take a considerable amount of the total time (require, deamon and timeout take > 20s over 66s).
Passing -fp
shows the time per describe
and it
:
Profiling info:
Top 100 describes:
5.110 - Kernel.require (concurrently)
5.104 - Kernel#require (concurrently)
3.003 - Timeout.timeout
2.121 - Process.daemon when the second argument is false
2.112 - Process.daemon when the second argument is not given
2.111 - Process.daemon when the second argument is nil
1.266 - Process.kill
1.002 - IO.select
...
Top 100 its:
1.501 - Kernel.require (concurrently) blocks a 3rd require if the 1st raises an exception and the 2nd is still running
1.501 - Kernel#require (concurrently) blocks a 3rd require if the 1st raises an exception and the 2nd is still running
1.501 - Kernel.require (concurrently) allows a 2nd require if the 1st raised an exception
1.501 - Kernel#require (concurrently) allows a 2nd require if the 1st raised an exception
1.101 - Kernel#require (concurrently) blocks based on the path
1.101 - Kernel.require (concurrently) blocks based on the path
1.035 - Process.daemon when the second argument is false redirects stderr to /dev/null
1.030 - Process.daemon when the second argument is not given redirects stdout to /dev/null
1.029 - Process.daemon when the second argument is nil redirects stdout to /dev/null
1.029 - Process.daemon when the second argument is false redirects stdout to /dev/null
1.028 - Process.daemon when the second argument is not given redirects stderr to /dev/null
1.028 - Process.daemon when the second argument is nil redirects stderr to /dev/null
1.001 - Kernel#require (concurrently) blocks a second thread from returning while the 1st is still requiring
1.001 - Kernel.require (concurrently) blocks a second thread from returning while the 1st is still requiring
1.001 - Timeout.timeout raises Timeout::Error when it times out with no specified error type
1.001 - Timeout.timeout does not wait too long
1.001 - Timeout.timeout raises specified error type when it times out
0.502 - IO.write on a FIFO writes correctly
0.501 - IO.select blocks for duration of timeout if there are no objects ready for I/O
0.501 - File#flock blocks if trying to lock an exclusively locked file
0.500 - IO.select returns supplied objects when they are ready for I/O
0.500 - Signal.trap accepts a block in place of a proc/command argument
0.500 - Process.times returns current cpu times
...
The require
have a long justification in source code but maybe we could use a Mutex instead to check if inside the require and the other thread use Mutex#locked?
or similar usage to coordinate the threads.
I did not investigate yet the others.
@hsbt Is the migration to ruby/rubyspec complete?
It looks good to me!
spec for File::TMPFILE fails on Amazon Linux:
http://rubyci.org/logs/rubyci.s3.amazonaws.com/amazon/ruby-trunk/log/20161207T063002Z.fail.html.gz
O_TMPFILE is defined in glibc 2.19 or later:
It may be better to test the behavior if File::TMPFILE is defined, instead of testing
the existence of the constant.
I removed 2.0.0 from the Travis build matrix as MRI 2.0.0 is no longer supported.
I plan to drop support for 2.0.0 gradually, since it gives some overhead when writing new specs (more guards) and makes more complicated to test 2.1 syntax like keyword arguments.
I made a tag drop_support_for_2.0.0 to note the last compatible version.
If somebody has a strong objection, I am open to reconsider.
Currently mspec is run without any argument, which means it defaults to :files
, which is:
get(:language) + get(:core) + get(:library)
Should we run :optional
(the same as :capi
)?
Should we run command_line
?
Both pass on rubies 2.0 - trunk.
I think we should run them by default since they do not take long and improve coverage.
Hi.
I'm experiencing a segfault in my test suite for my rather complicated and probably overly ambitious project at https://github.com/markburns/delfos
(Project aim - record method calls and runtime type information in a ruby application)
I'd like to know how best to report this, given that I now have very limited time for open source and have not followed the changes in depth for the move to ruby/spec
. So apologies up front.
#install neo4j on your system
git clone [email protected]:markburns/delfos.git
git checkout origin/support_recording_class_method_inherited
bundle install
# BEWARE this currently deletes all data in the neo4j instance at localhost:7474
bundle exec rspec lib/delfos/integration_spec.rb
Output - https://gist.github.com/markburns/e26481b83c91c3726a0aa497f96b16ab
I have some questions - because I didn't see mention in the CONTRIBUTING.MD
:
https://travis-ci.org/ruby/ruby/jobs/78775640
33)
An exception occurred during: loading /Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/typed_data_spec.rb ERROR
RuntimeError: Unable to compile "/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/ext/typed_data_spec.c"
/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/spec_helper.rb:79:in `compile_extension'
/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/spec_helper.rb:108:in `load_extension'
/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/typed_data_spec.rb:3:in `<top (required)>'
/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/ext/util_spec.c:1:18: fatal error: ruby.h: No such file or directory
#include "ruby.h"
^
compilation terminated.
ERROR:
incflags=-I/Users/travis/build/ruby/ruby/spec/rubyspec/optional/capi/ext -I/./include -I/.ext/include/x86_64-darwin13 -I/./include/ruby
cflags= -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wunused-variable -Werror=pointer-arith -Werror=write-strings -Werror=declaration-after-statement -Werror=implicit-function-declaration -Werror=deprecated-declarations -Wno-packed-bitfield-compat -pipe -fPIC
we added new method named "Array#bsearch_index"
Now that Travis is green again #55 (comment),
I think we should enable notifications when the build is broken.
Of course there will be false negatives if the change applies on trunk but Travis's trunk is not yet updated. But it would be quite good for any other change.
Who wants to get notified? The committer and author, Travis' default?
On Debian GNU/Linux 7.8 (amd64) and on Solaris 10 (sparc), the following failure is observed when running the spec on NFS network file system.
1)
File::Stat#<=> is able to compare files by the same modification times FAILED
Expected -1
to equal 0
/XXXXX-trunk-51815/spec/rubyspec/core/file/stat/comparison_spec.rb:21:in `block (2 levels) in <top (required)>'
/XXXXX-trunk-51815/spec/rubyspec/core/file/stat/comparison_spec.rb:3:in `<top (required)>'
This failure is caused by NFS cache. NFS cache lifetime is 1 second, and time stamps of a file is used to determine whether the cache for the file should be flushed or not.
The following simple patch to set 1 second ago timestamps solved the problem.
It seems that setting the file timestamps to more than 1 second ago would tell the system to expire the cache immediately.
--- a/core/file/stat/comparison_spec.rb
+++ b/core/file/stat/comparison_spec.rb
@@ -15,7 +15,7 @@ describe "File::Stat#<=>" do
end
it "is able to compare files by the same modification times" do
- now = Time.now
+ now = Time.now - 1 # 1 second ago to avoid NFS cache issue
File.utime(now, now, @name1)
File.utime(now, now, @name2)
(@file1.stat <=> @file2.stat).should == 0
https://github.com/ruby/spec/blob/master/library/socket/basicsocket/setsockopt_spec.rb#L189
Above expectation fails on FreeBSD, Solaris environments on Ruby CI. ref http://rubyci.s3.amazonaws.com/freebsd82-64/ruby-trunk/log/20160904T030301Z.fail.html.gz
On Solaris, some specs about File#flock failed.
On Solaris, the following specs in core/file/flock_spec.rb failed.
3)
File#flock returns false if trying to lock an exclusively locked file FAILED
Expected 0
to equal false
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:34:in `block (3 levels) in <top (required)>'
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:33:in `open'
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:33:in `block (2 levels) in <top (required)>'
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:3:in `<top (required)>'
4)
File#flock blocks if trying to lock an exclusively locked file FAILED
Expected [:before, :after]
to equal [:before]
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:59:in `block (2 levels) in <top (required)>'
/XXXXX/trunk-51741/spec/rubyspec/core/file/flock_spec.rb:3:in `<top (required)>'
On Solaris, file locks by using File#flock belong to its process, not belong to file descriptors.
So, flock to the same file in a single process always succeeds.
To examine flock on Solaris, locking a file from different process is needed.
Such as
https://github.com/ruby/rubyspec/blob/master/language/method_spec.rb#L734
evaluate <<-ruby do
def m(a=1, *) a end
ruby
m().should == 1
m(2, 3, 4).should == 2
end
These generate very awkward spec name such as
"A method assigns local variables from method parameters for definition \n def m(*, **k) k end"
Ruby 2.4.0 has been unified bignum/fixnum to Integer at ruby/ruby@f9727c1
spec is boken now. We need to solve it. ref https://travis-ci.org/ruby/ruby/builds/130761218
moved from ruby/mspec#12 and modified.
$ MSPECOPT="--verbose" make test-rubyspec
(snip)
/XXXXX/trunk-51741/spec/rubyspec/language/predefined/data_spec.rb /XXXXX/trunk-51741/spec/rubyspec/language/predefined/fixtures/data_flock.rb:1:in `flock': Bad file number @ rb_file_flock - /XXXXX/trunk-51741/spec/rubyspec/language/predefined/fixtures/data_flock.rb (Errno::EBADF)
from /XXXXX/trunk-51741/spec/rubyspec/language/predefined/fixtures/data_flock.rb:1:in `<main>'
F
(snip)
1)
The DATA constant succeeds in locking the file DATA came from FAILED
Expected ""
to equal "0"
/XXXXX/trunk-51741/spec/rubyspec/language/predefined/data_spec.rb:31:in `block (3 levels) in <top (required)>'
/XXXXX/trunk-51741/spec/rubyspec/language/predefined/data_spec.rb:3:in `<top (required)>'
(snip)
On Solaris, write access is needed to lock a file.
This is well known difference of behavior between Solaris and Linux.
As far as I know, no official spec is defined for flock and the behavior of flock is platform specific.
DATA is opened by Ruby interpreter with read-only mode.
So, I think the spec should be skipped on Solaris.
References:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/291535
https://rails.lighthouseapp.com/projects/8994/tickets/6662-fileflock-cant-lock-read-only-file-for-exclusive-access
This test doesn't restore the value of $,
back to nil. IO.print
uses it internally for joining passed strings into a single one. If interpreter allows to set this global variable, the whole test suite breaks.
rubyspec should contain this spec.
https://bugs.ruby-lang.org/issues/10943
mspec
provides the -R
flag to run specs multiple times. From the perspective of JIT Compiler testing, this is quite useful, as it will allow exercising many more paths than possible with just compile-before-execution.
However, ruby/spec
currently explodes if -R
is passed: ie,
$ ./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems -r./x86_64-linux-fake ./spec/mspec/bin/mspec run -B ./spec/default.mspec -R 2
Puts out huge quantities of error.
I think there's a philosophical question here about spec: Should specs be modified to be idempotent to support repetition?
Or, is this instead a problem that should be addressed via mspec
's runner?
For jruby/jruby#3405, a bug reporter noted behavioral issues in JRuby's handling of String keys to Hash instances.
The correct behaviors are listed below, with JRuby 9.0.4.0 behavior noted where it differs:
I'm not sure how to specify these behaviors properly in RubySpec, given that the original test script provided in jruby/jruby#3405 expects the script's default encoding to be ASCII-8BIT and the strings entered into the test Hashes must be direct, literal keys. RubySpec files are typically UTF-8, and it would not be appropriate to modify e.g. language/hash_spec.rb to be ASCII-8BIT throughout.
I do not believe we should specify the deduplication effects, since they are not intended to be specified behavior in any Ruby. Any test we write should only be confirming that the various ways to enter String keys into Hash do not lose encoding information.
I wrote a spec based on the original code, but then realized it did not actually use literal strings in literal hashes where it should, and was not testing what we want to test. I've included it below as an example.
it "preserves original string keys' encodings after insertion" do
ascii_str = 'hello'.force_encoding('ASCII-8BIT')
str = ascii_str.dup
hash1 = {ascii_str.dup => 1}
hash2 = {str => 2}
str.encoding.should == hash1.keys.first.encoding
str.encoding.should == hash2.keys.first.encoding
str = ascii_str.dup.force_encoding('UTF-8')
hash1 = {'hello'.force_encoding('UTF-8') => 1}
hash2 = {str => 2}
str.encoding.should == hash1.keys.first.encoding
str.encoding.should == hash2.keys.first.encoding
str = ascii_str.dup.force_encoding('UTF-8')
hash1 = {}
hash1[str] = 1
hash2 = {}
hash2[str] = 2
str.encoding.should == hash1.keys.first.encoding
str.encoding.should == hash2.keys.first.encoding
ascii_str2 = 'world'.force_encoding('ASCII-8BIT')
str = ascii_str2.dup.force_encoding('UTF-8')
hash1 = {ascii_str2.dup.force_encoding('UTF-8') => 1}
hash2 = {str => 2}
str.encoding.should == hash1.keys.first.encoding
str.encoding.should == hash2.keys.first.encoding
ascii_str3 = 'unused'.force_encoding('ASCII-8BIT')
str = ascii_str3.dup
hash1 = {ascii_str3.dup.force_encoding('UTF-8') => 1}
hash2 = {str.force_encoding('UTF-8') => 2}
str.encoding.should == hash1.keys.first.encoding
str.encoding.should == hash2.keys.first.encoding
end
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.