antonmi / espec Goto Github PK

Impement randomization of specs order with predefined seed

https://github.com/rspec/rspec-expectations

I have been running in the problem that whenever I run espec from an iex prompt, I always get the same result from the first run:

Mix.Task.reenable "espec"
Mix.Task.run "espec", ["file1_spec.exs"]
# output of file1_spec.exs
Mix.Task.reenable "espec"
Mix.Task.run "espec", ["file2_spec.exs"]
# still output of file1_spec.exs

I don't know if did something wrong, but even restarting the application with Application.stop(Mix.Project.config[:app]) did not help.

I also noticed that the return of the call increases for every run of Mix.Task.run:

After run 1:

[#Function<3.4688896/1 in Mix.Tasks.Espec.run/1>]

After run 2:

[#Function<3.4688896/1 in Mix.Tasks.Espec.run/1>,
 #Function<3.4688896/1 in Mix.Tasks.Espec.run/1>]

and so on

Currently if you try to mock, e.g. :gen_tcp.connect/4 you will end up with an error like

** (EXIT from #PID<0.49.0>) :module_is_sticky
12:44:42.805 [error] Can't load module that resides in sticky dir

meck has option to overcome this called unstick but there's no way to call it as in espec options passed to :espec.new are hardcoded.

Add a possibility to specify any line in the file. ESpec should find respective spec

expect(dict).to be_equal(another_dict)
expect(dict).to have_key(key)
expect(dict).to have_value(value)
expect(dict).to have_size(value)

Make possible to set custom tags to examples and context groups.
A user should be able to run examples with some tag (or tag with some value) or exclude tests.
example

context 'Context', my_tag: "tag_value" do
#example
end

RSpec tags
commad line examples

Consider the following:

defmodule Foo
  defp bar do
    :quux
  end
end

How am I supposed to write a spec against Foo.bar? Obviously the following doesn't work:

defmodule FooSpec do
  use ESpec

  it do
    expect(Foo.bar).to be :quux
  end
end

It does fail with an UndefinedFunctionError.

mix espec spec/some_spec:35
35 is the line number of context

There's a problem using a shared example together with 'finally' block, for example:

defmodule SharedSpec do
  use ESpec, shared: true

  it "should be true" do
    expect(1).to be(1)
  end
end

describe "testing shared spec" do
  finally do
    { :ok, foo: "bar" }
  end

  it_behaves_like(SharedSpec)
end

It raised the following error when I ran the specs:

(EXIT from #PID<0.49.0>) an exception was raised:
** (UndefinedFunctionError) undefined function: SharedSpec.finally_ztrpwibagckdomsylvuqfjexnh/1
SharedSpec.finally_ztrpwibagckdomsylvuqfjexnh(%{conn: %{assigns: %{current_user: nil}, params: %{}}})
(espec) lib/espec/example_runner.ex:99: anonymous fn/3 in ESpec.ExampleRunner.run_finallies/2
(elixir) lib/enum.ex:1261: Enum."-reduce/3-lists^foldl/2-0-"/3
(espec) lib/espec/example_runner.ex:51: ESpec.ExampleRunner."-run_example/2-after$^0/0-0-"/2
(espec) lib/espec/example_runner.ex:32: ESpec.ExampleRunner.run_example/2
(elixir) lib/enum.ex:977: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1261: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:977: Enum.map/2

The specs were fine when I removed the 'finally' block, and the 'finally' block doesn't have to be in the corresponding block to cause the issue (like the example above), it could be the one defined before the shared module.

This will be more descriptive to use :shared key.

To be consistent with underscored variables convention find a new name for the variable.
I think 'shared' is a good option.

Task should create spec folder, spec_helper.exs file with some config hints and some_spec.exs with some hints

Happened across this, could be good to know: elixir-lang/elixir#3254

Pavlov has a nice take on RSpec-like syntax in Elixir without making use of tuple modules.

defmodule SomeSpec do
  use ESpec
  let :a, do: 1
  let :b, do: a + 1
  it do: b |> should eq 2
end 

raises (CompileError) nofile:5: undefined function a/0 in Code.eval_quoted
but with

  let! :b, do: a + 1
  #or
  let :b, do: SomeSpec.a + 1

it doesn't

Something like
expect(self).to receive_message(message, timeout)
or for arbitrary process
expect(pid).to receive_message(message, timeout)

Be careful with 'let' and mocks

https://github.com/fishcakez/dbg
Possible to make some pretty expectations for function calls and message communications between processes

expect(string).to contain(smth)
expect(string).to start_with(smth)
expect(string).to end_with(smth)
expect(string).to be_printable(smth)
expect(string).to be_valid(smth)

https://github.com/chrismccord/linguist

Add the info to output.

I'm trying to figure this out and will raise a PR if I come up with a good solution, but am keen for your feedback if you have any.

-e, --example STRING (Run examples whose full nested names include STRING (may be used more than once)

I am trying to test code that basically looks like this

defmodule MyApp.SomethingImportant.Supervisor do
  use Supervisor

  def start_link(options \\ []) do
    Supervisor.start_link(__MODULE__, options)
  end
end

I want to ensure that start_link really spawns supervisor process. So I write:

defmodule MyApp.SomethingImportantTest do
  alias MyApp.SomethingImportant.Supervisor, as: This

  use ESpec, async: false

  context ".start_link/0" do
    before do
      allow(Supervisor).to accept(:start_link, fn(_, _) -> {:ok, self()} end)
    end

    it "calls Supervisor.start_link with MyApp.SomethingImportant.Supervisor as arg, passes [] as options and returns {:ok, pid}" do
      This.start_link

      expect(Supervisor).to accepted(:start_link, [])
    end
  end
end

This test should fail, because parameters passed to accepted are invalid but I even don't reach that stage because the VM crashes with:

** (EXIT from #PID<0.49.0>) {:compile_forms, {:error, [{'lib/supervisor.ex', [{459, :erl_lint, {:type_syntax, :integer}}, {459, :erl_lint, {:type_syntax, :integer}}, {459, :erl_lint, {:type_syntax, :integer}}, {459, :erl_lint, {:type_syntax, :integer}}]}], []}}

I was unable to find clear pattern what exactly causes this error.

Ubuntu 14.04 64-bit
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]
Elixir 1.0.5
espec 0.7.0

Try to implement a different behaviour for 'it_behaves_like' and 'include_examples' like in RSpec https://www.relishapp.com/rspec/rspec-core/docs/example-groups/shared-examples

'include_examples' just inserts examples into current context (like 'it_behaves_like' does now).
'it_behaves_like' should create nested context.

Not so important but nice to have

Looks like this commit breaks in umbrella projects because :seed is added to Configuration as an integer, and String.to_integer is then called on it.