Coder Social home page Coder Social logo

ahmed-ali / cucumberish Goto Github PK

View Code? Open in Web Editor NEW
343.0 26.0 84.0 346.18 MB

Cucumberish is test automation framework for Behaviour Driven Development. It is inspired by the amazing way of writing automated test cases introduced originally by Cucumber using Gherkin language

License: MIT License

Ruby 0.44% Objective-C 97.11% Gherkin 2.12% Swift 0.34%

cucumberish's Introduction

Cucumberish is Seeking New Maintainers/Contributors

Hello everyone,

I've had a great time working on Cucumberish, but I am currently unable to continue actively maintaining this project. I would like to take this opportunity to express my gratitude to everyone who has contributed to the development of this project over the years.

I'm hoping to find someone (or a group of people) who are interested in taking over as maintainers or contributors. If you are interested, here's how you can express your interest:

  1. Fork the repository and make some contributions. You can start with solving existing issues or adding new features.
  2. Send a pull request with your changes. I promise I will be more attentive to future PRs.
  3. After a few successful pull requests, create an issue titled "[Your Name] for maintainer" and in the description, reference your earlier contribution and any additional features/changes you would like to bring to the project, if any.

The community will have a chance to voice their thoughts on the issue. If the feedback is neutral or positive, and there is no clear blocker on why you wouldn't be a good contributor, I will add you to the project as maintainer.

I believe in the power of open source and the incredible community that drives it. I am confident that with your support, Cucumberish can continue to grow and evolve.

Thank you for your understanding and for your continued support of Cucumberish.

Best regards,

Ahmed


Cucumberish

Cucumberish is a test automation framework for Behaviour Driven Development (BDD). It is inspired by the amazing way of writing automated test cases introduced originally in Cucumber using the Gherkin language.

If you are not sure whether Cucumberish is for you or not checkout the FAQ section

Features

  • Full integration with Xcode Test Navigator.
  • Can be used with continues integration servers, or cloud services; just like XC unit tests!
  • When any test fail, Xcode Test Navigator will point to your .feature file on the line that failed.
  • Your test reports will appear in your Reports Navigator just like any XC unit tests.
  • No Ruby, "command line-only", or any other non-ios related languages or chaotic dependency chain needed to install or use Cucumberish at all!
  • Few minutes installation!
  • Step implementations are done directly in Objective-C or Swift! Which means you can breakpoint and debug your steps implementations easily.
  • Can be used with Unit Test as well as UI Test targets!
Here is a quick animated GIF showing Cucumberish in action:

Cucumberish In Action

Install Manually

Install with CocoaPods

Add the following to your Podfile

use_frameworks!
target 'YourAppTestTarget' do
  pod 'Cucumberish'
end

And follow the rest for the setup steps:

Install with Carthage

Getting started

Now you have Cucumberish in place and you followed all the installation and post-installation instructions; it is time to write your first simple feature and scenario in just a few more steps! Since the exact step implementations will differ between one project and another, we will not dig to deeply into it; we will just outline the general approach on how to get there. I will assume your test target is an Objective-C one for the sake of demonstration; but the same principles can be applied on Swift targets.

Start by creating a new file in your features folder; we will call it example.feature.

Note: You can have only one feature per file, but as many scenarios as you want.

Open this file to edit in Xcode (or any text editor you prefer) and write the following for your very first feature:

Feature: Example
# This is a free text description as an inline documentation for your features, you can omit it if you want.
# However, it is advisable to describe your features well.
As someone who plans to automate the iOS project test cases, I will use Cucumberish.

# First scenario is the scenario name, which will also appear in a proper format in Xcode test navigator
Scenario: First scenario

    # This is the first step in the scenario
    # Also noteworthy; a "Given" step should be treated as the step that defines the app state before going into the rest of the scenario
    # Or consider it as a precondition for the scenario;
    # For example if the user must be logged in to post a comment, then you should say something like "Given the user is logged in" as your Given step.
    Given I have a very cool app
    
    
    # The grammar being used is completely defined by you and your team; it is up to you to find the best way to define your functionality.
    # Only keep in mind that every step must start with "Given", "When", "Then", "And", or "But".
    When I automate it with "Cucumberish"
    Then I will be more confident about the quality of the project and its releases

Now you have one cool feature in place, it is time to implement its steps. You only need to care about the steps, not the feature or the scenario itself. So your step implementations are done without context.

In the body of the CucumberishInit C function and before calling the -[Cucumberish beginExecution] method, add the following:

    Given(@"I have a very cool app", ^(NSArray<NSString *> *args, NSDictionary *userInfo) {
        //Now it is expected that you will do whatever necessary to make sure "I have a very cool app" condition is satisfied :)
        NSLog(@"\"Given I have a very cool app\" step is implemented");
    });
    
    // The step implementation matching text can be any valid regular expression text
    // Your regex capturing groups will be added to the args array in the same order they have been captured by your regex string
    When(@"^I automate it with \"(.*)\"$", ^(NSArray<NSString *> *args, NSDictionary *userInfo) {
        NSLog(@"I am gonna automate my test cases with \"%@\"", args[0]);
    });
    
    Then(@"I will be more confident about the quality of the project and its releases", ^(NSArray<NSString *> *args, NSDictionary *userInfo) {
        NSLog(@"Implemented step for the sake of the example");
    });

And that's it! You have implemented your first feature scenario steps! For more information about the steps, the blocks that are passed to it and the hooks, check the CCIBlockDefinitions.h file and Cucumberish Wiki.

Examples

Beside all the information you can find on the wiki and specifically the Steps Definition page, seeing examples in action is the best way to demonstrate something. You can clone this repository and open the file CucumberishLibraryTest/CucumberishLibrary.xcodeproj.

In the CucumberishLibrary project there are five targets:

  1. CucumberishLibrary target:

    • This is the main target that includes the Cucumberish library, we use it as our testable target to test it in the test targets.
  2. CucumberishFeatureDefinition target:

    • This target is mainly a demonstration of different ways to define a step implemnetation and to get you familiar with most of Cucumberish APIs.
    • Most of the steps in this target uses bit complicated regular expressions strings, but you should not worry about that. As long as you know some of the basics of regular expressions, you should be fine. The Steps Definition page can be of great help if you are not so familiar with regular expressions.
  3. CucumberishTests Target:

    • In this target we use the Cucumberish library to test its own behaviour.

Unit Tests

The CucumberishLibrary project, located in the CucumberLibraryTest folder, contains two targets, CucumberishTests and CucumberishFeatureDefinition, which are used to test Cucumberish itself.

Troubleshooting

My test case scenario failed because the step is not implemented, but I am sure I implemented it — now what?

This can happen if your step definition text can not be matched against the step written in the feature file. It can happen because of a mistake in your regex string, or maybe an additional unwated space has been added to either the definition of the step or the step itself. A good place to put a breakpoint and see what is going on is in Cucumberish/Core/Managers/CCIStepsManagers.m in the method findDefinitionForStep:amongDefinitions:

My test case failed, but when I click on the failed case in the Xcode test navigator it does not open the file where the failure occured.

This is known to happen for two reasons:

  1. If there is a a runtime crash in your code, for example when you try to access the fifth element of an array that has only four elements. It is believed to be meaningless to point to one of your feature files in such a case. However, you still need to find the issue; you can put an exception breakpoint or open the Report navigator in Xcode and click on the first Test item at the top of your reports. Inside this report the failed case will be expandable and will show you all the available information.

  2. If you missed one of the first two steps of the Manual installation, in case you installed Cucumberish manually, or you missed the second step in the Post installation steps.

If you believe you've identified another type of failure, please report it as an issue.

FAQ

  1. How does Cucumberish work in brief?

    • It is very simple. You describe your features in .feature files and define a set of scenarios for each feature where each scenario has a set of steps. Then Cucumberish will create an XC test suite for each feature and an XCTestCase for each scenario, and will map the steps to your code. So it ends up with human-readable features and still implemented in a way we, as developers, prefer.
  2. I want to use Swift in my test target, is it possible?

    • Yes, please checkout the UI test target in the example application. It is completely Swift!
  3. I prefer to use Apple UI test framework for long-term support, is it still possible to do this with Cucumberish?

    • YES! You can combine Cucumberish with your prefered frameworks and tools! This also means Cucumberish is not limited to UI.
  4. I still don't know if Cucumberish is right for me. Is it? You have some options:

    • You can read the wiki, it is short and to the point.
    • You can read a bit about the original Cucumber to get an idea how it has started and what it is all about
    • Finally you can always open an issue if you have a specific issue, or post your questions on StackOverflow (and ping me), and I will be happy to help out.

Continuous Integration

A continuous integration (CI) process is highly recommended and is extremely useful in ensuring that your application stays functional. The easiest way to do this will be either using Bots, or Jenkins or another tool that uses xcodebuild. For tools using xcodebuild, review the manpage for instructions on using test destinations.

Known Issues

When executing individual scenario or feature from the test navigator, the rest of test cases will disappear. You will have to run all the tests by pressing CMD+U (or from the Tests button) in order for the test cases to appear again.

Contributing

I am so glad that you are interested in contributing to Cucumberish. When you have made some improvements or additions that you would like to see in the master repository, please create a pull request and I will merge it as soon as possible.

License

Cucumberish is available under the MIT license. See the LICENSE file for more information.

cucumberish's People

Contributors

ahmed-ali avatar asdf-mobile avatar brentleyjones avatar dgommers avatar elu-dawn avatar enricode avatar ereynolds-agamatrix avatar idelfonsog2 avatar mazyod avatar parfeon avatar patmalt avatar rocklobster avatar sidekickr avatar stevedaontuc avatar titouanvanbelle avatar vincentkjj avatar vladoun avatar wayne-morgan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cucumberish's Issues

Including release notes in Github

Usually I directly open up the Releases tab to see what is new in the latest version. However, when I manually opened up the diff between v1.0.4 and v1.0.5, I found out that there is an actual CHANGELOG.md.

Can we start using Github's native tab to be consistent with other projects? Quick is a prefect example. I would be happy to help.

Support for dry-run to print undefined steps

It would be nice to be able to scan through all the scenarios to make sure all steps are defined.

Given the following scenario

Feature: Test
  Scenario: Test
    Given this step is undefined

When I set Cucumberish.instance().dryRun to true
And I run the tests
Then it should pass and print the following in the console

We found some undefined steps in your scenarios. Use the following snippets to fine them:

Given("this step is undefined") { (args, userInfo) in
    
}

Bridging headers not imported when Podfile has 'use_frameworks!'

Cocoapod installation seems to have an issue when Podfile has

use_frameworks!

Not sure, it's issue with Cocoapods or Cucumberish but it's not working

Error says
``
/Users/Shashi/Desktop/CucumberishPodTests/CucumberishPodTestsCucumberTests/CucumberishPodTestsCucumberTests-Bridging-Header.h:14:9: error: 'Cucumberish.h' file not found

import "Cucumberish.h"

    ^

:0: error: failed to import bridging header '/Users/Shashi/Desktop/CucumberishPodTests/CucumberishPodTestsCucumberTests/CucumberishPodTestsCucumberTests-Bridging-Header.h'
``

Steps to Reproduce

Add use_frameworks! to Podfile and try running tests

addUIInterruptionMonitor gives an exception Internal error: no current scope to add handler

When I am trying to interact with a system dialog which comes up in the application before accessing contacts, i try to addUIInterruptionMonitor before the system alert is expected and write a handler for interacting with the same.
On run time i get the exception: caught "NSInternalInconsistencyException", "Internal error: no current scope to add handler Permission Dialog to."

Has anyone else faced a similar problem?

i must restart xcode everytime in order for the tests to run properly

given i have written several features to be tested
and I run UI tests
and the tests run successfully
when I try to run the UI tests again
then
expected the tests should run successfully
actual all the tests are skipped.. and i get a "tests succeeded" all the time
workaround restart xcode everytime, in order to successfully run tests

note: the test that actually runs and is marked successfully shows as "null" in the Xcode test navigator

screen shot 2016-04-27 at 9 22 36 am

Real iOS 10 device throws Linker command failed error

I tried running the tests on a real iPhone with iOS 10.2.1 and I get the Linker command failed error, but I do not get this error for a simulator of same iOS

Apple Mach-O Linker (ld) Error Group
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Is this a bug or known issue? Can it be fixed?

Adding steps in the execution report

@Ahmed-Ali Thanks for your framework. Is there any way by which we can list down the steps as well in the execution report. Right now i am using Xcpretty which is listing only feature and scenario names not the step names. Is there any feature?

Steps with accent character crash

Hi,

Writing steps with accented character like 'é' or 'ô' make the test crash. I got this error

failed: caught "NSRangeException", "*** -[NSRegularExpression enumerateMatchesInString:options:range:usingBlock:]: Range or index out of bounds"

I dont have time currently to dig into it but if you could that would be awesome.

Aside from that it works perfectly :)

tvOS Support

Currently the podfile does not allow installation if it is a tvOS project.
Adding something like s.tvos.deployment_target = "10.0" to the podspec should solve this.

I personally do not think that any other special changes to Cucumberish is needed, but have not looked deeper. I was able to run the sample tests provided on a Apple TV simulator. XCode/XCTest supports both iOS and tvOS.

Lexical or Preprocessor Issue .h file

Hi,

After setting up Cucumberish in the project and trying to run the first test on Xcode 9 I'm getting this error:

lexical or proprocessor issue

This file is definitely located in the project in Supporting Files folder.

On Xcode 8.3 I'm not getting this error.

Any ideas what may be the cause of that?
Bart

weird error when having extended steps

these gherkin steps work

Feature: view user details
As a user I should be able to see all user details

Scenario: displaying user rows
  Given the app is running
  Then A table with rows of type "users" is displayed

Scenario Outline: displaying user names on rows
    Given the app is running
    Then the row with text "<name>" shows up at index "<index>"

Examples:
|     name   | index |
|  Abdullah  |      0      |
|  Rima      |      1      |
|  Craig     |      2      |

however when i want to combine these steps into this

Feature: view user details
As a user I should be able to see all user details

Scenario Outline: displaying user rows
  Given the app is running
  When A table with rows of type "users" is displayed
  Then the row with text "<name>" shows up at index "<index>"

Examples:
|     name   | index |
|  Abdullah  |      0      |
|  Rima      |      1      |
|  Craig     |      2      |

then this error happens

2016-04-27 15:51:45.053 XCTRunner[96459:6134523] Running tests...
2016-04-27 15:51:45.054 XCTRunner[96459:6134523] Looking for test bundles in /Users/abdullahbakhach/Library/Developer/CoreSimulator/Devices/9F30BAE1-E232-4C92-A053-B872A862B10B/data/Containers/Bundle/Application/8E345FC2-E04D-41A9-BB6D-625786D6A43E/listoUITests-Runner.app/PlugIns
2016-04-27 15:51:45.055 XCTRunner[96459:6134523] Found test bundle at /Users/abdullahbakhach/Library/Developer/CoreSimulator/Devices/9F30BAE1-E232-4C92-A053-B872A862B10B/data/Containers/Bundle/Application/8E345FC2-E04D-41A9-BB6D-625786D6A43E/listoUITests-Runner.app/PlugIns/listoUITests.xctest
2016-04-27 15:51:45.055 XCTRunner[96459:6134523] Looking for configurations in /Users/abdullahbakhach/Library/Developer/CoreSimulator/Devices/9F30BAE1-E232-4C92-A053-B872A862B10B/data/Containers/Bundle/Application/8E345FC2-E04D-41A9-BB6D-625786D6A43E/listoUITests-Runner.app/PlugIns/listoUITests.xctest
2016-04-27 15:51:45.058 XCTRunner[96459:6134523] Found configuration <XCTestConfiguration: 0x7fb17623c130>
                      testBundleURL:file:///Users/abdullahbakhach/Library/Developer/Xcode/DerivedData/listo-bqafxjqqguipqehhmcyiembvvjcy/Build/Products/Debug-iphonesimulator/listoUITests-Runner.app/PlugIns/listoUITests.xctest/
                  productModuleName:listoUITests
                        testsToSkip:(null)
                         testsToRun:(null)
                 reportResultsToIDE:YES
                  sessionIdentifier:<__NSConcreteUUID 0x7fb17623a620> A22C6378-F597-46E0-BFDD-90EEC799BBF9
         pathToXcodeReportingSocket:(null)
          disablePerformanceMetrics:no
    treatMissingBaselinesAsFailures:no
                    baselineFileURL:(null)
              targetApplicationPath:/Users/abdullahbakhach/Library/Developer/Xcode/DerivedData/listo-bqafxjqqguipqehhmcyiembvvjcy/Build/Products/Debug-iphonesimulator/listo.app
          targetApplicationBundleID:Tohtt.listo
                   reportActivities:YES
           testsMustRunOnMainThread:YES
15:51:45.441 XCTRunner[96459:6134639] _XCT_testBundleReadyWithProtocolVersion:minimumVersion: reply received
15:51:45.445 XCTRunner[96459:6134639] _IDE_startExecutingTestPlanWithProtocolVersion:16
Test Suite 'All tests' started at 2016-04-27 15:51:45.469
Test Suite 'listoUITests-Runner.app' started at 2016-04-27 15:51:45.470
Test Suite 'CCI_ViewUserDetails' started at 2016-04-27 15:51:45.471
Test Case '-[CCI_ViewUserDetails displayingUserRowsExample1]' started.
    t =     0.00s     Start Test
    t =     0.00s     Set Up
2016-04-27 15:51:45.474 XCTRunner[96459:6134523] Currently executing: "Given the app is running"
fatal error: unexpectedly found nil while unwrapping an Optional value

here is the trace:

* thread #1: tid = 0x5d9afb, 0x000000011993fe78 libswiftCore.dylib`function signature specialization <Arg[0] = Exploded, Arg[1] = Exploded, Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage (Swift.StaticString, Swift.StaticString, Swift.StaticString, Swift.UInt) -> () + 40, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x000000011993fe78 libswiftCore.dylib`function signature specialization <Arg[0] = Exploded, Arg[1] = Exploded, Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage (Swift.StaticString, Swift.StaticString, Swift.StaticString, Swift.UInt) -> () + 40
  * frame #1: 0x000000011977c66b listoUITests`static listoUITests.CucumberishInitializer.(args=nil, userInfo=nil, application=nil) -> () -> ()).(closure #2) + 107 at listoUITests.swift:22
    frame #2: 0x000000011977afde listoUITests`reabstraction thunk helper from @callee_owned (@owned Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, @owned Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>) -> (@unowned ()) to @callee_owned (@in (Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>)) -> (@out ()) + 46 at CCIStepDefinitions.swift:0
    frame #3: 0x000000011977c091 listoUITests`partial apply forwarder for reabstraction thunk helper from @callee_owned (@owned Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, @owned Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>) -> (@unowned ()) to @callee_owned (@in (Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>)) -> (@out ()) + 81 at listoUITests.swift:0
    frame #4: 0x000000011977b016 listoUITests`reabstraction thunk helper from @callee_owned (@in (Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>)) -> (@out ()) to @callee_owned (@owned Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, @owned Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>) -> (@unowned ()) + 38 at CCIStepDefinitions.swift:0
    frame #5: 0x000000011977b11f listoUITests`reabstraction thunk helper from @callee_owned (@owned Swift.ImplicitlyUnwrappedOptional<Swift.Array<Swift.String>>, @owned Swift.ImplicitlyUnwrappedOptional<Swift.Dictionary<__ObjC.NSObject, Swift.AnyObject>>) -> (@unowned ()) to @callee_unowned @convention(block) (@unowned Swift.ImplicitlyUnwrappedOptional<__ObjC.NSArray>, @unowned Swift.ImplicitlyUnwrappedOptional<__ObjC.NSDictionary>) -> (@unowned ()) + 255 at CCIStepDefinitions.swift:0
    frame #6: 0x0000000119759392 listoUITests`-[CCIStepsManager executeStep:](self=0x00007fb176116730, _cmd="executeStep:", step=0x00007fb172d74c90) + 1042 at CCIStepsManager.m:148
    frame #7: 0x00000001197728bf listoUITests`executeSteps(testCase=0x00007fb172dfa2d0, steps=0x00007fb172dc0d40, parentScenario=0x00007fb172df9a10) + 959 at Cucumberish.m:446
    frame #8: 0x0000000119772e9a listoUITests`__executeScenario_block_invoke(.block_descriptor=0x00007fff539a2e60) + 90 at Cucumberish.m:424
    frame #9: 0x000000011976f89f listoUITests`-[Cucumberish executeAroundHocksWithScenario:executionBlock:](self=0x00007fb172c0c200, _cmd="executeAroundHocksWithScenario:executionBlock:", scenario=0x00007fb172df9a10, executionBlock=(listoUITests`__executeScenario_block_invoke at Cucumberish.m:423)) + 1791 at Cucumberish.m:224
    frame #10: 0x0000000119771449 listoUITests`executeScenario(self=0x00007fb172dfa2d0, _cmd="displayingUserRowsExample1", scenario=0x00007fb172df9a10, feature=0x00007fb1761c8ac0) + 1353 at Cucumberish.m:423
    frame #11: 0x000000010ccc41cc CoreFoundation`__invoking___ + 140
    frame #12: 0x000000010ccc401e CoreFoundation`-[NSInvocation invoke] + 286
    frame #13: 0x000000010c2eb080 XCTest`__24-[XCTestCase invokeTest]_block_invoke_2 + 159
    frame #14: 0x000000010c31eb14 XCTest`-[XCTestContext performInScope:] + 184
    frame #15: 0x000000010c2eafd0 XCTest`-[XCTestCase invokeTest] + 169
    frame #16: 0x000000010c2eb46b XCTest`-[XCTestCase performTest:] + 443
    frame #17: 0x000000010c2e9131 XCTest`-[XCTestSuite performTest:] + 377
    frame #18: 0x000000010c2e9131 XCTest`-[XCTestSuite performTest:] + 377
    frame #19: 0x000000010c2e9131 XCTest`-[XCTestSuite performTest:] + 377
    frame #20: 0x000000010c2d6181 XCTest`__25-[XCTestDriver _runSuite]_block_invoke + 51
    frame #21: 0x000000010c2f6b5b XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 615
    frame #22: 0x000000010c2d60cd XCTest`-[XCTestDriver _runSuite] + 408
    frame #23: 0x000000010c2d6e2c XCTest`-[XCTestDriver _checkForTestManager] + 263
    frame #24: 0x000000010c31fe99 XCTest`_XCTestMain + 628
    frame #25: 0x000000010cd01a1c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #26: 0x000000010ccf76a5 CoreFoundation`__CFRunLoopDoBlocks + 341
    frame #27: 0x000000010ccf6e02 CoreFoundation`__CFRunLoopRun + 850
    frame #28: 0x000000010ccf6828 CoreFoundation`CFRunLoopRunSpecific + 488
    frame #29: 0x000000010e987ad2 GraphicsServices`GSEventRunModal + 161
    frame #30: 0x000000010d183610 UIKit`UIApplicationMain + 171
    frame #31: 0x000000010c25b94a XCTRunner`___lldb_unnamed_function9$$XCTRunner + 42
    frame #32: 0x000000010f19692d libdyld.dylib`start + 1
    frame #33: 0x000000010f19692d libdyld.dylib`start + 1

  thread #2: tid = 0x5d9b6c, 0x000000010f4ebfde libsystem_kernel.dylib`kevent64 + 10, queue = 'com.apple.libdispatch-manager'
    frame #0: 0x000000010f4ebfde libsystem_kernel.dylib`kevent64 + 10
    frame #1: 0x000000010f1568c0 libdispatch.dylib`_dispatch_mgr_invoke + 260
    frame #2: 0x000000010f15665a libdispatch.dylib`_dispatch_mgr_thread + 54

  thread #3: tid = 0x5d9b6e, 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #0: 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #1: 0x000000010f4af729 libsystem_pthread.dylib`_pthread_wqthread + 1283
    frame #2: 0x000000010f4ad365 libsystem_pthread.dylib`start_wqthread + 13

  thread #4: tid = 0x5d9b6f, 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #0: 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #1: 0x000000010f4af729 libsystem_pthread.dylib`_pthread_wqthread + 1283
    frame #2: 0x000000010f4ad365 libsystem_pthread.dylib`start_wqthread + 13

  thread #5: tid = 0x5d9b73, 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #0: 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #1: 0x000000010f4af729 libsystem_pthread.dylib`_pthread_wqthread + 1283
    frame #2: 0x000000010f4ad365 libsystem_pthread.dylib`start_wqthread + 13

  thread #6: tid = 0x5d9b74, 0x000000010f4e53c2 libsystem_kernel.dylib`semaphore_wait_trap + 10, queue = 'parsing queue'
    frame #0: 0x000000010f4e53c2 libsystem_kernel.dylib`semaphore_wait_trap + 10
    frame #1: 0x000000010f15340b libdispatch.dylib`_dispatch_semaphore_wait_slow + 224
    frame #2: 0x000000011aa2baf5 DTXConnectionServices`-[DTXMessageParser waitForMoreData:incrementalBuffer:] + 87
    frame #3: 0x000000011aa2b5a0 DTXConnectionServices`-[DTXMessageParser parseMessageWithExceptionHandler:] + 61
    frame #4: 0x000000011aa2b316 DTXConnectionServices`__68-[DTXMessageParser initWithMessageHandler:andParseExceptionHandler:]_block_invoke + 39
    frame #5: 0x000000010f14ad59 libdispatch.dylib`_dispatch_call_block_and_release + 12
    frame #6: 0x000000010f1664a7 libdispatch.dylib`_dispatch_client_callout + 8
    frame #7: 0x000000010f1500e8 libdispatch.dylib`_dispatch_queue_drain + 1048
    frame #8: 0x000000010f14faa0 libdispatch.dylib`_dispatch_queue_invoke + 595
    frame #9: 0x000000010f1513b8 libdispatch.dylib`_dispatch_root_queue_drain + 565
    frame #10: 0x000000010f15117c libdispatch.dylib`_dispatch_worker_thread3 + 98
    frame #11: 0x000000010f4af68f libsystem_pthread.dylib`_pthread_wqthread + 1129
    frame #12: 0x000000010f4ad365 libsystem_pthread.dylib`start_wqthread + 13

  thread #7: tid = 0x5d9b7b, 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #0: 0x000000010f4eb6de libsystem_kernel.dylib`__workq_kernreturn + 10
    frame #1: 0x000000010f4af729 libsystem_pthread.dylib`_pthread_wqthread + 1283
    frame #2: 0x000000010f4ad365 libsystem_pthread.dylib`start_wqthread + 13

Access current test case within step definition

How can I access the instance of XCTestCase that is currently running in order to call methods like the following?

- (void)waitForExpectations:(NSArray<XCTestExpectation *> *)expectations timeout:(NSTimeInterval)seconds;

BDD tests pass even though their definitions are missing.

Steps:

  1. Create a test scenario with a Given step as "I have a very cool app" in feature file.
  2. Implement its definition in the native code.
  3. Create an another test scenario with a Given step as "I have a very cool app and website" in feature file.
  4. Don't implement its definition in native code.
  5. Run BDD tests.
  6. All tests pass even though another test scenario was never implemented in native code.

This happens only when the implemented step string acts as the substring of the step in feature file.

Autonomous tests

Hi,

Calabash framework can be linked to an app and that will enable the embedded HTTP server thus allowing for instrumenting the app on the physical device.

Can Cucumberish be built into another (testing) app to be installed on the same device and then connect to Calabash http server to run the tests completely autonomously?

Thanks

`beforeStart` and `afterFinish` should run in their own test cases

Currently beforeStart will run in the first test case, and afterFinish will run in the last test case (I think, I don't have an afterFinish hook). If these are long running (like they are for our setup) it will incorrectly inflate the running time of those test cases. If instead they became their own test case (like [Hooks beforeStart] and [Hooks afterFinish]) they will be reported on separately. Of course, we need to make sure the name doesn't clash, so maybe make it configurable.

Escape characters

This is one of my scenarios:

Feature: Login Feature

Scenario: Login Scenario

Given the app is running
And I wait to see "Login" button
And I wait to see "Login" field
And I wait to see "Password" secure field
And I write "[email protected]" in the "Login" field
And I write "Password" in the "Password" secure field
And I tap the "Login" button 1 time
Then I wait to see "notificationsView" view

[email protected] is written in the field instead [email protected]

screen shot 2016-11-18 at 18 01 05

Any ideas?

Adding sequence to feature files

I have got into a need where I want to execute my feature files in sequence or say in a specific order to reduce the execution time.
I tried calling executeFeatures twice with the specific tags like below but the scenarios in each feature files are executed in a jumbled manner and feature execution is also not in sequence.

    Cucumberish.executeFeatures(inDirectory: "Features", from: bundle,
                                includeTags: ["FirstTimeUse"], excludeTags: [])
    
    Cucumberish.executeFeatures(inDirectory: "Features", from: bundle,
                                includeTags: nil, excludeTags: ["FirstTimeUse"])

Does cucumberish support something like this?

Support subdirectories

Currently if the feature directory has subdirectories the features are not picked up. When you have lots of feature files is makes sense to be able to organize them on the filesystem in subdirectories,

Include and Exclude tags don't consider scenarios

parserFeaturesInDirectory:fromBundle:includeTags:excludeTags: only looks at the tags at the feature level. The correct behavior should be that it looks at all of the scenarios, since a tag at the feature level is just shorthand for applying the tag to all of the scenarios under it.

Currently, if a tag only exists on a scenario, like @Skip, it's not considered.

Implement using data table when calling a step within a step definition

I'd like to be able to call a step within a step definition, but one that uses a data table. I've seen examples in Cucumber and Ruby, but not for this framework. Something similar to below, possibly?

        self.performStep("I enter this data into the following fields:", in: dataTable(%{
      | userValue | field   | 
      | myName   | input field  |)

Accessing the JSON reports

Hi all,
I have managed to integrate cucumberish into my project successfully and can't wait to use. Now I am looking to extract the generated JSON reports out from iOS Simulator to report as HTML in Jenkins. As it stored in NSDocumentDirectory its bit hard to get out. I am not sure if there is an easy way of extracting the generated report.

Sorry for creating this as an issue. I couldn't find any easy way of clarifying.

No such module 'Cucumberish'

I am getting this error No such module 'Cucumberish'

image

Cucumberish is successfully installed via Pods and Project builds fine as well. Cleaned the project , deleted derived data and still no luck.

Anyone experienced that issue?

how does regex work in Cucumberish?

@Ahmed-Ali
I was trying to understand this syntax you have

 MatchAll(@"^I should see \"([^\\\"]*)\" in (?:the )?\"([^\\\"]*)\" (?:button|view|label)$" ,  ^void(NSArray *args, id userInfo) {
     //Step implementation goes here
     //The definition string in this case can match all the following steps
     //When I should see "Button Title" in "MyButton" button
     //Then I should see "Button Title" in "MyButton" button
     //And I should see "Button Title" in "MyButton" button
     //But I should see "Button Title" in "MyButton" button
 });

b/c some of my own expressions weren't recognized properly..
So i created this playground.. and your patterns aren't recognized by the regular swift regex matchers.. can you please elaborate a bit? are you using a different regex engine under the hood or something?

Reporting: Can't click into the source file from Xcode report navigator when there is a space in app name

I've discovered with v0.0.7 that we are not able to click into the report navigator after a case fails.
We have the setting SRC_ROOT=@"$(SRCROOT)"

On debugging we've noticed that parseFeatureFiles method of CCIFeaturesManager.m is not handling such case correctly (the original string has URL encoded spaces into %20, but not handled in code)

NSString * localPath = [[filePath.absoluteString stringByReplacingOccurrencesOfString:[[NSBundle bundleForClass:[self class]] bundlePath] withString:@""] stringByReplacingOccurrencesOfString:@"file://" withString:@""];

This following logic works for us

NSString * testBundlePath = [[NSBundle bundleForClass:[self class]] bundlePath];
NSString * localPath = [[[filePath.absoluteString stringByRemovingPercentEncoding]
                                     stringByReplacingOccurrencesOfString:testBundlePath withString:@""]
                                    stringByReplacingOccurrencesOfString:@"file://" withString:@""];

If this looks good to you, I'll open a PR.

Option for changing/disabling class prefix

Currently we prefix the test case names with CCI/CCI_. Can we have an option to disable that prefix and/or change it? For our use case we only use Cucumberish so we don't need the prefix, and it would make our reports look cleaner without them.

JSON output

Hi!

I'm creating a project with several features files and its corresponding steps in Swift. The integration is OK and I have achieved that all tests passed. However, when I try to get the results report, I discover that this report is not generated.

In more detail, I have 3 feature files, and 3 swift files with steps definition, and I call these definitions from my cucumberish initializer. Debugging the library, I have discovered that, in the last test, after run the steps of this test, testing process is over, without run after methods neither write the output file (Cucumberish.m -> executeScenario() ). I run my tests with cmd+U. Also, I detect that when I run only one of the features (no matter which) from test navigator, Cucumberish creates the file, but the rest of the features not tested are skipped or failed.

I hope you can solve this soon! Thank you for the library!

Number of executed tests is wrong in the console

When running the tests for the following .feature file that contains 1 scenario the console prints

Test Suite 'All tests' failed at 2017-04-06 12:27:22.018.
	 Executed 2 tests, with 1 failure (0 unexpected) in 0.008 (0.012) seconds

Feature

@run
Feature: Test

@smoke
Scenario: As a user, ...
     When I do ..
     Then I should ...

This seem to be counted as a Test Case too

Test Case '-[CCISDKStartingTheSDK cucumberishCleanupScenario]' started.
    t =     0.00s     Start Test at 2017-04-06 12:27:22.014
    t =     0.00s     Set Up
    t =     0.00s     Tear Down
Test Case '-[CCISDKStartingTheSDK cucumberishCleanupScenario]' passed (0.004 seconds).

Not a big deal but I still think it should be fixed if it is possible.

Multiple Examples tables for a scenario outline

I supply different Examples to the same scenario based on tags. The tags may be for different test durations (a brief daily test vs a more comprehensive weekly test), or for platforms with different capabilities (desktop platform may support different file formats to android platform). I already use this functionality on other platforms, and would like to use the same feature files on iOS to reduce duplication.
e.g.

Scenario Outline: Scenario with multiple examples tables
Given the test equipment is set up
When <test_data> is applied
Then the result will be <test_result>

@daily_test
Examples:
  | test_data             | test_result             |
  | short_daily_test_data | short_daily_test_result |

@weekly_test
Examples:
  | test_data               | test_result               |
  | longer_weekly_test_data | longer_weekly_test_result |

I've noticed two problems with this:

  1. Although the scenario is run twice (once for each item in each table), the data is always taken from the first table. If subsequent tables have more items than the first, then there will be a crash due to an attempt to reference beyond the bounds of the first table.
    I've tried fixing this locally by passing the example table CCIExample* to invocationForScenarioOutline, but problem 2 remains.

  2. It seems that tags for the example tables are ignored. After the fix for problem1, then the above test will execute both daily and weekly tests regardless of the tags supplied.

Incorrect "name" in "Scenario outline"

I noticed that in the JSON output results file, the example's data is included in the Scenario Outline 'name' (as is required in the 'id' field), but in the documentation I found, just the scenario name is used (without examples data):
https://relishapp.com/cucumber/cucumber/docs/formatters/json-output-formatter#scenario-outline
http://rmpestano.github.io/cukedoctor/cucumber/cucumber-documentation.html#_scenario_outline

        "id": "an-outline-feature;outline;examples1;2",
        "name": "outline",
        "keyword": "Scenario Outline",

But with Cucumberish, a Scenario Outline named 'My Scenario' returns this output:

        "id" : "an-outline-feature;my-scenario-example-1",
        "name" : "My Scenario Example 1",
        "keyword" : "Scenario Outline",

Instead of:
"name" : "My Scenario",

Is it an issue or there is an updated JSON output formatter documentation?

PD. thanks for this great tool, the best/¿only? option for making great BDD & ATDD in iOS using Cucumber

*** Assertion failure in -[XCTestContext addUIInterruptionMonitorWithDescription:handler:]

There's a problem with adding UIInterruptionMonitor in iOS UITests using Swift.

Steps created in Cucumberish do not have XCTestContextScope and this causes an assertion.

Here's what I mean:
Before running addUIInterruptionMonitor(withDescription: "") { ...} I have this in my XCTestCase

myappCucumberTests.MyappCommonUITestBase
|>XCFit.XCFit XCFit.XCFit
-|>XCTest.XCTestCase XCTestCase
--|>baseXCTest@0 XCTest
---|>_internalImplementation _XCTestCaseImplementation * 0x6080003609c0
----|>_testContext XCTestContext * 0x608000422460
-----|>_currentScope id 0x0

Note: _currentScope is nil
This causes the assertion:

Currently executing: "And I add interruption"
*** Assertion failure in -[XCTestContext addUIInterruptionMonitorWithDescription:handler:], /Library/Caches/com.apple.xbs/Sources/XCTest_Sim/XCTest-11753/XCTestFramework/Classes/XCTestContext.m:113

When I manually set XCTestContextScope everything works as expected.
Maybe I'm doing something wrong?

Xcode Version 8.2.1 (8C1002)

CucumberishInitializer and CucumberishSwiftInit not recognised in .m file

I was following the wiki regarding the installation using cocoapods. Everytime I tried to run the tests the .m file was giving me errors on line [CucumberishInitializer CucumberishSwiftInit]; I think they didn't recognise CucumberishInitializer and CucumberishSwiftInit. Making them both public fixed it. Maybe you could update the wiki?

import Foundation
import Cucumberish

public class CucumberishInitializer: NSObject {
  public class func CucumberishSwiftInit()
  { ... }
}

Why has "And" been deprecated

Thanks for a cool library. The MatchAll function doesn't recognize when you use an "And" in your feature file. I see that "And" has been deprecated too. This doesn't make sense to me, can you please explain what the thinking behind this is?

scenario outline that launches app cannot be first scenario

bug repro

given the fist scenario is a scenario outline (ie one that has a data table like so):

Feature: view user details
As a user I should be able to see all user details

Scenario Outline: displaying user rows
  Given the app is running
  When A table with rows of type "users" is displayed
  Then the row with text "<name>" shows up at index "<index>"

Examples:
|     name   | index |
|  Abdullah  |      0      |
|  Rima      |      1      |
|  Craig     |      2      |

when the user runs the tests
then
expected the app should launch and tests must be executed as normal
actual the app crashes with this error

fatal error: unexpectedly found nil while unwrapping an Optional value

related

see this issue for more details #11

analysis

see this comment #11 (comment)

Add possibility to use the around hook without tags

Add the possibility to add an around hook around every scenarios no matter what tag they have.

around { (scenario, scenarioExecutionBlock) in
    // executed before the scenario
    scenarioExecutionBlock!()
    // executed after the scenario
}

Release 1.0.4 with regex fixes

Hi,

Can we get develop merged into master and cut a new CocoaPods release? I'm currently pinned to the top of the develop branch in my local environment, but would like to use a tagged release to make CocoaPods run faster.

Thanks!

Where do I add the objc.m file?

I am interested in trying out your cucumber solution. I installed the framework via carthage into a swift test target.

but I am getting the following error:
CucumberishInitilizer.m:16:6: error: use of undeclared identifier 'CucumberishInitializer'; did you mean 'CucumberishInit'?
CucumberishInitilizer.m:16:6: error: bad receiver type 'void (*)()'

I am having trouble getting this up and running after following the installation tutorial

JSON output, Scenario outlines are not counted in Jenkins Cucumber Report

We are using the Jenkins Cucumber Report plugin (https://github.com/jenkinsci/cucumber-reports-plugin) to generate a html report from the Cucumberish JSON output. We use one feature file to execute scenarios on iOS Using Cucumberish and Android using Cucumber-JVM. The feature file contains Scenarios and Scenario Outlines.
I noticed that the number of tests in the iOS results are lower than in the Android results.
It seems that the Cucumber Report does not count scenarios in Scenario Outlines.
A difference was noticed in the json output. Cucumber-JVM logs Scenario Outline results as:
"keyword": "Scenario Outline",
"type": "scenario",

where Cucumberish logs it as:
"keyword" : "Scenario Outline",
"type" : "scenario_outline",

Could it be that the "type" parameter should be set to "scenario" instead of "scenario_outline" in the Cucumberish JSON output?

Incorrect 'id' in Scenario Outline JSON output results

I noticed that in the JSON output results file, the Scenario Outline 'id' field includes the 'EXAMPLES' data separated by a '-' character, but looks like it must be a ';'
https://relishapp.com/cucumber/cucumber/docs/formatters/json-output-formatter#scenario-outline-expanded

       "id": "an-outline-feature;outline;examples1;2",
        "name": "outline",
        "keyword": "Scenario Outline",

But with Cucumberish returns this output:

        "id" : "an-outline-feature;my-scenario-example-1",
        "name" : "My Scenario Example 1",
        "keyword" : "Scenario Outline",

Instead of:
"id" : "an-outline-feature;my-scenario;example-1",

This could be related to issue 80, because the Scenario Outline 'name' field is incorrect (it must be just 'My Scenario', ).
"name" : "My Scenario",

Scenario fails silently if Chinese is used

I have a test case, which contains Chinese for matching label, fails silently. It crashes at

NSTextCheckingResult * match = [[regex matchesInString:step.text options:NSMatchingReportCompletion range:searchRange] firstObject];
.

The reason is searchRange contains incorrect length. With [step.text lengthOfBytesUsingEncoding:NSUTF8StringEncoding], the length of the string in UTF8 encoding is used instead of its 'native' length. For characters other than plain English characters, the length would be two or more times than its native length. Taking 活動 (meaning "activity") as example, their respective unicode information can be found at

And both of their length in UTF-8 is 3 (bytes), which means the incorrect length of 6 will be used instead of correct length of 2 (characters) in current implementation.

I've modified a scenario in Activity.feature:

Scenario: Validating added 活動
    Given it is home screen
    When I tap "活動" button
    Then I should see 4 rows at section 0 in "Activities List" table
    But I should see 2 rows at section 1 in "Activities List" table

which should be able to reproduce the problem.

And I've confirmed that #43 could fix the problem.

Step matching definition that only contains a part of the step's name

I have the following scenario

Scenario: As a publisher, I want to start the SDK with AppID and UserID
     When I start the SDK with an AppID and a UserID
     Then the SDK should start

When the first step is executed it matches the following definition which only partially contains the step's name:

Match(["When", "And"], "I start the SDK with an AppID") { (args, userInfo) in
    //
}

Is that normal or is it a bug? This only way around it at the moment is to write the step definition with ^ and $

Match(["When", "And"], "^I start the SDK with an AppID$") { (args, userInfo) in
    //
}

Contextual "And" handling

Cucumberish currently parses the gherkin literally, so it makes it hard for me to compose tests where a test might be written Given I am in a state or And I am in a state. I will have to duplicate the entry with a Given("I am in a state") and an And("I am in a state") as the And will never find the Given.

Other cucumber implementations understand the context of the And based on the last used token. Is this something that could be easily added?

Carthage support

Any chance of getting support for Carthage dependency manager?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.