Archive

Archive for April, 2014

Test Execution Report in SpecFlow

April 27, 2014 6 comments

It would be very good to generate test reports related to Acceptance tests for satisfying our requirements and providing the test statuses to Customers & Project Managers. This report provides a formatted HTML report of a test execution. The report contains a summary about the executed tests and the result and also a detailed report for the individual scenario executions.

NUnit Test Execution Report

Let me create a Test Execution Report for my previous blog on SpecFlow. In order to generate this report you have to execute the acceptance tests with the nunit-console runner. This tool generates an XML summary about the test executions. To have the detailed scenario execution traces visible, you also need to capture the test output using the /out and the /labels options as it can be seen in the following example.

Step 1:

nunit-console.exe /labels /out=TestResult.txt /xml=TestResult.xml bin\Debug\SpecflowTest.dll

The two generated file can be used to invoke the SpecFlow report generation. If you use the output file names shown above it is enough to specify the project file path containing the feature files.

Step 2:

specflow.exe nunitexecutionreport SpecflowTest.csproj /out:MyResult.html

Step 1 is already executed as we are running the tests using the NUnit GUI and this automatically generates a TestResult.xml file in the bin\Debug folder as shown in the snapshot below:

 

Now, let me jump to the Step 2. Step 2 can be executed by using the External tools in the Visual Studio.

  • Go to Tools -> External Tools
  • Click on the ‘Add’ button in the ‘External Tools’ window
  • Input a Title (say in my case ‘SpecFlow Test Result’)
  • Input the location of SpecFlow application in the ‘Command’ field (say in my case ‘C:\Program Files (x86)\TechTalk\SpecFlow\specflow.exe’)
  • Input the argument in the ‘Arguments’ field (say in my case ‘nunitexecutionreport SpecflowTest.csproj /xmlTestResult:bin\debug\TestResult.xml’)
  • Input the directory info in ‘Initial directory’ field (say in my case ‘$(ProjectDir)’)
  • Set the ‘Close on exit’ checkbox and hit the ‘Apply’ & ‘OK’ buttons. A new option ‘SpecFlow Test Result’ under the ‘Tools’ menu.
 
Generating the Test Report
  • Debug the project and run the test using NUnit GUI window.
  • Click on the option ‘SpecFlow Test Result’ under the ‘Tools’ menu.
  • Open the ‘TestResult.html’ file present in the ‘Projects’ folder using the browser as in the snapshot below.
  • Create a short-cut to the ‘TestResult.html’ file by adding the file to the Visual Studio Solution Explorer for quick viewing as in the snapshot below.
 

Behaviour Driven Development (BDD) in .Net using SpecFlow, Selenium and NUnit

April 19, 2014 20 comments

I have been working on the Behaviour Driven Development (BDD) using SpecFlow for .Net and I felt it is fun. According to me, it is a Tester’s tool rather than a Developer’s tool and I felt it is worth sharing. Kindly refer the wiki for more theoretical info on BDD: http://en.wikipedia.org/wiki/Behavior_Driven_Development

SOFTWARE REQUIREMENTS:

  • Windows 7 as OS
  • Microsoft Visual Studio 2010 as IDE
  • SpecFlow as BDD tool for .Net
  • NUnit as Unit Testing Tool
  • Selenium as a Test Automation Tool

SET-UP INSTRUCTIONS:

To install the Selenium WebDriver project:
  • Create a new Project with C# Class Library and save it (‘SpecflowTest‘ in my example).
  • Download and install NuGet Package Manager using Tools -> Extension Manager. Restart MS Visual Studio in order for the changes to take effect.
  • Go to Package Manager Console install the latest version of Selenium WebDriver by running the command Install-Package Selenium.WebDriver -Version 2.20.0
  • Install the latest version of Selenium WebDriver Support Classes by running the command Install-Package Selenium.Support
 
Referencing the NUnit Library:
  • Download the latest NUnit framework from the site: http://www.nunit.org
  • Install the NUnit software on your machine.
  • In Visual Studio, Go to the Project -> Add Reference menu item.
  • When the Add Reference dialog appears, click on ‘Browse‘ and navigate to C:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\framework and select nunit.framework.dll.
Referencing the SpecFlow:
  • Download the latest NUnit framework from the site: http://www.specflow.org/
  • Install the SpecFlow software on your machine.
  • In Visual Studio, Go to the Project -> Add Reference menu item.
  • When the Add Reference dialog appears, click on ‘Browse‘ and navigate to C:\Program Files (x86)\TechTalk\SpecFlow and select TechTalk.SpecFlow.dll.
  • Finally, the solution Explorer should look like this:

Consider a sample user story:

As an end user,
I would like to visit the google search page 
And then I would like to search an item so that
I can view the search results

Let me write a test scenario for the above story:

Given I am on the Google home page
When I search for text selenium
Then I should see the search results

Creating a new SpecFlow feature file:
  • Context-click on the project name ‘SpecflowTest‘ in the Solution Explorer
  • Select Add -> New Item
  • Select the ‘SpecFlow Feature File‘ and save it as ‘GoogleSearch.feature
 
 
 
 
 
  •  Specflow creates a new file “GoogleSearch.feature” and a designer file “GoogleSearch.feature.cs“. The default content of the Specflow file is in the Gherkin format.
  • The default content of the file is as shown is below:
Feature: Addition
 In order to avoid silly mistakes
 As a math idiot
 I want to be told the sum of two numbers

@mytag
Scenario: Add two numbers
 Given I have entered 50 into the calculator
 And I have entered 70 into the calculator
 When I press add
 Then the result should be 120 on the screen
  • Change the contents of the file to the new scenario as below and the Specflow generates a feature file ‘GoogleSearch.feature.cs’:
Feature: Google Search
 As an end user,
 I would like to visit the google search page
 And then I would like to search an item so that
 I can view the search results

@mytag
Scenario: Google Search
 Given I am on the Google home page
 When I search for text selenium
 Then I should see the search results

Creating a new SpecFlow Step Definition file:
  • Rename the class ‘Class1.cs‘ to ‘GoogleSearchStepDefinition.cs‘.
  • Context-click on the feature file ‘GoogleSearch.feature‘ and select …… System generates a specification file ‘…..’ as shown below.
  • Debug the project. Go to Projects -> Project Properties (‘SpecflowTest Properties‘ in my example).
  • Click on the ‘Debug‘ tab and set the ‘Start external program‘ to the location of NUnit exe file (C:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\nunit-x86.exe).
  • Build the solution, Go to ‘Build -> Build Solution‘ (hit the F6 key) in Visual Studio.
  • Execute the test, Go to ‘Debug -> Start Debugging‘ (hit the F5 key) in Visual Studio. Visual Studio invokes the NUnit application.
  • In NUnit, click on ‘File -> Open Project‘ and choose the location of the SpecflowTest.dll file.
  • In NUnit, click on the ‘Run‘ button to run the tests. Test shows Inconclusive result and the result is displayed in NUnit ‘Text Output’ tab.
  • Copy the contents of the NUnit  ‘Text Output‘ tab and paste it in the ‘GoogleSearchStepDefinition.cs‘ step definition file as shown below. The key point here is that the class needs to be annotated with the binding attribute and we can use the (.*) expression to represent variables coming from the feature file.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using TechTalk.SpecFlow;
namespace SpecflowTest
{
 [Binding]
 class GoogleSearchStepDefinition
  {

  [Given(@"I am on the Google home page")]
  public void GivenIAmOnTheGoogleHomePage()
  {
  ScenarioContext.Current.Pending();
  }

  [When(@"I search for text (.*)")]
  public void WhenISearchForATextSelenium()
  {
  ScenarioContext.Current.Pending();
  }

  [Then(@"I should see the search results")]
  public void ThenIShouldSeeTheSearchResultsForSelenium()
  {
  ScenarioContext.Current.Pending();
  }
}
}
 
Creating a Static Browser class for defining the browser selection:
  • This class is a static helper class for selecting the browser and Selenium has support for Internet Explorer, Firefox and Chrome.
  • Context-click on the project name ‘SpecflowTest‘ in the Solution Explorer and select Add -> New Item
  • Create a C# file with name ‘WebBrowser.cs‘.
  • Now, copy and paste the code below to the ‘WebBrowser.cs‘.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using TechTalk.SpecFlow;
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Chrome;
namespace SpecflowTest
{
   [Binding]
   public class WebBrowser
   {
     public static IWebDriver Current
     {
       get
       {
          if (!ScenarioContext.Current.ContainsKey("browser"))
          {
          //Select IE browser
          ScenarioContext.Current["browser"] = new InternetExplorerDriver();

          //Select Firefox browser
          //ScenarioContext.Current["browser"] = new FirefoxDriver();

          //Select Chrome browser
          //ScenarioContext.Current["browser"] = new ChromeDriver();
          }
        return (IWebDriver)ScenarioContext.Current["browser"];
        }
      }

    [AfterScenario]
    public static void Close()
    {
     if (ScenarioContext.Current.ContainsKey("browser"))
      {
          Current.Dispose();
      }
    }
  }
}

Implementing the Steps:
  • Remove all the methods ‘ScenarioContext.Current.Pending()
  • Now, copy and paste the code below to the ‘GoogleSearchStepDefinition.cs‘ as shown below.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using TechTalk.SpecFlow;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Support.UI;
namespace SpecflowTest
{
 [Binding]
 public class StepDefinition1
 {

 [Given(@"I am on the Google home page")]
 public void GivenIAmOnTheGoogleHomePage()
 {
 //Navigate to the site
 WebBrowser.Current.Navigate().GoToUrl("http://www.google.com");
 //Check that the Title is what we are expecting
 Assert.AreEqual("Google", WebBrowser.Current.Title);
 }

 [When(@"I search for text (.*)")]
 public void WhenISearchForATextSelenium(string keyword)
 {
 // Find the text input element by its name
 IWebElement query = WebBrowser.Current.FindElement(By.Name("q"));
 // Input the search text
 query.SendKeys(keyword);
 // Now submit the form
 query.Submit();

  // Google's search is rendered dynamically with JavaScript.
  // Wait for the page to load, timeout after 5 seconds
  WebDriverWait wait = new WebDriverWait(WebBrowser.Current, TimeSpan.FromSeconds(5));
  IWebElement title = wait.Until<IWebElement>((d) =>
  {
  return d.FindElement(By.ClassName("ab_button"));
  });
 }

 [Then(@"I should see the search results")]
 public void ThenIShouldSeeTheSearchResultsForSelenium()
 {
 //Check that the Title is what we are expecting
 Assert.AreEqual("selenium - Google Search", WebBrowser.Current.Title);
 }

}
}
 
Executing the Tests:
  • Build the solution, Go to ‘Build -> Build Solution‘ (hit the F6 key) in Visual Studio.
  • Execute the test, Go to ‘Debug -> Start Debugging‘ (hit the F5 key) in Visual Studio. Visual Studio invokes the NUnit application.
  • In NUnit, click on ‘File -> Open Project‘ and choose the location of the SpecflowTest.dll file.
  • In NUnit, click on the ‘Run‘ button to run the tests. Test is executed using NUnit and the result will be displayed in NUnit GUI window as shown in the snapshot below.
%d bloggers like this: