Merge pull request #213 from JakeGinnivan/BuildUpdates

Build updates

Author: JakeGinnivan

.gitignore

@@ -18,3 +18,4 @@ TestResult.xml
 TestStack.BDDfy.sln.ide/graph
 _NCrunch_TestStack.BDDfy/
 TestStack.BDDfy.sln.ide/
+src/packages/

TestStack.BDDfy.Samples.nuspec

@@ -0,0 +1,28 @@
+install:
+  - choco install gitversion.portable -pre -y
+  
+platform:
+  - Any CPU
+
+configuration:
+  - Release
+
+assembly_info:
+  patch: false
+
+before_build:
+  - nuget restore src\
+  - ps: gitversion /l console /output buildserver /updateAssemblyInfo
+
+build:
+  project: src\TestStack.BDDfy.sln
+
+after_build:
+  - cmd: nuget pack src\TestStack.BDDfy\TestStack.BDDfy.nuspec -BasePath src\TestStack.BDDfy\ -version "%GitVersion_NuGetVersion%" -prop "configuration=%CONFIGURATION%"
+  - cmd: appveyor PushArtifact "TestStack.BDDfy.%GitVersion_NuGetVersion%.nupkg"
+
+  - cmd: 7z a "TestStack.BDDfy_%GitVersion_NuGetVersion%.zip" -r src\TestStack.BDDfy\bin\%CONFIGURATION%\*.*
+  - cmd: appveyor PushArtifact "TestStack.BDDfy_%GitVersion_NuGetVersion%.zip"
+
+cache:
+  - src\packages -> **\packages.config  # preserve "packages" directory in the root of build folder but will reset it if packages.config is modified

appveyor.yml

@@ -0,0 +1,100 @@
+# Architecture Overview
+## Introduction ##
+This post provides an overview of the main components of the BDDfy architecture to provide some context for the rest of this section and to illustrate the extensibility points.
+
+![BDDfy functional decomposition](/img/BDDfy/Customizing/bddfy-functional-decomposition.png)
+
+The unit of operation in BDDfy is the Story. A Story has metadata (information about the Story) and a collection of Scenarios. Each Scenario represents a test class and contains metadata and a collection of Execution Steps, which are the methods on the test class. There are three types of architectural components in BDDfy: Scanners, Processors and Batch Processors. For each test class BDDfy composes a Story unit with various Scanners and passes it to the Processors in a processor pipeline. Once all of the test classes have been scanned and processed the Batch Processors run aggregate operations against all of the Stories.
+
+Scanners turn a call to BDDfy (from a method) into a Scenario which could potentially be related to a Story. BDDfy doesn't need Stories but if there is one it uses it. If a Scenario is not related to a Story then it is associated with a dummy placeholder. Each Story is then passed to the Processors, which perform various operations, including executing the tests, and populate the Stories, Scenarios and Steps with the test execution results. Once all of the tests have been scanned and processed, the Batch Processors take the collection of Stories and process their results. This could be any sort of aggregate operation, but currently all the batch processors are reports.
+
+## Scanners ##
+Most of the BDDfy in Action series so far has covered the various Scanners, so I won’t go into much detail here. Suffice to say, BDDfy uses Scanners to scan each test class to find all of the methods on it and turn the test class into a Scenario. The different Scanners are shown here:
+
+![BDDfy scanners](/img/BDDfy/Customizing/bddfy-scanners.png)
+
+### Story Scanner ###
+BDDfy creates a Story Scanner for each test object. This is the Scanner that actually scans the test object and turns it into a Story. It composes together the Story Metadata Scanner and the appropriate Scenario Scanner – [Fluent](/BDDfy/Usage/FluentAPI.html) or [Reflective](/BDDfy/Usage/MethodNameConventions.html).
+
+The Story Metadata Scanner gets information from the Story attribute, if one exists on the class.
+
+A **Story** has the following properties:
+
+- **Story Metadata**: Information about the Story such as Title, As a, I want, So that
+- **Scenarios**: The collection of Scenarios related to the Story
+- **Result**: A Story’s Result is a Step Execution Result and is determined by the highest Step Execution Result of its Scenarios.
+- **Category**: The Story Category
+
+**Step Execution Results** have a numerical hierarchy and can be (in ascending order):
+
+- Not Executed (0)
+- Passed (1)
+- Not Implemented (2)
+- Inconclusive (3)
+- Failed (4)
+
+The Test Runner Processor assigns a numerical Step Execution Result to every Execution Step. The result of a Scenario is then determined by the highest value of from its Steps and the result of a Story is determined by the highest result of its Scenarios. For example, if a Step fails, then its parent Scenario and Story will also have a result of Failed.
+
+### Scenario Scanners ###
+Scenario Scanners scan the test class and use the information they find to create a Scenario. There is a Fluent Scenario Scanner and a Reflective Scenario Scanner.
+
+A **Scenario** has the following properties:
+
+- **Title**: The Scenario Title
+- **Steps**: The collection of Steps (test class methods) related to the Scenario
+- **Result**:	A Scenario’s Result is a Step Execution Result and is determined by the highest Step Execution Result of its Steps.
+- **Duration**: How long the Scenario took to execute. Used by Diagnostics.
+
+### Step Scanners ###
+Step Scanners turn methods into Execution Steps. The Reflective Scanners (the Executable Attribute Step Scanner and the Method Name Scanner) scan the test class to find all the methods on it and turns them into Execution Steps. The Fluent Step Scanner is only a registry and in practice doesn't do any scanning.
+
+An **Execution Step** has the following properties:
+
+- **Title**: The Step Title
+- **Result**: The result of executing the Step.
+- **Duration**: How long the Step took to execute. Used by Diagnostics.
+- **Asserts**: Whether or not the Step is an Assertion Step.
+- **Should Report**: Whether the Step should be displayed in reports.
+- **Execution Order**: The order that the step should run in relative to the other steps. Can be (in ascending order)
+   - Initialize (for example, "Context", "Setup)
+   - Setup State ("Given")
+   - Consecutive Setup State ("And Given")
+   - Transition ("When)
+   - Consecutive Transition "And When")
+   - Assertion ("Then")
+   - Consecutive Assertion ("And Then")
+   - Tear Down ("TearDown")
+ 
+## Processors ##
+Once a test class has been scanned into a Story, the Story is passed into a **Processor pipeline** where a series of processing steps are performed on it. The Processors are categorized by Type and the order they run in is determined by this Type.
+
+The various **Process Types**, in order, are:
+
+1. Firstly
+1. Execute
+1. Before Report
+1. Report
+1. After Report
+1. Process Exceptions
+1. Finally
+
+![BDDfy processor pipeline](/img/BDDfy/Customizing/bddfy-processor-pipeline.png)
+
+
+1. **Test Runner (Execute)**: Executes the tests.
+1. **Console Reporter (Report)**: Displays the test result in the console.
+1. **Exception Processor (Process Exceptions)**:	Handles exceptions.
+1. **Story Cache (Finally)**: Saves each Story for later processing by the Batch Processors.
+1. **Disposer	(Finally)**: Cleans up the Story and its Scenarios.
+
+## Batch Processors ##
+Once all of the tests have been scanned and processed, the Batch Processors take all of the Stories and process their results (technically speaking they run in the [AppDomain DomainUnload event](http://msdn.microsoft.com/en-GB/library/system.appdomain.domainunload.aspx)). This could be any sort of result processing, but currently all the batch processors are reports. The built-in Batch Processors are displayed in the diagram below.
+
+![BDDfy batch processor pipeline](/img/BDDfy/Customizing/bddfy-batch-processor-pipeline.png)
+
+- **HTML Reporter**: Creates the HTML report
+- **Markdown Reporter**: Creates the Markdown report
+- **Diagnostics Reporter**: Creates the Diagnostics report
+
+## Configurator ##
+The static Configurator class allows you to configure Scanners, Processors and Batch Processors. It lets you enable,  disable, or replace individual components and it also allows you to add custom implementations.

docs/Customizing/ArchitectureOverview.md

@@ -0,0 +1,175 @@
+# Customizing Reports
+This post looks at how to customize the HTML Report and how to create your own custom reports. You can customize the HTML report via configuration or by applying custom CSS or JavaScript. You can create custom reports either by implementing a new Processor or Batch Processor and adding them into their respective pipelines (see the [Architecture Overview](/BDDfy/Customizing/ArchitectureOverview.html) for details about the differences between these).
+
+## Customizing the HTML Report ##
+The HTML report is the most sophisticated report in [BDDfy](http://teststack.github.io/pages/BDDfy.html) and therefore provides a lot more things that you can configure. Its configuration is defined by the IHtmlReportConfiguration interface.
+
+	public interface IHtmlReportConfiguration
+	{
+	    string ReportHeader { get; }
+	    string ReportDescription { get; }
+	    string OutputPath { get; }
+	    string OutputFileName { get; }
+	    bool RunsOn(Story story);
+	}
+
+You can create a new configuration by implementing that interface or you can inherit from the  DefaultHtmlReportConfiguration class, used to configure the standard HTML Report, and just override specific properties.  Here is an example of a custom configuration, taken from the ATM sample, available on github [here](https://github.com/TestStack/TestStack.BDDfy/tree/master/TestStack.BDDfy.Samples/Atm).
+
+    public class HtmlReportConfig : DefaultHtmlReportConfiguration
+    {
+        public override bool RunsOn(Core.Story story)
+        {
+            return story.MetaData.Type.Namespace != null && story.MetaData.Type.Namespace.EndsWith("Atm");
+        }
+
+        /// <summary>
+        /// Change the output file name
+        /// </summary>
+        public override string OutputFileName
+        {
+            get
+            {
+                return "ATM.html";
+            }
+        }
+
+        /// <summary>
+        /// Change the report header to your project
+        /// </summary>
+        public override string ReportHeader
+        {
+            get
+            {
+                return "ATM Solutions";
+            }
+        }
+
+        /// <summary>
+        /// Change the report description
+        /// </summary>
+        public override string ReportDescription
+        {
+            get
+            {
+                return "A reliable solution for your offline banking needs";
+            }
+        }
+    }
+
+which produces the following customised report, which you will find in your bin directory named ATM.html:
+
+![BDDfy functional decomposition](/img/BDDfy/Customizing/bddfy-sample-atm-html-custom.png)
+
+
+The HTML report is a [Batch Processor](/BDDfy/Customizing/ArchitectureOverview.html) and is implemented by the HtmlReporter class. To plug the new report into BDDfy you need to create a new HtmlReporter and pass the custom configuration into its constructor. As I explained in the [Reports post](/BDDfy/Usage/Reports.html), the place to apply that configuration to BDDfy is the Configurator class, which is called before the tests run.
+
+	Configurator.BatchProcessors.Add(new HtmlReporter(new HtmlReportConfig()));
+
+The use of the Add method means this is adding a second HTML Report processor into the Batch Processor pipeline, so the default report runner will still run. If you actually want the new report to replace the default report, then you will also need to disable the default report.
+
+	Configurator.BatchProcessors.HtmlReport.Disable();
+
+## Custom CSS and JavaScript ##
+You can customize a lot more about the HTML report. BDDfy uses the BDDfy.css file to style the report and BDDfy.js and jQuery to add interactivity to it. You will find these files in the bin directory alongside the HTML report. You can customise the styles by adding a bddifyCustom.css class and the behaviour by adding a bddifyCustom.js file. These files also need to be in the same directory as the HTML report file. This will affect all the reports in the project.
+
+
+## Create a custom report by creating a new Processor  ##
+One way to create a custom report is to implement a new Processor and plug it into the Processor pipeline. You just have to implement the one Process() method and set the Process Type to Report.
+
+An example of doing this is provided in the BDDfy Tic Tac Toe sample project with the Custom Text Reporter. The sample is available on [github](https://github.com/TestStack/TestStack.BDDfy/tree/master/TestStack.BDDfy.Samples) or [nuget](http://nuget.org/packages/TestStack.BDDfy.Samples/).
+
+    /// <summary>
+    /// This is a custom reporter that shows you how easily you can create a custom report.
+    /// Just implement IProcessor and you are done
+    /// </summary>
+    public class CustomTextReporter : IProcessor
+    {
+        private static readonly string Path;
+
+        private static string OutputDirectory
+        {
+            get
+            {
+                string codeBase = typeof(CustomTextReporter).Assembly.CodeBase;
+                var uri = new UriBuilder(codeBase);
+                string path = Uri.UnescapeDataString(uri.Path);
+                return System.IO.Path.GetDirectoryName(path);
+            }
+        }
+
+        static CustomTextReporter()
+        {
+            Path = System.IO.Path.Combine(OutputDirectory, "BDDfy-text-report.txt");
+
+            if(File.Exists(Path))
+                File.Delete(Path);
+
+            var header =
+                " A custom report created from your test assembly with no required configuration " +
+                Environment.NewLine +
+                Environment.NewLine +
+                Environment.NewLine +
+                Environment.NewLine;
+            File.AppendAllText(Path, header);
+        }
+
+        public void Process(Story story)
+        {
+            // use this report only for tic tac toe stories
+            if (!story.MetaData.Type.Name.Contains("TicTacToe"))
+                return;
+
+            var scenario = story.Scenarios.First();
+            var scenarioReport = new StringBuilder();
+            scenarioReport.AppendLine(string.Format(" SCENARIO: {0}  ", scenario.Title));
+
+            if (scenario.Result != StepExecutionResult.Passed && scenario.Steps.Any(s => s.Exception != null))
+            {
+                scenarioReport.Append(string.Format("    {0} : ", scenario.Result));
+                scenarioReport.AppendLine(scenario.Steps.First(s => s.Result == scenario.Result).Exception.Message);
+            }
+
+            scenarioReport.AppendLine();
+
+            foreach (var step in scenario.Steps)
+                scenarioReport.AppendLine(string.Format("   [{1}] {0}", step.StepTitle, step.Result));
+
+            scenarioReport.AppendLine("--------------------------------------------------------------------------------");
+            scenarioReport.AppendLine();
+
+            File.AppendAllText(Path, scenarioReport.ToString());
+        }
+
+        public ProcessType ProcessType
+        {
+            get { return ProcessType.Report; }
+        }
+    }
+
+This produces the BDDfy-text-report.txt text file report which is output to the bin directory:
+
+![BDDfy custom text report](/img/BDDfy/Customizing/bddfy-custom-text-report.png)
+
+## Create a custom report by creating a new Batch Processor  ##
+While that is one way that you can create a custom report, probably the better way to do it is to implement a new Batch Processor. The Processor runs as each test is being executed and allows you to build up the report, whereas a Batch Processor has the advantage of running after all of the tests have finished, meaning that you have access to total and summary information, such as diagnostics.
+
+As an example of creating a custom report by creating a new Batch Processor, I was recently messing around with running tests in parallel and, as you might expect, the normal console report was quite jumbled. Multiple console report Processors were writing to the console at the same time and different test results were overlapping. The solution was to run the console report after all of the tests had run by creating a new Console Reporter as a Batch Processor rather than a Processor.
+
+    public class MyConsoleReporter : IBatchProcessor
+    {
+        public void Process(IEnumerable<Story> stories)
+        {
+            var reporter = new ConsoleReporter();
+            stories
+                .ToList()
+                .ForEach(story => reporter.Process(story));
+        }
+    }
+
+Then I just needed to add it to the Batch Processor pipeline and disable the built-in console report:
+
+    Configurator.Processors.ConsoleReport.Disable();
+    Configurator.BatchProcessors.Add(new MyConsoleReporter());
+
+
+That's a bit of a hack for demo purposes. The HTML, MarkDown, and Diagnostics reports are all implemented as Batch Processors and I would recommend checking them out for examples of how to create a new report.