C plus plus Behaviour Driven Development tools
Ranked #963 in Internet, #51,604 overall
Behaviour Driven Development in C plus plus
What is behaviour driven development? It is a method for developing software where the behaviour of a bit of code, or an app, is described first, in a formal but as plain english way as possible. This description, or specification actually forms an executable set of tests against the software, so initially, they would fail, but as the software is built, these will start to pass. If the software needs to be refactored, then it would be assumed that these specifications of behaviour should still pass. If a new team member needs to understand what some code should do, they need only look at this specification to see. Then for them to know how much of this it does do, they can run this specification and see the the result.
BDD (the common acronym used) has evolved from a few disciplines in software practice. Those include the unit testing, evolving into TDD (Test Driven Development), and the automation of many tests. It comes from bringing testers, and product designers back in close to development teams and encouraging these team members to start speaking the same language, by keeping the language as plain as possible. In terms of automation, BDD frameworks fit into CI frameworks so the set of specifications can be run on code on a regular basis, a team can then see feedback on their progress, and quickly be aware of any regressions. For a developer, running the specs on their own desk gives them a quick scaffold to build their code in. For a developer to start thinking and writing specs, in the plain English and using the right words - like "This thing should say hello", it both closes communication gaps between them and the rest of the team, and means that the code itself starts to be more readable and maintainable.
This article is not intended to evangelise BDD, TDD or testing in general. It is intended so I can share my experiences on them with C (I've used them plenty with Ruby RSpec and Jasmine for JS, and built an AS2 BDD framework), where the frameworks to facilitate this are a little less well developed. It is also intended as a call to action for those with interests in this realm, and perhaps for developers of some of the solutions to combine efforts for a really good result. It is a mention of where they could improve, and includes tutorials for developers to get in there and actually try building examples with them so they can start being used.
BDD (the common acronym used) has evolved from a few disciplines in software practice. Those include the unit testing, evolving into TDD (Test Driven Development), and the automation of many tests. It comes from bringing testers, and product designers back in close to development teams and encouraging these team members to start speaking the same language, by keeping the language as plain as possible. In terms of automation, BDD frameworks fit into CI frameworks so the set of specifications can be run on code on a regular basis, a team can then see feedback on their progress, and quickly be aware of any regressions. For a developer, running the specs on their own desk gives them a quick scaffold to build their code in. For a developer to start thinking and writing specs, in the plain English and using the right words - like "This thing should say hello", it both closes communication gaps between them and the rest of the team, and means that the code itself starts to be more readable and maintainable.
This article is not intended to evangelise BDD, TDD or testing in general. It is intended so I can share my experiences on them with C (I've used them plenty with Ruby RSpec and Jasmine for JS, and built an AS2 BDD framework), where the frameworks to facilitate this are a little less well developed. It is also intended as a call to action for those with interests in this realm, and perhaps for developers of some of the solutions to combine efforts for a really good result. It is a mention of where they could improve, and includes tutorials for developers to get in there and actually try building examples with them so they can start being used.
Contents
- Getting the Words Right
- Current C++ Books
- Actual tools for C++ BDD and Specification
- Screencast -- using Igloo in XCode
- Hold on - Whats a DSL?
- Documentation for Igloo and CppSpec
- Igloo, meet CppSpec, and CppSpec meet Igloo
- Extending Igloo to expect exceptions
- Mocking or Faking
- The other BDD for C++
- My other lenses on coding
- Give me a thumbs up for testing in C++
- What testing do you use for your programming?
- Share these C++ programming tools!
- About Danny Staple
Getting the Words Right
So here is an example describing a bit of code using Jasmine - a very well developed and supported BDD framework for Javascript.
require 'calendar'
describe Calendar, "finding first gap" do
include CalendarHelperMethods
it "Should find the the first gap in the calendar longer than 30 minutes" do
cal = Calendar.new()
addTestEvents(cal)
cal.firstGap(:withSize=>30.minutes).should == gapList[0].startTime
end
end
This is a (very!) abridged snippet of a set of specs, which I will at some point add for download, along with some code that they run with for readers to look at, although the focus here is on the C++.
Quickly describing what this does, it is a specification to describe how a user of a calendar would find the first gap, and what this function should do. Somebody wants to find a gap in their calendar of 30 minutes or more.
It includes a set of helper methods, to cut down on the boilerplate needed for testing and leave the developer less to repeat. In the one specification this description holds, a calendar object is instantiated (Ruby people will spot that this double bracket is superfluous and not needed). A helper event "addTestEvents" sets up a given and known set of events, and the same helper provides a gapList, a list of the gaps, by start time and duration.
The actual expectation here is pretty clear - when firstGap has been called, with a parameter specifying that the gap should have a size of 30 minutes, it should return a start time that is equal to the first listed gap - since the helper provided a set of events and gaps.
In other words,
Given a known set of events, The Calendars First Gap with size of 30 minutes should return a start time equal to the start time of the first gap in the known list
Here, the ruby syntax adds things like ".should" instead of simply " should", and should equal becomes '.should ==' (there is an equal but it compares object identities instead of content, not useful for our purposes).
Current C++ Books
Actual tools for C++ BDD and Specification
There is one heavyweight tool that can do C++ testing, FitNesse. I've not spent time on it here, as getting some lightweight tools on the go and well developed will hopefully give faster gains to somebody looking to employ these methods. FitNesse requires a bit more investment in infrastructure, time and set up than the other tools. Fitness also requires a bit more in the way of code preparation than the other frameworks.
Next up is CppSpec. I will get plenty of detail in later, but in short it is a relatively lightweight BDD tool for C++ that is based on the Boost library and needs to be downloaded as source then built using Cmake. It is a project that is very much a live with quite a responsive developer on it. There are plenty of features for matchers and expectations, however, as we will see, some of the boilerplate needed to get it moving could do with reduction.
Also great is Igloo. This is a purely header based library, with no other dependancies, making it somewhat easier to fit into a project than CppSpec. It has some nice features around keeping spec declarations simple, however, it lacks one or two features in its matchers.
I see some similar tools, different efforts to build a decent Cplusplus spec tool. I've been looking at these two, and see they both have some real strengths, and a couple of weaknesses, but I think pooling resources could really get something great.
Next up is CppSpec. I will get plenty of detail in later, but in short it is a relatively lightweight BDD tool for C++ that is based on the Boost library and needs to be downloaded as source then built using Cmake. It is a project that is very much a live with quite a responsive developer on it. There are plenty of features for matchers and expectations, however, as we will see, some of the boilerplate needed to get it moving could do with reduction.
Also great is Igloo. This is a purely header based library, with no other dependancies, making it somewhat easier to fit into a project than CppSpec. It has some nice features around keeping spec declarations simple, however, it lacks one or two features in its matchers.
I see some similar tools, different efforts to build a decent Cplusplus spec tool. I've been looking at these two, and see they both have some real strengths, and a couple of weaknesses, but I think pooling resources could really get something great.
Screencast -- using Igloo in XCode
I've created this screencast, showing how Igloo can be used to test things in C++ and using the XCode environment. As Igloo is a header only setup, it is not difficult to integrate into any tool, Visual Studio, Makefiles, CMake, Boost Build (BJam) etc...
powered by Youtube
Hold on - Whats a DSL?
A DSL is a Domain Specific Language. It is a way of using more plain English, or at least Jargon specific to a domain, to program or script over perhaps more complicated or less plain English native constructs. Constructing one well will lead to more self documenting and easy to understand code, where the intent is stated clearly and not well hidden in syntax and subscripts. In the case of BDD consider these examples:
The second one is longer, but definitely states exactly what was intended here. It certainly means that a developer can read the code and should be able to gather what is going on. Another example:
In the first case here, the 1800 could be replaced with 60 * 30 perhaps, but either way it is still unclear. The second example is again stating its intent plainly.
A DSL can be brought into a development environment in one of two ways. The first type is an "External DSL", based on building a parser with its own lexicon, grammar and interpreter. This would mean getting to know things like Lexx and Yacc, and divorces that DSL from the power of a fully fledged language. Word to the wise, if you've done this before, you'll know that being able to import a well known interpreter like Python or Ruby will get a lot more done than trying to build that from scratch.
The alternative way, and IMHO better, is an internal DSL, building it into the language you are using. This is ideal for test, as you can use constructs that belong to your language thus being in familiar territory. It picks up more of your languages natural syntactic noise, and may mean some interesting problems to be solve in libraries to make that grammar work, but the results can be impressive. Both igloo and CppSpec are using an internal DSL style to get their results.
More information on DSL's
Test(a,2);
Expect.that(a, Equals(2));
The second one is longer, but definitely states exactly what was intended here. It certainly means that a developer can read the code and should be able to gather what is going on. Another example:
Test(duration, 1800);
Expect.that(duration, Equals(60, minutes));
In the first case here, the 1800 could be replaced with 60 * 30 perhaps, but either way it is still unclear. The second example is again stating its intent plainly.
A DSL can be brought into a development environment in one of two ways. The first type is an "External DSL", based on building a parser with its own lexicon, grammar and interpreter. This would mean getting to know things like Lexx and Yacc, and divorces that DSL from the power of a fully fledged language. Word to the wise, if you've done this before, you'll know that being able to import a well known interpreter like Python or Ruby will get a lot more done than trying to build that from scratch.
The alternative way, and IMHO better, is an internal DSL, building it into the language you are using. This is ideal for test, as you can use constructs that belong to your language thus being in familiar territory. It picks up more of your languages natural syntactic noise, and may mean some interesting problems to be solve in libraries to make that grammar work, but the results can be impressive. Both igloo and CppSpec are using an internal DSL style to get their results.
More information on DSL's
Documentation for Igloo and CppSpec
The two frameworks mentioned, Igloo and CppSpec are both early-ish projects. Their documentation, and library maturity is currently in early stages, and they need some community usage and support to really get them going. Getting them into more developers projects, running in hudson and some of their pain felt and boilerplate smashed further into simpler reusable DSL macro's will serve to improve them. I also recommend their users get to know either a little ruby and rspec, or a little JS and jasmine - because this gives an idea of how pleasing a BDD DSL can be.
I am also cooking up screencasts for some example projects to get people rolling with either.
Documentation Resources for CppSpec:
CppSpec Examples
Documentation For Igloo
Creating a Test Application
I am also cooking up screencasts for some example projects to get people rolling with either.
Documentation Resources for CppSpec:
CppSpec Examples
Documentation For Igloo
Creating a Test Application
Igloo, meet CppSpec, and CppSpec meet Igloo
Why they should perhaps link up, combine forces and code for something truly awesome.
CppSpec and Igloo look like they are aiming at a very similar goal, while FitNesse is in a league of its own. These lightweight tools however have real potential to bounce ideas from each other.
As I have mentioned, a great model (IMHO) for BDD tools is probably rspec - it kind of flows nicely. A DSL in C++ may not be quite as cute as that, but it can be not far off. Another inspiration for great spec tools is Jasmine, from the Javascript world - which is getting closer to what a C++ spec may look like.
Igloo and CppSpec are the closest I am seeing to rspec and Jasmine. But here's where joined efforts could really close this gap beautifully.
Igloo has as its first strength being a header only library - which is pretty handy, while CppSpec requires boost and compilation with CMake. Perhaps some greater automation of getting this running would be nice, although the support from the developer was second to none.
Igloo has got the simplest DSL for actually declaring Specs. I would probably map Context to "Describe" and "Spec" to "It" to bring it closer to the rspec/Jasmine world. CppSpec currently needs a lot of boilerplate to build specs and fixtures, with template instantiation and then registration functions being typed out in each spec file.
They have relatively like for like richness in their matcher DSL - comparing them:
CppSpec:
Igloo:
Assert::That(myValue, Is().EqualTo("Hello"));
Declaring a spec in with Igloo:
Declaring the same spec in CppSpec:
However, where CppSpec really shines is it is able to run/invoke a function, and expect exceptions, something missing from Igloo (unless I just haven't found it yet). CppSpec has more of an extendable matcher system compared with igloo.
Taken straight from the CppSpec examples:
specify(invoking(&Stack<int>::pop).should.raise.exception<std::exception>());
Igloo also does not catch exceptions in its specs, letting them bubble up to the top, it really could be nicer if it was to capture them, and specify which spec they were thrown in.
If the reduced boilerplate around declaring specs and fluency from igloo, could be combine with the exception matching of CppSpec, then this may be a great step towards having very easy to use Cpp tools for BDD.
Packages, or simpler installers for CppSpec would make it easier to put in a C++ project (ah - were there something like Maven for a C++ project and its dependancies).
As I have mentioned, a great model (IMHO) for BDD tools is probably rspec - it kind of flows nicely. A DSL in C++ may not be quite as cute as that, but it can be not far off. Another inspiration for great spec tools is Jasmine, from the Javascript world - which is getting closer to what a C++ spec may look like.
Igloo and CppSpec are the closest I am seeing to rspec and Jasmine. But here's where joined efforts could really close this gap beautifully.
Igloo has as its first strength being a header only library - which is pretty handy, while CppSpec requires boost and compilation with CMake. Perhaps some greater automation of getting this running would be nice, although the support from the developer was second to none.
Igloo has got the simplest DSL for actually declaring Specs. I would probably map Context to "Describe" and "Spec" to "It" to bring it closer to the rspec/Jasmine world. CppSpec currently needs a lot of boilerplate to build specs and fixtures, with template instantiation and then registration functions being typed out in each spec file.
They have relatively like for like richness in their matcher DSL - comparing them:
CppSpec:
specify(myValue, should.equal("Hello"));
Igloo:
Assert::That(myValue, Is().EqualTo("Hello"));
Declaring a spec in with Igloo:
Context(describeHello) {
Spec(getGreetingShouldReturnHello) {
Hello hello;
Assert::That(hello.getGreeting(), Equals("Hello"));
}
};
Declaring the same spec in CppSpec:
class DescribeHello : public CppSpec::Specification<Hello, DescribeHello> {
public:
DescribeHello() {
REGISTER_BEHAVIOUR(DescribeHello, getGreetingShouldReturnHello);
}
void getGreetingShouldReturnHello() {
specify(context().getGreeting(), should.equal("Hello"));
}
} describeHello;
However, where CppSpec really shines is it is able to run/invoke a function, and expect exceptions, something missing from Igloo (unless I just haven't found it yet). CppSpec has more of an extendable matcher system compared with igloo.
Taken straight from the CppSpec examples:
specify(invoking(&Stack<int>::pop).should.raise.exception<std::exception>());
Igloo also does not catch exceptions in its specs, letting them bubble up to the top, it really could be nicer if it was to capture them, and specify which spec they were thrown in.
If the reduced boilerplate around declaring specs and fluency from igloo, could be combine with the exception matching of CppSpec, then this may be a great step towards having very easy to use Cpp tools for BDD.
Packages, or simpler installers for CppSpec would make it easier to put in a C++ project (ah - were there something like Maven for a C++ project and its dependancies).
Extending Igloo to expect exceptions
Because a project I am working on required this, and I noted it's lack in Igloo, I created a small header only helper that expects an exception to have been thrown. I will not claim it is perfect, only that it is good enough.
It can be downloaded from my Orionrobots site at: IglooHelper.hpp.
What do you get?
When passed in a block of code, this will fail an internal Igloo expect if that code did not throw any exception. It is promiscuous in that any exception will do. To use it, download it, and add the header to the top of your Spec code:
Where it is needed for part of your spec:
The other macro that can be used here is ShouldThrowWhatError, which is more selective in that an exception derived from std::exception, which matches a given "what" should have been fired:
The usage, as above starts with including the same header. Then when using in your spec:
This means that another exception fired will either not be caught at all (and bubble up), or if it is a std::exception derivative, it will fail on not being equal to the expected what.
It can be downloaded from my Orionrobots site at: IglooHelper.hpp.
What do you get?
/*
Checks that an exception was thrown, doesn't care what.
@param block A block of code that should throw an exception
For use inside an Igloo Spec
*/
#define ShouldThrowError(block)
When passed in a block of code, this will fail an internal Igloo expect if that code did not throw any exception. It is promiscuous in that any exception will do. To use it, download it, and add the header to the top of your Spec code:
#include "IglooHelper.hpp"
Where it is needed for part of your spec:
Spec(shouldRejectOversizeMinutes) {
ShouldThrowError({ DateTime dt("08:70"); });
}
The other macro that can be used here is ShouldThrowWhatError, which is more selective in that an exception derived from std::exception, which matches a given "what" should have been fired:
/*
Checks that an exception was thrown, with a given "what" value.
@param block A block of code that should throw an exception
@param expected_what A string to compare the what field of the error with.
For use inside an Igloo Spec.
*/
#define ShouldThrowWhatError(block, expected_what)
The usage, as above starts with including the same header. Then when using in your spec:
Spec(shouldRejectANonTimeString) {
ShouldThrowWhatError({ DateTime dt("foobar"); }, "Invalid datetime");
ShouldThrowWhatError({ DateTime dt("08:45:55"); }, "Invalid datetime");
}
This means that another exception fired will either not be caught at all (and bubble up), or if it is a std::exception derivative, it will fail on not being equal to the expected what.
Mocking or Faking
Mocking is a technique used in TDD and BDD to provide fake versions of objects that the code under test is expected to interact with.
This can firstly be used to avoid the code interacting with real database writes or drivers and similar. This means that the code can be better isolated.
Secondly, the fakes/mocks can be spied on to ensure that the interface was interacted with in a certain way.
These are to be used carefully, as while they have the ability to help, if they are overdone, you may end up with tests that are too fragile.
Currently, I am still yet to research what provision there is for these in C++. Neither Igloo or CppSpec are that far forward with this, however, Googlemock might well be usable with this. This brings me to wandering how advanced a Mocking + BDD framework built on the concepts of GoogleTest and GoogleMock might be and if those two projects could be used to "booststrap" a more advance C++ BDD testing setup?
Also now there is Turtle - a mocking library based on the Boost framework, that may also have a real candidate for the de-facto mocking library for Boost. I am yet to play with this to see how powerful it is.
This can firstly be used to avoid the code interacting with real database writes or drivers and similar. This means that the code can be better isolated.
Secondly, the fakes/mocks can be spied on to ensure that the interface was interacted with in a certain way.
These are to be used carefully, as while they have the ability to help, if they are overdone, you may end up with tests that are too fragile.
Currently, I am still yet to research what provision there is for these in C++. Neither Igloo or CppSpec are that far forward with this, however, Googlemock might well be usable with this. This brings me to wandering how advanced a Mocking + BDD framework built on the concepts of GoogleTest and GoogleMock might be and if those two projects could be used to "booststrap" a more advance C++ BDD testing setup?
Also now there is Turtle - a mocking library based on the Boost framework, that may also have a real candidate for the de-facto mocking library for Boost. I am yet to play with this to see how powerful it is.
The other BDD for C++
There is another BDD - Binary Decision Diagrams - which has tools like BuDDy. This is a completely different use of the acronym, and is not the same as Behaviour Driven Development. I am not sure which of these acronyms have precedent, but if you were looking for information on the other kind of BDD please try this:
BuDDy - BDD C++ Package, related to JBDD and CUDD.
BuDDy - BDD C++ Package, related to JBDD and CUDD.
My other lenses on coding
Coding tips, coding challenges and coding examples - either by me, or that I like on Squidoo.
Give me a thumbs up for testing in C++
This module only appears with actual data when viewed on a live lens. The favorite and lensroll options will appear on a live lens if the viewer is a member of Squidoo and logged in.
What testing do you use for your programming?
If you have good tools you use to automate testing C++, especially test first, test driven or behaviour driven ones, I'd love to hear about them. General comments on the lens are welcome too.
HTML and links are permitted, but all comments are moderated - no spam or explicit content please!
-
-
EditorDave Apr 16, 2012 @ 5:25 pm | delete
- Wow. Interesting lens. I've been a tech writer for many, many, many (too many) years... and one of the gigs I had was working for 2 years with a software company that developed "Requirements Driven Development" (RDD-100) applications software for engineering design and development. The programmers used "SmallTalk" rather than C++ to create their code and the applications. Fascinating stuff. (But I'm a *biology major*/*chemistry minor* ... so the only bugs I can fix are the ones you can spray pesticide on!) Congratulations on a Squidoo masterpiece!
-
-
-
curious0927
Mar 2, 2012 @ 1:53 pm | delete
- MMMM, I just downloaded a free for 30 days program--notepad c++, something like that. and I am clueless how to begin. Am I supposed to copy my source code into the progam from my squidoo lens? I know a little HTML but want to do a lot more visually with my lenses. Like, centering pictures for instance. I'm wondering too if I should save all my pics used for squidoo on Pinterest or flickr. When trying to center a photo (all my photo's are saved to my HD) the code want's a domain name---should I move everything to flicker? then I could avoid upload photo and use code with pics stored somewhere with a domain name. Hope you understand this! Great lens...I've book marked for later study! Blessed!
-
-
-
dannystaple Mar 2, 2012 @ 3:36 pm | delete
- Hi there curious. I know of notepad++, which is a pretty good source code editor for Windows - and totally free. If you want to learn HTML (kind of a different computing topic from C++ programming), then I'd suggets playing with w3cschools - they have good tutorials which you can test drive right on the web page, and learn the html stuff, then the CSS stuff. You can embed css bits into style attribute on lens tags.
-
-
-
ana_nimoss
Oct 17, 2011 @ 1:19 am | delete
- Very interesting but way over my head! I will come back later, maybe?
-
-
-
efriedman
Jun 23, 2011 @ 1:13 pm | delete
- I have featured this lens on my Mahout in Action page
-
- Load More
About Danny Staple
by dannystaple
I've been programming computers since I was a kid in the 80's. I started with Basic (and a little assembly) on a Commodore 64, then went on to doing stuff... more »
- 78 featured lenses
- Winner of 18 trophies!
- Top lens » The Lego RCX, Inside And Out
Feeling creative?
Create a Lens!
Explore related pages
- How to make Flash games with Actionscript 3 How to make Flash games with Actionscript 3
- Which Programming Language Is Best? Which Programming Language Is Best?
- How to Become a Freelance Software Developer How to Become a Freelance Software Developer
- Twuring - The Twitter Turing Test Twuring - The Twitter Turing Test
- Python Programming - Why Python? Python Programming - Why Python?
- Dannys Programming Tips Dannys Programming Tips
Behaviour Driven
The RSpec Book: Behaviour Driven Development with Rspec, Cucumber, and Friends (The Facets of Ruby Series)
Amazon Price: $17.99 (as of 05/30/2012)
There are few books on this topic as it is currently still a very new branch of computing. However, this book is a good reference, it shows some of the best tools and pulls some of the heavyweight contributors to this methodology, notably including Dan North.
