Integration platform testing strategies

From Google Summer of Code Mentor Wiki

Integration platform testing strategies:

Big problem when you test your application is to setup the different environnment in which you applicatoin can be run. XWiki for example need to test with different configuration of server, of web browser. In their case, they use JUnit for unitesting the java API they develop and selenium for the UI part. The problem they encounter now with selenium is that there is more and more JS frameworks which complexify the way you write your test. For web frameworks they need to test their app with different SQL database.

For the ogre project the problem is, how to test that the image you render is OK. Currently they usea kind of diff tools. They generate the image they should generate with one function, and the test is does the image generated is the same than the one expected. But if they modify the colour, they need to generate another test with another image to base the test on.

Write a test every time you correct a bug in order to be sure that next time you modify some code you will not reproduce this bug. Unittests need to be close to bug tracker

Random stuff is not always the good way to perform test. There is only one thing which is buggy, you need to test this buggy thing. Thanks to random test you will be able to find new bugs because you create new environment but most of the time your random test will not fire the bug and will pass most of the time. You need the both.

Ask people who used your libraries and have written some unittest which involed your library to give it to you so you can improve your test set. It will help both projects.

On TurboGears they also unittest a lot the tutorials they have written in order to be sure they are functionnal.

Several tools to be able to do continuous integration in several environment:

- bitten
- buildbot
- cmake