Ürgo Ringo's blob

Set up test state using API when running tests via UI

Ürgo Ringo — Fri, 16 Dec 2011 08:38:09 +0000

When writing tests via UI it usually makes sense not to use UI for setting up application state. First, using UI for setup makes your UI tests even slower. Second, you may need to write some additional UI automation code simply for setup.

Alternative option is to put test data directly into database. However, for things that are not just part of stable reference data (e.g list of all countries) this is quite risky and error prone. Direct database access means that you are bypassing your domain logic and hence can easily end up using invalid state in your tests. Better alternative that I have seen in many projects is to use special remote API for setting up application state. In many cases RESTful API is the most suitable since it can be accessed using very simple client. Writing such API should pay off even when not considering test execution time since the amount of code you need for it should in most cases be less than required by UI automation script.

 @Test
  public void createBlogPostOnBehalfOfTopManager() {
    RemoteUser topManager = new RemoteUser("top manager");
    soapService.addUser(topManager);

    BlogPostPage createBlogPostPage = dashboard.goToCreateBlogPostPage();
    createBlogPostPage.setTitle("test").setContent("test content");
    ViewPage viewBlogPost = createBlogPostPage.setAuthor("top manager").save();

    assertEquals("top manager", viewBlogPost.getAddedBy());
    assertEquals("top manager", viewBlogPost.getLastModifiedBy());
  }

One team – one language

Ürgo Ringo — Fri, 11 Nov 2011 11:05:33 +0000

In many multicultural teams it happens that people start talking to each other using their native language. Main problem is that this way we effectively cut off those in the team who do not understand given language. This essentially means that osmotic communication doesn’t work. If topic is discussed in a language that everybody understands then everyone has a chance to join conversation even if he/she was not addressed when discussion was started. However, if non-common language is used then there is high risk that full potential of the team is not used when decisions are made.

I think main reason why people do this is not because they want to be impolite or want to exclude someone from discussion but it just feels more natural. If two team members have language X as their native tongue then they feel awkward using any other language when speaking to each other. To avoid this awkwardness there could be clear team rule that all work related talk should be in the language that everybody can understand. Of course there could also be exceptions like doing pair-programming. However, such exceptions make it harder to follow the general idea and osmotic communication is useful even when doing pair-programming.

High vs low level testing

Ürgo Ringo — Fri, 11 Nov 2011 07:50:24 +0000

What is more important – automating tests on unit (class) level or automating the tests that verify behaviour of the system as a whole?

When going from lowest level of testing to highest then increase of following properties can be noticed:

time needed to write tests
time needed to maintain (keep up-to-date) tests
time needed to run test
potential for discovering bugs
usefulness as documentation of system functionality

Since lowest level of testing (unit testing of classes) is the cheapest then main automation effort is often put there. However, I believe that depending on the type of system and set of technologies the optimal point considering cost and benefit may well be above unit testing. Often it is the UI layer that adds significant extra cost to automation. If that is the case I think it may be good strategy to bypass it. In case of web apps this means that all client side logic will not be covered by functional tests. However, often client side logic is specific for each page so each page is quite independent from others and we don’t need much regression testing.

I have noticed that in projects with automated functional tests the amount of unit tests is much smaller than in projects where there are only unit tests. This makes sense because in terms of giving you the safety net for future changes there is quite a bit of overlap in these types of tests.

No matter whether you have automated functional tests or not I only see point in writing unit tests in following cases:

when you implement some complex algorithm and want to verify that it works as expected before putting it into play with the rest of your system
when you want to document some (ugly) behaviour that cannot be easily tracked back from any system requirement
when you are more or less following the test first practice

Otherwise, if you have implemented the logic and manually verified that it works correctly then the value of writing unit tests for given code isn’t very high. I have often seen that such tests tend to be more complex (less readable) so they will soon become quite a headache to maintain.

So functional tests don’t necessarily have to mean driving tests through UI. Without UI functional tests can be significantly cheaper while still being more useful in terms of coverage and documentation compared to low level unit tests.

Agile Saturday VI

Ürgo Ringo — Mon, 31 Oct 2011 08:25:17 +0000

Deep bow to enthusiasts at Agile Estonia who are organizing these events. I think that for me the main value of Agile Saturday has always been that it gives you quite good picture of current state of software development in Estonia.

This was the 2nd time for me to speak at Agile Saturday and 3rd time to visit it. One of the positive undercurrents I have noticed is that more and more people want to look beyond the basic guidelines of some process/methodology/framework. That does not mean that basic rules/patterns are not important. It is important to first master the technique before going about synthesising it into something new (as per the Shu-Ha-Ri). For western people like myself it is often very hard to understand that journey is more important that the goal itself. We often think that being an apprentice or journeyman is something lesser than being a craftsman.

It is totally OK to start with something like Scrum and then go on from there. It is well as long as it is not about perfecting Scrum but perfecting the solutions that you offer to customer.

Checking OSGI package imports

Ürgo Ringo — Mon, 10 Oct 2011 08:35:31 +0000

In one of our projects where we use OSGI we generate package imports automatically with maven-bundle-plugin. It works quite nicely most of the time. However, the problem is that if you automate imports then you give up some control over your dependencies which goes a bit against the whole idea why package imports exist in OSGI.

So we came up with simple test that checks generated imports against our expectations. This test gives us possibility to review each change in our imports. Main benefit compared to adding these imports manually and not using any generator plugin is that this way imports will always be up-to-date and there will be no unused imports that we forgot to remove.

import org.springframework.osgi.test.internal.util.jar.JarUtils;

...

public static void assertPackageImports() {
  //retrieve jar file
  Resource jarFile = ...

  Manifest mf = JarUtils.getManifest(jarFile);
  String actualImports = mf.getMainAttributes().getValue("Import-Package");

  assertEquals(expectedImports, actualImports);
}

How, what and why thinking

Ürgo Ringo — Tue, 30 Aug 2011 06:56:29 +0000

One way of thinking about one’s competence is using three levels: the “how” level, the “what” level and the “why” level.

How level – in this level the main question one asks is how something can be done. One will feel totally satisfied if something starts working no matter how ugly it is. Getting it working is the main goal on this level.

What level – in this level one already has more experience so one doesn’t need to be told how something should be done. One just wants to know what needs to be done. Also emphasis moves more towards how something can be done well.

Why level – in this level one starts asking more why something is needed at all and what are the actual goals that this or that feature needs to be added. One manifestation of this level is when one is able to convince the customer that some new feature is not needed at all or that the same goals can be achieved with existing tools.

About dedicated QA

Ürgo Ringo — Wed, 22 Jun 2011 06:35:44 +0000

Do we need to have a dedicated QA in development team and what happens if we do not have dedicated QA?

Having someone specialized in single role doesn’t matter if it is QA or any other increases the risk of bottlenecks. There is the danger that some area of project knowledge is concentrated on one person which makes it harder to optimize team as a whole. I have often seen that tester will sort of start living in a world of his own with his special tools and reports and stuff that nobody else has the time to immerse into (see more here). In a way that is of course also good because that provides additional independent view of the system.

What will happen if there is no dedicated tester? First, developers will be very eager to automate as much testing as possible. That means not only more unit testing but automation on functional and system testing levels (actually the latter will often result in less unit testing). We all know that automation of tedious, repetitive tasks is good so what do we actually loose if there is no dedicated QA?

I think that main thing that is hard to compensate is that different view of the system. I mean any developer can learn the tools and theories of QA and indeed should learn but there is always a difference in results when doing explorative testing for fun and doing it because you have to.

Interesting behaviour in Apache Camel Bean Invocation

Ürgo Ringo — Fri, 17 Jun 2011 05:42:21 +0000

When using bean invocation in Apache Camel with method parameter in URI like: uri="bean:accountService?method=updateEmail" then you have to specify method the same way for all subsequent invocation of any beans.

Trying to use annotations or having only one method in your bean won’t make any difference you still have to specify the method parameter. This is so because when using method parameter Camel adds camelbeanmethodname header to Exchange and will not remove this header once bean has been invoked. Unfortunately camelbeanmethodname header will take precedence over any other bean invocation option as specified here. So this is not a bug but a feature.

Thoughts from Mobile Monday Estonia event

Ürgo Ringo — Tue, 07 Jun 2011 06:30:46 +0000

Scrum is for beginners

This is not meant in a patronising way but rather that Scrum is one good methodology to start with not necessarily your final destination.

Measuring success

How can you analyze how successful your organization has been in improving itself? You should not count the number of Scrum teams as it only shows how many teams are on the first level (see the previous point). Neither should you analyze how precise estimates your teams are able to produce since estimation is a tool not a goal. Most interesting indicator is how fast you can deliver new functionality – how short is your release cycle. Second thing that is a bit harder to measure but is quite important is the happiness of your developers.

What customer wants vs what customer needs

In most cases we should not be trying to deliver features that customer wants but features that customer actually needs. This topic is quite well covered in Jim Coplien’s Lean Architecture book.

Big thanks to organizers.
Event details: http://www.momoestonia.com/2011/05/mobilemonday-event-mobile-development.html

Asynchronous assertions

Ürgo Ringo — Tue, 10 May 2011 05:52:14 +0000

In our current project we needed to test asynchronously executed logic. At the beginning we had only few of such tests so Thread.sleep was good enough for us. However, when the number of such tests started to grow we decided to implement better solution which is partly based on the idea described by Martin Fowler in his post.

    protected void waitForSuccess(Runnable runnable) {
        long startTime = System.currentTimeMillis();
        long waitLimit = 2000;
        long pollingInterval = 100;

        while (System.currentTimeMillis() - startTime < waitLimit) {
            try {
                runnable.run();
                return;
            } catch (Exception e) {
                //ignore
            } catch (AssertionError e) {
                //ignore
            }
            try {
                Thread.sleep(pollingInterval);
            } catch (InterruptedException e) {
                //ignore
            }
        }
            
        runnable.run();
    }

And here is how we use it in our tests:


waitForSuccess(new Runnable() {
  public void run() {
    assertEquals(...);
    assertEquals(...);
    ...
  }
});

Of course in a language that supports closures this solution would be more elegant but it’s not that bad even in Java.

In addition to server side testing I imagine that this is quite useful for testing Ajax UIs (e.g with WebDriver) where parts of UI are updated asynchronously. Of course that could also be solved by checking existence of some DOM node but that tends to pollute the test code and cannot be applied in a generic fashion.