Updated. Thanks to Arun for giving me a pointer to pstools.

This will be part of Marathon User Guide when it comes out. I am writing these snippets so that you can benefit from them now.

Now that you wrote a lot tests using marathon how do you run them? The problem with GUI testing is that your desktop is tied up for the duration of the tests. The monitor blinks like a tube-light with a starter problem with Windows popping up all over with crazy titles – ‘testStartAndStopRecordingWithSetupAndTearDown’ or ‘testDeleteFilesWithSomeFailures’. Even if you can live with this – all these popping up windows will be stealing your application’s focus (beside stealing yours) and some of your keystrokes and mouse clicks will go there. In the end you end up not knowing whether a test case failed because of you using the keyboard or a bug in the application under test.

Using java.awt.headless

Java since 1.4.2 supports a headless mode that is half-baked and not useful for the purposes we are talking about. Googling for ‘java headless’ turns up this page, so I am pointing this out. Java support for headless is really for applications that need to run without a display and keyboard. Anyhow if you do not need a display then the whole of this article does not make sense.

Running headless on Linux

An elegant solution I found to run GUI test suites on Linux is to use Xvbf – a X server that simulates a display, mouse and keyboard. On ubuntu, the platform of choice for me, the Xvfb package comes with a script xvfb-run. Executing an application under Xvfb is just typing:

xvfb-run marathon <path-to-your-mpf>

The standard error and output will be on your terminal, so you can see the progress of your test case execution and any errors thrown up by the test run.

Running headless on Windows

Using psexec

psexec is a small utility from Sysinternals from the pstools package. You can Download it here.

psexec allows you to run applications on remote machines. We will use it to run marathon headless on Windows. Download the pstools archive and install on the machine. Add the installed directory to the path. There are two ways you can execute Marathon – with a console and without a console.

For using it with a console (so that you can see the progress):

psexec -s cmd /c marathon -batch <path-to-your-mpf>

If you are not interested in a console:

psexec -s -d cmd /c marathon -batch <path-to-your-mpf>

Note that -s uses the system account (i.e the administrator) to execute the command. I found it easier to create a batch file that runs Marathon with all options and execute the batch file using psexec:

psexec -s cmd /c <path-to-batch-file>

Using telnet

You can run the tests headless on Windows using the telnet service that is available on Windows XP Professional. You can enable telnet service from Control Panel -> Services. Once you enable the service log-in using the command:

telnet localhost

Give your user name and password. You will land up at C:> prompt. Run your command:

marathon -batch <path-to-your-mpf>

That is it. Your display, keyboard and mouse belong to you, no annoying pop-up windows to disturb you and Marathon chugs along running the application tests. You can use -text, -html or -xml arguments along with -batch so that the test results are captured in a report.

Note: Running a telnet server is insecure. Modify your windows firewall settings to disable incoming connections. Default is to disable incoming connections from Internet – it should be OK.

Note: It will be possible to automate the log-in part provided we have a telnet client that can take input from a file and send the commands to the server. I don’t know of any. Drop me a line if you know such a client for Windows.

I am writing this post so that it can be a reminder to myself as well as giving information to all of you waiting for a new release of Marathon. Since beginning of August I am full time working for my own company, so I am able to put in lot more effort in the development of Marathon. So what is going to be the release plan? The following are the tentative release dates for new versions of Marathon:

1. Version 0.91 – August 15th
2. Version 0.92 – August 28th
3. Version 0.93 – September 4th
4. Version 1.0beta – September 11th
5. Version 1.0 – September 18th

Version 0.91

The following features are added to Version 0.91:

1. Tabbed editor

   Now you can open multple testscripts/capturescripts and fixtures in the editor multiple tabs. Consider the current functionality as rudimentary. Some more enhancements are planned for 0.92. I faced a funny problem while developing this bit. Somehow, when we put up JEditTextArea components into a tabbed pane, the content of all the tabbed panes with JEditTextArea becomes the same. It turns out that JEditTextArea’s (from jedit-syntax package) uses a static TextAreaDefault object if default constructor is used. And the TextAreaDefaults has a document also!!

2. Docking

   The result window and console are docked to the bottom pane. You can switch between the two. The JUnit panel is docked to the Navigator window. You can open it by clicking on the JUnit button.

3. Capturing output from the Application under test

   The output (stderr/stdout) from the application as well as the test scripts is captured and displayed in the console window. The output from script (using system.err or system.out) is italicised. The stderr output is in red color and stdout is in blue color.

4. Runtime port

   Marathon uses RMI to communicate with the AUT. Setting up of this port was a major headache in the earlier versions. Now this port is dynamically allocated by Marathon, so you do not need to set marathon.runtime.port property.

5. Playing scripts with a delay

   Now you can execute the script with a delay from the toolbar using Slow Play command. The delay can be changed by setting marathon.runtime.default.delay to a different value (default: 1000 – i.e 1second).

6. Edit menu

   Undo/Redo/Cut/Copy/Paste are added to the Edit menu item.

Version 0.92

The concentration for 0.92 is on hardening of Marathon and providing support for Java 5.

1. Run unit tests in a loop.

   I added a junitrunner script and facility so that we can execute the UTs in a loop. Found multiple failures when the UTs are run like this and fixed all of them. At this time all UTs are passing.

2. Support for Java 5

   All the UTs should pass under Java 5. Should be able to run all of them in a loop. Found a problem with Java 5. See Window.dispose() can be a recursive call.

3. ComponentResolver and DefaultNamingConvention need to be updated

   I am planning for simplifying the naming conventions used. If we take into consideration that only components on the toplevel, focused window need to be named at any time – we may not need to consider the container objects within the window. Need to check this out. This will also solve the problem where-in different names are being generated for components in Java 2 and Java 5.

4. Fix some bugs

   Now when a play is stopped by the user by quitting the application, the display window can be restored to normal state by clicking on the ‘stop’ button. Similarly, while recording if the application is exited, the display window comes back to the normal state. (However, the last few clicks might not be recorded).

Version 0.93

1. Floating toolbar and hiding Marathon main window while recording

   I am excited with this feature. I do not know about others, but I am always irritated that the screen real estate is occupied by the Marathon main window while recording test cases. I am planning to hide the main window if you press the record button. A floating (always on top) tool bar will be displayed with stop/insertscript/pause etc. relevant buttons.

   It will also be nice to have a small text area in the toolbar where the current component name hovered on is displayed.

2. Assertions

   I might leave this for the next version (post 1.0), but I will mention the feature here. There are always requests for new assertions to be added to Marathon. If we can provide a mechanism by which the tester can define a new assertion while recording it might be of great value. While recording, we already know the current focused component. When the user opens the context menu, we can provide a option to record a new assertion. That can provide a list of all getters/is<XX> functions from which the user can select an option and can record a new assertion. Giving this a name make this assertion available when recording on similar components.

1.0 Versions

Mostly documentation and more documentation.

So what do you people think? If I missed out something that you feel should be there, please drop a mail to either the list or add a comment here.

Remember that Marathon can be improved only by feedback from its users.

While working on porting Marathon to Java 5, I found a funny problem. We use a TestDialog(derived from JDialog) for running most of the component related test cases. When run under Java 5, the dispose() of TestDialog is being called multiple times. Since TestDialog.dispose() calls super.dispose() this goes into a loop. The number of times TestDialog.dispose() is called is random. I dug into the sources and found that since some developers might not call super.dispose() in classes derived from Window, the VM calls dispose() for them.. So a little bit of sloppy programming would have helped.

For some time I am planning to update this website. I am using WordPress as the backend and wanted to use it also as a CMS for this site. WP supports a concept of pages and page heirarchy, but I could not find any WP themes that makes use of this as I want it.

I started with an excellent opensource web template by Andreas. (BTW, head for his site if you want to see some excellent CSS/XHTML compliant templates). I hacked a bit in wordpress to get what I want. Then I went ahead and created a template for this site (what you see currently for this page).

Both the templates are available for download @ WordPress themes page.