TestTastic

Java Client Embedded Browser QTP Headaches

2009-05-07T19:30:00.000Z

The thick Java client that we test using QTP has an embedded browser hosting the Kana Knowledge Management System (KMS). You can configure QTP to play with the embedded browser. It works but it's a bit awkward because you can only use the Object Repository and Spy if you disable the QTP Java addin and even then QTP only recognises a subset of the objects in the embedded browser.

The subset of objects that QTP was willing to recognise in the embedded browser was enough to verify the test points that we were interested in but a recent update of the KMS has changed things, now QTP recognises fewer of the objects and this is causing us a bit of a headache.

My suspicion is that there's yet another object embedded in the embedded browser but I've not been able to get any more info on that yet.

Updated Hudson version

2009-05-05T19:42:00.003Z

I updated our Hudson version last night. This has fixed a minor issue that was making life tedious for us. We have around 15 views (tabs) in Hudson. This is too many to fit on the screen and Hudson has no method of Wrapping or Stacking them so we get a horizontal scroll bar.

We use Hudson to run GUI Integration tests (using QTP) and it's very typical for us to manually kick of a single test. The problem with the many views and horizontal scroll bar is the the Hudson 'run build' button is off the right hand side of the screen and you have to scroll across to it. This becomes tedious when you're doing it a lot.

This new version of Hudson, 1.3.0.3 allows us to add, remove and relocate the Hudson columns so now the run build button is in the first column rather than the last. A simple problem but a sweet fix.

Results Signatures and Auto Outcome Adjust

2009-04-13T20:43:00.000Z

One of the area's I work in is automated regression testing of our CRM solution. We've been improving our test execution solution since since we brought the reponsibility for execution and reporting back in house in Jan 09.

In our world we have test environments that may be unreliable or defective, bad test data causing test failures and, God forbid, maybe even faults in the automated test solution... ;) So there's a fair bit of investigation required before we decide that a test failure really is a fault with the product under test.

This blog entry discusses our Automatic Adjust Solution.

A Fail Signature is a unique collection of failed test point ID's.

We've been working with a Fail Signature Auto Adjust solution for a wee while and it's saving us from tedious, repetitive and wasteful root cause investigation.

Every test point within our automated tests (pass or fail) has a unique ID. So If we run a single test case, as well as a big log file telling us what happened during the test run we'll also have a list of all the failed test points... the Fail Signature.

Starting from a premise that ALL automated test failures need to be investigated before a REAL fail can be reported to the customer of our testing, we use our Auto Adjust to handle predictable, repeatable fail patterns to save us the time and effort looking into every fail.

Example: If our java client fails to load during the test the test will fail. The fail record will contain a unique ID for this test point. The Fail Signature is the collection of all the fail points. If the Fail Signature contains ONLY the 'client failed to load' fail point then the test results will be automatically adjusted to an 'EXEC FAILURE' with a comment that the client failed to load. We don't need to drill into the test logs to derive this conclusion.

When a tester investigates the root cause of a test fail and decides that the behaviour is likley to recur he adds the Fail Signature to our Auto Adjust database so that the same root cause analysis work is not needed next time. The tester decides on the outcome and comment that the Auto Adjust solution should add to the test report.

This automatic analysis of Fail Signature has been a real help in reducing the effort required for initial test fail root cause investigation. It has enabled us to significantly reduce the time taken to deliver the results of a test run to our customers.

Subversion Commit Monitor

2008-03-20T23:53:00.007Z

Commit Monitor is a tool developed by Stefan's Tools | A small collection of tools and utilities. It sits in your systray watching your subversion repository(s).

When files are commited to repositories / projects you're interested in it alerts you and allows you to check out the commit comments and even do a unified diff.

We previously used to send emails to the team when ever we commited changes that had significance for others. It's much easier to use the Commit Monitor as people can see the changes being made without relying on someone remembering / bothering to send an email.

This tool has made it easy for us to keep upto date with code commits with minimum effort for all team members.

Thanks to Stefan for this superb tool.

Cool Tools

2008-03-20T23:51:00.003Z

Well...... I've not managed to do much with this blog. I've decided I might blog a bit about the tools I use for test automation. I'm always discovering little tools to solve little problems so I'll write a little about them I think.

Glenn

Automated Code Documentation for QTP Actions

2007-03-08T14:38:00.005Z

Goal:

My goal was to be able to automatically produce code documentation for my QTP Test Actions to make it easier for others to re-use the actions and avoid having to manually create and maintain the documentation.

Background:

I had already selected NaturalDocs (http://www.naturaldocs.org/) for documenting my VBScript support functions. I wanted to be able to produce similar documentation for my re-usable QTP Test Actions.

Developing the solution:

Note: I often use the term Function to refer to a QTP Action.

On inspecting the QTP test script directory I found that each Action has a directory beneath the QTP Test directory and each Action directory contains a script file: Script.mts.

NaturalDocs recursively searches the specified directory looking for code files. It expects a file header and function headers in the code file. I added a file header and function header to the first Action of my QTP Test and added a function header to each of the remaining Actions. When I ran NaturalDocs it found each of the QTP Action script files and produced the documentation from the File and Function headers but because each Action has a separate script file only the first Action has the file header and hence the other Actions do not get associated with the QTP script name. To fix this problem I wrote a small perl script that concatenates all the individual Action scripts into a single file named after the QTP Test. These concatenated script files are placed in a dedicated directory and this directory is processed by NaturalDocs. This approach ensures all the QTP Actions are documented as part of their QTP Test. I added a call to this perl script in the batch file that is used to run NaturalDocs so that the concatenated files are produced each time I run NaturalDocs.

I extended the NaturalDocs configuration a little to recognise the keyword Action and document it in the same was as a Function. I had already added the keyword Parameter for documenting function parameters in my VBScript support libraries.

Problems

Keyword View: The large number of comment lines in the Action Header are all shown in the Keyword View. This doesn't look very good and just wastes screen space. perhaps there’s a way to configure Keyword View to no show the comment lines?

Example of documented QTP Test Actions: Screen Shot

Setup QTP automated code documentation

Download and install Perl

NaturalDocs requires Perl
Free download from ActiveState: http://www.activestate.com/Products/ActivePerl/

Download and install NaturalDocs

http://www.naturaldocs.org/download.html

Setup the file structure for your project
Add File and Action headers to your QTP tests
Run NaturalDocs
Revel in your lovely documentation

Example

Email me for QtpTest_AutoDoc.zip which contains a working setup including a dummy QTP tests which has File and Action headers and the scripts and folders required to produce the code documentation. You'll still need to install Perl and NauralDocs.

The First Post

2007-02-23T22:53:00.000Z

This is my first post here.

I've been thinkng about a blog for a while. I've been working in software test automation for around 5 years. Mostly non gui system test in the telecoms industry but some gui web app uat recently.

Before working in software test I did a year or so in manufacturing systems development and a few years before that in manufacturing automation. Software for manufacturing robots..... now thats fun...... if you get the code wrong you'd better get your head out of the way pretty damn fast.

I've lurked on many software test groups, commenting from time to time but not very much.

I worked on a scrum (agile) project for a big telecoms instrument and monitoring system developer recently. I enjoyed that very much , it was a real eye opener working on an agile project for real - it takes a lot of work and discipline. It was fun finding ways to give test results back to the developers as rapudly as possible when they released code and it made me realise that in that environment the testers serve the developers first and foremost - giving them feedback on the code they've produced as rappidly as possible.

Even more recently I've been developing automated uat regression testing for an airline web booking app. This was my first time testing through the GUI. I've been using Mercury QTP as the main test tool for this. I've found it to be a pretty good tool for controlling the GUI and managing the test scripts but i'd have got on just as well with Watir or some other open source tool if the Mercury tools hadn't been available. It's good to have mercury test tool experience on your cv in the test automation job market though :-)

I'm starting a new job soon with a company who also use Mercury QTP. I think much of my blogging may be around getting the mercury tools to do what I want them to. For exampe this morning I successfully created automated code documentation for the QTP script actions. I've been using the open source NaturalDocs for automated code documentaion of my QTP script support functions. I finally made time to add my QTP re-usable actions to the code docs.

Well, thats enough drivel for now - I managed to acuire a snotty cold today so I've been treating it with Irish Red Breast Malt Wiskey for most of the evening - probably the cause of me actually getting this ramble off the ground.