What is bumbleTest
The bumbleTest project is an open-source framework for executing and recording the results of benchmarks. As it was primarily built for performing benchmarks it is designed to do concurrency and scaling tests in a consistent and repeatable fashion. bumbleTest currently supports the following types of tests: 

  • DB Operations: Such as DML and DDL operations e.g.
  • OS Calls: Any command locally available to the server running bumbleTest.
  • Java Method Calls: Similar to jUnit but simpler.
  • jUnit Tests Cases: See jUnit for more information.
Who would use bumbleTest
Anyone who wants to run a series of tests and gather the results into a database for analysis. People have used bumbleTest to do the following: 

  • Execute repeatable database benchmarks
  • Run parallel jUnit test cases
  • Gather OS call timings

How do I get started

  1. Download the complete package from sourceForge.
  2. Unzip the contents of the file to the machine you want to run it on.
  3. Check that you have java 1.6 installed.
  4. java -version

  5. (Optional) Run a simulation, using the example test, by typing the following
  6. java -cp bin/bumble.jar -Dsimulate -Dlabel=”test” TestHarness

    If everything goes well a series of tests will run in simulation mode and you’ll see an output that ends with.

    ... [INFO]Executing test case: Q3 [INFO]Completed test case: Q3 [INFO]Executing test case: Q1 [INFO]Completed test case: Q1 Time: 19.636 OK (24 tests)
  7. Modify the files in the xml folder to reflect the tests you want to do; see below for a simple example.

How do I make my first test

  1. Open the example named helloWorld.xml,in the XML folder, in your favorite xml editor.
  2. Modify the url, username, driver and password to point to the database of your choice. Oracle is used here but as long as you have the JDBC driver for your database, you’ll be able to continue to the next step.
  3. Place a copy of the jdbc driver on the machine.
  4. Modify the Hello sqlcase sql statement to something that works on the target database.
  5. Run the test by typing the following:
    java -cp bin/bumble.jar  -Dlabel=”My First Test” TestHarness

    If everything goes well a series of tests will run and you’ll see a success message.

  6. To view the results type the following:

    java -cp bin/bumble.jar org.h2.tools.Console

    A browser window will popup asking you to connect to the H2 database. <ake sure the the connection URL is jdbc:h2:test and press connect.

  7. The test schema consists of four tables.
    • Test_Run_Config – Represents a test suite execution (One row for each Test Run declared in the xml file)
    • Test_Run – Represents a collection of test groups (One row for each test execution requested. The threads field represents the concurrency)
    • Test_Group_Instance – Represents a collection of tests (Each concurrent thread is a test group, as each thread may run a series of SQL statements)
    • Test_Case – An execution of a test, contained within a test group (The individual sql statements)

Why Use Eclipse

bumbleTest under the covers uses jUnit, because of this integration with eclipse is relatively simple. The main reason to use eclipse over the command line version is the jUnit progress monitor and the built in XML editor. If you aren’t familiar with eclipse, then it might be best to keep to the command line version.

How do I configure this

  1. Create a new java eclipse project.
  2. Import the bumble.zip, using the eclipse import wizard.
  3. Create a new jUnit run configuration named TestHarness Simulation.
    1. Class name is TestHarness
    2. VM arguments are -Dsimulate  -Dlabel=”simulation”
  4. Create a new jUnit run configuration named TestHarness Live
    1. Class name is TestHarness
    2. VM arguments are -server -Dlabel=”live”
  5. Run a simulation by executing TestHarness Simulate. When you do this the jUnit monitor will appear.
  6. Run a live test by executing TestHarness Live

 

Simple Parameters

Most basic form of parameters allows for a single value to be reused many times. If the value needs to be changed it only have to be changed in one place.

Declaration

Define a parameters group and give it a name, the id attribute. Declare one or more parameters; CDATA declarations are supported.

<parameters id="basic">
 <parameter id="username">scott</parameter>
</parameters>

Usage

Reference the parameter group by calling it using the parameters attribute. Call the individual parameter using the following syntax “${parameter_name}”.

<testgroup id="TestGroup" parameters="basic">
 <sqlcase id="Hello" description="Something simple">select count(*) from all_users where username = '${username}'</sqlcase>
</testgroup>

Advanced Parameters – Inheritance

Parameter groups can inherit other parameter groups, this allows a hierarchy of parameters to be setup. A good example of this is a database connection. The username and password might be different but the connection url would be the same.

Declaration

<parameters id="connection">
 <parameter id="url">jdbc:db@localhost</parameter>
 <parameter id="driver">org.database.driver</parameter>
</parameters>

<parameters id="defaultPassword">
 <parameter id="password">password</parameter>
</parameters>

<parameters id="connect_as_scott" inherit="connection,defaultPassword">
<parameter id="username">scott</parameter>
</parameters>

<parameters id="connect_as_bob" inherit="connection,defaultPassword">
 <parameter id="username">bob</parameter>
</parameters>

Usage

<testgroup id="TestGroupScott" parameters="connect_as_scott">
 <sqlcase id="Hello" description="Something simple">select count(*) from all_users where username = '${username}'</sqlcase>
</testgroup>

<testgroup id="TestGroupBob" parameters="connect_as_bob">
 <sqlcase id="Hello" description="Something simple">select count(*) from all_users where username = '${username}'</sqlcase>
</testgroup>

Advanced Parameters – Functions

bumbleTest has a few built in functions which allow for dynamic parameters.

  • randomDate(min date, max date) – Generates a date between the given range.
    Syntax ${randomDate(YYYYMMDD,YYYYMMDD)}
  • random(min value, max value) – Generate a number between the given range.
    Syntax ${randomDate(#,#)}
  • seed – Generates a random number between 0 and 100
    Syntax ${seed}

Declaration

<parameters id="basic">
 <parameter id="start_date">${randomDate(20100101,20100131)}</parameter>
 <parameter id="random_number">${random(23,100)}</parameter>
 <parameter id="random_between_0_and_100">${seed}</parameter>
</parameters>
© 2011 bumbleTest Suffusion theme by Sayontan Sinha