« Adding a comment feed | Main | Dead code elimination »

Testing thread safety - updated

Listen to this articleListen to this article

Not much this time except to say that I took the previous examples and made them a bit more generic. The example provided shows the simplest method of using the classes but it can easily be extended for more complex requirements. In fact, I've so far used these classes to successfully test some in-memory database code I'd been writing so it definitely works for other than tivial examples.

/**
 * Based on article http://www.npac.syr.edu/projects/cps615fall95/students/jgyip5/public_html/cps616/conflict.html
 */
public final class SimpleSample {
    private int _common;

    /**
     * Increment the common variable
     * @return true if we managed to update it "atomically", otherwise false to indicate failure
     */
    public synchronized boolean increment() {
        int common = _common;

        Thread.yield();

        boolean successfull = (_common == common);

        _common = common + 1;

        return successfull;
    }
}

import java.util.LinkedList;
import java.util.List;

public class SimpleSampleTest extends CustomTestCase {
    public SimpleSampleTest() {
        super(SimpleSample.class);
    }

    public void test() {
        final SimpleSample sample = new SimpleSample();

        List targets = new LinkedList();

        for (int i = 0; i < 5; ++i) {
            targets.add(new CustomRunnable() {
                public boolean run() {
                    return sample.increment();
                }
            });
        }

        execute(targets);
    }
}

import org.objectweb.asm.Attribute;
import org.objectweb.asm.ClassAdapter;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.CodeVisitor;
import org.objectweb.asm.Constants;

/**
 * Removes synchronization from a method. Currently only removes the synchronized access flag from the
 * method declaration.
 */
final class ClassModifier extends ClassAdapter {
    public ClassModifier(ClassVisitor visitor) {
        super(visitor);
    }

    public CodeVisitor visitMethod(int access, String name, String desc, String[] exceptions, Attribute attributes) {
        int newAccess = access;

        if ((access & Constants.ACC_SYNCHRONIZED) != 0) {
            newAccess -= Constants.ACC_SYNCHRONIZED;
        }

        return super.visitMethod(newAccess, name, desc, exceptions, attributes);
    }
}

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.ClassWriter;

import java.io.IOException;

/**
 * Forces certain classes to be loaded into this class loader. In addition, performs byte-code modification to remove
 * synchronization from specific classes.
 */
final class CustomClassLoader extends ClassLoader {
    /** Name of the class under test */
    private final String _subjectClassName;

    /** Name of the test class */
    private final String _testClassName;

    /**
     * Constructor.
     * @param subjectClassName Name of the class under test
     * @param testClassName Name of the test class
     */
    public CustomClassLoader(String subjectClassName, String testClassName) {
        _subjectClassName = subjectClassName;
        _testClassName = testClassName;
    }

    public synchronized Class loadClass(String name) throws ClassNotFoundException {
        Class c = null;

        if (name.startsWith(_subjectClassName)) {
            c = defineClass(name, true);
        } else if (name.startsWith(_testClassName)) {
            c = defineClass(name, false);
        } else {
            c = super.loadClass(name);
        }

        return c;
    }

    /**
     * Forces the loading of a class into this class loader
     * @param name The fully qualified class name
     * @param modify
     * @return The newly defined class
     * @throws ClassNotFoundException If an error ocurrs during loading
     */
    private Class defineClass(String name, boolean modify) throws ClassNotFoundException {
        // Setup the class file to read
        ClassReader reader = null;

        try {
            reader = new ClassReader(getResourceAsStream(name.replace('.', '/') + ".class"));
        } catch (IOException e) {
            throw new ClassNotFoundException(name, e);
        }

        // Setup an in-memory writer for the byte-code
        ClassWriter writer = new ClassWriter(false);

        // Determine if we need to modify the class
        ClassVisitor visitor = writer;

        if (modify) {
            visitor = new ClassModifier(writer);
        }

        // And load it
        reader.accept(visitor, false);

        byte[] byteCode = writer.toByteArray();

        return defineClass(name, byteCode, 0, byteCode.length);
    }
}

/**
 * Convenience interface for constructing simple threaded tests.
 * @see CustomTestCase#execute(java.util.List)
 */
public interface CustomRunnable {
    /**
     * Performs the test.
     * @return true to indicate success, otherwise false to indicate failure
     */
    public boolean run();
}

import junit.framework.TestCase;
import junit.framework.TestResult;
import junit.framework.TestSuite;

import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;

/**
 * Base class for testing thread safety. Extend this class and simply implement any tests you need. Each test will be
 * run once ASIS and then again with synchronisation removed to, hopefylly, induce failure. When run with synchrisation
 * remove, each test is checked to ensure that it failed.
 * 
* There is also a convenience method execute(List) to assist in writing simple tests. * @see #execute(java.util.List) */ public class CustomTestCase extends TestCase { /** Names of classes to modify */ private final String _subjectClassName; /** * Constructor. * @param subjectClass Names of class under test */ public CustomTestCase(Class subjectClass) { _subjectClassName = subjectClass.getName(); } /** * As the name implies, re-runs all tests (except itself) with synchronisation removed. */ public final void testAllUnsynchronized() throws Exception { // Ignore recursive invocations of this particualar test if (getClass().getClassLoader() instanceof CustomClassLoader) { return; } // We'll need our custom class loader to perform byte-code manipulation CustomClassLoader loader = new CustomClassLoader(_subjectClassName, getClass().getName()); // We need an instance of this test within the new class loader TestSuite suite = new TestSuite(loader.loadClass(getClass().getName())); // Run the tests TestResult result = new TestResult(); suite.run(result); // And and ensure they ALL failed // TODO: This is not sufficient for reporting purposes. We need to check and report on each test assertFalse(result.wasSuccessful()); } /** * Convenience method if a simple threaded model is all you require. This method executes the specified * CustomRunnables and asserts that each thread completed successfully. * @param targets Collection of CustomRunnables to execute in parallells */ protected final void execute(List targets) { // All threads are to share the same thread group final ThreadGroup group = new ThreadGroup(getName()); // Create a bunch of threads, one for each target final List threads = new LinkedList(); for (Iterator i = targets.iterator(); i.hasNext(); ) { threads.add(new CustomThread(group, (CustomRunnable) i.next())); } // Start up the threads for (Iterator i = threads.iterator(); i.hasNext();) { ((CustomThread) i.next()).start(); } // Wait for them to finish and determine if they were all successful or not boolean successful = true; for (Iterator i = threads.iterator(); i.hasNext();) { CustomThread thread = (CustomThread) i.next(); try { thread.join(); } catch (InterruptedException e) { // Ignore it } successful &= thread.wasSuccessful(); } assertTrue(successful); } }

/**
 * Executes a CustomRunnable and reports on the success or failure.
 * @see CustomRunnable
 */
final class CustomThread extends Thread {
    private final CustomRunnable _target;
    private boolean _successful = false;

    public CustomThread(ThreadGroup group, CustomRunnable target) {
        super(group, "");
        _target = target;
    }

    public void run() {
        try {
            _successful = _target.run();
        } finally {
            if (!_successful) {
                // Fail-fast - interrupts all sleeping threads
                Thread.currentThread().getThreadGroup().interrupt();
            }
        }
    }

    public boolean wasSuccessful() {
        return _successful;
    }
}

Comments

That's an awful lot of test code. At a certain point you have to wonder if the development and maintence of complex tests like this is worth it.

Fair point. However, it is re-usable code. So I never have to write it again. The only code that I have really written for the test is the SimpleSampleTest. You could argue that JUnit contains an awful lot of code but again, you never have to write it yourself.

Also, I did state upfront that I was really trying to prove a point. Most peoples argument is that it can't be done. I'd rather people admit to me (and themselves) that really it comes down either feeling it's not worth it, or they can't be bothered. I don't want people to even have the option of using the cop-out that its impossible.

Cheers,

Simon

Post a comment