Story of a Java 8 Compiler Bug (JDK-8064803)

Story begins with a cleanup done on Hazelcast IQueue interface by one of my colleagues. What’s done actually was just removing unneeded/redundant overridden methods those are already defined in java.util.Queue and java.util.concurrent.BlockingQueue interfaces. It was a very simple cleanup, but…

Full commit diff can be seen here: 2d9eabe82a0a0568d2c92c7d53a5383efb32cf7d#diff-fc20765c5b3aa6eafd6348c5a004447b

Then suddenly, we started to see build errors on our Jenkins CI compatibility builds which are using java 8. There was no compile errors but only a single test using a parameterized queue was failing with java.lang.NoSuchMethodError: com.hazelcast.core.IQueue.poll()Ljava/lang/String;.

Here is the failing test method which has nothing special:

@Test
public void testQueueItemListener() throws InterruptedException {
    final CountDownLatch latch = new CountDownLatch(8);
    final String value = "hello";
    final HazelcastInstance instance = createHazelcastInstance();
    IQueue<String> queue = instance.getQueue("testQueueItemListener");

    queue.addItemListener(new ItemListener<String>() {
        public void itemAdded(ItemEvent<String> itemEvent) {
            assertEquals(value, itemEvent.getItem());
            latch.countDown();
        }

        public void itemRemoved(ItemEvent<String> itemEvent) {
            assertEquals(value, itemEvent.getItem());
            latch.countDown();
        }
    }, true);

    queue.offer(value);
    assertEquals(value, queue.poll()); // <---- error is thrown here
    queue.offer(value);
    assertTrue(queue.remove(value));
    queue.add(value);
    assertEquals(value, queue.remove());
    queue.put(value);
    assertEquals(value, queue.take());

    assertTrue(latch.await(5, TimeUnit.SECONDS));
    assertTrue(queue.isEmpty());
}

My gut feeling was saying that, this is certainly a compiler bug. But I needed to prove that using a simple and easily reproducible test case. At the beginning I wrote a dummy implementation of IQueue interface and executed the same test against it. Result was the same failure again.

Then I decided to emulate the Hazelcast IQueue implementation using a few single abstract method interfaces. There were two parent interfaces ParentA and ParentB replacing java.util.Queue and java.util.concurrent.BlockingQueue. And the actual interface Child which is substitute of IQueue. And an empty implementation of Child interface:

public interface ParentA<T> {
    T process() throws Exception;
}

public interface ParentB<T> {
    T process() throws Exception;
}

public interface Child<T> extends ParentA<T>, ParentB<T> {
}

public class ChildImpl<T> implements Child<T> {
    @Override
    public T process() {
        return null;
    }
}

Test method was very simple, just instantiate a ChildImpl with String generic type by assigning it to a Child reference and call empty process() method:

public class Test {
    public static void main(String[] args) throws Exception {
        Child<String> child = new ChildImpl<String>();
        String result = child.process();
        System.err.println(result);
    }
}

Compiled these with javac 8 and ran the main method. Result was as expected: failure!

Exception in thread "main" java.lang.NoSuchMethodError: Child.process()Ljava/lang/String;

When I disassembled the generated bytecode using javap tool (javap -c Test.class), output was:

public class Test {
  public Test();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public static void main(java.lang.String[]) throws java.lang.Exception;
    Code:
       0: new           #2                  // class ChildImpl
       3: dup
       4: invokespecial #3                  // Method ChildImpl."<init>":()V
       7: astore_1
       8: aload_1
       9: invokeinterface #4,  1            // InterfaceMethod Child.process:()Ljava/lang/String;
      14: astore_2
      15: getstatic     #5                  // Field java/lang/System.err:Ljava/io/PrintStream;
      18: aload_2
      19: invokevirtual #6                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
      22: return
}

Code was trying to invoke an interface method which is returning a String, see this line

9: invokeinterface #4,  1   // InterfaceMethod Child.process:()Ljava/lang/String;

Problem was Child interface doesn’t have a process() method returning a String. It’s a generic method and because of type erasure it simply returns a plain Object. Compiler itself should add a cast instruction where needed.

Then I compiled the same interfaces/classes using javac 7. When I disassembled the Test.class again, it was invoking the righ interface method and checking returned type using checkcast bytecode:

public class Test {
  public Test();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public static void main(java.lang.String[]) throws java.lang.Exception;
    Code:
       0: new           #2                  // class failure/ChildImpl
       3: dup
       4: invokespecial #3                  // Method failure/ChildImpl."<init>":()V
       7: astore_1
       8: aload_1
       9: invokeinterface #4,  1            // InterfaceMethod failure/Child.process:()Ljava/lang/Object;
      14: checkcast     #5                  // class java/lang/String
      17: astore_2
      18: getstatic     #6                  // Field java/lang/System.err:Ljava/io/PrintStream;
      21: aload_2
      22: invokevirtual #7                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
      25: return
}

See these two lines:

9: invokeinterface #4,  1            // InterfaceMethod failure/Child.process:()Ljava/lang/Object;
14: checkcast     #5                  // class java/lang/String

It was known that everything was OK before we cleaned up overridden interface methods from IQueue interface. To verify that, I added an overriding method declaration to Child interface, which then became:

public interface Child<T> extends ParentA<T>, ParentB<T> {
    T process() throws Exception;
}

As I was expecting this version generated the correct bytecode and worked fine using java 8 too.

To prevent this issue happening for our java 8 users, we added back some of the overriding generic methods to the IQueue interface with commit 81581a2d11dfcc535c2549a222a8dbd054f8669d. Now our IQueue looks like:

public interface IQueue<E> extends BlockingQueue<E>, BaseQueue<E>, ICollection<E> {

    /*
     * Added poll(), poll(long timeout, TimeUnit unit) and take()
     * methods here to prevent wrong method return type issue when
     * compiled with java 8.
     *
     * For additional details see;
     *
     * http://mail.openjdk.java.net/pipermail/compiler-dev/2014-November/009139.html
     * https://bugs.openjdk.java.net/browse/JDK-8064803
     *
     */

    E poll();

    E poll(long timeout, TimeUnit unit) throws InterruptedException;

    E take() throws InterruptedException;

    /**
     * Returns LocalQueueStats for this queue.
     * LocalQueueStats is the statistics for the local portion of this
     * queue.
     *
     * @return this queue's local statistics.
     */
    LocalQueueStats getLocalQueueStats();
}

I also reported this issue to compiler-dev mailgroup. See http://mail.openjdk.java.net/pipermail/compiler-dev/2014-November/009139.html. My report was replied back by Maurizio Cimadamore promptly. It was definitely a javac bug and he filed an issue on openjdk issue tracker, JDK-8064803. Issue was solved in a few days for JDK-9 target but sadly this bug still exists in javac 8 as of version java 1.8.0_31.