The mentioned JCK-test doesn't really hang - it just needs quadratic time now. This new behavior is caused by the Compressed Oops change ("6420645: Create a VM that uses compressed oops for up to 32GB heapsizes" (http://hg.openjdk.java.net/jdk7/jdk7/hotspot/rev/ba764ed4b6f2)) which changed the sorting algorithm for class methods from the default quicksort implementation to bubblesort. Before this change, bubblesort was only used for classes with less than 8 methods, after the changed it is always used if CompressedOops is turned on. The mentioned JCK-Test loads a class with 65535 methods which exceeds the default time frame of 10min for the test.

The solution is change the 'method_compare' function in methodOop.cpp to handle compressed oops and use the quicksort algorithm for all classes with more than 8 methods in 'methodOopDesc::sort_methods'.

The enclosed test is not suitable for automatic testing. It can be used as an evidence during manual testing. The test repeatedly generates classes with 6, 60, 600, 6000 and 60000 methods and loads them in a custom class loader. If the time ratio between the time required to load the 6000 and the 60000 methods classes exceeds 80 times, an exception is thrown which indicates that class loading probably requires quadratic time with regard to the number of class methods.

Following some measurements which point out the time ratio between the 6000 and 60000 methods case on various platforms with and without compressed oops:

             | +CompressedOops | -CompressedOops | +CompressedOops with fix |
-------------|-----------------|-----------------|--------------------------|
linux/x86_64 |      375x       |        18x      |            15x           |
             |                 |                 |                          |
sol/sparcv9  |      568x       |        17x      |            18x           |