Run the JVM with -XX:MaxHeapSize=512m (or any big number as you need) (or -Xmx512m for short)
This can also be caused by setting something too large on a 32-bit HotSpot vm, for example:
-Xms1536m -Xmx1536m
where this might/would work:
-Xms1336m -Xmx1336m