| 212 |
|
* The table is resized when occupancy exceeds a percentage |
| 213 |
|
* threshold (nominally, 0.75, but see below). Any thread |
| 214 |
|
* noticing an overfull bin may assist in resizing after the |
| 215 |
< |
* initiating thread allocates and sets up the replacement |
| 216 |
< |
* array. However, rather than stalling, these other threads may |
| 217 |
< |
* proceed with insertions etc. The use of TreeBins shields us |
| 218 |
< |
* from the worst case effects of overfilling while resizes are in |
| 215 |
> |
* initiating thread allocates and sets up the replacement array. |
| 216 |
> |
* However, rather than stalling, these other threads may proceed |
| 217 |
> |
* with insertions etc. The use of TreeBins shields us from the |
| 218 |
> |
* worst case effects of overfilling while resizes are in |
| 219 |
|
* progress. Resizing proceeds by transferring bins, one by one, |
| 220 |
< |
* from the table to the next table. To enable concurrency, the |
| 221 |
< |
* next table must be (incrementally) prefilled with place-holders |
| 222 |
< |
* serving as reverse forwarders to the old table. Because we are |
| 223 |
< |
* using power-of-two expansion, the elements from each bin must |
| 224 |
< |
* either stay at same index, or move with a power of two |
| 225 |
< |
* offset. We eliminate unnecessary node creation by catching |
| 226 |
< |
* cases where old nodes can be reused because their next fields |
| 227 |
< |
* won't change. On average, only about one-sixth of them need |
| 228 |
< |
* cloning when a table doubles. The nodes they replace will be |
| 229 |
< |
* garbage collectable as soon as they are no longer referenced by |
| 230 |
< |
* any reader thread that may be in the midst of concurrently |
| 231 |
< |
* traversing table. Upon transfer, the old table bin contains |
| 232 |
< |
* only a special forwarding node (with hash field "MOVED") that |
| 233 |
< |
* contains the next table as its key. On encountering a |
| 234 |
< |
* forwarding node, access and update operations restart, using |
| 235 |
< |
* the new table. |
| 220 |
> |
* from the table to the next table. However, threads claim small |
| 221 |
> |
* blocks of indices to transfer (via field transferIndex) before |
| 222 |
> |
* doing so, reducing contention. Because we are using |
| 223 |
> |
* power-of-two expansion, the elements from each bin must either |
| 224 |
> |
* stay at same index, or move with a power of two offset. We |
| 225 |
> |
* eliminate unnecessary node creation by catching cases where old |
| 226 |
> |
* nodes can be reused because their next fields won't change. On |
| 227 |
> |
* average, only about one-sixth of them need cloning when a table |
| 228 |
> |
* doubles. The nodes they replace will be garbage collectable as |
| 229 |
> |
* soon as they are no longer referenced by any reader thread that |
| 230 |
> |
* may be in the midst of concurrently traversing table. Upon |
| 231 |
> |
* transfer, the old table bin contains only a special forwarding |
| 232 |
> |
* node (with hash field "MOVED") that contains the next table as |
| 233 |
> |
* its key. On encountering a forwarding node, access and update |
| 234 |
> |
* operations restart, using the new table. |
| 235 |
|
* |
| 236 |
|
* Each bin transfer requires its bin lock, which can stall |
| 237 |
|
* waiting for locks while resizing. However, because other |
| 239 |
|
* locks, average aggregate waits become shorter as resizing |
| 240 |
|
* progresses. The transfer operation must also ensure that all |
| 241 |
|
* accessible bins in both the old and new table are usable by any |
| 242 |
< |
* traversal. This is arranged by proceeding from the last bin |
| 243 |
< |
* (table.length - 1) up towards the first. Upon seeing a |
| 244 |
< |
* forwarding node, traversals (see class Traverser) arrange to |
| 245 |
< |
* move to the new table without revisiting nodes. However, to |
| 246 |
< |
* ensure that no intervening nodes are skipped, bin splitting can |
| 247 |
< |
* only begin after the associated reverse-forwarders are in |
| 248 |
< |
* place. |
| 242 |
> |
* traversal. This is arranged in part by proceeding from the |
| 243 |
> |
* last bin (table.length - 1) up towards the first. Upon seeing |
| 244 |
> |
* a forwarding node, traversals (see class Traverser) arrange to |
| 245 |
> |
* move to the new table without revisiting nodes. To ensure that |
| 246 |
> |
* no intervening nodes are skipped even when moved out of order, |
| 247 |
> |
* a stack (see class TableStack) is created on first encounter of |
| 248 |
> |
* a forwarding node during a traversal, to maintain its place if |
| 249 |
> |
* later processing the current table. The need for these |
| 250 |
> |
* save/restore mechanics is relatively rare, but when one |
| 251 |
> |
* forwarding node is encountered, typically many more will be. |
| 252 |
> |
* So Traversers use a simple caching scheme to avoid creating so |
| 253 |
> |
* many new TableStack nodes. (Thanks to Peter Levart for |
| 254 |
> |
* suggesting use of a stack here.) |
| 255 |
|
* |
| 256 |
|
* The traversal scheme also applies to partial traversals of |
| 257 |
|
* ranges of bins (via an alternate Traverser constructor) |
| 324 |
|
* unused "Segment" class that is instantiated in minimal form |
| 325 |
|
* only when serializing. |
| 326 |
|
* |
| 327 |
+ |
* Also, solely for compatibility with previous versions of this |
| 328 |
+ |
* class, it extends AbstractMap, even though all of its methods |
| 329 |
+ |
* are overridden, so it is just useless baggage. |
| 330 |
+ |
* |
| 331 |
|
* This file is organized to make things a little easier to follow |
| 332 |
|
* while reading than they might otherwise: First the main static |
| 333 |
|
* declarations and utilities, then fields, then main public |
| 336 |
|
* bulk operations. |
| 337 |
|
*/ |
| 338 |
|
|
| 339 |
+ |
|
| 340 |
|
/* ---------------- Constants -------------- */ |
| 341 |
|
|
| 342 |
|
/** |
| 620 |
|
private transient volatile int transferIndex; |
| 621 |
|
|
| 622 |
|
/** |
| 613 |
– |
* The least available table index to split while resizing. |
| 614 |
– |
*/ |
| 615 |
– |
private transient volatile int transferOrigin; |
| 616 |
– |
|
| 617 |
– |
/** |
| 623 |
|
* Spinlock (locked via CAS) used when resizing and/or creating CounterCells. |
| 624 |
|
*/ |
| 625 |
|
private transient volatile int cellsBusy; |
| 1597 |
|
while (s >= (long)(sc = sizeCtl) && (tab = table) != null && |
| 1598 |
|
tab.length < MAXIMUM_CAPACITY) { |
| 1599 |
|
if (sc < 0) { |
| 1600 |
< |
if (sc == -1 || transferIndex <= transferOrigin || |
| 1600 |
> |
if (sc == -1 || transferIndex <= 0 || |
| 1601 |
|
(nt = nextTable) == null) |
| 1602 |
|
break; |
| 1603 |
|
if (U.compareAndSwapInt(this, SIZECTL, sc, sc - 1)) |
| 1617 |
|
Node<K,V>[] nextTab; int sc; |
| 1618 |
|
if ((f instanceof ForwardingNode) && |
| 1619 |
|
(nextTab = ((ForwardingNode<K,V>)f).nextTable) != null) { |
| 1620 |
< |
if (nextTab == nextTable && tab == table && |
| 1621 |
< |
transferIndex > transferOrigin && (sc = sizeCtl) < -1 && |
| 1622 |
< |
U.compareAndSwapInt(this, SIZECTL, sc, sc - 1)) |
| 1623 |
< |
transfer(tab, nextTab); |
| 1620 |
> |
while (transferIndex > 0 && nextTab == nextTable && |
| 1621 |
> |
(sc = sizeCtl) < -1) { |
| 1622 |
> |
if (U.compareAndSwapInt(this, SIZECTL, sc, sc - 1)) { |
| 1623 |
> |
transfer(tab, nextTab); |
| 1624 |
> |
break; |
| 1625 |
> |
} |
| 1626 |
> |
} |
| 1627 |
|
return nextTab; |
| 1628 |
|
} |
| 1629 |
|
return table; |
| 1674 |
|
if (nextTab == null) { // initiating |
| 1675 |
|
try { |
| 1676 |
|
@SuppressWarnings("unchecked") |
| 1677 |
< |
Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n << 1]; |
| 1677 |
> |
Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n << 1]; |
| 1678 |
|
nextTab = nt; |
| 1679 |
|
} catch (Throwable ex) { // try to cope with OOME |
| 1680 |
|
sizeCtl = Integer.MAX_VALUE; |
| 1681 |
|
return; |
| 1682 |
|
} |
| 1683 |
|
nextTable = nextTab; |
| 1676 |
– |
transferOrigin = n; |
| 1684 |
|
transferIndex = n; |
| 1678 |
– |
ForwardingNode<K,V> rev = new ForwardingNode<K,V>(tab); |
| 1679 |
– |
for (int k = n; k > 0;) { // progressively reveal ready slots |
| 1680 |
– |
int nextk = (k > stride) ? k - stride : 0; |
| 1681 |
– |
for (int m = nextk; m < k; ++m) |
| 1682 |
– |
nextTab[m] = rev; |
| 1683 |
– |
for (int m = n + nextk; m < n + k; ++m) |
| 1684 |
– |
nextTab[m] = rev; |
| 1685 |
– |
U.putOrderedInt(this, TRANSFERORIGIN, k = nextk); |
| 1686 |
– |
} |
| 1685 |
|
} |
| 1686 |
|
int nextn = nextTab.length; |
| 1687 |
|
ForwardingNode<K,V> fwd = new ForwardingNode<K,V>(nextTab); |
| 1688 |
|
boolean advance = true; |
| 1689 |
+ |
boolean finishing = false; // to ensure sweep before committing nextTab |
| 1690 |
|
for (int i = 0, bound = 0;;) { |
| 1691 |
< |
int nextIndex, nextBound, fh; Node<K,V> f; |
| 1691 |
> |
Node<K,V> f; int fh; |
| 1692 |
|
while (advance) { |
| 1693 |
< |
if (--i >= bound) |
| 1693 |
> |
int nextIndex, nextBound; |
| 1694 |
> |
if (--i >= bound || finishing) |
| 1695 |
|
advance = false; |
| 1696 |
< |
else if ((nextIndex = transferIndex) <= transferOrigin) { |
| 1696 |
> |
else if ((nextIndex = transferIndex) <= 0) { |
| 1697 |
|
i = -1; |
| 1698 |
|
advance = false; |
| 1699 |
|
} |
| 1707 |
|
} |
| 1708 |
|
} |
| 1709 |
|
if (i < 0 || i >= n || i + n >= nextn) { |
| 1710 |
< |
for (int sc;;) { |
| 1711 |
< |
if (U.compareAndSwapInt(this, SIZECTL, sc = sizeCtl, ++sc)) { |
| 1712 |
< |
if (sc == -1) { |
| 1713 |
< |
nextTable = null; |
| 1714 |
< |
table = nextTab; |
| 1715 |
< |
sizeCtl = (n << 1) - (n >>> 1); |
| 1716 |
< |
} |
| 1717 |
< |
return; |
| 1718 |
< |
} |
| 1710 |
> |
int sc; |
| 1711 |
> |
if (finishing) { |
| 1712 |
> |
nextTable = null; |
| 1713 |
> |
table = nextTab; |
| 1714 |
> |
sizeCtl = (n << 1) - (n >>> 1); |
| 1715 |
> |
return; |
| 1716 |
|
} |
| 1717 |
< |
} |
| 1718 |
< |
else if ((f = tabAt(tab, i)) == null) { |
| 1719 |
< |
if (casTabAt(tab, i, null, fwd)) { |
| 1720 |
< |
setTabAt(nextTab, i, null); |
| 1721 |
< |
setTabAt(nextTab, i + n, null); |
| 1725 |
< |
advance = true; |
| 1717 |
> |
if (U.compareAndSwapInt(this, SIZECTL, sc = sizeCtl, ++sc)) { |
| 1718 |
> |
if (sc != -1) |
| 1719 |
> |
return; |
| 1720 |
> |
finishing = advance = true; |
| 1721 |
> |
i = n; // recheck before commit |
| 1722 |
|
} |
| 1723 |
|
} |
| 1724 |
+ |
else if ((f = tabAt(tab, i)) == null) |
| 1725 |
+ |
advance = casTabAt(tab, i, null, fwd); |
| 1726 |
|
else if ((fh = f.hash) == MOVED) |
| 1727 |
|
advance = true; // already processed |
| 1728 |
|
else { |
| 1754 |
|
else |
| 1755 |
|
hn = new Node<K,V>(ph, pk, pv, hn); |
| 1756 |
|
} |
| 1757 |
+ |
setTabAt(nextTab, i, ln); |
| 1758 |
+ |
setTabAt(nextTab, i + n, hn); |
| 1759 |
+ |
setTabAt(tab, i, fwd); |
| 1760 |
+ |
advance = true; |
| 1761 |
|
} |
| 1762 |
|
else if (f instanceof TreeBin) { |
| 1763 |
|
TreeBin<K,V> t = (TreeBin<K,V>)f; |
| 1789 |
|
(hc != 0) ? new TreeBin<K,V>(lo) : t; |
| 1790 |
|
hn = (hc <= UNTREEIFY_THRESHOLD) ? untreeify(hi) : |
| 1791 |
|
(lc != 0) ? new TreeBin<K,V>(hi) : t; |
| 1792 |
+ |
setTabAt(nextTab, i, ln); |
| 1793 |
+ |
setTabAt(nextTab, i + n, hn); |
| 1794 |
+ |
setTabAt(tab, i, fwd); |
| 1795 |
+ |
advance = true; |
| 1796 |
|
} |
| 1791 |
– |
else |
| 1792 |
– |
ln = hn = null; |
| 1793 |
– |
setTabAt(nextTab, i, ln); |
| 1794 |
– |
setTabAt(nextTab, i + n, hn); |
| 1795 |
– |
setTabAt(tab, i, fwd); |
| 1796 |
– |
advance = true; |
| 1797 |
|
} |
| 1798 |
|
} |
| 1799 |
|
} |
| 2466 |
|
/* ----------------Table Traversal -------------- */ |
| 2467 |
|
|
| 2468 |
|
/** |
| 2469 |
+ |
* Records the table, its length, and current traversal index for a |
| 2470 |
+ |
* traverser that must process a region of a forwarded table before |
| 2471 |
+ |
* proceeding with current table. |
| 2472 |
+ |
*/ |
| 2473 |
+ |
static final class TableStack<K,V> { |
| 2474 |
+ |
int length; |
| 2475 |
+ |
int index; |
| 2476 |
+ |
Node<K,V>[] tab; |
| 2477 |
+ |
TableStack<K,V> next; |
| 2478 |
+ |
} |
| 2479 |
+ |
|
| 2480 |
+ |
/** |
| 2481 |
|
* Encapsulates traversal for methods such as containsValue; also |
| 2482 |
< |
* serves as a base class for other iterators. |
| 2482 |
> |
* serves as a base class for other iterators and spliterators. |
| 2483 |
|
* |
| 2484 |
|
* Method advance visits once each still-valid node that was |
| 2485 |
|
* reachable upon iterator construction. It might miss some that |
| 2501 |
|
static class Traverser<K,V> { |
| 2502 |
|
Node<K,V>[] tab; // current table; updated if resized |
| 2503 |
|
Node<K,V> next; // the next entry to use |
| 2504 |
+ |
TableStack<K,V> stack, spare; // to save/restore on ForwardingNodes |
| 2505 |
|
int index; // index of bin to use next |
| 2506 |
|
int baseIndex; // current index of initial table |
| 2507 |
|
int baseLimit; // index bound for initial table |
| 2523 |
|
if ((e = next) != null) |
| 2524 |
|
e = e.next; |
| 2525 |
|
for (;;) { |
| 2526 |
< |
Node<K,V>[] t; int i, n; K ek; // must use locals in checks |
| 2526 |
> |
Node<K,V>[] t; int i, n; // must use locals in checks |
| 2527 |
|
if (e != null) |
| 2528 |
|
return next = e; |
| 2529 |
|
if (baseIndex >= baseLimit || (t = tab) == null || |
| 2530 |
|
(n = t.length) <= (i = index) || i < 0) |
| 2531 |
|
return next = null; |
| 2532 |
< |
if ((e = tabAt(t, index)) != null && e.hash < 0) { |
| 2532 |
> |
if ((e = tabAt(t, i)) != null && e.hash < 0) { |
| 2533 |
|
if (e instanceof ForwardingNode) { |
| 2534 |
|
tab = ((ForwardingNode<K,V>)e).nextTable; |
| 2535 |
|
e = null; |
| 2536 |
+ |
pushState(t, i, n); |
| 2537 |
|
continue; |
| 2538 |
|
} |
| 2539 |
|
else if (e instanceof TreeBin) |
| 2541 |
|
else |
| 2542 |
|
e = null; |
| 2543 |
|
} |
| 2544 |
< |
if ((index += baseSize) >= n) |
| 2545 |
< |
index = ++baseIndex; // visit upper slots if present |
| 2544 |
> |
if (stack != null) |
| 2545 |
> |
recoverState(n); |
| 2546 |
> |
else if ((index = i + baseSize) >= n) |
| 2547 |
> |
index = ++baseIndex; // visit upper slots if present |
| 2548 |
> |
} |
| 2549 |
> |
} |
| 2550 |
> |
|
| 2551 |
> |
/** |
| 2552 |
> |
* Saves traversal state upon encountering a forwarding node. |
| 2553 |
> |
*/ |
| 2554 |
> |
private void pushState(Node<K,V>[] t, int i, int n) { |
| 2555 |
> |
TableStack<K,V> s = spare; // reuse if possible |
| 2556 |
> |
if (s != null) |
| 2557 |
> |
spare = s.next; |
| 2558 |
> |
else |
| 2559 |
> |
s = new TableStack<K,V>(); |
| 2560 |
> |
s.tab = t; |
| 2561 |
> |
s.length = n; |
| 2562 |
> |
s.index = i; |
| 2563 |
> |
s.next = stack; |
| 2564 |
> |
stack = s; |
| 2565 |
> |
} |
| 2566 |
> |
|
| 2567 |
> |
/** |
| 2568 |
> |
* Possibly pops traversal state. |
| 2569 |
> |
* |
| 2570 |
> |
* @param n length of current table |
| 2571 |
> |
*/ |
| 2572 |
> |
private void recoverState(int n) { |
| 2573 |
> |
TableStack<K,V> s; int len; |
| 2574 |
> |
while ((s = stack) != null && (index += (len = s.length)) >= n) { |
| 2575 |
> |
n = len; |
| 2576 |
> |
index = s.index; |
| 2577 |
> |
tab = s.tab; |
| 2578 |
> |
s.tab = null; |
| 2579 |
> |
TableStack<K,V> next = s.next; |
| 2580 |
> |
s.next = spare; // save for reuse |
| 2581 |
> |
stack = next; |
| 2582 |
> |
spare = s; |
| 2583 |
|
} |
| 2584 |
+ |
if (s == null && (index += baseSize) >= n) |
| 2585 |
+ |
index = ++baseIndex; |
| 2586 |
|
} |
| 2587 |
|
} |
| 2588 |
|
|
| 3238 |
|
private static final sun.misc.Unsafe U; |
| 3239 |
|
private static final long SIZECTL; |
| 3240 |
|
private static final long TRANSFERINDEX; |
| 3188 |
– |
private static final long TRANSFERORIGIN; |
| 3241 |
|
private static final long BASECOUNT; |
| 3242 |
|
private static final long CELLSBUSY; |
| 3243 |
|
private static final long CELLVALUE; |
| 3252 |
|
(k.getDeclaredField("sizeCtl")); |
| 3253 |
|
TRANSFERINDEX = U.objectFieldOffset |
| 3254 |
|
(k.getDeclaredField("transferIndex")); |
| 3203 |
– |
TRANSFERORIGIN = U.objectFieldOffset |
| 3204 |
– |
(k.getDeclaredField("transferOrigin")); |
| 3255 |
|
BASECOUNT = U.objectFieldOffset |
| 3256 |
|
(k.getDeclaredField("baseCount")); |
| 3257 |
|
CELLSBUSY = U.objectFieldOffset |
