ViewVC Help
View File | Revision Log | Show Annotations | Download File | Root Listing
root/jsr166/jsr166/src/jsr166y/LinkedTransferQueue.java
(Generate patch)

Comparing jsr166/src/jsr166y/LinkedTransferQueue.java (file contents):
Revision 1.52 by dl, Sat Oct 24 14:57:32 2009 UTC vs.
Revision 1.77 by jsr166, Wed Sep 1 23:40:29 2010 UTC

# Line 15 | Line 15 | import java.util.Iterator;
15   import java.util.NoSuchElementException;
16   import java.util.Queue;
17   import java.util.concurrent.locks.LockSupport;
18 +
19   /**
20   * An unbounded {@link TransferQueue} based on linked nodes.
21   * This queue orders elements FIFO (first-in-first-out) with respect
# Line 206 | Line 207 | public class LinkedTransferQueue<E> exte
207       * additional GC bookkeeping ("write barriers") that are sometimes
208       * more costly than the writes themselves because of contention).
209       *
209     * Removal of interior nodes (due to timed out or interrupted
210     * waits, or calls to remove(x) or Iterator.remove) can use a
211     * scheme roughly similar to that described in Scherer, Lea, and
212     * Scott's SynchronousQueue. Given a predecessor, we can unsplice
213     * any node except the (actual) tail of the queue. To avoid
214     * build-up of cancelled trailing nodes, upon a request to remove
215     * a trailing node, it is placed in field "cleanMe" to be
216     * unspliced upon the next call to unsplice any other node.
217     * Situations needing such mechanics are not common but do occur
218     * in practice; for example when an unbounded series of short
219     * timed calls to poll repeatedly time out but never otherwise
220     * fall off the list because of an untimed call to take at the
221     * front of the queue. Note that maintaining field cleanMe does
222     * not otherwise much impact garbage retention even if never
223     * cleared by some other call because the held node will
224     * eventually either directly or indirectly lead to a self-link
225     * once off the list.
226     *
210       * *** Overview of implementation ***
211       *
212       * We use a threshold-based approach to updates, with a slack
# Line 239 | Line 222 | public class LinkedTransferQueue<E> exte
222       * per-thread one available, but even ThreadLocalRandom is too
223       * heavy for these purposes.
224       *
225 <     * With such a small slack threshold value, it is rarely
226 <     * worthwhile to augment this with path short-circuiting; i.e.,
227 <     * unsplicing nodes between head and the first unmatched node, or
228 <     * similarly for tail, rather than advancing head or tail
246 <     * proper. However, it is used (in awaitMatch) immediately before
247 <     * a waiting thread starts to block, as a final bit of helping at
248 <     * a point when contention with others is extremely unlikely
249 <     * (since if other threads that could release it are operating,
250 <     * then the current thread wouldn't be blocking).
225 >     * With such a small slack threshold value, it is not worthwhile
226 >     * to augment this with path short-circuiting (i.e., unsplicing
227 >     * interior nodes) except in the case of cancellation/removal (see
228 >     * below).
229       *
230       * We allow both the head and tail fields to be null before any
231       * nodes are enqueued; initializing upon first append.  This
# Line 329 | Line 307 | public class LinkedTransferQueue<E> exte
307       *    versa) compared to their predecessors receive additional
308       *    chained spins, reflecting longer paths typically required to
309       *    unblock threads during phase changes.
310 +     *
311 +     *
312 +     * ** Unlinking removed interior nodes **
313 +     *
314 +     * In addition to minimizing garbage retention via self-linking
315 +     * described above, we also unlink removed interior nodes. These
316 +     * may arise due to timed out or interrupted waits, or calls to
317 +     * remove(x) or Iterator.remove.  Normally, given a node that was
318 +     * at one time known to be the predecessor of some node s that is
319 +     * to be removed, we can unsplice s by CASing the next field of
320 +     * its predecessor if it still points to s (otherwise s must
321 +     * already have been removed or is now offlist). But there are two
322 +     * situations in which we cannot guarantee to make node s
323 +     * unreachable in this way: (1) If s is the trailing node of list
324 +     * (i.e., with null next), then it is pinned as the target node
325 +     * for appends, so can only be removed later after other nodes are
326 +     * appended. (2) We cannot necessarily unlink s given a
327 +     * predecessor node that is matched (including the case of being
328 +     * cancelled): the predecessor may already be unspliced, in which
329 +     * case some previous reachable node may still point to s.
330 +     * (For further explanation see Herlihy & Shavit "The Art of
331 +     * Multiprocessor Programming" chapter 9).  Although, in both
332 +     * cases, we can rule out the need for further action if either s
333 +     * or its predecessor are (or can be made to be) at, or fall off
334 +     * from, the head of list.
335 +     *
336 +     * Without taking these into account, it would be possible for an
337 +     * unbounded number of supposedly removed nodes to remain
338 +     * reachable.  Situations leading to such buildup are uncommon but
339 +     * can occur in practice; for example when a series of short timed
340 +     * calls to poll repeatedly time out but never otherwise fall off
341 +     * the list because of an untimed call to take at the front of the
342 +     * queue.
343 +     *
344 +     * When these cases arise, rather than always retraversing the
345 +     * entire list to find an actual predecessor to unlink (which
346 +     * won't help for case (1) anyway), we record a conservative
347 +     * estimate of possible unsplice failures (in "sweepVotes").
348 +     * We trigger a full sweep when the estimate exceeds a threshold
349 +     * ("SWEEP_THRESHOLD") indicating the maximum number of estimated
350 +     * removal failures to tolerate before sweeping through, unlinking
351 +     * cancelled nodes that were not unlinked upon initial removal.
352 +     * We perform sweeps by the thread hitting threshold (rather than
353 +     * background threads or by spreading work to other threads)
354 +     * because in the main contexts in which removal occurs, the
355 +     * caller is already timed-out, cancelled, or performing a
356 +     * potentially O(n) operation (e.g. remove(x)), none of which are
357 +     * time-critical enough to warrant the overhead that alternatives
358 +     * would impose on other threads.
359 +     *
360 +     * Because the sweepVotes estimate is conservative, and because
361 +     * nodes become unlinked "naturally" as they fall off the head of
362 +     * the queue, and because we allow votes to accumulate even while
363 +     * sweeps are in progress, there are typically significantly fewer
364 +     * such nodes than estimated.  Choice of a threshold value
365 +     * balances the likelihood of wasted effort and contention, versus
366 +     * providing a worst-case bound on retention of interior nodes in
367 +     * quiescent queues. The value defined below was chosen
368 +     * empirically to balance these under various timeout scenarios.
369 +     *
370 +     * Note that we cannot self-link unlinked interior nodes during
371 +     * sweeps. However, the associated garbage chains terminate when
372 +     * some successor ultimately falls off the head of the list and is
373 +     * self-linked.
374       */
375  
376      /** True if on multiprocessor */
# Line 355 | Line 397 | public class LinkedTransferQueue<E> exte
397      private static final int CHAINED_SPINS = FRONT_SPINS >>> 1;
398  
399      /**
400 +     * The maximum number of estimated removal failures (sweepVotes)
401 +     * to tolerate before sweeping through the queue unlinking
402 +     * cancelled nodes that were not unlinked upon initial
403 +     * removal. See above for explanation. The value must be at least
404 +     * two to avoid useless sweeps when removing trailing nodes.
405 +     */
406 +    static final int SWEEP_THRESHOLD = 32;
407 +
408 +    /**
409       * Queue nodes. Uses Object, not E, for items to allow forgetting
410       * them after use.  Relies heavily on Unsafe mechanics to minimize
411 <     * unnecessary ordering constraints: Writes that intrinsically
412 <     * precede or follow CASes use simple relaxed forms.  Other
362 <     * cleanups use releasing/lazy writes.
411 >     * unnecessary ordering constraints: Writes that are intrinsically
412 >     * ordered wrt other accesses or CASes use simple relaxed forms.
413       */
414      static final class Node {
415          final boolean isData;   // false if this is a request node
# Line 373 | Line 423 | public class LinkedTransferQueue<E> exte
423          }
424  
425          final boolean casItem(Object cmp, Object val) {
426 +            //            assert cmp == null || cmp.getClass() != Node.class;
427              return UNSAFE.compareAndSwapObject(this, itemOffset, cmp, val);
428          }
429  
430          /**
431 <         * Creates a new node. Uses relaxed write because item can only
432 <         * be seen if followed by CAS.
431 >         * Constructs a new node.  Uses relaxed write because item can
432 >         * only be seen after publication via casNext.
433           */
434          Node(Object item, boolean isData) {
435              UNSAFE.putObject(this, itemOffset, item); // relaxed write
# Line 394 | Line 445 | public class LinkedTransferQueue<E> exte
445          }
446  
447          /**
448 <         * Sets item to self (using a releasing/lazy write) and waiter
449 <         * to null, to avoid garbage retention after extracting or
450 <         * cancelling.
448 >         * Sets item to self and waiter to null, to avoid garbage
449 >         * retention after matching or cancelling. Uses relaxed writes
450 >         * because order is already constrained in the only calling
451 >         * contexts: item is forgotten only after volatile/atomic
452 >         * mechanics that extract items.  Similarly, clearing waiter
453 >         * follows either CAS or return from park (if ever parked;
454 >         * else we don't care).
455           */
456          final void forgetContents() {
457 <            UNSAFE.putOrderedObject(this, itemOffset, this);
458 <            UNSAFE.putOrderedObject(this, waiterOffset, null);
457 >            UNSAFE.putObject(this, itemOffset, this);
458 >            UNSAFE.putObject(this, waiterOffset, null);
459          }
460  
461          /**
# Line 409 | Line 464 | public class LinkedTransferQueue<E> exte
464           */
465          final boolean isMatched() {
466              Object x = item;
467 <            return x == this || (x != null) != isData;
467 >            return (x == this) || ((x == null) == isData);
468 >        }
469 >
470 >        /**
471 >         * Returns true if this is an unmatched request node.
472 >         */
473 >        final boolean isUnmatchedRequest() {
474 >            return !isData && item == null;
475          }
476  
477          /**
# Line 427 | Line 489 | public class LinkedTransferQueue<E> exte
489           * Tries to artificially match a data node -- used by remove.
490           */
491          final boolean tryMatchData() {
492 +            //            assert isData;
493              Object x = item;
494              if (x != null && x != this && casItem(x, null)) {
495                  LockSupport.unpark(waiter);
# Line 448 | Line 511 | public class LinkedTransferQueue<E> exte
511      }
512  
513      /** head of the queue; null until first enqueue */
514 <    private transient volatile Node head;
452 <
453 <    /** predecessor of dangling unspliceable node */
454 <    private transient volatile Node cleanMe; // decl here to reduce contention
514 >    transient volatile Node head;
515  
516      /** tail of the queue; null until first append */
517      private transient volatile Node tail;
518  
519 +    /** The number of apparent failures to unsplice removed nodes */
520 +    private transient volatile int sweepVotes;
521 +
522      // CAS methods for fields
523      private boolean casTail(Node cmp, Node val) {
524          return UNSAFE.compareAndSwapObject(this, tailOffset, cmp, val);
# Line 465 | Line 528 | public class LinkedTransferQueue<E> exte
528          return UNSAFE.compareAndSwapObject(this, headOffset, cmp, val);
529      }
530  
531 <    private boolean casCleanMe(Node cmp, Node val) {
532 <        return UNSAFE.compareAndSwapObject(this, cleanMeOffset, cmp, val);
531 >    private boolean casSweepVotes(int cmp, int val) {
532 >        return UNSAFE.compareAndSwapInt(this, sweepVotesOffset, cmp, val);
533      }
534  
535      /*
536 <     * Possible values for "how" argument in xfer method. Beware that
474 <     * the order of assigned numerical values matters.
536 >     * Possible values for "how" argument in xfer method.
537       */
538 <    private static final int NOW     = 0; // for untimed poll, tryTransfer
539 <    private static final int ASYNC   = 1; // for offer, put, add
540 <    private static final int SYNC    = 2; // for transfer, take
541 <    private static final int TIMEOUT = 3; // for timed poll, tryTransfer
538 >    private static final int NOW   = 0; // for untimed poll, tryTransfer
539 >    private static final int ASYNC = 1; // for offer, put, add
540 >    private static final int SYNC  = 2; // for transfer, take
541 >    private static final int TIMED = 3; // for timed poll, tryTransfer
542 >
543 >    @SuppressWarnings("unchecked")
544 >    static <E> E cast(Object item) {
545 >        //        assert item == null || item.getClass() != Node.class;
546 >        return (E) item;
547 >    }
548  
549      /**
550       * Implements all queuing methods. See above for explanation.
551       *
552       * @param e the item or null for take
553       * @param haveData true if this is a put, else a take
554 <     * @param how NOW, ASYNC, SYNC, or TIMEOUT
555 <     * @param nanos timeout in nanosecs, used only if mode is TIMEOUT
554 >     * @param how NOW, ASYNC, SYNC, or TIMED
555 >     * @param nanos timeout in nanosecs, used only if mode is TIMED
556       * @return an item if matched, else e
557       * @throws NullPointerException if haveData mode but e is null
558       */
559 <    private Object xfer(Object e, boolean haveData, int how, long nanos) {
559 >    private E xfer(E e, boolean haveData, int how, long nanos) {
560          if (haveData && (e == null))
561              throw new NullPointerException();
562          Node s = null;                        // the node to append, if needed
# Line 503 | Line 571 | public class LinkedTransferQueue<E> exte
571                          break;
572                      if (p.casItem(item, e)) { // match
573                          for (Node q = p; q != h;) {
574 <                            Node n = q.next;  // update head by 2
575 <                            if (n != null)    // unless singleton
508 <                                q = n;
509 <                            if (head == h && casHead(h, q)) {
574 >                            Node n = q.next;  // update by 2 unless singleton
575 >                            if (head == h && casHead(h, n == null? q : n)) {
576                                  h.forgetNext();
577                                  break;
578                              }                 // advance and retry
# Line 515 | Line 581 | public class LinkedTransferQueue<E> exte
581                                  break;        // unless slack < 2
582                          }
583                          LockSupport.unpark(p.waiter);
584 <                        return item;
584 >                        return this.<E>cast(item);
585                      }
586                  }
587                  Node n = p.next;
588                  p = (p != n) ? n : (h = head); // Use head if p offlist
589              }
590  
591 <            if (how >= ASYNC) {               // No matches available
591 >            if (how != NOW) {                 // No matches available
592                  if (s == null)
593                      s = new Node(e, haveData);
594                  Node pred = tryAppend(s, haveData);
595                  if (pred == null)
596                      continue retry;           // lost race vs opposite mode
597 <                if (how >= SYNC)
598 <                    return awaitMatch(s, pred, e, how, nanos);
597 >                if (how != ASYNC)
598 >                    return awaitMatch(s, pred, e, (how == TIMED), nanos);
599              }
600              return e; // not waiting
601          }
# Line 545 | Line 611 | public class LinkedTransferQueue<E> exte
611       * predecessor
612       */
613      private Node tryAppend(Node s, boolean haveData) {
614 <        for (Node t = tail, p = t;;) { // move p to last node and append
614 >        for (Node t = tail, p = t;;) {        // move p to last node and append
615              Node n, u;                        // temps for reads of next & tail
616              if (p == null && (p = head) == null) {
617                  if (casHead(null, s))
# Line 578 | Line 644 | public class LinkedTransferQueue<E> exte
644       * predecessor, or null if unknown (the null case does not occur
645       * in any current calls but may in possible future extensions)
646       * @param e the comparison value for checking match
647 <     * @param how either SYNC or TIMEOUT
648 <     * @param nanos timeout value
647 >     * @param timed if true, wait only until timeout elapses
648 >     * @param nanos timeout in nanosecs, used only if timed is true
649       * @return matched item, or e if unmatched on interrupt or timeout
650       */
651 <    private Object awaitMatch(Node s, Node pred, Object e,
652 <                              int how, long nanos) {
587 <        long lastTime = (how == TIMEOUT) ? System.nanoTime() : 0L;
651 >    private E awaitMatch(Node s, Node pred, E e, boolean timed, long nanos) {
652 >        long lastTime = timed ? System.nanoTime() : 0L;
653          Thread w = Thread.currentThread();
654          int spins = -1; // initialized after first item and cancel checks
655          ThreadLocalRandom randomYields = null; // bound if needed
# Line 592 | Line 657 | public class LinkedTransferQueue<E> exte
657          for (;;) {
658              Object item = s.item;
659              if (item != e) {                  // matched
660 +                //                assert item != s;
661                  s.forgetContents();           // avoid garbage
662 <                return item;
662 >                return this.<E>cast(item);
663              }
664 <            if ((w.isInterrupted() || (how == TIMEOUT && nanos <= 0)) &&
665 <                     s.casItem(e, s)) {       // cancel
664 >            if ((w.isInterrupted() || (timed && nanos <= 0)) &&
665 >                    s.casItem(e, s)) {        // cancel
666                  unsplice(pred, s);
667                  return e;
668              }
# Line 606 | Line 672 | public class LinkedTransferQueue<E> exte
672                      randomYields = ThreadLocalRandom.current();
673              }
674              else if (spins > 0) {             // spin
675 <                if (--spins == 0)
676 <                    shortenHeadPath();        // reduce slack before blocking
611 <                else if (randomYields.nextInt(CHAINED_SPINS) == 0)
675 >                --spins;
676 >                if (randomYields.nextInt(CHAINED_SPINS) == 0)
677                      Thread.yield();           // occasionally yield
678              }
679              else if (s.waiter == null) {
680                  s.waiter = w;                 // request unpark then recheck
681              }
682 <            else if (how == TIMEOUT) {
682 >            else if (timed) {
683                  long now = System.nanoTime();
684                  if ((nanos -= now - lastTime) > 0)
685                      LockSupport.parkNanos(this, nanos);
# Line 622 | Line 687 | public class LinkedTransferQueue<E> exte
687              }
688              else {
689                  LockSupport.park(this);
625                s.waiter = null;
626                spins = -1;                   // spin if front upon wakeup
690              }
691          }
692      }
# Line 644 | Line 707 | public class LinkedTransferQueue<E> exte
707          return 0;
708      }
709  
710 +    /* -------------- Traversal methods -------------- */
711 +
712      /**
713 <     * Tries (once) to unsplice nodes between head and first unmatched
714 <     * or trailing node; failing on contention.
715 <     */
716 <    private void shortenHeadPath() {
717 <        Node h, hn, p, q;
718 <        if ((p = h = head) != null && h.isMatched() &&
719 <            (q = hn = h.next) != null) {
655 <            Node n;
656 <            while ((n = q.next) != q) {
657 <                if (n == null || !q.isMatched()) {
658 <                    if (hn != q && h.next == hn)
659 <                        h.casNext(hn, q);
660 <                    break;
661 <                }
662 <                p = q;
663 <                q = n;
664 <            }
665 <        }
713 >     * Returns the successor of p, or the head node if p.next has been
714 >     * linked to self, which will only be true if traversing with a
715 >     * stale pointer that is now off the list.
716 >     */
717 >    final Node succ(Node p) {
718 >        Node next = p.next;
719 >        return (p == next) ? head : next;
720      }
721  
668    /* -------------- Traversal methods -------------- */
669
722      /**
723       * Returns the first unmatched node of the given mode, or null if
724       * none.  Used by methods isEmpty, hasWaitingConsumer.
725       */
726 <    private Node firstOfMode(boolean data) {
727 <        for (Node p = head; p != null; ) {
726 >    private Node firstOfMode(boolean isData) {
727 >        for (Node p = head; p != null; p = succ(p)) {
728              if (!p.isMatched())
729 <                return (p.isData == data) ? p : null;
678 <            Node n = p.next;
679 <            p = (n != p) ? n : head;
729 >                return (p.isData == isData) ? p : null;
730          }
731          return null;
732      }
733  
734      /**
735       * Returns the item in the first unmatched node with isData; or
736 <     * null if none. Used by peek.
736 >     * null if none.  Used by peek.
737       */
738 <    private Object firstDataItem() {
739 <        for (Node p = head; p != null; ) {
690 <            boolean isData = p.isData;
738 >    private E firstDataItem() {
739 >        for (Node p = head; p != null; p = succ(p)) {
740              Object item = p.item;
741 <            if (item != p && (item != null) == isData)
742 <                return isData ? item : null;
743 <            Node n = p.next;
744 <            p = (n != p) ? n : head;
741 >            if (p.isData) {
742 >                if (item != null && item != p)
743 >                    return this.<E>cast(item);
744 >            }
745 >            else if (item == null)
746 >                return null;
747          }
748          return null;
749      }
# Line 723 | Line 774 | public class LinkedTransferQueue<E> exte
774  
775      final class Itr implements Iterator<E> {
776          private Node nextNode;   // next node to return item for
777 <        private Object nextItem; // the corresponding item
777 >        private E nextItem;      // the corresponding item
778          private Node lastRet;    // last returned node, to support remove
779 +        private Node lastPred;   // predecessor to unlink lastRet
780  
781          /**
782           * Moves to next node after prev, or first node if prev null.
783           */
784          private void advance(Node prev) {
785 +            lastPred = lastRet;
786              lastRet = prev;
787 <            Node p;
788 <            if (prev == null || (p = prev.next) == prev)
736 <                p = head;
737 <            while (p != null) {
787 >            for (Node p = (prev == null) ? head : succ(prev);
788 >                 p != null; p = succ(p)) {
789                  Object item = p.item;
790                  if (p.isData) {
791                      if (item != null && item != p) {
792 <                        nextItem = item;
792 >                        nextItem = LinkedTransferQueue.this.<E>cast(item);
793                          nextNode = p;
794                          return;
795                      }
796                  }
797                  else if (item == null)
798                      break;
748                Node n = p.next;
749                p = (n != p) ? n : head;
799              }
800              nextNode = null;
801          }
# Line 762 | Line 811 | public class LinkedTransferQueue<E> exte
811          public final E next() {
812              Node p = nextNode;
813              if (p == null) throw new NoSuchElementException();
814 <            Object e = nextItem;
814 >            E e = nextItem;
815              advance(p);
816 <            return (E) e;
816 >            return e;
817          }
818  
819          public final void remove() {
820              Node p = lastRet;
821              if (p == null) throw new IllegalStateException();
822 <            lastRet = null;
823 <            findAndRemoveNode(p);
822 >            if (p.tryMatchData())
823 >                unsplice(lastPred, p);
824          }
825      }
826  
# Line 781 | Line 830 | public class LinkedTransferQueue<E> exte
830       * Unsplices (now or later) the given deleted/cancelled node with
831       * the given predecessor.
832       *
833 <     * @param pred predecessor of node to be unspliced
833 >     * @param pred a node that was at one time known to be the
834 >     * predecessor of s, or null or s itself if s is/was at head
835       * @param s the node to be unspliced
836       */
837 <    private void unsplice(Node pred, Node s) {
838 <        s.forgetContents(); // clear unneeded fields
837 >    final void unsplice(Node pred, Node s) {
838 >        s.forgetContents(); // forget unneeded fields
839          /*
840 <         * At any given time, exactly one node on list cannot be
841 <         * unlinked -- the last inserted node. To accommodate this, if
842 <         * we cannot unlink s, we save its predecessor as "cleanMe",
843 <         * processing the previously saved version first. Because only
844 <         * one node in the list can have a null next, at least one of
795 <         * node s or the node previously saved can always be
796 <         * processed, so this always terminates.
840 >         * See above for rationale. Briefly: if pred still points to
841 >         * s, try to unlink s.  If s cannot be unlinked, because it is
842 >         * trailing node or pred might be unlinked, and neither pred
843 >         * nor s are head or offlist, add to sweepVotes, and if enough
844 >         * votes have accumulated, sweep.
845           */
846 <        if (pred != null && pred != s) {
847 <            while (pred.next == s) {
848 <                Node oldpred = (cleanMe == null) ? null : reclean();
849 <                Node n = s.next;
850 <                if (n != null) {
851 <                    if (n != s)
852 <                        pred.casNext(s, n);
853 <                    break;
846 >        if (pred != null && pred != s && pred.next == s) {
847 >            Node n = s.next;
848 >            if (n == null ||
849 >                (n != s && pred.casNext(s, n) && pred.isMatched())) {
850 >                for (;;) {               // check if at, or could be, head
851 >                    Node h = head;
852 >                    if (h == pred || h == s || h == null)
853 >                        return;          // at head or list empty
854 >                    if (!h.isMatched())
855 >                        break;
856 >                    Node hn = h.next;
857 >                    if (hn == null)
858 >                        return;          // now empty
859 >                    if (hn != h && casHead(h, hn))
860 >                        h.forgetNext();  // advance head
861 >                }
862 >                if (pred.next != pred && s.next != s) { // recheck if offlist
863 >                    for (;;) {           // sweep now if enough votes
864 >                        int v = sweepVotes;
865 >                        if (v < SWEEP_THRESHOLD) {
866 >                            if (casSweepVotes(v, v + 1))
867 >                                break;
868 >                        }
869 >                        else if (casSweepVotes(v, 0)) {
870 >                            sweep();
871 >                            break;
872 >                        }
873 >                    }
874                  }
807                if (oldpred == pred ||      // Already saved
808                    (oldpred == null && casCleanMe(null, pred)))
809                    break;                  // Postpone cleaning
875              }
876          }
877      }
878  
879      /**
880 <     * Tries to unsplice the deleted/cancelled node held in cleanMe
881 <     * that was previously uncleanable because it was at tail.
817 <     *
818 <     * @return current cleanMe node (or null)
880 >     * Unlinks matched (typically cancelled) nodes encountered in a
881 >     * traversal from head.
882       */
883 <    private Node reclean() {
884 <        /*
885 <         * cleanMe is, or at one time was, predecessor of a cancelled
886 <         * node s that was the tail so could not be unspliced.  If it
887 <         * is no longer the tail, try to unsplice if necessary and
888 <         * make cleanMe slot available.  This differs from similar
889 <         * code in unsplice() because we must check that pred still
827 <         * points to a matched node that can be unspliced -- if not,
828 <         * we can (must) clear cleanMe without unsplicing.  This can
829 <         * loop only due to contention.
830 <         */
831 <        Node pred;
832 <        while ((pred = cleanMe) != null) {
833 <            Node s = pred.next;
834 <            Node n;
835 <            if (s == null || s == pred || !s.isMatched())
836 <                casCleanMe(pred, null); // already gone
837 <            else if ((n = s.next) != null) {
838 <                if (n != s)
839 <                    pred.casNext(s, n);
840 <                casCleanMe(pred, null);
841 <            }
842 <            else
883 >    private void sweep() {
884 >        for (Node p = head, s, n; p != null && (s = p.next) != null; ) {
885 >            if (p == s)                    // stale
886 >                p = head;
887 >            else if (!s.isMatched())
888 >                p = s;
889 >            else if ((n = s.next) == null) // trailing node is pinned
890                  break;
891 <        }
892 <        return pred;
846 <    }
847 <
848 <    /**
849 <     * Main implementation of Iterator.remove(). Find
850 <     * and unsplice the given node.
851 <     */
852 <    final void findAndRemoveNode(Node s) {
853 <        if (s.tryMatchData()) {
854 <            Node pred = null;
855 <            Node p = head;
856 <            while (p != null) {
857 <                if (p == s) {
858 <                    unsplice(pred, p);
859 <                    break;
860 <                }
861 <                if (!p.isData && !p.isMatched())
862 <                    break;
863 <                pred = p;
864 <                if ((p = p.next) == pred) { // stale
865 <                    pred = null;
866 <                    p = head;
867 <                }
868 <            }
891 >            else
892 >                p.casNext(s, n);
893          }
894      }
895  
# Line 874 | Line 898 | public class LinkedTransferQueue<E> exte
898       */
899      private boolean findAndRemove(Object e) {
900          if (e != null) {
901 <            Node pred = null;
878 <            Node p = head;
879 <            while (p != null) {
901 >            for (Node pred = null, p = head; p != null; ) {
902                  Object item = p.item;
903                  if (p.isData) {
904                      if (item != null && item != p && e.equals(item) &&
# Line 888 | Line 910 | public class LinkedTransferQueue<E> exte
910                  else if (item == null)
911                      break;
912                  pred = p;
913 <                if ((p = p.next) == pred) {
913 >                if ((p = p.next) == pred) { // stale
914                      pred = null;
915                      p = head;
916                  }
# Line 1016 | Line 1038 | public class LinkedTransferQueue<E> exte
1038       */
1039      public boolean tryTransfer(E e, long timeout, TimeUnit unit)
1040          throws InterruptedException {
1041 <        if (xfer(e, true, TIMEOUT, unit.toNanos(timeout)) == null)
1041 >        if (xfer(e, true, TIMED, unit.toNanos(timeout)) == null)
1042              return true;
1043          if (!Thread.interrupted())
1044              return false;
# Line 1024 | Line 1046 | public class LinkedTransferQueue<E> exte
1046      }
1047  
1048      public E take() throws InterruptedException {
1049 <        Object e = xfer(null, false, SYNC, 0);
1049 >        E e = xfer(null, false, SYNC, 0);
1050          if (e != null)
1051 <            return (E)e;
1051 >            return e;
1052          Thread.interrupted();
1053          throw new InterruptedException();
1054      }
1055  
1056      public E poll(long timeout, TimeUnit unit) throws InterruptedException {
1057 <        Object e = xfer(null, false, TIMEOUT, unit.toNanos(timeout));
1057 >        E e = xfer(null, false, TIMED, unit.toNanos(timeout));
1058          if (e != null || !Thread.interrupted())
1059 <            return (E)e;
1059 >            return e;
1060          throw new InterruptedException();
1061      }
1062  
1063      public E poll() {
1064 <        return (E)xfer(null, false, NOW, 0);
1064 >        return xfer(null, false, NOW, 0);
1065      }
1066  
1067      /**
# Line 1096 | Line 1118 | public class LinkedTransferQueue<E> exte
1118      }
1119  
1120      public E peek() {
1121 <        return (E) firstDataItem();
1121 >        return firstDataItem();
1122      }
1123  
1124      /**
# Line 1105 | Line 1127 | public class LinkedTransferQueue<E> exte
1127       * @return {@code true} if this queue contains no elements
1128       */
1129      public boolean isEmpty() {
1130 <        return firstOfMode(true) == null;
1130 >        for (Node p = head; p != null; p = succ(p)) {
1131 >            if (!p.isMatched())
1132 >                return !p.isData;
1133 >        }
1134 >        return true;
1135      }
1136  
1137      public boolean hasWaitingConsumer() {
# Line 1192 | Line 1218 | public class LinkedTransferQueue<E> exte
1218          }
1219      }
1220  
1195
1221      // Unsafe mechanics
1222  
1223      private static final sun.misc.Unsafe UNSAFE = getUnsafe();
# Line 1200 | Line 1225 | public class LinkedTransferQueue<E> exte
1225          objectFieldOffset(UNSAFE, "head", LinkedTransferQueue.class);
1226      private static final long tailOffset =
1227          objectFieldOffset(UNSAFE, "tail", LinkedTransferQueue.class);
1228 <    private static final long cleanMeOffset =
1229 <        objectFieldOffset(UNSAFE, "cleanMe", LinkedTransferQueue.class);
1228 >    private static final long sweepVotesOffset =
1229 >        objectFieldOffset(UNSAFE, "sweepVotes", LinkedTransferQueue.class);
1230  
1231      static long objectFieldOffset(sun.misc.Unsafe UNSAFE,
1232                                    String field, Class<?> klazz) {
# Line 1215 | Line 1240 | public class LinkedTransferQueue<E> exte
1240          }
1241      }
1242  
1243 <    private static sun.misc.Unsafe getUnsafe() {
1243 >    /**
1244 >     * Returns a sun.misc.Unsafe.  Suitable for use in a 3rd party package.
1245 >     * Replace with a simple call to Unsafe.getUnsafe when integrating
1246 >     * into a jdk.
1247 >     *
1248 >     * @return a sun.misc.Unsafe
1249 >     */
1250 >    static sun.misc.Unsafe getUnsafe() {
1251          try {
1252              return sun.misc.Unsafe.getUnsafe();
1253          } catch (SecurityException se) {

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines