ViewVC Help
View File | Revision Log | Show Annotations | Download File | Root Listing
root/jsr166/jsr166/src/jsr166y/Phaser.java
(Generate patch)

Comparing jsr166/src/jsr166y/Phaser.java (file contents):
Revision 1.1 by dl, Mon Jul 7 16:53:30 2008 UTC vs.
Revision 1.49 by dl, Fri Nov 5 23:01:47 2010 UTC

# Line 5 | Line 5
5   */
6  
7   package jsr166y;
8 < import jsr166y.forkjoin.*;
8 >
9   import java.util.concurrent.*;
10 < import java.util.concurrent.atomic.*;
10 > import java.util.concurrent.atomic.AtomicReference;
11   import java.util.concurrent.locks.LockSupport;
12  
13   /**
14 < * A reusable synchronization barrier, similar in functionality to a
15 < * {@link java.util.concurrent.CyclicBarrier}, but supporting more
16 < * flexible usage.
14 > * A reusable synchronization barrier, similar in functionality to
15 > * {@link java.util.concurrent.CyclicBarrier CyclicBarrier} and
16 > * {@link java.util.concurrent.CountDownLatch CountDownLatch}
17 > * but supporting more flexible usage.
18 > *
19 > * <p> <b>Registration.</b> Unlike the case for other barriers, the
20 > * number of parties <em>registered</em> to synchronize on a phaser
21 > * may vary over time.  Tasks may be registered at any time (using
22 > * methods {@link #register}, {@link #bulkRegister}, or forms of
23 > * constructors establishing initial numbers of parties), and
24 > * optionally deregistered upon any arrival (using {@link
25 > * #arriveAndDeregister}).  As is the case with most basic
26 > * synchronization constructs, registration and deregistration affect
27 > * only internal counts; they do not establish any further internal
28 > * bookkeeping, so tasks cannot query whether they are registered.
29 > * (However, you can introduce such bookkeeping by subclassing this
30 > * class.)
31 > *
32 > * <p> <b>Synchronization.</b> Like a {@code CyclicBarrier}, a {@code
33 > * Phaser} may be repeatedly awaited.  Method {@link
34 > * #arriveAndAwaitAdvance} has effect analogous to {@link
35 > * java.util.concurrent.CyclicBarrier#await CyclicBarrier.await}. Each
36 > * generation of a {@code Phaser} has an associated phase number. The
37 > * phase number starts at zero, and advances when all parties arrive
38 > * at the barrier, wrapping around to zero after reaching {@code
39 > * Integer.MAX_VALUE}. The use of phase numbers enables independent
40 > * control of actions upon arrival at a barrier and upon awaiting
41 > * others, via two kinds of methods that may be invoked by any
42 > * registered party:
43   *
44   * <ul>
45   *
46 < * <li> The number of parties synchronizing on the barrier may vary
47 < * over time.  A task may register to be a party in a barrier at any
48 < * time, and may deregister upon arriving at the barrier.  As is the
49 < * case with most basic synchronization constructs, registration
50 < * and deregistration affect only internal counts; they do not
51 < * establish any further internal bookkeeping, so tasks cannot query
52 < * whether they are registered.
53 < *
54 < * <li> Each generation has an associated phase value, starting at
55 < * zero, and advancing when all parties reach the barrier (wrapping
56 < * around to zero after reaching <tt>Integer.MAX_VALUE</tt>).
57 < *
58 < * <li> Like a CyclicBarrier, a Phaser may be repeatedly awaited.
59 < * Method <tt>arriveAndAwaitAdvance</tt> has effect analogous to
60 < * <tt>CyclicBarrier.await</tt>.  However, Phasers separate two
61 < * aspects of coordination, that may be invoked independently:
62 < *
63 < * <ul>
46 > *   <li> <b>Arrival.</b> Methods {@link #arrive} and
47 > *       {@link #arriveAndDeregister} record arrival at a
48 > *       barrier. These methods do not block, but return an associated
49 > *       <em>arrival phase number</em>; that is, the phase number of
50 > *       the barrier to which the arrival applied. When the final
51 > *       party for a given phase arrives, an optional barrier action
52 > *       is performed and the phase advances.  Barrier actions,
53 > *       performed by the party triggering a phase advance, are
54 > *       arranged by overriding method {@link #onAdvance(int, int)},
55 > *       which also controls termination. Overriding this method is
56 > *       similar to, but more flexible than, providing a barrier
57 > *       action to a {@code CyclicBarrier}.
58 > *
59 > *   <li> <b>Waiting.</b> Method {@link #awaitAdvance} requires an
60 > *       argument indicating an arrival phase number, and returns when
61 > *       the barrier advances to (or is already at) a different phase.
62 > *       Unlike similar constructions using {@code CyclicBarrier},
63 > *       method {@code awaitAdvance} continues to wait even if the
64 > *       waiting thread is interrupted. Interruptible and timeout
65 > *       versions are also available, but exceptions encountered while
66 > *       tasks wait interruptibly or with timeout do not change the
67 > *       state of the barrier. If necessary, you can perform any
68 > *       associated recovery within handlers of those exceptions,
69 > *       often after invoking {@code forceTermination}.  Phasers may
70 > *       also be used by tasks executing in a {@link ForkJoinPool},
71 > *       which will ensure sufficient parallelism to execute tasks
72 > *       when others are blocked waiting for a phase to advance.
73   *
39 *   <li> Arriving at a barrier. Methods <tt>arrive</tt> and
40 *       <tt>arriveAndDeregister</tt> do not block, but return
41 *       the phase value on entry to the method.
42 *
43 *   <li> Awaiting others. Method <tt>awaitAdvance</tt> requires an
44 *       argument indicating the entry phase, and returns when the
45 *       barrier advances to a new phase.
74   * </ul>
75   *
76 + * <p> <b>Termination.</b> A {@code Phaser} may enter a
77 + * <em>termination</em> state in which all synchronization methods
78 + * immediately return without updating phaser state or waiting for
79 + * advance, and indicating (via a negative phase value) that execution
80 + * is complete.  Termination is triggered when an invocation of {@code
81 + * onAdvance} returns {@code true}.  As illustrated below, when
82 + * phasers control actions with a fixed number of iterations, it is
83 + * often convenient to override this method to cause termination when
84 + * the current phase number reaches a threshold. Method {@link
85 + * #forceTermination} is also available to abruptly release waiting
86 + * threads and allow them to terminate.
87 + *
88 + * <p> <b>Tiering.</b> Phasers may be <em>tiered</em> (i.e., arranged
89 + * in tree structures) to reduce contention. Phasers with large
90 + * numbers of parties that would otherwise experience heavy
91 + * synchronization contention costs may instead be set up so that
92 + * groups of sub-phasers share a common parent.  This may greatly
93 + * increase throughput even though it incurs greater per-operation
94 + * overhead.
95 + *
96 + * <p><b>Monitoring.</b> While synchronization methods may be invoked
97 + * only by registered parties, the current state of a phaser may be
98 + * monitored by any caller.  At any given moment there are {@link
99 + * #getRegisteredParties} parties in total, of which {@link
100 + * #getArrivedParties} have arrived at the current phase ({@link
101 + * #getPhase}).  When the remaining ({@link #getUnarrivedParties})
102 + * parties arrive, the phase advances.  The values returned by these
103 + * methods may reflect transient states and so are not in general
104 + * useful for synchronization control.  Method {@link #toString}
105 + * returns snapshots of these state queries in a form convenient for
106 + * informal monitoring.
107 + *
108 + * <p><b>Sample usages:</b>
109 + *
110 + * <p>A {@code Phaser} may be used instead of a {@code CountDownLatch}
111 + * to control a one-shot action serving a variable number of parties.
112 + * The typical idiom is for the method setting this up to first
113 + * register, then start the actions, then deregister, as in:
114 + *
115 + *  <pre> {@code
116 + * void runTasks(List<Runnable> tasks) {
117 + *   final Phaser phaser = new Phaser(1); // "1" to register self
118 + *   // create and start threads
119 + *   for (Runnable task : tasks) {
120 + *     phaser.register();
121 + *     new Thread() {
122 + *       public void run() {
123 + *         phaser.arriveAndAwaitAdvance(); // await all creation
124 + *         task.run();
125 + *       }
126 + *     }.start();
127 + *   }
128   *
129 < * <li> Barrier actions, performed by the task triggering a phase
130 < * advance while others may be waiting, are arranged by overriding
131 < * method <tt>onAdvance</tt>, that also controls termination.
132 < *
133 < * <li> Phasers may enter a <em>termination</em> state in which all
134 < * await actions immediately return, indicating (via a negative phase
135 < * value) that execution is complete.  Termination is triggered by
136 < * executing the overridable <tt>onAdvance</tt> method that is invoked
137 < * each time the barrier is tripped. When a Phaser is controlling an
138 < * action with a fixed number of iterations, it is often convenient to
139 < * override this method to cause termination when the current phase
140 < * number reaches a threshold.  Method <tt>forceTermination</tt> is
141 < * also available to assist recovery actions upon failure.
142 < *
143 < * <li> Unlike most synchronizers, a Phaser may also be used with
144 < * ForkJoinTasks (as well as plain threads).
145 < *
146 < * <li> By default, <tt>awaitAdvance</tt> continues to wait even if
147 < * the current thread is interrupted. And unlike the case in
148 < * CyclicBarriers, exceptions encountered while tasks wait
149 < * interruptibly or with timeout do not change the state of the
150 < * barrier. If necessary, you can perform any associated recovery
151 < * within handlers of those exceptions.
152 < *
153 < * </ul>
129 > *   // allow threads to start and deregister self
130 > *   phaser.arriveAndDeregister();
131 > * }}</pre>
132 > *
133 > * <p>One way to cause a set of threads to repeatedly perform actions
134 > * for a given number of iterations is to override {@code onAdvance}:
135 > *
136 > *  <pre> {@code
137 > * void startTasks(List<Runnable> tasks, final int iterations) {
138 > *   final Phaser phaser = new Phaser() {
139 > *     protected boolean onAdvance(int phase, int registeredParties) {
140 > *       return phase >= iterations || registeredParties == 0;
141 > *     }
142 > *   };
143 > *   phaser.register();
144 > *   for (final Runnable task : tasks) {
145 > *     phaser.register();
146 > *     new Thread() {
147 > *       public void run() {
148 > *         do {
149 > *           task.run();
150 > *           phaser.arriveAndAwaitAdvance();
151 > *         } while (!phaser.isTerminated());
152 > *       }
153 > *     }.start();
154 > *   }
155 > *   phaser.arriveAndDeregister(); // deregister self, don't wait
156 > * }}</pre>
157   *
158 < * <p><b>Sample usage:</b>
158 > * If the main task must later await termination, it
159 > * may re-register and then execute a similar loop:
160 > *  <pre> {@code
161 > *   // ...
162 > *   phaser.register();
163 > *   while (!phaser.isTerminated())
164 > *     phaser.arriveAndAwaitAdvance();}</pre>
165 > *
166 > * <p>Related constructions may be used to await particular phase numbers
167 > * in contexts where you are sure that the phase will never wrap around
168 > * {@code Integer.MAX_VALUE}. For example:
169 > *
170 > *  <pre> {@code
171 > * void awaitPhase(Phaser phaser, int phase) {
172 > *   int p = phaser.register(); // assumes caller not already registered
173 > *   while (p < phase) {
174 > *     if (phaser.isTerminated())
175 > *       // ... deal with unexpected termination
176 > *     else
177 > *       p = phaser.arriveAndAwaitAdvance();
178 > *   }
179 > *   phaser.arriveAndDeregister();
180 > * }}</pre>
181   *
77 * <p>[todo: non-FJ example]
182   *
183 < * <p> A Phaser may be used to support a style of programming in
184 < * which a task waits for others to complete, without otherwise
185 < * needing to keep track of which tasks it is waiting for. This is
186 < * similar to the "sync" construct in Cilk and "clocks" in X10.
187 < * Special constructions based on such barriers are available using
188 < * the <tt>LinkedAsyncAction</tt> and <tt>CyclicAction</tt> classes,
189 < * but they can be useful in other contexts as well.  For a simple
190 < * (but not very useful) example, here is a variant of Fibonacci:
191 < *
192 < * <pre>
193 < * class BarrierFibonacci extends RecursiveAction {
90 < *   int argument, result;
91 < *   final Phaser parentBarrier;
92 < *   BarrierFibonacci(int n, Phaser parentBarrier) {
93 < *     this.argument = n;
94 < *     this.parentBarrier = parentBarrier;
95 < *     parentBarrier.register();
96 < *   }
97 < *   protected void compute() {
98 < *     int n = argument;
99 < *     if (n &lt;= 1)
100 < *        result = n;
101 < *     else {
102 < *        Phaser childBarrier = new Phaser(1);
103 < *        BarrierFibonacci f1 = new BarrierFibonacci(n - 1, childBarrier);
104 < *        BarrierFibonacci f2 = new BarrierFibonacci(n - 2, childBarrier);
105 < *        f1.fork();
106 < *        f2.fork();
107 < *        childBarrier.arriveAndAwait();
108 < *        result = f1.result + f2.result;
183 > * <p>To create a set of tasks using a tree of phasers,
184 > * you could use code of the following form, assuming a
185 > * Task class with a constructor accepting a phaser that
186 > * it registers with upon construction:
187 > *
188 > *  <pre> {@code
189 > * void build(Task[] actions, int lo, int hi, Phaser ph) {
190 > *   if (hi - lo > TASKS_PER_PHASER) {
191 > *     for (int i = lo; i < hi; i += TASKS_PER_PHASER) {
192 > *       int j = Math.min(i + TASKS_PER_PHASER, hi);
193 > *       build(actions, i, j, new Phaser(ph));
194   *     }
195 < *     parentBarrier.arriveAndDeregister();
195 > *   } else {
196 > *     for (int i = lo; i < hi; ++i)
197 > *       actions[i] = new Task(ph);
198 > *       // assumes new Task(ph) performs ph.register()
199   *   }
200   * }
201 < * </pre>
201 > * // .. initially called, for n tasks via
202 > * build(new Task[n], 0, n, new Phaser());}</pre>
203 > *
204 > * The best value of {@code TASKS_PER_PHASER} depends mainly on
205 > * expected barrier synchronization rates. A value as low as four may
206 > * be appropriate for extremely small per-barrier task bodies (thus
207 > * high rates), or up to hundreds for extremely large ones.
208   *
209   * <p><b>Implementation notes</b>: This implementation restricts the
210 < * maximum number of parties to 65535. Attempts to register
211 < * additional parties result in IllegalStateExceptions.  
210 > * maximum number of parties to 65535. Attempts to register additional
211 > * parties result in {@code IllegalStateException}. However, you can and
212 > * should create tiered phasers to accommodate arbitrarily large sets
213 > * of participants.
214 > *
215 > * @since 1.7
216 > * @author Doug Lea
217   */
218   public class Phaser {
219      /*
220       * This class implements an extension of X10 "clocks".  Thanks to
221 <     * Vijay Saraswat for the idea of applying it to ForkJoinTasks,
222 <     * and to Vivek Sarkar for enhancements to extend functionality.
221 >     * Vijay Saraswat for the idea, and to Vivek Sarkar for
222 >     * enhancements to extend functionality.
223       */
224  
225      /**
226       * Barrier state representation. Conceptually, a barrier contains
227       * four values:
228 <     *
228 >     *
229       * * parties -- the number of parties to wait (16 bits)
230       * * unarrived -- the number of parties yet to hit barrier (16 bits)
231       * * phase -- the generation of the barrier (31 bits)
232       * * terminated -- set if barrier is terminated (1 bit)
233       *
234       * However, to efficiently maintain atomicity, these values are
235 <     * packed into a single AtomicLong. Termination uses the sign bit
236 <     * of 32 bit representation of phase, so phase is set to -1 on
237 <     * termination.
238 <     */
239 <    private final AtomicLong state;
240 <
241 <    /**
143 <     * Head of Treiber stack for waiting nonFJ threads.
235 >     * packed into a single (atomic) long. Termination uses the sign
236 >     * bit of 32 bit representation of phase, so phase is set to -1 on
237 >     * termination. Good performance relies on keeping state decoding
238 >     * and encoding simple, and keeping race windows short.
239 >     *
240 >     * Note: there are some cheats in arrive() that rely on unarrived
241 >     * count being lowest 16 bits.
242       */
243 <    private final AtomicReference<QNode> head = new AtomicReference<QNode>();
243 >    private volatile long state;
244  
245 <    private static final int ushortBits = 16;
246 <    private static final int ushortMask =  (1 << ushortBits) - 1;
149 <    private static final int phaseMask = 0x7fffffff;
245 >    private static final int ushortMask = 0xffff;
246 >    private static final int phaseMask  = 0x7fffffff;
247  
248      private static int unarrivedOf(long s) {
249 <        return (int)(s & ushortMask);
249 >        return (int) (s & ushortMask);
250      }
251  
252      private static int partiesOf(long s) {
253 <        return (int)(s & (ushortMask << 16)) >>> 16;
253 >        return ((int) s) >>> 16;
254      }
255  
256      private static int phaseOf(long s) {
257 <        return (int)(s >>> 32);
257 >        return (int) (s >>> 32);
258      }
259  
260      private static int arrivedOf(long s) {
# Line 165 | Line 262 | public class Phaser {
262      }
263  
264      private static long stateFor(int phase, int parties, int unarrived) {
265 <        return (((long)phase) << 32) | ((parties << 16) | unarrived);
265 >        return ((((long) phase) << 32) | (((long) parties) << 16) |
266 >                (long) unarrived);
267      }
268  
269 <    private static IllegalStateException badBounds(int parties, int unarrived) {
270 <        return new IllegalStateException("Attempt to set " + unarrived +
271 <                                         " unarrived of " + parties + " parties");
269 >    private static long trippedStateFor(int phase, int parties) {
270 >        long lp = (long) parties;
271 >        return (((long) phase) << 32) | (lp << 16) | lp;
272      }
273  
274      /**
275 <     * Creates a new Phaser without any initially registered parties,
276 <     * and initial phase number 0.
275 >     * Returns message string for bad bounds exceptions.
276 >     */
277 >    private static String badBounds(int parties, int unarrived) {
278 >        return ("Attempt to set " + unarrived +
279 >                " unarrived of " + parties + " parties");
280 >    }
281 >
282 >    /**
283 >     * The parent of this phaser, or null if none
284 >     */
285 >    private final Phaser parent;
286 >
287 >    /**
288 >     * The root of phaser tree. Equals this if not in a tree.  Used to
289 >     * support faster state push-down.
290 >     */
291 >    private final Phaser root;
292 >
293 >    // Wait queues
294 >
295 >    /**
296 >     * Heads of Treiber stacks for waiting threads. To eliminate
297 >     * contention when releasing some threads while adding others, we
298 >     * use two of them, alternating across even and odd phases.
299 >     * Subphasers share queues with root to speed up releases.
300 >     */
301 >    private final AtomicReference<QNode> evenQ = new AtomicReference<QNode>();
302 >    private final AtomicReference<QNode> oddQ  = new AtomicReference<QNode>();
303 >
304 >    private AtomicReference<QNode> queueFor(int phase) {
305 >        Phaser r = root;
306 >        return ((phase & 1) == 0) ? r.evenQ : r.oddQ;
307 >    }
308 >
309 >    /**
310 >     * Returns current state, first resolving lagged propagation from
311 >     * root if necessary.
312 >     */
313 >    private long getReconciledState() {
314 >        return (parent == null) ? state : reconcileState();
315 >    }
316 >
317 >    /**
318 >     * Recursively resolves state.
319 >     */
320 >    private long reconcileState() {
321 >        Phaser par = parent;
322 >        long s = state;
323 >        if (par != null) {
324 >            int phase, rootPhase;
325 >            while ((phase = phaseOf(s)) >= 0 &&
326 >                   (rootPhase = phaseOf(root.state)) != phase &&
327 >                   (rootPhase < 0 || unarrivedOf(s) == 0)) {
328 >                long parentState = par.getReconciledState();
329 >                int parentPhase = phaseOf(parentState);
330 >                int parties = partiesOf(s);
331 >                long next = trippedStateFor(parentPhase, parties);
332 >                if (phaseOf(root.state) == rootPhase &&
333 >                    parentPhase != phase &&
334 >                    state == s && casState(s, next)) {
335 >                    releaseWaiters(phase);
336 >                    if (parties == 0) // exit if the final deregistration
337 >                        break;
338 >                }
339 >                s = state;
340 >            }
341 >        }
342 >        return s;
343 >    }
344 >
345 >    /**
346 >     * Creates a new phaser without any initially registered parties,
347 >     * initial phase number 0, and no parent. Any thread using this
348 >     * phaser will need to first register for it.
349       */
350      public Phaser() {
351 <        state = new AtomicLong(stateFor(0, 0, 0));
351 >        this(null);
352      }
353  
354      /**
355 <     * Creates a new Phaser with the given numbers of registered
356 <     * unarrived parties and initial phase number 0.
357 <     * @param parties the number of parties required to trip barrier.
355 >     * Creates a new phaser with the given number of registered
356 >     * unarrived parties, initial phase number 0, and no parent.
357 >     *
358 >     * @param parties the number of parties required to trip barrier
359       * @throws IllegalArgumentException if parties less than zero
360 <     * or greater than the maximum number of parties supported.
360 >     * or greater than the maximum number of parties supported
361       */
362      public Phaser(int parties) {
363 +        this(null, parties);
364 +    }
365 +
366 +    /**
367 +     * Creates a new phaser with the given parent, without any
368 +     * initially registered parties. If parent is non-null this phaser
369 +     * is registered with the parent and its initial phase number is
370 +     * the same as that of parent phaser.
371 +     *
372 +     * @param parent the parent phaser
373 +     */
374 +    public Phaser(Phaser parent) {
375 +        int phase = 0;
376 +        this.parent = parent;
377 +        if (parent != null) {
378 +            this.root = parent.root;
379 +            phase = parent.register();
380 +        }
381 +        else
382 +            this.root = this;
383 +        this.state = trippedStateFor(phase, 0);
384 +    }
385 +
386 +    /**
387 +     * Creates a new phaser with the given parent and number of
388 +     * registered unarrived parties. If parent is non-null, this phaser
389 +     * is registered with the parent and its initial phase number is
390 +     * the same as that of parent phaser.
391 +     *
392 +     * @param parent the parent phaser
393 +     * @param parties the number of parties required to trip barrier
394 +     * @throws IllegalArgumentException if parties less than zero
395 +     * or greater than the maximum number of parties supported
396 +     */
397 +    public Phaser(Phaser parent, int parties) {
398          if (parties < 0 || parties > ushortMask)
399              throw new IllegalArgumentException("Illegal number of parties");
400 <        state = new AtomicLong(stateFor(0, parties, parties));
400 >        int phase = 0;
401 >        this.parent = parent;
402 >        if (parent != null) {
403 >            this.root = parent.root;
404 >            phase = parent.register();
405 >        }
406 >        else
407 >            this.root = this;
408 >        this.state = trippedStateFor(phase, parties);
409      }
410  
411      /**
412       * Adds a new unarrived party to this phaser.
413 <     * @return the current barrier phase number upon registration
413 >     * If an ongoing invocation of {@link #onAdvance} is in progress,
414 >     * this method waits until its completion before registering.
415 >     *
416 >     * @return the arrival phase number to which this registration applied
417       * @throws IllegalStateException if attempting to register more
418 <     * than the maximum supported number of parties.
418 >     * than the maximum supported number of parties
419       */
420 <    public int register() { // increment both parties and unarrived
421 <        final AtomicLong state = this.state;
422 <        for (;;) {
423 <            long s = state.get();
424 <            int phase = phaseOf(s);
425 <            int parties = partiesOf(s) + 1;
426 <            int unarrived = unarrivedOf(s) + 1;
427 <            if (parties > ushortMask || unarrived > ushortMask)
428 <                throw badBounds(parties, unarrived);
429 <            if (state.compareAndSet(s, stateFor(phase, parties, unarrived)))
430 <                return phase;
420 >    public int register() {
421 >        return doRegister(1);
422 >    }
423 >
424 >    /**
425 >     * Adds the given number of new unarrived parties to this phaser.
426 >     * If an ongoing invocation of {@link #onAdvance} is in progress,
427 >     * this method waits until its completion before registering.
428 >     *
429 >     * @param parties the number of additional parties required to trip barrier
430 >     * @return the arrival phase number to which this registration applied
431 >     * @throws IllegalStateException if attempting to register more
432 >     * than the maximum supported number of parties
433 >     * @throws IllegalArgumentException if {@code parties < 0}
434 >     */
435 >    public int bulkRegister(int parties) {
436 >        if (parties < 0)
437 >            throw new IllegalArgumentException();
438 >        if (parties == 0)
439 >            return getPhase();
440 >        return doRegister(parties);
441 >    }
442 >
443 >    /**
444 >     * Shared code for register, bulkRegister
445 >     */
446 >    private int doRegister(int registrations) {
447 >        Phaser par = parent;
448 >        long s;
449 >        int phase;
450 >        while ((phase = phaseOf(s = par==null? state:reconcileState())) >= 0) {
451 >            int p = partiesOf(s);
452 >            int u = unarrivedOf(s);
453 >            int unarrived = u + registrations;
454 >            int parties = p + registrations;
455 >            if (par == null || phase == phaseOf(root.state)) {
456 >                if (parties > ushortMask || unarrived > ushortMask)
457 >                    throw new IllegalStateException(badBounds(parties,
458 >                                                              unarrived));
459 >                else if (p != 0 && u == 0)       // back off if advancing
460 >                    Thread.yield();              // not worth actually blocking
461 >                else if (casState(s, stateFor(phase, parties, unarrived)))
462 >                    break;
463 >            }
464          }
465 +        return phase;
466      }
467  
468      /**
469       * Arrives at the barrier, but does not wait for others.  (You can
470 <     * in turn wait for others via {@link #awaitAdvance}).
470 >     * in turn wait for others via {@link #awaitAdvance}).  It is an
471 >     * unenforced usage error for an unregistered party to invoke this
472 >     * method.
473       *
474 <     * @return the current barrier phase number upon entry to
475 <     * this method, or a negative value if terminated;
476 <     * @throws IllegalStateException if the number of unarrived
477 <     * parties would become negative.
478 <     */
479 <    public int arrive() { // decrement unarrived. If zero, trip
480 <        final AtomicLong state = this.state;
481 <        for (;;) {
482 <            long s = state.get();
230 <            int phase = phaseOf(s);
474 >     * @return the arrival phase number, or a negative value if terminated
475 >     * @throws IllegalStateException if not terminated and the number
476 >     * of unarrived parties would become negative
477 >     */
478 >    public int arrive() {
479 >        Phaser par = parent;
480 >        long s;
481 >        int phase;
482 >        while ((phase = phaseOf(s = par==null? state:reconcileState())) >= 0) {
483              int parties = partiesOf(s);
484              int unarrived = unarrivedOf(s) - 1;
485 <            if (unarrived < 0)
486 <                throw badBounds(parties, unarrived);
487 <            if (unarrived == 0 && phase >= 0) {
488 <                trip(phase, parties);
489 <                return phase;
485 >            if (parties == 0 || unarrived < 0)
486 >                throw new IllegalStateException(badBounds(parties,
487 >                                                          unarrived));
488 >            else if (unarrived > 0) {           // Not the last arrival
489 >                if (casState(s, s - 1))         // s-1 adds one arrival
490 >                    break;
491 >            }
492 >            else if (par == null) {             // directly trip
493 >                if (casState(s, trippedStateFor(onAdvance(phase, parties) ? -1 :
494 >                                                ((phase + 1) & phaseMask),
495 >                                                parties))) {
496 >                    releaseWaiters(phase);
497 >                    break;
498 >                }
499 >            }
500 >            else if (phaseOf(root.state) == phase && casState(s, s - 1)) {
501 >                par.arrive();                   // cascade to parent
502 >                reconcileState();
503 >                break;
504              }
239            if (state.compareAndSet(s, stateFor(phase, parties, unarrived)))
240                return phase;
505          }
506 +        return phase;
507      }
508  
509      /**
510 <     * Arrives at the barrier, and deregisters from it, without
511 <     * waiting for others.
510 >     * Arrives at the barrier and deregisters from it without waiting
511 >     * for others. Deregistration reduces the number of parties
512 >     * required to trip the barrier in future phases.  If this phaser
513 >     * has a parent, and deregistration causes this phaser to have
514 >     * zero parties, this phaser also arrives at and is deregistered
515 >     * from its parent.  It is an unenforced usage error for an
516 >     * unregistered party to invoke this method.
517       *
518 <     * @return the current barrier phase number upon entry to
519 <     * this method, or a negative value if terminated;
520 <     * @throws IllegalStateException if the number of registered or
521 <     * unarrived parties would become negative.
522 <     */
523 <    public int arriveAndDeregister() { // Same as arrive, plus decrement parties
524 <        final AtomicLong state = this.state;
525 <        for (;;) {
526 <            long s = state.get();
527 <            int phase = phaseOf(s);
518 >     * @return the arrival phase number, or a negative value if terminated
519 >     * @throws IllegalStateException if not terminated and the number
520 >     * of registered or unarrived parties would become negative
521 >     */
522 >    public int arriveAndDeregister() {
523 >        // similar to arrive, but too different to merge
524 >        Phaser par = parent;
525 >        long s;
526 >        int phase;
527 >        while ((phase = phaseOf(s = par==null? state:reconcileState())) >= 0) {
528              int parties = partiesOf(s) - 1;
529              int unarrived = unarrivedOf(s) - 1;
530              if (parties < 0 || unarrived < 0)
531 <                throw badBounds(parties, unarrived);
532 <            if (unarrived == 0 && phase >= 0) {
533 <                trip(phase, parties);
534 <                return phase;
531 >                throw new IllegalStateException(badBounds(parties,
532 >                                                          unarrived));
533 >            else if (unarrived > 0) {
534 >                if (casState(s, stateFor(phase, parties, unarrived)))
535 >                    break;
536 >            }
537 >            else if (par == null) {
538 >                if (casState(s, trippedStateFor(onAdvance(phase, parties)? -1:
539 >                                                (phase + 1) & phaseMask,
540 >                                                parties))) {
541 >                    releaseWaiters(phase);
542 >                    break;
543 >                }
544 >            }
545 >            else if (phaseOf(root.state) == phase &&
546 >                     casState(s, stateFor(phase, parties, 0))) {
547 >                if (parties == 0)
548 >                    par.arriveAndDeregister();
549 >                else
550 >                    par.arrive();
551 >                reconcileState();
552 >                break;
553              }
266            if (state.compareAndSet(s, stateFor(phase, parties, unarrived)))
267                return phase;
554          }
555 +        return phase;
556      }
557  
558      /**
559 <     * Arrives at the barrier and awaits others. Unlike other arrival
560 <     * methods, this method returns the arrival index of the
561 <     * caller. The caller tripping the barrier returns zero, the
562 <     * previous caller 1, and so on.
563 <     * @return the arrival index
564 <     * @throws IllegalStateException if the number of unarrived
565 <     * parties would become negative.
559 >     * Arrives at the barrier and awaits others. Equivalent in effect
560 >     * to {@code awaitAdvance(arrive())}.  If you need to await with
561 >     * interruption or timeout, you can arrange this with an analogous
562 >     * construction using one of the other forms of the {@code
563 >     * awaitAdvance} method.  If instead you need to deregister upon
564 >     * arrival, use {@link #arriveAndDeregister}. It is an unenforced
565 >     * usage error for an unregistered party to invoke this method.
566 >     *
567 >     * @return the arrival phase number, or a negative number if terminated
568 >     * @throws IllegalStateException if not terminated and the number
569 >     * of unarrived parties would become negative
570       */
571      public int arriveAndAwaitAdvance() {
572 <        final AtomicLong state = this.state;
282 <        for (;;) {
283 <            long s = state.get();
284 <            int phase = phaseOf(s);
285 <            int parties = partiesOf(s);
286 <            int unarrived = unarrivedOf(s) - 1;
287 <            if (unarrived < 0)
288 <                throw badBounds(parties, unarrived);
289 <            if (unarrived == 0 && phase >= 0) {
290 <                trip(phase, parties);
291 <                return 0;
292 <            }
293 <            if (state.compareAndSet(s, stateFor(phase, parties, unarrived))) {
294 <                awaitAdvance(phase);
295 <                return unarrived;
296 <            }
297 <        }
572 >        return awaitAdvance(arrive());
573      }
574  
575      /**
576 <     * Awaits the phase of the barrier to advance from the given
577 <     * value, or returns immediately if this barrier is terminated
578 <     * @param phase the phase on entry to this method
579 <     * @return the phase on exit from this method
576 >     * Awaits the phase of the barrier to advance from the given phase
577 >     * value, returning immediately if the current phase of the
578 >     * barrier is not equal to the given phase value or this barrier
579 >     * is terminated.  It is an unenforced usage error for an
580 >     * unregistered party to invoke this method.
581 >     *
582 >     * @param phase an arrival phase number, or negative value if
583 >     * terminated; this argument is normally the value returned by a
584 >     * previous call to {@code arrive} or its variants
585 >     * @return the next arrival phase number, or a negative value
586 >     * if terminated or argument is negative
587       */
588      public int awaitAdvance(int phase) {
589          if (phase < 0)
590              return phase;
591 <        Thread current = Thread.currentThread();
592 <        if (current instanceof ForkJoinWorkerThread)
593 <            return helpingWait(phase);
594 <        if (untimedWait(current, phase, false))
313 <            current.interrupt();
314 <        return phaseOf(state.get());
591 >        int p = getPhase();
592 >        if (p != phase)
593 >            return p;
594 >        return untimedWait(phase);
595      }
596  
597      /**
598 <     * Awaits the phase of the barrier to advance from the given
599 <     * value, or returns immediately if this barrier is terminated, or
600 <     * throws InterruptedException if interrupted while waiting.
601 <     * @param phase the phase on entry to this method
602 <     * @return the phase on exit from this method
598 >     * Awaits the phase of the barrier to advance from the given phase
599 >     * value, throwing {@code InterruptedException} if interrupted
600 >     * while waiting, or returning immediately if the current phase of
601 >     * the barrier is not equal to the given phase value or this
602 >     * barrier is terminated. It is an unenforced usage error for an
603 >     * unregistered party to invoke this method.
604 >     *
605 >     * @param phase an arrival phase number, or negative value if
606 >     * terminated; this argument is normally the value returned by a
607 >     * previous call to {@code arrive} or its variants
608 >     * @return the next arrival phase number, or a negative value
609 >     * if terminated or argument is negative
610       * @throws InterruptedException if thread interrupted while waiting
611       */
612 <    public int awaitAdvanceInterruptibly(int phase) throws InterruptedException {
612 >    public int awaitAdvanceInterruptibly(int phase)
613 >        throws InterruptedException {
614          if (phase < 0)
615              return phase;
616 <        Thread current = Thread.currentThread();
617 <        if (current instanceof ForkJoinWorkerThread)
618 <            return helpingWait(phase);
619 <        else if (Thread.interrupted() || untimedWait(current, phase, true))
332 <            throw new InterruptedException();
333 <        else
334 <            return phaseOf(state.get());
616 >        int p = getPhase();
617 >        if (p != phase)
618 >            return p;
619 >        return interruptibleWait(phase);
620      }
621  
622      /**
623 <     * Awaits the phase of the barrier to advance from the given value
624 <     * or the given timeout elapses, or returns immediately if this
625 <     * barrier is terminated
626 <     * @param phase the phase on entry to this method
627 <     * @return the phase on exit from this method
623 >     * Awaits the phase of the barrier to advance from the given phase
624 >     * value or the given timeout to elapse, throwing {@code
625 >     * InterruptedException} if interrupted while waiting, or
626 >     * returning immediately if the current phase of the barrier is
627 >     * not equal to the given phase value or this barrier is
628 >     * terminated.  It is an unenforced usage error for an
629 >     * unregistered party to invoke this method.
630 >     *
631 >     * @param phase an arrival phase number, or negative value if
632 >     * terminated; this argument is normally the value returned by a
633 >     * previous call to {@code arrive} or its variants
634 >     * @param timeout how long to wait before giving up, in units of
635 >     *        {@code unit}
636 >     * @param unit a {@code TimeUnit} determining how to interpret the
637 >     *        {@code timeout} parameter
638 >     * @return the next arrival phase number, or a negative value
639 >     * if terminated or argument is negative
640       * @throws InterruptedException if thread interrupted while waiting
641       * @throws TimeoutException if timed out while waiting
642       */
643 <    public int awaitAdvanceInterruptibly(int phase, long timeout, TimeUnit unit)
643 >    public int awaitAdvanceInterruptibly(int phase,
644 >                                         long timeout, TimeUnit unit)
645          throws InterruptedException, TimeoutException {
646 +        long nanos = unit.toNanos(timeout);
647          if (phase < 0)
648              return phase;
649 <        long nanos = unit.toNanos(timeout);
650 <        Thread current = Thread.currentThread();
651 <        if (current instanceof ForkJoinWorkerThread)
652 <            return timedHelpingWait(phase, nanos);
354 <        timedWait(current, phase, nanos);
355 <        return phaseOf(state.get());
649 >        int p = getPhase();
650 >        if (p != phase)
651 >            return p;
652 >        return timedWait(phase, nanos);
653      }
654  
655      /**
656       * Forces this barrier to enter termination state. Counts of
657 <     * arrived and registered parties are unaffected. This method may
658 <     * be useful for coordinating recovery after one or more tasks
659 <     * encounter unexpected exceptions.
657 >     * arrived and registered parties are unaffected. If this phaser
658 >     * has a parent, it too is terminated. This method may be useful
659 >     * for coordinating recovery after one or more tasks encounter
660 >     * unexpected exceptions.
661       */
662      public void forceTermination() {
663 <        final AtomicLong state = this.state;
664 <        for (;;) {
665 <            long s = state.get();
666 <            int phase = phaseOf(s);
667 <            int parties = partiesOf(s);
668 <            int unarrived = unarrivedOf(s);
669 <            if (phase < 0 ||
372 <                state.compareAndSet(s, stateFor(-1, parties, unarrived))) {
373 <                if (head.get() != null)
374 <                    releaseWaiters(-1);
375 <                return;
376 <            }
377 <        }
378 <    }
379 <
380 <    /**
381 <     * Resets the barrier with the given numbers of registered unarrived
382 <     * parties and phase number 0. This method allows repeated reuse
383 <     * of this barrier, but only if it is somehow known not to be in
384 <     * use for other purposes.
385 <     * @param parties the number of parties required to trip barrier.
386 <     * @throws IllegalArgumentException if parties less than zero
387 <     * or greater than the maximum number of parties supported.
388 <     */
389 <    public void reset(int parties) {
390 <        if (parties < 0 || parties > ushortMask)
391 <            throw new IllegalArgumentException("Illegal number of parties");
392 <        state.set(stateFor(0, parties, parties));
393 <        if (head.get() != null)
394 <            releaseWaiters(0);
663 >        Phaser r = root;    // force at root then reconcile
664 >        long s;
665 >        while (phaseOf(s = r.state) >= 0)
666 >            r.casState(s, stateFor(-1, partiesOf(s), unarrivedOf(s)));
667 >        reconcileState();
668 >        releaseWaiters(0);  // ensure wakeups on both queues
669 >        releaseWaiters(1);
670      }
671  
672      /**
673       * Returns the current phase number. The maximum phase number is
674 <     * <tt>Integer.MAX_VALUE</tt>, after which it restarts at
674 >     * {@code Integer.MAX_VALUE}, after which it restarts at
675       * zero. Upon termination, the phase number is negative.
676 +     *
677       * @return the phase number, or a negative value if terminated
678       */
679 <    public int getPhase() {
680 <        return phaseOf(state.get());
679 >    public final int getPhase() {
680 >        return phaseOf(getReconciledState());
681      }
682  
683      /**
684       * Returns the number of parties registered at this barrier.
685 +     *
686       * @return the number of parties
687       */
688      public int getRegisteredParties() {
689 <        return partiesOf(state.get());
689 >        return partiesOf(getReconciledState());
690      }
691  
692      /**
693 <     * Returns the number of parties that have arrived at the current
694 <     * phase of this barrier.
693 >     * Returns the number of registered parties that have arrived at
694 >     * the current phase of this barrier.
695 >     *
696       * @return the number of arrived parties
697       */
698      public int getArrivedParties() {
699 <        return arrivedOf(state.get());
699 >        return arrivedOf(getReconciledState());
700      }
701  
702      /**
703       * Returns the number of registered parties that have not yet
704       * arrived at the current phase of this barrier.
705 +     *
706       * @return the number of unarrived parties
707       */
708      public int getUnarrivedParties() {
709 <        return unarrivedOf(state.get());
709 >        return unarrivedOf(getReconciledState());
710 >    }
711 >
712 >    /**
713 >     * Returns the parent of this phaser, or {@code null} if none.
714 >     *
715 >     * @return the parent of this phaser, or {@code null} if none
716 >     */
717 >    public Phaser getParent() {
718 >        return parent;
719      }
720  
721      /**
722 <     * Returns true if this barrier has been terminated
723 <     * @return true if this barrier has been terminated
722 >     * Returns the root ancestor of this phaser, which is the same as
723 >     * this phaser if it has no parent.
724 >     *
725 >     * @return the root ancestor of this phaser
726 >     */
727 >    public Phaser getRoot() {
728 >        return root;
729 >    }
730 >
731 >    /**
732 >     * Returns {@code true} if this barrier has been terminated.
733 >     *
734 >     * @return {@code true} if this barrier has been terminated
735       */
736      public boolean isTerminated() {
737 <        return phaseOf(state.get()) < 0;
737 >        return getPhase() < 0;
738      }
739  
740      /**
741 <     * Overridable method to perform an action upon phase advance, and
742 <     * to control termination. This method is invoked whenever the
743 <     * barrier is tripped (and thus all other waiting parties are
744 <     * dormant). If it returns true, then, rather than advance the
745 <     * phase number, this barrier will be set to a final termination
746 <     * state, and subsequent calls to <tt>isTerminated</tt> will
747 <     * return true.
748 <     *
749 <     * <p> The default version returns true when the number of
741 >     * Overridable method to perform an action upon impending phase
742 >     * advance, and to control termination. This method is invoked
743 >     * upon arrival of the party tripping the barrier (when all other
744 >     * waiting parties are dormant).  If this method returns {@code
745 >     * true}, then, rather than advance the phase number, this barrier
746 >     * will be set to a final termination state, and subsequent calls
747 >     * to {@link #isTerminated} will return true. Any (unchecked)
748 >     * Exception or Error thrown by an invocation of this method is
749 >     * propagated to the party attempting to trip the barrier, in
750 >     * which case no advance occurs.
751 >     *
752 >     * <p>The arguments to this method provide the state of the phaser
753 >     * prevailing for the current transition. (When called from within
754 >     * an implementation of {@code onAdvance} the values returned by
755 >     * methods such as {@code getPhase} may or may not reliably
756 >     * indicate the state to which this transition applies.)
757 >     *
758 >     * <p>The default version returns {@code true} when the number of
759       * registered parties is zero. Normally, overrides that arrange
760       * termination for other reasons should also preserve this
761       * property.
762       *
763       * @param phase the phase number on entering the barrier
764 <     * @param registeredParties the current number of registered
765 <     * parties.
458 <     * @return true if this barrier should terminate
764 >     * @param registeredParties the current number of registered parties
765 >     * @return {@code true} if this barrier should terminate
766       */
767      protected boolean onAdvance(int phase, int registeredParties) {
768          return registeredParties <= 0;
769      }
770  
771      /**
772 <     * Returns a string identifying this barrier, as well as its
772 >     * Returns a string identifying this phaser, as well as its
773       * state.  The state, in brackets, includes the String {@code
774 <     * "phase ="} followed by the phase number, {@code "parties ="}
774 >     * "phase = "} followed by the phase number, {@code "parties = "}
775       * followed by the number of registered parties, and {@code
776 <     * "arrived ="} followed by the number of arrived parties
776 >     * "arrived = "} followed by the number of arrived parties.
777       *
778       * @return a string identifying this barrier, as well as its state
779       */
780      public String toString() {
781 <        long s = state.get();
782 <        return super.toString() + "[phase = " + phaseOf(s) + " parties = " + partiesOf(s) + " arrived = " + arrivedOf(s) + "]";
781 >        long s = getReconciledState();
782 >        return super.toString() +
783 >            "[phase = " + phaseOf(s) +
784 >            " parties = " + partiesOf(s) +
785 >            " arrived = " + arrivedOf(s) + "]";
786      }
787  
788 <    // methods for tripping and waiting
788 >    // methods for waiting
789  
790      /**
791 <     * Advance the current phase (or terminate)
791 >     * Wait nodes for Treiber stack representing wait queue
792       */
793 <    private void trip(int phase, int parties) {
794 <        int next = onAdvance(phase, parties)? -1 : ((phase + 1) & phaseMask);
795 <        state.set(stateFor(next, parties, parties));
796 <        if (head.get() != null)
797 <            releaseWaiters(next);
798 <    }
793 >    static final class QNode implements ForkJoinPool.ManagedBlocker {
794 >        final Phaser phaser;
795 >        final int phase;
796 >        final long startTime;
797 >        final long nanos;
798 >        final boolean timed;
799 >        final boolean interruptible;
800 >        volatile boolean wasInterrupted = false;
801 >        volatile Thread thread; // nulled to cancel wait
802 >        QNode next;
803  
804 <    private int helpingWait(int phase) {
805 <        final AtomicLong state = this.state;
806 <        int p;
807 <        while ((p = phaseOf(state.get())) == phase) {
808 <            ForkJoinTask<?> t = ForkJoinWorkerThread.pollTask();
809 <            if (t != null) {
810 <                if ((p = phaseOf(state.get())) == phase)
811 <                    t.exec();
812 <                else {   // push task and exit if barrier advanced
813 <                    t.fork();
814 <                    break;
815 <                }
804 >        QNode(Phaser phaser, int phase, boolean interruptible,
805 >              boolean timed, long startTime, long nanos) {
806 >            this.phaser = phaser;
807 >            this.phase = phase;
808 >            this.timed = timed;
809 >            this.interruptible = interruptible;
810 >            this.startTime = startTime;
811 >            this.nanos = nanos;
812 >            thread = Thread.currentThread();
813 >        }
814 >
815 >        public boolean isReleasable() {
816 >            return (thread == null ||
817 >                    phaser.getPhase() != phase ||
818 >                    (interruptible && wasInterrupted) ||
819 >                    (timed && (nanos - (System.nanoTime() - startTime)) <= 0));
820 >        }
821 >
822 >        public boolean block() {
823 >            if (Thread.interrupted()) {
824 >                wasInterrupted = true;
825 >                if (interruptible)
826 >                    return true;
827 >            }
828 >            if (!timed)
829 >                LockSupport.park(this);
830 >            else {
831 >                long waitTime = nanos - (System.nanoTime() - startTime);
832 >                if (waitTime <= 0)
833 >                    return true;
834 >                LockSupport.parkNanos(this, waitTime);
835              }
836 +            return isReleasable();
837          }
504        return p;
505    }
838  
839 <    private int timedHelpingWait(int phase, long nanos) throws TimeoutException {
840 <        final AtomicLong state = this.state;
509 <        long lastTime = System.nanoTime();
510 <        int p;
511 <        while ((p = phaseOf(state.get())) == phase) {
512 <            long now = System.nanoTime();
513 <            nanos -= now - lastTime;
514 <            lastTime = now;
515 <            if (nanos <= 0) {
516 <                if ((p = phaseOf(state.get())) == phase)
517 <                    throw new TimeoutException();
518 <                else
519 <                    break;
520 <            }
521 <            ForkJoinTask<?> t = ForkJoinWorkerThread.pollTask();
839 >        void signal() {
840 >            Thread t = thread;
841              if (t != null) {
842 <                if ((p = phaseOf(state.get())) == phase)
843 <                    t.exec();
525 <                else {   // push task and exit if barrier advanced
526 <                    t.fork();
527 <                    break;
528 <                }
842 >                thread = null;
843 >                LockSupport.unpark(t);
844              }
845          }
531        return p;
532    }
533
534    /**
535     * Wait nodes for Treiber stack representing wait queue for non-FJ
536     * tasks. The waiting scheme is an adaptation of the one used in
537     * forkjoin.PoolBarrier.
538     */
539    static final class QNode {
540        QNode next;
541        volatile Thread thread; // nulled to cancel wait
542        final int phase;
543        QNode(Thread t, int c) {
544            thread = t;
545            phase = c;
546        }
547    }
846  
847 <    private void releaseWaiters(int currentPhase) {
848 <        final AtomicReference<QNode> head = this.head;
849 <        QNode p;
850 <        while ((p = head.get()) != null && p.phase != currentPhase) {
851 <            if (head.compareAndSet(p, null)) {
852 <                do {
853 <                    Thread t = p.thread;
556 <                    if (t != null) {
557 <                        p.thread = null;
558 <                        LockSupport.unpark(t);
559 <                    }
560 <                } while ((p = p.next) != null);
847 >        boolean doWait() {
848 >            if (thread != null) {
849 >                try {
850 >                    ForkJoinPool.managedBlock(this);
851 >                } catch (InterruptedException ie) {
852 >                    wasInterrupted = true; // can't currently happen
853 >                }
854              }
855 +            return wasInterrupted;
856          }
857      }
858  
565    /** The number of CPUs, for spin control */
566    static final int NCPUS = Runtime.getRuntime().availableProcessors();
567
859      /**
860 <     * The number of times to spin before blocking in timed waits.
570 <     * The value is empirically derived
860 >     * Removes and signals waiting threads from wait queue.
861       */
862 <    static final int maxTimedSpins = (NCPUS < 2)? 0 : 32;
862 >    private void releaseWaiters(int phase) {
863 >        AtomicReference<QNode> head = queueFor(phase);
864 >        QNode q;
865 >        while ((q = head.get()) != null) {
866 >            if (head.compareAndSet(q, q.next))
867 >                q.signal();
868 >        }
869 >    }
870  
871      /**
872 <     * The number of times to spin before blocking in untimed waits.
873 <     * This is greater than timed value because untimed waits spin
874 <     * faster since they don't need to check times on each spin.
872 >     * Tries to enqueue given node in the appropriate wait queue.
873 >     *
874 >     * @return true if successful
875       */
876 <    static final int maxUntimedSpins = maxTimedSpins * 32;
876 >    private boolean tryEnqueue(QNode node) {
877 >        AtomicReference<QNode> head = queueFor(node.phase);
878 >        return head.compareAndSet(node.next = head.get(), node);
879 >    }
880  
881      /**
882 <     * The number of nanoseconds for which it is faster to spin
883 <     * rather than to use timed park. A rough estimate suffices.
882 >     * Enqueues node and waits unless aborted or signalled.
883 >     *
884 >     * @return current phase
885       */
886 <    static final long spinForTimeoutThreshold = 1000L;
886 >    private int untimedWait(int phase) {
887 >        QNode node = null;
888 >        boolean queued = false;
889 >        boolean interrupted = false;
890 >        int p;
891 >        while ((p = getPhase()) == phase) {
892 >            if (Thread.interrupted())
893 >                interrupted = true;
894 >            else if (node == null)
895 >                node = new QNode(this, phase, false, false, 0, 0);
896 >            else if (!queued)
897 >                queued = tryEnqueue(node);
898 >            else if (node.doWait())
899 >                interrupted = true;
900 >        }
901 >        if (node != null)
902 >            node.thread = null;
903 >        releaseWaiters(phase);
904 >        if (interrupted)
905 >            Thread.currentThread().interrupt();
906 >        return p;
907 >    }
908  
909      /**
910 <     * Enqueues node and waits unless aborted or signalled.
910 >     * Interruptible version
911 >     * @return current phase
912       */
913 <    private boolean untimedWait(Thread thread, int currentPhase,
591 <                               boolean abortOnInterrupt) {
592 <        final AtomicReference<QNode> head = this.head;
593 <        final AtomicLong state = this.state;
594 <        boolean wasInterrupted = false;
913 >    private int interruptibleWait(int phase) throws InterruptedException {
914          QNode node = null;
915          boolean queued = false;
916 <        int spins = maxUntimedSpins;
917 <        while (phaseOf(state.get()) == currentPhase) {
918 <            QNode h;
919 <            if (node != null && queued) {
920 <                if (node.thread != null) {
921 <                    LockSupport.park();
922 <                    if (Thread.interrupted()) {
923 <                        wasInterrupted = true;
924 <                        if (abortOnInterrupt)
925 <                            break;
926 <                    }
608 <                }
609 <            }
610 <            else if ((h = head.get()) != null && h.phase != currentPhase) {
611 <                if (phaseOf(state.get()) == currentPhase) { // must recheck
612 <                    if (head.compareAndSet(h, h.next)) {
613 <                        Thread t = h.thread; // help clear out old waiters
614 <                        if (t != null) {
615 <                            h.thread = null;
616 <                            LockSupport.unpark(t);
617 <                        }
618 <                    }
619 <                }
620 <                else
621 <                    break;
622 <            }
623 <            else if (node != null)
624 <                queued = head.compareAndSet(node.next = h, node);
625 <            else if (spins <= 0)
626 <                node = new QNode(thread, currentPhase);
627 <            else
628 <                --spins;
916 >        boolean interrupted = false;
917 >        int p;
918 >        while ((p = getPhase()) == phase && !interrupted) {
919 >            if (Thread.interrupted())
920 >                interrupted = true;
921 >            else if (node == null)
922 >                node = new QNode(this, phase, true, false, 0, 0);
923 >            else if (!queued)
924 >                queued = tryEnqueue(node);
925 >            else if (node.doWait())
926 >                interrupted = true;
927          }
928          if (node != null)
929              node.thread = null;
930 <        return wasInterrupted;
930 >        if (p != phase || (p = getPhase()) != phase)
931 >            releaseWaiters(phase);
932 >        if (interrupted)
933 >            throw new InterruptedException();
934 >        return p;
935      }
936  
937      /**
938 <     * Messier timeout version
938 >     * Timeout version.
939 >     * @return current phase
940       */
941 <    private void timedWait(Thread thread, int currentPhase, long nanos)
941 >    private int timedWait(int phase, long nanos)
942          throws InterruptedException, TimeoutException {
943 <        final AtomicReference<QNode> head = this.head;
641 <        final AtomicLong state = this.state;
642 <        long lastTime = System.nanoTime();
943 >        long startTime = System.nanoTime();
944          QNode node = null;
945          boolean queued = false;
946 <        int spins = maxTimedSpins;
947 <        while (phaseOf(state.get()) == currentPhase) {
948 <            QNode h;
949 <            long now = System.nanoTime();
950 <            nanos -= now - lastTime;
951 <            lastTime = now;
952 <            if (nanos <= 0) {
953 <                if (node != null)
954 <                    node.thread = null;
955 <                if (phaseOf(state.get()) == currentPhase)
956 <                    throw new TimeoutException();
957 <                else
958 <                    break;
658 <            }
659 <            else if (node != null && queued) {
660 <                if (node.thread != null &&
661 <                    nanos > spinForTimeoutThreshold) {
662 <                    //                LockSupport.parkNanos(this, nanos);
663 <                    LockSupport.parkNanos(nanos);
664 <                    if (Thread.interrupted()) {
665 <                        node.thread = null;
666 <                        throw new InterruptedException();
667 <                    }
668 <                }
669 <            }
670 <            else if ((h = head.get()) != null && h.phase != currentPhase) {
671 <                if (phaseOf(state.get()) == currentPhase) { // must recheck
672 <                    if (head.compareAndSet(h, h.next)) {
673 <                        Thread t = h.thread; // help clear out old waiters
674 <                        if (t != null) {
675 <                            h.thread = null;
676 <                            LockSupport.unpark(t);
677 <                        }
678 <                    }
679 <                }
680 <                else
681 <                    break;
682 <            }
683 <            else if (node != null)
684 <                queued = head.compareAndSet(node.next = h, node);
685 <            else if (spins <= 0)
686 <                node = new QNode(thread, currentPhase);
687 <            else
688 <                --spins;
946 >        boolean interrupted = false;
947 >        int p;
948 >        while ((p = getPhase()) == phase && !interrupted) {
949 >            if (Thread.interrupted())
950 >                interrupted = true;
951 >            else if (nanos - (System.nanoTime() - startTime) <= 0)
952 >                break;
953 >            else if (node == null)
954 >                node = new QNode(this, phase, true, true, startTime, nanos);
955 >            else if (!queued)
956 >                queued = tryEnqueue(node);
957 >            else if (node.doWait())
958 >                interrupted = true;
959          }
960          if (node != null)
961              node.thread = null;
962 +        if (p != phase || (p = getPhase()) != phase)
963 +            releaseWaiters(phase);
964 +        if (interrupted)
965 +            throw new InterruptedException();
966 +        if (p == phase)
967 +            throw new TimeoutException();
968 +        return p;
969      }
970  
971 < }
971 >    // Unsafe mechanics
972 >
973 >    private static final sun.misc.Unsafe UNSAFE = getUnsafe();
974 >    private static final long stateOffset =
975 >        objectFieldOffset("state", Phaser.class);
976  
977 +    private final boolean casState(long cmp, long val) {
978 +        return UNSAFE.compareAndSwapLong(this, stateOffset, cmp, val);
979 +    }
980 +
981 +    private static long objectFieldOffset(String field, Class<?> klazz) {
982 +        try {
983 +            return UNSAFE.objectFieldOffset(klazz.getDeclaredField(field));
984 +        } catch (NoSuchFieldException e) {
985 +            // Convert Exception to corresponding Error
986 +            NoSuchFieldError error = new NoSuchFieldError(field);
987 +            error.initCause(e);
988 +            throw error;
989 +        }
990 +    }
991 +
992 +    /**
993 +     * Returns a sun.misc.Unsafe.  Suitable for use in a 3rd party package.
994 +     * Replace with a simple call to Unsafe.getUnsafe when integrating
995 +     * into a jdk.
996 +     *
997 +     * @return a sun.misc.Unsafe
998 +     */
999 +    private static sun.misc.Unsafe getUnsafe() {
1000 +        try {
1001 +            return sun.misc.Unsafe.getUnsafe();
1002 +        } catch (SecurityException se) {
1003 +            try {
1004 +                return java.security.AccessController.doPrivileged
1005 +                    (new java.security
1006 +                     .PrivilegedExceptionAction<sun.misc.Unsafe>() {
1007 +                        public sun.misc.Unsafe run() throws Exception {
1008 +                            java.lang.reflect.Field f = sun.misc
1009 +                                .Unsafe.class.getDeclaredField("theUnsafe");
1010 +                            f.setAccessible(true);
1011 +                            return (sun.misc.Unsafe) f.get(null);
1012 +                        }});
1013 +            } catch (java.security.PrivilegedActionException e) {
1014 +                throw new RuntimeException("Could not initialize intrinsics",
1015 +                                           e.getCause());
1016 +            }
1017 +        }
1018 +    }
1019 + }

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines