ViewVC Help
View File | Revision Log | Show Annotations | Download File | Root Listing
root/jsr166/jsr166/src/jsr166y/Phaser.java
(Generate patch)

Comparing jsr166/src/jsr166y/Phaser.java (file contents):
Revision 1.38 by dl, Mon Aug 24 12:11:00 2009 UTC vs.
Revision 1.80 by jsr166, Sun Sep 13 16:28:14 2015 UTC

# Line 1 | Line 1
1   /*
2   * Written by Doug Lea with assistance from members of JCP JSR-166
3   * Expert Group and released to the public domain, as explained at
4 < * http://creativecommons.org/licenses/publicdomain
4 > * http://creativecommons.org/publicdomain/zero/1.0/
5   */
6  
7   package jsr166y;
8  
9 < import java.util.concurrent.*;
10 <
9 > import java.util.concurrent.TimeUnit;
10 > import java.util.concurrent.TimeoutException;
11   import java.util.concurrent.atomic.AtomicReference;
12   import java.util.concurrent.locks.LockSupport;
13  
# Line 17 | Line 17 | import java.util.concurrent.locks.LockSu
17   * {@link java.util.concurrent.CountDownLatch CountDownLatch}
18   * but supporting more flexible usage.
19   *
20 < * <p> <b>Registration.</b> Unlike the case for other barriers, the
20 > * <p><b>Registration.</b> Unlike the case for other barriers, the
21   * number of parties <em>registered</em> to synchronize on a phaser
22   * may vary over time.  Tasks may be registered at any time (using
23   * methods {@link #register}, {@link #bulkRegister}, or forms of
# Line 30 | Line 30 | import java.util.concurrent.locks.LockSu
30   * (However, you can introduce such bookkeeping by subclassing this
31   * class.)
32   *
33 < * <p> <b>Synchronization.</b> Like a {@code CyclicBarrier}, a {@code
33 > * <p><b>Synchronization.</b> Like a {@code CyclicBarrier}, a {@code
34   * Phaser} may be repeatedly awaited.  Method {@link
35   * #arriveAndAwaitAdvance} has effect analogous to {@link
36   * java.util.concurrent.CyclicBarrier#await CyclicBarrier.await}. Each
37 < * generation of a {@code Phaser} has an associated phase number. The
38 < * phase number starts at zero, amd advances when all parties arrive
39 < * at the barrier, wrapping around to zero after reaching {@code
37 > * generation of a phaser has an associated phase number. The phase
38 > * number starts at zero, and advances when all parties arrive at the
39 > * phaser, wrapping around to zero after reaching {@code
40   * Integer.MAX_VALUE}. The use of phase numbers enables independent
41 < * control of actions upon arrival at a barrier and upon awaiting
41 > * control of actions upon arrival at a phaser and upon awaiting
42   * others, via two kinds of methods that may be invoked by any
43   * registered party:
44   *
45   * <ul>
46   *
47 < *   <li> <b>Arrival.</b> Methods {@link #arrive} and
48 < *       {@link #arriveAndDeregister} record arrival at a
49 < *       barrier. These methods do not block, but return an associated
50 < *       <em>arrival phase number</em>; that is, the phase number of
51 < *       the barrier to which the arrival applied. When the final
52 < *       party for a given phase arrives, an optional barrier action
53 < *       is performed and the phase advances.  Barrier actions,
54 < *       performed by the party triggering a phase advance, are
55 < *       arranged by overriding method {@link #onAdvance(int, int)},
56 < *       which also controls termination. Overriding this method is
57 < *       similar to, but more flexible than, providing a barrier
58 < *       action to a {@code CyclicBarrier}.
47 > *   <li><b>Arrival.</b> Methods {@link #arrive} and
48 > *       {@link #arriveAndDeregister} record arrival.  These methods
49 > *       do not block, but return an associated <em>arrival phase
50 > *       number</em>; that is, the phase number of the phaser to which
51 > *       the arrival applied. When the final party for a given phase
52 > *       arrives, an optional action is performed and the phase
53 > *       advances.  These actions are performed by the party
54 > *       triggering a phase advance, and are arranged by overriding
55 > *       method {@link #onAdvance(int, int)}, which also controls
56 > *       termination. Overriding this method is similar to, but more
57 > *       flexible than, providing a barrier action to a {@code
58 > *       CyclicBarrier}.
59   *
60 < *   <li> <b>Waiting.</b> Method {@link #awaitAdvance} requires an
60 > *   <li><b>Waiting.</b> Method {@link #awaitAdvance} requires an
61   *       argument indicating an arrival phase number, and returns when
62 < *       the barrier advances to (or is already at) a different phase.
62 > *       the phaser advances to (or is already at) a different phase.
63   *       Unlike similar constructions using {@code CyclicBarrier},
64   *       method {@code awaitAdvance} continues to wait even if the
65 < *       waiting thread is interrupted. (Interruptible and timeout
66 < *       versions are also available.)  Exceptions encountered while
65 > *       waiting thread is interrupted. Interruptible and timeout
66 > *       versions are also available, but exceptions encountered while
67   *       tasks wait interruptibly or with timeout do not change the
68 < *       state of the barrier. If necessary, you can perform any
68 > *       state of the phaser. If necessary, you can perform any
69   *       associated recovery within handlers of those exceptions,
70   *       often after invoking {@code forceTermination}.  Phasers may
71   *       also be used by tasks executing in a {@link ForkJoinPool},
# Line 74 | Line 74 | import java.util.concurrent.locks.LockSu
74   *
75   * </ul>
76   *
77 < * <p> <b>Termination.</b> A {@code Phaser} may enter a
78 < * <em>termination</em> state in which all actions immediately return
79 < * without updating phaser state or waiting for advance, and
80 < * indicating (via a negative phase value) that execution is complete.
77 > * <p><b>Termination.</b> A phaser may enter a <em>termination</em>
78 > * state, that may be checked using method {@link #isTerminated}. Upon
79 > * termination, all synchronization methods immediately return without
80 > * waiting for advance, as indicated by a negative return value.
81 > * Similarly, attempts to register upon termination have no effect.
82   * Termination is triggered when an invocation of {@code onAdvance}
83 < * returns {@code true}.  As illustrated below, when phasers control
83 > * returns {@code true}. The default implementation returns {@code
84 > * true} if a deregistration has caused the number of registered
85 > * parties to become zero.  As illustrated below, when phasers control
86   * actions with a fixed number of iterations, it is often convenient
87   * to override this method to cause termination when the current phase
88   * number reaches a threshold. Method {@link #forceTermination} is
89   * also available to abruptly release waiting threads and allow them
90   * to terminate.
91   *
92 < * <p> <b>Tiering.</b> Phasers may be <em>tiered</em> (i.e., arranged
93 < * in tree structures) to reduce contention. Phasers with large
94 < * numbers of parties that would otherwise experience heavy
92 > * <p><b>Tiering.</b> Phasers may be <em>tiered</em> (i.e.,
93 > * constructed in tree structures) to reduce contention. Phasers with
94 > * large numbers of parties that would otherwise experience heavy
95   * synchronization contention costs may instead be set up so that
96   * groups of sub-phasers share a common parent.  This may greatly
97   * increase throughput even though it incurs greater per-operation
98   * overhead.
99   *
100 + * <p>In a tree of tiered phasers, registration and deregistration of
101 + * child phasers with their parent are managed automatically.
102 + * Whenever the number of registered parties of a child phaser becomes
103 + * non-zero (as established in the {@link #Phaser(Phaser,int)}
104 + * constructor, {@link #register}, or {@link #bulkRegister}), the
105 + * child phaser is registered with its parent.  Whenever the number of
106 + * registered parties becomes zero as the result of an invocation of
107 + * {@link #arriveAndDeregister}, the child phaser is deregistered
108 + * from its parent.
109 + *
110   * <p><b>Monitoring.</b> While synchronization methods may be invoked
111   * only by registered parties, the current state of a phaser may be
112   * monitored by any caller.  At any given moment there are {@link
113 < * #getRegisteredParties}, where {@link #getArrivedParties} have
114 < * arrived at the current phase ({@link #getPhase}). When the
115 < * remaining {@link #getUnarrivedParties}) arrive, the phase
116 < * advances. Method {@link #toString} returns snapshots of these state
117 < * queries in a form convenient for informal monitoring.
113 > * #getRegisteredParties} parties in total, of which {@link
114 > * #getArrivedParties} have arrived at the current phase ({@link
115 > * #getPhase}).  When the remaining ({@link #getUnarrivedParties})
116 > * parties arrive, the phase advances.  The values returned by these
117 > * methods may reflect transient states and so are not in general
118 > * useful for synchronization control.  Method {@link #toString}
119 > * returns snapshots of these state queries in a form convenient for
120 > * informal monitoring.
121   *
122   * <p><b>Sample usages:</b>
123   *
124   * <p>A {@code Phaser} may be used instead of a {@code CountDownLatch}
125 < * to control a one-shot action serving a variable number of
126 < * parties. The typical idiom is for the method setting this up to
127 < * first register, then start the actions, then deregister, as in:
125 > * to control a one-shot action serving a variable number of parties.
126 > * The typical idiom is for the method setting this up to first
127 > * register, then start the actions, then deregister, as in:
128   *
129   *  <pre> {@code
130   * void runTasks(List<Runnable> tasks) {
131   *   final Phaser phaser = new Phaser(1); // "1" to register self
132   *   // create and start threads
133 < *   for (Runnable task : tasks) {
133 > *   for (final Runnable task : tasks) {
134   *     phaser.register();
135   *     new Thread() {
136   *       public void run() {
# Line 139 | Line 155 | import java.util.concurrent.locks.LockSu
155   *     }
156   *   };
157   *   phaser.register();
158 < *   for (Runnable task : tasks) {
158 > *   for (final Runnable task : tasks) {
159   *     phaser.register();
160   *     new Thread() {
161   *       public void run() {
162   *         do {
163   *           task.run();
164   *           phaser.arriveAndAwaitAdvance();
165 < *         } while(!phaser.isTerminated();
165 > *         } while (!phaser.isTerminated());
166   *       }
167   *     }.start();
168   *   }
# Line 155 | Line 171 | import java.util.concurrent.locks.LockSu
171   *
172   * If the main task must later await termination, it
173   * may re-register and then execute a similar loop:
174 < * <pre> {@code
174 > *  <pre> {@code
175   *   // ...
176   *   phaser.register();
177   *   while (!phaser.isTerminated())
178 < *     phaser.arriveAndAwaitAdvance();
163 < * }</pre>
178 > *     phaser.arriveAndAwaitAdvance();}</pre>
179   *
180 < * Related constructions may be used to await particular phase numbers
180 > * <p>Related constructions may be used to await particular phase numbers
181   * in contexts where you are sure that the phase will never wrap around
182   * {@code Integer.MAX_VALUE}. For example:
183   *
184 < * <pre> {@code
185 < *   void awaitPhase(Phaser phaser, int phase) {
186 < *     int p = phaser.register(); // assumes caller not already registered
187 < *     while (p < phase) {
188 < *       if (phaser.isTerminated())
189 < *         // ... deal with unexpected termination
190 < *       else
191 < *         p = phaser.arriveAndAwaitAdvance();
177 < *     }
178 < *     phaser.arriveAndDeregister();
184 > *  <pre> {@code
185 > * void awaitPhase(Phaser phaser, int phase) {
186 > *   int p = phaser.register(); // assumes caller not already registered
187 > *   while (p < phase) {
188 > *     if (phaser.isTerminated())
189 > *       // ... deal with unexpected termination
190 > *     else
191 > *       p = phaser.arriveAndAwaitAdvance();
192   *   }
193 < * }</pre>
193 > *   phaser.arriveAndDeregister();
194 > * }}</pre>
195 > *
196   *
197 + * <p>To create a set of {@code n} tasks using a tree of phasers, you
198 + * could use code of the following form, assuming a Task class with a
199 + * constructor accepting a {@code Phaser} that it registers with upon
200 + * construction. After invocation of {@code build(new Task[n], 0, n,
201 + * new Phaser())}, these tasks could then be started, for example by
202 + * submitting to a pool:
203   *
183 * <p>To create a set of tasks using a tree of phasers,
184 * you could use code of the following form, assuming a
185 * Task class with a constructor accepting a phaser that
186 * it registers for upon construction:
204   *  <pre> {@code
205 < * void build(Task[] actions, int lo, int hi, Phaser b) {
206 < *   int step = (hi - lo) / TASKS_PER_PHASER;
207 < *   if (step > 1) {
208 < *     int i = lo;
209 < *     while (i < hi) {
193 < *       int r = Math.min(i + step, hi);
194 < *       build(actions, i, r, new Phaser(b));
195 < *       i = r;
205 > * void build(Task[] tasks, int lo, int hi, Phaser ph) {
206 > *   if (hi - lo > TASKS_PER_PHASER) {
207 > *     for (int i = lo; i < hi; i += TASKS_PER_PHASER) {
208 > *       int j = Math.min(i + TASKS_PER_PHASER, hi);
209 > *       build(tasks, i, j, new Phaser(ph));
210   *     }
211   *   } else {
212   *     for (int i = lo; i < hi; ++i)
213 < *       actions[i] = new Task(b);
214 < *       // assumes new Task(b) performs b.register()
213 > *       tasks[i] = new Task(ph);
214 > *       // assumes new Task(ph) performs ph.register()
215   *   }
216 < * }
203 < * // .. initially called, for n tasks via
204 < * build(new Task[n], 0, n, new Phaser());}</pre>
216 > * }}</pre>
217   *
218   * The best value of {@code TASKS_PER_PHASER} depends mainly on
219 < * expected barrier synchronization rates. A value as low as four may
220 < * be appropriate for extremely small per-barrier task bodies (thus
219 > * expected synchronization rates. A value as low as four may
220 > * be appropriate for extremely small per-phase task bodies (thus
221   * high rates), or up to hundreds for extremely large ones.
222   *
211 * </pre>
212 *
223   * <p><b>Implementation notes</b>: This implementation restricts the
224   * maximum number of parties to 65535. Attempts to register additional
225   * parties result in {@code IllegalStateException}. However, you can and
# Line 227 | Line 237 | public class Phaser {
237       */
238  
239      /**
240 <     * Barrier state representation. Conceptually, a barrier contains
231 <     * four values:
240 >     * Primary state representation, holding four bit-fields:
241       *
242 <     * * parties -- the number of parties to wait (16 bits)
243 <     * * unarrived -- the number of parties yet to hit barrier (16 bits)
244 <     * * phase -- the generation of the barrier (31 bits)
245 <     * * terminated -- set if barrier is terminated (1 bit)
246 <     *
247 <     * However, to efficiently maintain atomicity, these values are
248 <     * packed into a single (atomic) long. Termination uses the sign
249 <     * bit of 32 bit representation of phase, so phase is set to -1 on
250 <     * termination. Good performance relies on keeping state decoding
251 <     * and encoding simple, and keeping race windows short.
252 <     *
253 <     * Note: there are some cheats in arrive() that rely on unarrived
254 <     * count being lowest 16 bits.
242 >     * unarrived  -- the number of parties yet to hit barrier (bits  0-15)
243 >     * parties    -- the number of parties to wait            (bits 16-31)
244 >     * phase      -- the generation of the barrier            (bits 32-62)
245 >     * terminated -- set if barrier is terminated             (bit  63 / sign)
246 >     *
247 >     * Except that a phaser with no registered parties is
248 >     * distinguished by the otherwise illegal state of having zero
249 >     * parties and one unarrived parties (encoded as EMPTY below).
250 >     *
251 >     * To efficiently maintain atomicity, these values are packed into
252 >     * a single (atomic) long. Good performance relies on keeping
253 >     * state decoding and encoding simple, and keeping race windows
254 >     * short.
255 >     *
256 >     * All state updates are performed via CAS except initial
257 >     * registration of a sub-phaser (i.e., one with a non-null
258 >     * parent).  In this (relatively rare) case, we use built-in
259 >     * synchronization to lock while first registering with its
260 >     * parent.
261 >     *
262 >     * The phase of a subphaser is allowed to lag that of its
263 >     * ancestors until it is actually accessed -- see method
264 >     * reconcileState.
265       */
266      private volatile long state;
267  
268 <    private static final int ushortBits = 16;
269 <    private static final int ushortMask = 0xffff;
270 <    private static final int phaseMask  = 0x7fffffff;
268 >    private static final int  MAX_PARTIES     = 0xffff;
269 >    private static final int  MAX_PHASE       = Integer.MAX_VALUE;
270 >    private static final int  PARTIES_SHIFT   = 16;
271 >    private static final int  PHASE_SHIFT     = 32;
272 >    private static final int  UNARRIVED_MASK  = 0xffff;      // to mask ints
273 >    private static final long PARTIES_MASK    = 0xffff0000L; // to mask longs
274 >    private static final long COUNTS_MASK     = 0xffffffffL;
275 >    private static final long TERMINATION_BIT = 1L << 63;
276 >
277 >    // some special values
278 >    private static final int  ONE_ARRIVAL     = 1;
279 >    private static final int  ONE_PARTY       = 1 << PARTIES_SHIFT;
280 >    private static final int  ONE_DEREGISTER  = ONE_ARRIVAL|ONE_PARTY;
281 >    private static final int  EMPTY           = 1;
282 >
283 >    // The following unpacking methods are usually manually inlined
284  
285      private static int unarrivedOf(long s) {
286 <        return (int) (s & ushortMask);
286 >        int counts = (int)s;
287 >        return (counts == EMPTY) ? 0 : (counts & UNARRIVED_MASK);
288      }
289  
290      private static int partiesOf(long s) {
291 <        return ((int) s) >>> 16;
291 >        return (int)s >>> PARTIES_SHIFT;
292      }
293  
294      private static int phaseOf(long s) {
295 <        return (int) (s >>> 32);
295 >        return (int)(s >>> PHASE_SHIFT);
296      }
297  
298      private static int arrivedOf(long s) {
299 <        return partiesOf(s) - unarrivedOf(s);
300 <    }
301 <
269 <    private static long stateFor(int phase, int parties, int unarrived) {
270 <        return ((((long) phase) << 32) | (((long) parties) << 16) |
271 <                (long) unarrived);
272 <    }
273 <
274 <    private static long trippedStateFor(int phase, int parties) {
275 <        long lp = (long) parties;
276 <        return (((long) phase) << 32) | (lp << 16) | lp;
277 <    }
278 <
279 <    /**
280 <     * Returns message string for bad bounds exceptions.
281 <     */
282 <    private static String badBounds(int parties, int unarrived) {
283 <        return ("Attempt to set " + unarrived +
284 <                " unarrived of " + parties + " parties");
299 >        int counts = (int)s;
300 >        return (counts == EMPTY) ? 0 :
301 >            (counts >>> PARTIES_SHIFT) - (counts & UNARRIVED_MASK);
302      }
303  
304      /**
# Line 290 | Line 307 | public class Phaser {
307      private final Phaser parent;
308  
309      /**
310 <     * The root of phaser tree. Equals this if not in a tree.  Used to
294 <     * support faster state push-down.
310 >     * The root of phaser tree. Equals this if not in a tree.
311       */
312      private final Phaser root;
313  
298    // Wait queues
299
314      /**
315       * Heads of Treiber stacks for waiting threads. To eliminate
316 <     * contention while releasing some threads while adding others, we
316 >     * contention when releasing some threads while adding others, we
317       * use two of them, alternating across even and odd phases.
318 +     * Subphasers share queues with root to speed up releases.
319       */
320 <    private final AtomicReference<QNode> evenQ = new AtomicReference<QNode>();
321 <    private final AtomicReference<QNode> oddQ  = new AtomicReference<QNode>();
320 >    private final AtomicReference<QNode> evenQ;
321 >    private final AtomicReference<QNode> oddQ;
322  
323      private AtomicReference<QNode> queueFor(int phase) {
324          return ((phase & 1) == 0) ? evenQ : oddQ;
325      }
326  
327      /**
328 <     * Returns current state, first resolving lagged propagation from
314 <     * root if necessary.
328 >     * Returns message string for bounds exceptions on arrival.
329       */
330 <    private long getReconciledState() {
331 <        return (parent == null) ? state : reconcileState();
330 >    private String badArrive(long s) {
331 >        return "Attempted arrival of unregistered party for " +
332 >            stateToString(s);
333      }
334  
335      /**
336 <     * Recursively resolves state.
336 >     * Returns message string for bounds exceptions on registration.
337       */
338 <    private long reconcileState() {
339 <        Phaser p = parent;
340 <        long s = state;
341 <        if (p != null) {
342 <            while (unarrivedOf(s) == 0 && phaseOf(s) != phaseOf(root.state)) {
343 <                long parentState = p.getReconciledState();
344 <                int parentPhase = phaseOf(parentState);
345 <                int phase = phaseOf(s = state);
346 <                if (phase != parentPhase) {
347 <                    long next = trippedStateFor(parentPhase, partiesOf(s));
348 <                    if (casState(s, next)) {
338 >    private String badRegister(long s) {
339 >        return "Attempt to register more than " +
340 >            MAX_PARTIES + " parties for " + stateToString(s);
341 >    }
342 >
343 >    /**
344 >     * Main implementation for methods arrive and arriveAndDeregister.
345 >     * Manually tuned to speed up and minimize race windows for the
346 >     * common case of just decrementing unarrived field.
347 >     *
348 >     * @param adjust value to subtract from state;
349 >     *               ONE_ARRIVAL for arrive,
350 >     *               ONE_DEREGISTER for arriveAndDeregister
351 >     */
352 >    private int doArrive(int adjust) {
353 >        final Phaser root = this.root;
354 >        for (;;) {
355 >            long s = (root == this) ? state : reconcileState();
356 >            int phase = (int)(s >>> PHASE_SHIFT);
357 >            if (phase < 0)
358 >                return phase;
359 >            int counts = (int)s;
360 >            int unarrived = (counts == EMPTY) ? 0 : (counts & UNARRIVED_MASK);
361 >            if (unarrived <= 0)
362 >                throw new IllegalStateException(badArrive(s));
363 >            if (UNSAFE.compareAndSwapLong(this, stateOffset, s, s-=adjust)) {
364 >                if (unarrived == 1) {
365 >                    long n = s & PARTIES_MASK;  // base of next state
366 >                    int nextUnarrived = (int)n >>> PARTIES_SHIFT;
367 >                    if (root == this) {
368 >                        if (onAdvance(phase, nextUnarrived))
369 >                            n |= TERMINATION_BIT;
370 >                        else if (nextUnarrived == 0)
371 >                            n |= EMPTY;
372 >                        else
373 >                            n |= nextUnarrived;
374 >                        int nextPhase = (phase + 1) & MAX_PHASE;
375 >                        n |= (long)nextPhase << PHASE_SHIFT;
376 >                        UNSAFE.compareAndSwapLong(this, stateOffset, s, n);
377                          releaseWaiters(phase);
378 <                        s = next;
378 >                    }
379 >                    else if (nextUnarrived == 0) { // propagate deregistration
380 >                        phase = parent.doArrive(ONE_DEREGISTER);
381 >                        UNSAFE.compareAndSwapLong(this, stateOffset,
382 >                                                  s, s | EMPTY);
383 >                    }
384 >                    else
385 >                        phase = parent.doArrive(ONE_ARRIVAL);
386 >                }
387 >                return phase;
388 >            }
389 >        }
390 >    }
391 >
392 >    /**
393 >     * Implementation of register, bulkRegister
394 >     *
395 >     * @param registrations number to add to both parties and
396 >     * unarrived fields. Must be greater than zero.
397 >     */
398 >    private int doRegister(int registrations) {
399 >        // adjustment to state
400 >        long adjust = ((long)registrations << PARTIES_SHIFT) | registrations;
401 >        final Phaser parent = this.parent;
402 >        int phase;
403 >        for (;;) {
404 >            long s = (parent == null) ? state : reconcileState();
405 >            int counts = (int)s;
406 >            int parties = counts >>> PARTIES_SHIFT;
407 >            int unarrived = counts & UNARRIVED_MASK;
408 >            if (registrations > MAX_PARTIES - parties)
409 >                throw new IllegalStateException(badRegister(s));
410 >            phase = (int)(s >>> PHASE_SHIFT);
411 >            if (phase < 0)
412 >                break;
413 >            if (counts != EMPTY) {                  // not 1st registration
414 >                if (parent == null || reconcileState() == s) {
415 >                    if (unarrived == 0)             // wait out advance
416 >                        root.internalAwaitAdvance(phase, null);
417 >                    else if (UNSAFE.compareAndSwapLong(this, stateOffset,
418 >                                                       s, s + adjust))
419 >                        break;
420 >                }
421 >            }
422 >            else if (parent == null) {              // 1st root registration
423 >                long next = ((long)phase << PHASE_SHIFT) | adjust;
424 >                if (UNSAFE.compareAndSwapLong(this, stateOffset, s, next))
425 >                    break;
426 >            }
427 >            else {
428 >                synchronized (this) {               // 1st sub registration
429 >                    if (state == s) {               // recheck under lock
430 >                        phase = parent.doRegister(1);
431 >                        if (phase < 0)
432 >                            break;
433 >                        // finish registration whenever parent registration
434 >                        // succeeded, even when racing with termination,
435 >                        // since these are part of the same "transaction".
436 >                        while (!UNSAFE.compareAndSwapLong
437 >                               (this, stateOffset, s,
438 >                                ((long)phase << PHASE_SHIFT) | adjust)) {
439 >                            s = state;
440 >                            phase = (int)(root.state >>> PHASE_SHIFT);
441 >                            // assert (int)s == EMPTY;
442 >                        }
443 >                        break;
444                      }
445                  }
446              }
447          }
448 +        return phase;
449 +    }
450 +
451 +    /**
452 +     * Resolves lagged phase propagation from root if necessary.
453 +     * Reconciliation normally occurs when root has advanced but
454 +     * subphasers have not yet done so, in which case they must finish
455 +     * their own advance by setting unarrived to parties (or if
456 +     * parties is zero, resetting to unregistered EMPTY state).
457 +     *
458 +     * @return reconciled state
459 +     */
460 +    private long reconcileState() {
461 +        final Phaser root = this.root;
462 +        long s = state;
463 +        if (root != this) {
464 +            int phase, p;
465 +            // CAS to root phase with current parties, tripping unarrived
466 +            while ((phase = (int)(root.state >>> PHASE_SHIFT)) !=
467 +                   (int)(s >>> PHASE_SHIFT) &&
468 +                   !UNSAFE.compareAndSwapLong
469 +                   (this, stateOffset, s,
470 +                    s = (((long)phase << PHASE_SHIFT) |
471 +                         ((phase < 0) ? (s & COUNTS_MASK) :
472 +                          (((p = (int)s >>> PARTIES_SHIFT) == 0) ? EMPTY :
473 +                           ((s & PARTIES_MASK) | p))))))
474 +                s = state;
475 +        }
476          return s;
477      }
478  
479      /**
480 <     * Creates a new phaser without any initially registered parties,
481 <     * initial phase number 0, and no parent. Any thread using this
480 >     * Creates a new phaser with no initially registered parties, no
481 >     * parent, and initial phase number 0. Any thread using this
482       * phaser will need to first register for it.
483       */
484      public Phaser() {
485 <        this(null);
485 >        this(null, 0);
486      }
487  
488      /**
489 <     * Creates a new phaser with the given numbers of registered
490 <     * unarrived parties, initial phase number 0, and no parent.
489 >     * Creates a new phaser with the given number of registered
490 >     * unarrived parties, no parent, and initial phase number 0.
491       *
492 <     * @param parties the number of parties required to trip barrier
492 >     * @param parties the number of parties required to advance to the
493 >     * next phase
494       * @throws IllegalArgumentException if parties less than zero
495       * or greater than the maximum number of parties supported
496       */
# Line 362 | Line 499 | public class Phaser {
499      }
500  
501      /**
502 <     * Creates a new phaser with the given parent, without any
366 <     * initially registered parties. If parent is non-null this phaser
367 <     * is registered with the parent and its initial phase number is
368 <     * the same as that of parent phaser.
502 >     * Equivalent to {@link #Phaser(Phaser, int) Phaser(parent, 0)}.
503       *
504       * @param parent the parent phaser
505       */
506      public Phaser(Phaser parent) {
507 <        int phase = 0;
374 <        this.parent = parent;
375 <        if (parent != null) {
376 <            this.root = parent.root;
377 <            phase = parent.register();
378 <        }
379 <        else
380 <            this.root = this;
381 <        this.state = trippedStateFor(phase, 0);
507 >        this(parent, 0);
508      }
509  
510      /**
511 <     * Creates a new phaser with the given parent and numbers of
512 <     * registered unarrived parties. If parent is non-null, this phaser
513 <     * is registered with the parent and its initial phase number is
514 <     * the same as that of parent phaser.
511 >     * Creates a new phaser with the given parent and number of
512 >     * registered unarrived parties.  When the given parent is non-null
513 >     * and the given number of parties is greater than zero, this
514 >     * child phaser is registered with its parent.
515       *
516       * @param parent the parent phaser
517 <     * @param parties the number of parties required to trip barrier
517 >     * @param parties the number of parties required to advance to the
518 >     * next phase
519       * @throws IllegalArgumentException if parties less than zero
520       * or greater than the maximum number of parties supported
521       */
522      public Phaser(Phaser parent, int parties) {
523 <        if (parties < 0 || parties > ushortMask)
523 >        if (parties >>> PARTIES_SHIFT != 0)
524              throw new IllegalArgumentException("Illegal number of parties");
525          int phase = 0;
526          this.parent = parent;
527          if (parent != null) {
528 <            this.root = parent.root;
529 <            phase = parent.register();
528 >            final Phaser root = parent.root;
529 >            this.root = root;
530 >            this.evenQ = root.evenQ;
531 >            this.oddQ = root.oddQ;
532 >            if (parties != 0)
533 >                phase = parent.doRegister(1);
534          }
535 <        else
535 >        else {
536              this.root = this;
537 <        this.state = trippedStateFor(phase, parties);
537 >            this.evenQ = new AtomicReference<QNode>();
538 >            this.oddQ = new AtomicReference<QNode>();
539 >        }
540 >        this.state = (parties == 0) ? (long)EMPTY :
541 >            ((long)phase << PHASE_SHIFT) |
542 >            ((long)parties << PARTIES_SHIFT) |
543 >            ((long)parties);
544      }
545  
546      /**
547 <     * Adds a new unarrived party to this phaser.
548 <     *
549 <     * @return the arrival phase number to which this registration applied
547 >     * Adds a new unarrived party to this phaser.  If an ongoing
548 >     * invocation of {@link #onAdvance} is in progress, this method
549 >     * may await its completion before returning.  If this phaser has
550 >     * a parent, and this phaser previously had no registered parties,
551 >     * this child phaser is also registered with its parent. If
552 >     * this phaser is terminated, the attempt to register has
553 >     * no effect, and a negative value is returned.
554 >     *
555 >     * @return the arrival phase number to which this registration
556 >     * applied.  If this value is negative, then this phaser has
557 >     * terminated, in which case registration has no effect.
558       * @throws IllegalStateException if attempting to register more
559       * than the maximum supported number of parties
560       */
# Line 419 | Line 564 | public class Phaser {
564  
565      /**
566       * Adds the given number of new unarrived parties to this phaser.
567 <     *
568 <     * @param parties the number of parties required to trip barrier
569 <     * @return the arrival phase number to which this registration applied
567 >     * If an ongoing invocation of {@link #onAdvance} is in progress,
568 >     * this method may await its completion before returning.  If this
569 >     * phaser has a parent, and the given number of parties is greater
570 >     * than zero, and this phaser previously had no registered
571 >     * parties, this child phaser is also registered with its parent.
572 >     * If this phaser is terminated, the attempt to register has no
573 >     * effect, and a negative value is returned.
574 >     *
575 >     * @param parties the number of additional parties required to
576 >     * advance to the next phase
577 >     * @return the arrival phase number to which this registration
578 >     * applied.  If this value is negative, then this phaser has
579 >     * terminated, in which case registration has no effect.
580       * @throws IllegalStateException if attempting to register more
581       * than the maximum supported number of parties
582 +     * @throws IllegalArgumentException if {@code parties < 0}
583       */
584      public int bulkRegister(int parties) {
585          if (parties < 0)
# Line 434 | Line 590 | public class Phaser {
590      }
591  
592      /**
593 <     * Shared code for register, bulkRegister
594 <     */
595 <    private int doRegister(int registrations) {
596 <        int phase;
597 <        for (;;) {
598 <            long s = getReconciledState();
443 <            phase = phaseOf(s);
444 <            int unarrived = unarrivedOf(s) + registrations;
445 <            int parties = partiesOf(s) + registrations;
446 <            if (phase < 0)
447 <                break;
448 <            if (parties > ushortMask || unarrived > ushortMask)
449 <                throw new IllegalStateException(badBounds(parties, unarrived));
450 <            if (phase == phaseOf(root.state) &&
451 <                casState(s, stateFor(phase, parties, unarrived)))
452 <                break;
453 <        }
454 <        return phase;
455 <    }
456 <
457 <    /**
458 <     * Arrives at the barrier, but does not wait for others.  (You can
459 <     * in turn wait for others via {@link #awaitAdvance}).  It is an
460 <     * unenforced usage error for an unregistered party to invoke this
461 <     * method.
593 >     * Arrives at this phaser, without waiting for others to arrive.
594 >     *
595 >     * <p>It is a usage error for an unregistered party to invoke this
596 >     * method.  However, this error may result in an {@code
597 >     * IllegalStateException} only upon some subsequent operation on
598 >     * this phaser, if ever.
599       *
600       * @return the arrival phase number, or a negative value if terminated
601       * @throws IllegalStateException if not terminated and the number
602       * of unarrived parties would become negative
603       */
604      public int arrive() {
605 <        int phase;
469 <        for (;;) {
470 <            long s = state;
471 <            phase = phaseOf(s);
472 <            if (phase < 0)
473 <                break;
474 <            int parties = partiesOf(s);
475 <            int unarrived = unarrivedOf(s) - 1;
476 <            if (unarrived > 0) {        // Not the last arrival
477 <                if (casState(s, s - 1)) // s-1 adds one arrival
478 <                    break;
479 <            }
480 <            else if (unarrived == 0) {  // the last arrival
481 <                Phaser par = parent;
482 <                if (par == null) {      // directly trip
483 <                    if (casState
484 <                        (s,
485 <                         trippedStateFor(onAdvance(phase, parties) ? -1 :
486 <                                         ((phase + 1) & phaseMask), parties))) {
487 <                        releaseWaiters(phase);
488 <                        break;
489 <                    }
490 <                }
491 <                else {                  // cascade to parent
492 <                    if (casState(s, s - 1)) { // zeroes unarrived
493 <                        par.arrive();
494 <                        reconcileState();
495 <                        break;
496 <                    }
497 <                }
498 <            }
499 <            else if (phase != phaseOf(root.state)) // or if unreconciled
500 <                reconcileState();
501 <            else
502 <                throw new IllegalStateException(badBounds(parties, unarrived));
503 <        }
504 <        return phase;
605 >        return doArrive(ONE_ARRIVAL);
606      }
607  
608      /**
609 <     * Arrives at the barrier and deregisters from it without waiting
610 <     * for others. Deregistration reduces the number of parties
611 <     * required to trip the barrier in future phases.  If this phaser
609 >     * Arrives at this phaser and deregisters from it without waiting
610 >     * for others to arrive. Deregistration reduces the number of
611 >     * parties required to advance in future phases.  If this phaser
612       * has a parent, and deregistration causes this phaser to have
613 <     * zero parties, this phaser also arrives at and is deregistered
614 <     * from its parent.  It is an unenforced usage error for an
615 <     * unregistered party to invoke this method.
613 >     * zero parties, this phaser is also deregistered from its parent.
614 >     *
615 >     * <p>It is a usage error for an unregistered party to invoke this
616 >     * method.  However, this error may result in an {@code
617 >     * IllegalStateException} only upon some subsequent operation on
618 >     * this phaser, if ever.
619       *
620       * @return the arrival phase number, or a negative value if terminated
621       * @throws IllegalStateException if not terminated and the number
622       * of registered or unarrived parties would become negative
623       */
624      public int arriveAndDeregister() {
625 <        // similar code to arrive, but too different to merge
522 <        Phaser par = parent;
523 <        int phase;
524 <        for (;;) {
525 <            long s = state;
526 <            phase = phaseOf(s);
527 <            if (phase < 0)
528 <                break;
529 <            int parties = partiesOf(s) - 1;
530 <            int unarrived = unarrivedOf(s) - 1;
531 <            if (parties >= 0) {
532 <                if (unarrived > 0 || (unarrived == 0 && par != null)) {
533 <                    if (casState
534 <                        (s,
535 <                         stateFor(phase, parties, unarrived))) {
536 <                        if (unarrived == 0) {
537 <                            par.arriveAndDeregister();
538 <                            reconcileState();
539 <                        }
540 <                        break;
541 <                    }
542 <                    continue;
543 <                }
544 <                if (unarrived == 0) {
545 <                    if (casState
546 <                        (s,
547 <                         trippedStateFor(onAdvance(phase, parties) ? -1 :
548 <                                         ((phase + 1) & phaseMask), parties))) {
549 <                        releaseWaiters(phase);
550 <                        break;
551 <                    }
552 <                    continue;
553 <                }
554 <                if (par != null && phase != phaseOf(root.state)) {
555 <                    reconcileState();
556 <                    continue;
557 <                }
558 <            }
559 <            throw new IllegalStateException(badBounds(parties, unarrived));
560 <        }
561 <        return phase;
625 >        return doArrive(ONE_DEREGISTER);
626      }
627  
628      /**
629 <     * Arrives at the barrier and awaits others. Equivalent in effect
629 >     * Arrives at this phaser and awaits others. Equivalent in effect
630       * to {@code awaitAdvance(arrive())}.  If you need to await with
631       * interruption or timeout, you can arrange this with an analogous
632 <     * construction using one of the other forms of the awaitAdvance
633 <     * method.  If instead you need to deregister upon arrival use
634 <     * {@code arriveAndDeregister}. It is an unenforced usage error
635 <     * for an unregistered party to invoke this method.
632 >     * construction using one of the other forms of the {@code
633 >     * awaitAdvance} method.  If instead you need to deregister upon
634 >     * arrival, use {@code awaitAdvance(arriveAndDeregister())}.
635 >     *
636 >     * <p>It is a usage error for an unregistered party to invoke this
637 >     * method.  However, this error may result in an {@code
638 >     * IllegalStateException} only upon some subsequent operation on
639 >     * this phaser, if ever.
640       *
641 <     * @return the arrival phase number, or a negative number if terminated
641 >     * @return the arrival phase number, or the (negative)
642 >     * {@linkplain #getPhase() current phase} if terminated
643       * @throws IllegalStateException if not terminated and the number
644       * of unarrived parties would become negative
645       */
646      public int arriveAndAwaitAdvance() {
647 <        return awaitAdvance(arrive());
647 >        // Specialization of doArrive+awaitAdvance eliminating some reads/paths
648 >        final Phaser root = this.root;
649 >        for (;;) {
650 >            long s = (root == this) ? state : reconcileState();
651 >            int phase = (int)(s >>> PHASE_SHIFT);
652 >            if (phase < 0)
653 >                return phase;
654 >            int counts = (int)s;
655 >            int unarrived = (counts == EMPTY) ? 0 : (counts & UNARRIVED_MASK);
656 >            if (unarrived <= 0)
657 >                throw new IllegalStateException(badArrive(s));
658 >            if (UNSAFE.compareAndSwapLong(this, stateOffset, s,
659 >                                          s -= ONE_ARRIVAL)) {
660 >                if (unarrived > 1)
661 >                    return root.internalAwaitAdvance(phase, null);
662 >                if (root != this)
663 >                    return parent.arriveAndAwaitAdvance();
664 >                long n = s & PARTIES_MASK;  // base of next state
665 >                int nextUnarrived = (int)n >>> PARTIES_SHIFT;
666 >                if (onAdvance(phase, nextUnarrived))
667 >                    n |= TERMINATION_BIT;
668 >                else if (nextUnarrived == 0)
669 >                    n |= EMPTY;
670 >                else
671 >                    n |= nextUnarrived;
672 >                int nextPhase = (phase + 1) & MAX_PHASE;
673 >                n |= (long)nextPhase << PHASE_SHIFT;
674 >                if (!UNSAFE.compareAndSwapLong(this, stateOffset, s, n))
675 >                    return (int)(state >>> PHASE_SHIFT); // terminated
676 >                releaseWaiters(phase);
677 >                return nextPhase;
678 >            }
679 >        }
680      }
681  
682      /**
683 <     * Awaits the phase of the barrier to advance from the given phase
684 <     * value, returning immediately if the current phase of the
685 <     * barrier is not equal to the given phase value or this barrier
585 <     * is terminated.  It is an unenforced usage error for an
586 <     * unregistered party to invoke this method.
683 >     * Awaits the phase of this phaser to advance from the given phase
684 >     * value, returning immediately if the current phase is not equal
685 >     * to the given phase value or this phaser is terminated.
686       *
687       * @param phase an arrival phase number, or negative value if
688       * terminated; this argument is normally the value returned by a
689 <     * previous call to {@code arrive} or its variants
690 <     * @return the next arrival phase number, or a negative value
691 <     * if terminated or argument is negative
689 >     * previous call to {@code arrive} or {@code arriveAndDeregister}.
690 >     * @return the next arrival phase number, or the argument if it is
691 >     * negative, or the (negative) {@linkplain #getPhase() current phase}
692 >     * if terminated
693       */
694      public int awaitAdvance(int phase) {
695 +        final Phaser root = this.root;
696 +        long s = (root == this) ? state : reconcileState();
697 +        int p = (int)(s >>> PHASE_SHIFT);
698          if (phase < 0)
699              return phase;
700 <        long s = getReconciledState();
701 <        int p = phaseOf(s);
702 <        if (p != phase)
600 <            return p;
601 <        if (unarrivedOf(s) == 0 && parent != null)
602 <            parent.awaitAdvance(phase);
603 <        // Fall here even if parent waited, to reconcile and help release
604 <        return untimedWait(phase);
700 >        if (p == phase)
701 >            return root.internalAwaitAdvance(phase, null);
702 >        return p;
703      }
704  
705      /**
706 <     * Awaits the phase of the barrier to advance from the given phase
706 >     * Awaits the phase of this phaser to advance from the given phase
707       * value, throwing {@code InterruptedException} if interrupted
708 <     * while waiting, or returning immediately if the current phase of
709 <     * the barrier is not equal to the given phase value or this
710 <     * barrier is terminated. It is an unenforced usage error for an
613 <     * unregistered party to invoke this method.
708 >     * while waiting, or returning immediately if the current phase is
709 >     * not equal to the given phase value or this phaser is
710 >     * terminated.
711       *
712       * @param phase an arrival phase number, or negative value if
713       * terminated; this argument is normally the value returned by a
714 <     * previous call to {@code arrive} or its variants
715 <     * @return the next arrival phase number, or a negative value
716 <     * if terminated or argument is negative
714 >     * previous call to {@code arrive} or {@code arriveAndDeregister}.
715 >     * @return the next arrival phase number, or the argument if it is
716 >     * negative, or the (negative) {@linkplain #getPhase() current phase}
717 >     * if terminated
718       * @throws InterruptedException if thread interrupted while waiting
719       */
720      public int awaitAdvanceInterruptibly(int phase)
721          throws InterruptedException {
722 +        final Phaser root = this.root;
723 +        long s = (root == this) ? state : reconcileState();
724 +        int p = (int)(s >>> PHASE_SHIFT);
725          if (phase < 0)
726              return phase;
727 <        long s = getReconciledState();
728 <        int p = phaseOf(s);
729 <        if (p != phase)
730 <            return p;
731 <        if (unarrivedOf(s) == 0 && parent != null)
732 <            parent.awaitAdvanceInterruptibly(phase);
733 <        return interruptibleWait(phase);
727 >        if (p == phase) {
728 >            QNode node = new QNode(this, phase, true, false, 0L);
729 >            p = root.internalAwaitAdvance(phase, node);
730 >            if (node.wasInterrupted)
731 >                throw new InterruptedException();
732 >        }
733 >        return p;
734      }
735  
736      /**
737 <     * Awaits the phase of the barrier to advance from the given phase
737 >     * Awaits the phase of this phaser to advance from the given phase
738       * value or the given timeout to elapse, throwing {@code
739       * InterruptedException} if interrupted while waiting, or
740 <     * returning immediately if the current phase of the barrier is
741 <     * not equal to the given phase value or this barrier is
641 <     * terminated.  It is an unenforced usage error for an
642 <     * unregistered party to invoke this method.
740 >     * returning immediately if the current phase is not equal to the
741 >     * given phase value or this phaser is terminated.
742       *
743       * @param phase an arrival phase number, or negative value if
744       * terminated; this argument is normally the value returned by a
745 <     * previous call to {@code arrive} or its variants
745 >     * previous call to {@code arrive} or {@code arriveAndDeregister}.
746       * @param timeout how long to wait before giving up, in units of
747       *        {@code unit}
748       * @param unit a {@code TimeUnit} determining how to interpret the
749       *        {@code timeout} parameter
750 <     * @return the next arrival phase number, or a negative value
751 <     * if terminated or argument is negative
750 >     * @return the next arrival phase number, or the argument if it is
751 >     * negative, or the (negative) {@linkplain #getPhase() current phase}
752 >     * if terminated
753       * @throws InterruptedException if thread interrupted while waiting
754       * @throws TimeoutException if timed out while waiting
755       */
756      public int awaitAdvanceInterruptibly(int phase,
757                                           long timeout, TimeUnit unit)
758          throws InterruptedException, TimeoutException {
759 +        long nanos = unit.toNanos(timeout);
760 +        final Phaser root = this.root;
761 +        long s = (root == this) ? state : reconcileState();
762 +        int p = (int)(s >>> PHASE_SHIFT);
763          if (phase < 0)
764              return phase;
765 <        long s = getReconciledState();
766 <        int p = phaseOf(s);
767 <        if (p != phase)
768 <            return p;
769 <        if (unarrivedOf(s) == 0 && parent != null)
770 <            parent.awaitAdvanceInterruptibly(phase, timeout, unit);
771 <        return timedWait(phase, unit.toNanos(timeout));
765 >        if (p == phase) {
766 >            QNode node = new QNode(this, phase, true, true, nanos);
767 >            p = root.internalAwaitAdvance(phase, node);
768 >            if (node.wasInterrupted)
769 >                throw new InterruptedException();
770 >            else if (p == phase)
771 >                throw new TimeoutException();
772 >        }
773 >        return p;
774      }
775  
776      /**
777 <     * Forces this barrier to enter termination state. Counts of
778 <     * arrived and registered parties are unaffected. If this phaser
779 <     * has a parent, it too is terminated. This method may be useful
780 <     * for coordinating recovery after one or more tasks encounter
777 >     * Forces this phaser to enter termination state.  Counts of
778 >     * registered parties are unaffected.  If this phaser is a member
779 >     * of a tiered set of phasers, then all of the phasers in the set
780 >     * are terminated.  If this phaser is already terminated, this
781 >     * method has no effect.  This method may be useful for
782 >     * coordinating recovery after one or more tasks encounter
783       * unexpected exceptions.
784       */
785      public void forceTermination() {
786 <        for (;;) {
787 <            long s = getReconciledState();
788 <            int phase = phaseOf(s);
789 <            int parties = partiesOf(s);
790 <            int unarrived = unarrivedOf(s);
791 <            if (phase < 0 ||
792 <                casState(s, stateFor(-1, parties, unarrived))) {
793 <                releaseWaiters(0);
794 <                releaseWaiters(1);
687 <                if (parent != null)
688 <                    parent.forceTermination();
786 >        // Only need to change root state
787 >        final Phaser root = this.root;
788 >        long s;
789 >        while ((s = root.state) >= 0) {
790 >            if (UNSAFE.compareAndSwapLong(root, stateOffset,
791 >                                          s, s | TERMINATION_BIT)) {
792 >                // signal all threads
793 >                releaseWaiters(0); // Waiters on evenQ
794 >                releaseWaiters(1); // Waiters on oddQ
795                  return;
796              }
797          }
# Line 694 | Line 800 | public class Phaser {
800      /**
801       * Returns the current phase number. The maximum phase number is
802       * {@code Integer.MAX_VALUE}, after which it restarts at
803 <     * zero. Upon termination, the phase number is negative.
803 >     * zero. Upon termination, the phase number is negative,
804 >     * in which case the prevailing phase prior to termination
805 >     * may be obtained via {@code getPhase() + Integer.MIN_VALUE}.
806       *
807       * @return the phase number, or a negative value if terminated
808       */
809      public final int getPhase() {
810 <        return phaseOf(getReconciledState());
810 >        return (int)(root.state >>> PHASE_SHIFT);
811      }
812  
813      /**
814 <     * Returns the number of parties registered at this barrier.
814 >     * Returns the number of parties registered at this phaser.
815       *
816       * @return the number of parties
817       */
# Line 713 | Line 821 | public class Phaser {
821  
822      /**
823       * Returns the number of registered parties that have arrived at
824 <     * the current phase of this barrier.
824 >     * the current phase of this phaser. If this phaser has terminated,
825 >     * the returned value is meaningless and arbitrary.
826       *
827       * @return the number of arrived parties
828       */
829      public int getArrivedParties() {
830 <        return arrivedOf(state);
830 >        return arrivedOf(reconcileState());
831      }
832  
833      /**
834       * Returns the number of registered parties that have not yet
835 <     * arrived at the current phase of this barrier.
835 >     * arrived at the current phase of this phaser. If this phaser has
836 >     * terminated, the returned value is meaningless and arbitrary.
837       *
838       * @return the number of unarrived parties
839       */
840      public int getUnarrivedParties() {
841 <        return unarrivedOf(state);
841 >        return unarrivedOf(reconcileState());
842      }
843  
844      /**
# Line 751 | Line 861 | public class Phaser {
861      }
862  
863      /**
864 <     * Returns {@code true} if this barrier has been terminated.
864 >     * Returns {@code true} if this phaser has been terminated.
865       *
866 <     * @return {@code true} if this barrier has been terminated
866 >     * @return {@code true} if this phaser has been terminated
867       */
868      public boolean isTerminated() {
869 <        return getPhase() < 0;
869 >        return root.state < 0L;
870      }
871  
872      /**
873 <     * Overridable method to perform an action upon phase advance, and
874 <     * to control termination. This method is invoked whenever the
875 <     * barrier is tripped (and thus all other waiting parties are
876 <     * dormant). If it returns {@code true}, then, rather than advance
877 <     * the phase number, this barrier will be set to a final
878 <     * termination state, and subsequent calls to {@link #isTerminated}
879 <     * will return true.
880 <     *
881 <     * <p>The default version returns {@code true} when the number of
882 <     * registered parties is zero. Normally, overrides that arrange
883 <     * termination for other reasons should also preserve this
884 <     * property.
885 <     *
886 <     * <p>You may override this method to perform an action with side
887 <     * effects visible to participating tasks, but it is in general
888 <     * only sensible to do so in designs where all parties register
889 <     * before any arrive, and all {@link #awaitAdvance} at each phase.
890 <     * Otherwise, you cannot ensure lack of interference from other
891 <     * parties during the invocation of this method.
873 >     * Overridable method to perform an action upon impending phase
874 >     * advance, and to control termination. This method is invoked
875 >     * upon arrival of the party advancing this phaser (when all other
876 >     * waiting parties are dormant).  If this method returns {@code
877 >     * true}, this phaser will be set to a final termination state
878 >     * upon advance, and subsequent calls to {@link #isTerminated}
879 >     * will return true. Any (unchecked) Exception or Error thrown by
880 >     * an invocation of this method is propagated to the party
881 >     * attempting to advance this phaser, in which case no advance
882 >     * occurs.
883 >     *
884 >     * <p>The arguments to this method provide the state of the phaser
885 >     * prevailing for the current transition.  The effects of invoking
886 >     * arrival, registration, and waiting methods on this phaser from
887 >     * within {@code onAdvance} are unspecified and should not be
888 >     * relied on.
889 >     *
890 >     * <p>If this phaser is a member of a tiered set of phasers, then
891 >     * {@code onAdvance} is invoked only for its root phaser on each
892 >     * advance.
893 >     *
894 >     * <p>To support the most common use cases, the default
895 >     * implementation of this method returns {@code true} when the
896 >     * number of registered parties has become zero as the result of a
897 >     * party invoking {@code arriveAndDeregister}.  You can disable
898 >     * this behavior, thus enabling continuation upon future
899 >     * registrations, by overriding this method to always return
900 >     * {@code false}:
901 >     *
902 >     * <pre> {@code
903 >     * Phaser phaser = new Phaser() {
904 >     *   protected boolean onAdvance(int phase, int parties) { return false; }
905 >     * }}</pre>
906       *
907 <     * @param phase the phase number on entering the barrier
907 >     * @param phase the current phase number on entry to this method,
908 >     * before this phaser is advanced
909       * @param registeredParties the current number of registered parties
910 <     * @return {@code true} if this barrier should terminate
910 >     * @return {@code true} if this phaser should terminate
911       */
912      protected boolean onAdvance(int phase, int registeredParties) {
913 <        return registeredParties <= 0;
913 >        return registeredParties == 0;
914      }
915  
916      /**
# Line 795 | Line 920 | public class Phaser {
920       * followed by the number of registered parties, and {@code
921       * "arrived = "} followed by the number of arrived parties.
922       *
923 <     * @return a string identifying this barrier, as well as its state
923 >     * @return a string identifying this phaser, as well as its state
924       */
925      public String toString() {
926 <        long s = getReconciledState();
926 >        return stateToString(reconcileState());
927 >    }
928 >
929 >    /**
930 >     * Implementation of toString and string-based error messages
931 >     */
932 >    private String stateToString(long s) {
933          return super.toString() +
934              "[phase = " + phaseOf(s) +
935              " parties = " + partiesOf(s) +
936              " arrived = " + arrivedOf(s) + "]";
937      }
938  
939 <    // methods for waiting
939 >    // Waiting mechanics
940  
941      /**
942 <     * Wait nodes for Treiber stack representing wait queue
942 >     * Removes and signals threads from queue for phase.
943       */
944 <    static final class QNode implements ForkJoinPool.ManagedBlocker {
945 <        final Phaser phaser;
946 <        final int phase;
947 <        final long startTime;
948 <        final long nanos;
949 <        final boolean timed;
950 <        final boolean interruptible;
951 <        volatile boolean wasInterrupted = false;
952 <        volatile Thread thread; // nulled to cancel wait
822 <        QNode next;
823 <        QNode(Phaser phaser, int phase, boolean interruptible,
824 <              boolean timed, long startTime, long nanos) {
825 <            this.phaser = phaser;
826 <            this.phase = phase;
827 <            this.timed = timed;
828 <            this.interruptible = interruptible;
829 <            this.startTime = startTime;
830 <            this.nanos = nanos;
831 <            thread = Thread.currentThread();
832 <        }
833 <        public boolean isReleasable() {
834 <            return (thread == null ||
835 <                    phaser.getPhase() != phase ||
836 <                    (interruptible && wasInterrupted) ||
837 <                    (timed && (nanos - (System.nanoTime() - startTime)) <= 0));
838 <        }
839 <        public boolean block() {
840 <            if (Thread.interrupted()) {
841 <                wasInterrupted = true;
842 <                if (interruptible)
843 <                    return true;
844 <            }
845 <            if (!timed)
846 <                LockSupport.park(this);
847 <            else {
848 <                long waitTime = nanos - (System.nanoTime() - startTime);
849 <                if (waitTime <= 0)
850 <                    return true;
851 <                LockSupport.parkNanos(this, waitTime);
852 <            }
853 <            return isReleasable();
854 <        }
855 <        void signal() {
856 <            Thread t = thread;
857 <            if (t != null) {
858 <                thread = null;
944 >    private void releaseWaiters(int phase) {
945 >        QNode q;   // first element of queue
946 >        Thread t;  // its thread
947 >        AtomicReference<QNode> head = (phase & 1) == 0 ? evenQ : oddQ;
948 >        while ((q = head.get()) != null &&
949 >               q.phase != (int)(root.state >>> PHASE_SHIFT)) {
950 >            if (head.compareAndSet(q, q.next) &&
951 >                (t = q.thread) != null) {
952 >                q.thread = null;
953                  LockSupport.unpark(t);
954              }
955          }
862        boolean doWait() {
863            if (thread != null) {
864                try {
865                    ForkJoinPool.managedBlock(this, false);
866                } catch (InterruptedException ie) {
867                }
868            }
869            return wasInterrupted;
870        }
871
956      }
957  
958      /**
959 <     * Removes and signals waiting threads from wait queue.
959 >     * Variant of releaseWaiters that additionally tries to remove any
960 >     * nodes no longer waiting for advance due to timeout or
961 >     * interrupt. Currently, nodes are removed only if they are at
962 >     * head of queue, which suffices to reduce memory footprint in
963 >     * most usages.
964 >     *
965 >     * @return current phase on exit
966       */
967 <    private void releaseWaiters(int phase) {
968 <        AtomicReference<QNode> head = queueFor(phase);
969 <        QNode q;
970 <        while ((q = head.get()) != null) {
971 <            if (head.compareAndSet(q, q.next))
972 <                q.signal();
967 >    private int abortWait(int phase) {
968 >        AtomicReference<QNode> head = (phase & 1) == 0 ? evenQ : oddQ;
969 >        for (;;) {
970 >            Thread t;
971 >            QNode q = head.get();
972 >            int p = (int)(root.state >>> PHASE_SHIFT);
973 >            if (q == null || ((t = q.thread) != null && q.phase == p))
974 >                return p;
975 >            if (head.compareAndSet(q, q.next) && t != null) {
976 >                q.thread = null;
977 >                LockSupport.unpark(t);
978 >            }
979          }
980      }
981  
982 +    /** The number of CPUs, for spin control */
983 +    private static final int NCPU = Runtime.getRuntime().availableProcessors();
984 +
985      /**
986 <     * Tries to enqueue given node in the appropriate wait queue.
987 <     *
988 <     * @return true if successful
986 >     * The number of times to spin before blocking while waiting for
987 >     * advance, per arrival while waiting. On multiprocessors, fully
988 >     * blocking and waking up a large number of threads all at once is
989 >     * usually a very slow process, so we use rechargeable spins to
990 >     * avoid it when threads regularly arrive: When a thread in
991 >     * internalAwaitAdvance notices another arrival before blocking,
992 >     * and there appear to be enough CPUs available, it spins
993 >     * SPINS_PER_ARRIVAL more times before blocking. The value trades
994 >     * off good-citizenship vs big unnecessary slowdowns.
995       */
996 <    private boolean tryEnqueue(QNode node) {
892 <        AtomicReference<QNode> head = queueFor(node.phase);
893 <        return head.compareAndSet(node.next = head.get(), node);
894 <    }
996 >    static final int SPINS_PER_ARRIVAL = (NCPU < 2) ? 1 : 1 << 8;
997  
998      /**
999 <     * Enqueues node and waits unless aborted or signalled.
999 >     * Possibly blocks and waits for phase to advance unless aborted.
1000 >     * Call only on root phaser.
1001       *
1002 +     * @param phase current phase
1003 +     * @param node if non-null, the wait node to track interrupt and timeout;
1004 +     * if null, denotes noninterruptible wait
1005       * @return current phase
1006       */
1007 <    private int untimedWait(int phase) {
1008 <        QNode node = null;
1009 <        boolean queued = false;
1010 <        boolean interrupted = false;
1007 >    private int internalAwaitAdvance(int phase, QNode node) {
1008 >        // assert root == this;
1009 >        releaseWaiters(phase-1);          // ensure old queue clean
1010 >        boolean queued = false;           // true when node is enqueued
1011 >        int lastUnarrived = 0;            // to increase spins upon change
1012 >        int spins = SPINS_PER_ARRIVAL;
1013 >        long s;
1014          int p;
1015 <        while ((p = getPhase()) == phase) {
1016 <            if (Thread.interrupted())
1017 <                interrupted = true;
1018 <            else if (node == null)
1019 <                node = new QNode(this, phase, false, false, 0, 0);
1020 <            else if (!queued)
1021 <                queued = tryEnqueue(node);
1022 <            else
1023 <                interrupted = node.doWait();
1015 >        while ((p = (int)((s = state) >>> PHASE_SHIFT)) == phase) {
1016 >            if (node == null) {           // spinning in noninterruptible mode
1017 >                int unarrived = (int)s & UNARRIVED_MASK;
1018 >                if (unarrived != lastUnarrived &&
1019 >                    (lastUnarrived = unarrived) < NCPU)
1020 >                    spins += SPINS_PER_ARRIVAL;
1021 >                boolean interrupted = Thread.interrupted();
1022 >                if (interrupted || --spins < 0) { // need node to record intr
1023 >                    node = new QNode(this, phase, false, false, 0L);
1024 >                    node.wasInterrupted = interrupted;
1025 >                }
1026 >            }
1027 >            else if (node.isReleasable()) // done or aborted
1028 >                break;
1029 >            else if (!queued) {           // push onto queue
1030 >                AtomicReference<QNode> head = (phase & 1) == 0 ? evenQ : oddQ;
1031 >                QNode q = node.next = head.get();
1032 >                if ((q == null || q.phase == phase) &&
1033 >                    (int)(state >>> PHASE_SHIFT) == phase) // avoid stale enq
1034 >                    queued = head.compareAndSet(q, node);
1035 >            }
1036 >            else {
1037 >                try {
1038 >                    ForkJoinPool.managedBlock(node);
1039 >                } catch (InterruptedException ie) {
1040 >                    node.wasInterrupted = true;
1041 >                }
1042 >            }
1043 >        }
1044 >
1045 >        if (node != null) {
1046 >            if (node.thread != null)
1047 >                node.thread = null;       // avoid need for unpark()
1048 >            if (node.wasInterrupted && !node.interruptible)
1049 >                Thread.currentThread().interrupt();
1050 >            if (p == phase && (p = (int)(state >>> PHASE_SHIFT)) == phase)
1051 >                return abortWait(phase); // possibly clean up on abort
1052          }
916        if (node != null)
917            node.thread = null;
1053          releaseWaiters(phase);
919        if (interrupted)
920            Thread.currentThread().interrupt();
1054          return p;
1055      }
1056  
1057      /**
1058 <     * Interruptible version
926 <     * @return current phase
1058 >     * Wait nodes for Treiber stack representing wait queue
1059       */
1060 <    private int interruptibleWait(int phase) throws InterruptedException {
1061 <        QNode node = null;
1062 <        boolean queued = false;
1063 <        boolean interrupted = false;
1064 <        int p;
1065 <        while ((p = getPhase()) == phase && !interrupted) {
1066 <            if (Thread.interrupted())
1067 <                interrupted = true;
1068 <            else if (node == null)
1069 <                node = new QNode(this, phase, true, false, 0, 0);
1070 <            else if (!queued)
1071 <                queued = tryEnqueue(node);
1072 <            else
1073 <                interrupted = node.doWait();
1060 >    static final class QNode implements ForkJoinPool.ManagedBlocker {
1061 >        final Phaser phaser;
1062 >        final int phase;
1063 >        final boolean interruptible;
1064 >        final boolean timed;
1065 >        boolean wasInterrupted;
1066 >        long nanos;
1067 >        long lastTime;
1068 >        volatile Thread thread; // nulled to cancel wait
1069 >        QNode next;
1070 >
1071 >        QNode(Phaser phaser, int phase, boolean interruptible,
1072 >              boolean timed, long nanos) {
1073 >            this.phaser = phaser;
1074 >            this.phase = phase;
1075 >            this.interruptible = interruptible;
1076 >            this.nanos = nanos;
1077 >            this.timed = timed;
1078 >            this.lastTime = timed ? System.nanoTime() : 0L;
1079 >            thread = Thread.currentThread();
1080          }
943        if (node != null)
944            node.thread = null;
945        if (p != phase || (p = getPhase()) != phase)
946            releaseWaiters(phase);
947        if (interrupted)
948            throw new InterruptedException();
949        return p;
950    }
1081  
1082 <    /**
1083 <     * Timeout version.
1084 <     * @return current phase
1085 <     */
1086 <    private int timedWait(int phase, long nanos)
1087 <        throws InterruptedException, TimeoutException {
1088 <        long startTime = System.nanoTime();
959 <        QNode node = null;
960 <        boolean queued = false;
961 <        boolean interrupted = false;
962 <        int p;
963 <        while ((p = getPhase()) == phase && !interrupted) {
1082 >        public boolean isReleasable() {
1083 >            if (thread == null)
1084 >                return true;
1085 >            if (phaser.getPhase() != phase) {
1086 >                thread = null;
1087 >                return true;
1088 >            }
1089              if (Thread.interrupted())
1090 <                interrupted = true;
1091 <            else if (nanos - (System.nanoTime() - startTime) <= 0)
1092 <                break;
1093 <            else if (node == null)
1094 <                node = new QNode(this, phase, true, true, startTime, nanos);
1095 <            else if (!queued)
1096 <                queued = tryEnqueue(node);
1097 <            else
1098 <                interrupted = node.doWait();
1090 >                wasInterrupted = true;
1091 >            if (wasInterrupted && interruptible) {
1092 >                thread = null;
1093 >                return true;
1094 >            }
1095 >            if (timed) {
1096 >                if (nanos > 0L) {
1097 >                    long now = System.nanoTime();
1098 >                    nanos -= now - lastTime;
1099 >                    lastTime = now;
1100 >                }
1101 >                if (nanos <= 0L) {
1102 >                    thread = null;
1103 >                    return true;
1104 >                }
1105 >            }
1106 >            return false;
1107 >        }
1108 >
1109 >        public boolean block() {
1110 >            if (isReleasable())
1111 >                return true;
1112 >            else if (!timed)
1113 >                LockSupport.park(this);
1114 >            else if (nanos > 0)
1115 >                LockSupport.parkNanos(this, nanos);
1116 >            return isReleasable();
1117          }
975        if (node != null)
976            node.thread = null;
977        if (p != phase || (p = getPhase()) != phase)
978            releaseWaiters(phase);
979        if (interrupted)
980            throw new InterruptedException();
981        if (p == phase)
982            throw new TimeoutException();
983        return p;
1118      }
1119  
1120      // Unsafe mechanics
1121  
1122 <    private static final sun.misc.Unsafe UNSAFE = getUnsafe();
1123 <    private static final long stateOffset =
1124 <        objectFieldOffset("state", Phaser.class);
991 <
992 <    private final boolean casState(long cmp, long val) {
993 <        return UNSAFE.compareAndSwapLong(this, stateOffset, cmp, val);
994 <    }
995 <
996 <    private static long objectFieldOffset(String field, Class<?> klazz) {
1122 >    private static final sun.misc.Unsafe UNSAFE;
1123 >    private static final long stateOffset;
1124 >    static {
1125          try {
1126 <            return UNSAFE.objectFieldOffset(klazz.getDeclaredField(field));
1127 <        } catch (NoSuchFieldException e) {
1128 <            // Convert Exception to corresponding Error
1129 <            NoSuchFieldError error = new NoSuchFieldError(field);
1130 <            error.initCause(e);
1131 <            throw error;
1126 >            UNSAFE = getUnsafe();
1127 >            Class<?> k = Phaser.class;
1128 >            stateOffset = UNSAFE.objectFieldOffset
1129 >                (k.getDeclaredField("state"));
1130 >        } catch (Exception e) {
1131 >            throw new Error(e);
1132          }
1133      }
1134  
# Line 1014 | Line 1142 | public class Phaser {
1142      private static sun.misc.Unsafe getUnsafe() {
1143          try {
1144              return sun.misc.Unsafe.getUnsafe();
1145 <        } catch (SecurityException se) {
1146 <            try {
1147 <                return java.security.AccessController.doPrivileged
1148 <                    (new java.security
1149 <                     .PrivilegedExceptionAction<sun.misc.Unsafe>() {
1150 <                        public sun.misc.Unsafe run() throws Exception {
1151 <                            java.lang.reflect.Field f = sun.misc
1152 <                                .Unsafe.class.getDeclaredField("theUnsafe");
1153 <                            f.setAccessible(true);
1154 <                            return (sun.misc.Unsafe) f.get(null);
1155 <                        }});
1156 <            } catch (java.security.PrivilegedActionException e) {
1157 <                throw new RuntimeException("Could not initialize intrinsics",
1158 <                                           e.getCause());
1159 <            }
1145 >        } catch (SecurityException tryReflectionInstead) {}
1146 >        try {
1147 >            return java.security.AccessController.doPrivileged
1148 >            (new java.security.PrivilegedExceptionAction<sun.misc.Unsafe>() {
1149 >                public sun.misc.Unsafe run() throws Exception {
1150 >                    Class<sun.misc.Unsafe> k = sun.misc.Unsafe.class;
1151 >                    for (java.lang.reflect.Field f : k.getDeclaredFields()) {
1152 >                        f.setAccessible(true);
1153 >                        Object x = f.get(null);
1154 >                        if (k.isInstance(x))
1155 >                            return k.cast(x);
1156 >                    }
1157 >                    throw new NoSuchFieldError("the Unsafe");
1158 >                }});
1159 >        } catch (java.security.PrivilegedActionException e) {
1160 >            throw new RuntimeException("Could not initialize intrinsics",
1161 >                                       e.getCause());
1162          }
1163      }
1164   }

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines