ViewVC Help

View File | Revision Log | Show Annotations | Download File | Root Listing

root/jsr166/jsr166/src/jsr166y/ForkJoinPool.java

Comparing jsr166/src/jsr166y/ForkJoinPool.java (file contents):
Revision 1.57 by dl, Wed Jul 7 19:52:31 2010 UTC vs.
Revision 1.59 by dl, Fri Jul 23 14:09:17 2010 UTC

#	Line 60 | Line 60 | import java.util.concurrent.CountDownLat
60		* Runnable}- or {@code Callable}- based activities as well.  However,
61		* tasks that are already executing in a pool should normally
62		* <em>NOT</em> use these pool execution methods, but instead use the
63	<	* within-computation forms listed in the table. To avoid inadvertant
64	<	* cyclic task dependencies and to improve performance, task
65	<	* submissions to the current pool by an ongoing fork/join
66	<	* computations may be implicitly translated to the corresponding
67	<	* ForkJoinTask forms.
63	>	* within-computation forms listed in the table.
64		*
65		* <table BORDER CELLPADDING=3 CELLSPACING=1>
66		*  <tr>
#	Line 88 | Line 84 | import java.util.concurrent.CountDownLat
84		*    <td> {@link ForkJoinTask#fork} (ForkJoinTasks <em>are</em> Futures)</td>
85		*  </tr>
86		* </table>
87	<	*
87	>	*
88		* <p><b>Sample Usage.</b> Normally a single {@code ForkJoinPool} is
89		* used for all parallel task execution in a program or subsystem.
90		* Otherwise, use would not usually outweigh the construction and
#	Line 113 | Line 109 | import java.util.concurrent.CountDownLat
109		* {@code IllegalArgumentException}.
110		*
111		* <p>This implementation rejects submitted tasks (that is, by throwing
112	<	* {@link RejectedExecutionException}) only when the pool is shut down.
112	>	* {@link RejectedExecutionException}) only when the pool is shut down
113	>	* or internal resources have been exhuasted.
114		*
115		* @since 1.7
116		* @author Doug Lea
#	Line 140 | Line 137 | public class ForkJoinPool extends Abstra
137		* of tasks profit from cache affinities, but others are harmed by
138		* cache pollution effects.)
139		*
140	+	* Beyond work-stealing support and essential bookkeeping, the
141	+	* main responsibility of this framework is to arrange tactics for
142	+	* when one worker is waiting to join a task stolen (or always
143	+	* held by) another.  Becauae we are multiplexing many tasks on to
144	+	* a pool of workers, we can't just let them block (as in
145	+	* Thread.join).  We also cannot just reassign the joiner's
146	+	* run-time stack with another and replace it later, which would
147	+	* be a form of "continuation", that even if possible is not
148	+	* necessarily a good idea. Given that the creation costs of most
149	+	* threads on most systems mainly surrounds setting up runtime
150	+	* stacks, thread creation and switching is usually not much more
151	+	* expensive than stack creation and switching, and is more
152	+	* flexible). Instead we combine two tactics:
153	+	*
154	+	*   1. Arranging for the joiner to execute some task that it
155	+	*      would be running if the steal had not occurred.  Method
156	+	*      ForkJoinWorkerThread.helpJoinTask tracks joining->stealing
157	+	*      links to try to find such a task.
158	+	*
159	+	*   2. Unless there are already enough live threads, creating or
160	+	*      or re-activating a spare thread to compensate for the
161	+	*      (blocked) joiner until it unblocks.  Spares then suspend
162	+	*      at their next opportunity or eventually die if unused for
163	+	*      too long.  See below and the internal documentation
164	+	*      for tryAwaitJoin for more details about compensation
165	+	*      rules.
166	+	*
167	+	* Because the determining existence of conservatively safe
168	+	* helping targets, the availability of already-created spares,
169	+	* and the apparent need to create new spares are all racy and
170	+	* require heuristic guidance, joins (in
171	+	* ForkJoinWorkerThread.joinTask) interleave these options until
172	+	* successful.  Creating a new spare always succeeds, but also
173	+	* increases application footprint, so we try to avoid it, within
174	+	* reason.
175	+	*
176	+	* The ManagedBlocker extension API can't use option (1) so uses a
177	+	* special version of (2) in method awaitBlocker.
178	+	*
179		* The main throughput advantages of work-stealing stem from
180		* decentralized control -- workers mostly steal tasks from each
181		* other. We do not want to negate this by creating bottlenecks
182	<	* implementing the management responsibilities of this class. So
183	<	* we use a collection of techniques that avoid, reduce, or cope
184	<	* well with contention. These entail several instances of
185	<	* bit-packing into CASable fields to maintain only the minimally
186	<	* required atomicity. To enable such packing, we restrict maximum
187	<	* parallelism to (1<<15)-1 (enabling twice this to fit into a 16
188	<	* bit field), which is far in excess of normal operating range.
189	<	* Even though updates to some of these bookkeeping fields do
190	<	* sometimes contend with each other, they don't normally
191	<	* cache-contend with updates to others enough to warrant memory
192	<	* padding or isolation. So they are all held as fields of
193	<	* ForkJoinPool objects.  The main capabilities are as follows:
182	>	* implementing other management responsibilities. So we use a
183	>	* collection of techniques that avoid, reduce, or cope well with
184	>	* contention. These entail several instances of bit-packing into
185	>	* CASable fields to maintain only the minimally required
186	>	* atomicity. To enable such packing, we restrict maximum
187	>	* parallelism to (1<<15)-1 (enabling twice this (to accommodate
188	>	* unbalanced increments and decrements) to fit into a 16 bit
189	>	* field, which is far in excess of normal operating range.  Even
190	>	* though updates to some of these bookkeeping fields do sometimes
191	>	* contend with each other, they don't normally cache-contend with
192	>	* updates to others enough to warrant memory padding or
193	>	* isolation. So they are all held as fields of ForkJoinPool
194	>	* objects.  The main capabilities are as follows:
195		*
196		* 1. Creating and removing workers. Workers are recorded in the
197		* "workers" array. This is an array as opposed to some other data
#	Line 179 | Line 216 | public class ForkJoinPool extends Abstra
216		* that are neither blocked nor artifically suspended) as well as
217		* the total number.  These two values are packed into one field,
218		* "workerCounts" because we need accurate snapshots when deciding
219	<	* to create, resume or suspend.  To support these decisions,
220	<	* updates to spare counts must be prospective (not
221	<	* retrospective).  For example, the running count is decremented
222	<	* before blocking by a thread about to block as a spare, but
186	<	* incremented by the thread about to unblock it. Updates upon
187	<	* resumption ofr threads blocking in awaitJoin or awaitBlocker
188	<	* cannot usually be prospective, so the running count is in
189	<	* general an upper bound of the number of productively running
190	<	* threads Updates to the workerCounts field sometimes transiently
191	<	* encounter a fair amount of contention when join dependencies
192	<	* are such that many threads block or unblock at about the same
193	<	* time. We alleviate this by sometimes performing an alternative
194	<	* action on contention like releasing waiters or locating spares.
219	>	* to create, resume or suspend.  Note however that the
220	>	* correspondance of these counts to reality is not guaranteed. In
221	>	* particular updates for unblocked threads may lag until they
222	>	* actually wake up.
223		*
224		* 3. Maintaining global run state. The run state of the pool
225		* consists of a runLevel (SHUTDOWN, TERMINATING, etc) similar to
#	Line 249 | Line 277 | public class ForkJoinPool extends Abstra
277		* 5. Managing suspension of extra workers. When a worker is about
278		* to block waiting for a join (or via ManagedBlockers), we may
279		* create a new thread to maintain parallelism level, or at least
280	<	* avoid starvation (see below). Usually, extra threads are needed
281	<	* for only very short periods, yet join dependencies are such
282	<	* that we sometimes need them in bursts. Rather than create new
283	<	* threads each time this happens, we suspend no-longer-needed
284	<	* extra ones as "spares". For most purposes, we don't distinguish
285	<	* "extra" spare threads from normal "core" threads: On each call
286	<	* to preStep (the only point at which we can do this) a worker
280	>	* avoid starvation. Usually, extra threads are needed for only
281	>	* very short periods, yet join dependencies are such that we
282	>	* sometimes need them in bursts. Rather than create new threads
283	>	* each time this happens, we suspend no-longer-needed extra ones
284	>	* as "spares". For most purposes, we don't distinguish "extra"
285	>	* spare threads from normal "core" threads: On each call to
286	>	* preStep (the only point at which we can do this) a worker
287		* checks to see if there are now too many running workers, and if
288	<	* so, suspends itself.  Methods awaitJoin and awaitBlocker look
289	<	* for suspended threads to resume before considering creating a
290	<	* new replacement. We don't need a special data structure to
291	<	* maintain spares; simply scanning the workers array looking for
292	<	* worker.isSuspended() is fine because the calling thread is
293	<	* otherwise not doing anything useful anyway; we are at least as
294	<	* happy if after locating a spare, the caller doesn't actually
295	<	* block because the join is ready before we try to adjust and
296	<	* compensate.  Note that this is intrinsically racy.  One thread
297	<	* may become a spare at about the same time as another is
298	<	* needlessly being created. We counteract this and related slop
299	<	* in part by requiring resumed spares to immediately recheck (in
300	<	* preStep) to see whether they they should re-suspend. The only
301	<	* effective difference between "extra" and "core" threads is that
302	<	* we allow the "extra" ones to time out and die if they are not
303	<	* resumed within a keep-alive interval of a few seconds. This is
304	<	* implemented mainly within ForkJoinWorkerThread, but requires
288	>	* so, suspends itself.  Methods tryAwaitJoin and awaitBlocker
289	>	* look for suspended threads to resume before considering
290	>	* creating a new replacement. We don't need a special data
291	>	* structure to maintain spares; simply scanning the workers array
292	>	* looking for worker.isSuspended() is fine because the calling
293	>	* thread is otherwise not doing anything useful anyway; we are at
294	>	* least as happy if after locating a spare, the caller doesn't
295	>	* actually block because the join is ready before we try to
296	>	* adjust and compensate.  Note that this is intrinsically racy.
297	>	* One thread may become a spare at about the same time as another
298	>	* is needlessly being created. We counteract this and related
299	>	* slop in part by requiring resumed spares to immediately recheck
300	>	* (in preStep) to see whether they they should re-suspend. The
301	>	* only effective difference between "extra" and "core" threads is
302	>	* that we allow the "extra" ones to time out and die if they are
303	>	* not resumed within a keep-alive interval of a few seconds. This
304	>	* is implemented mainly within ForkJoinWorkerThread, but requires
305		* some coordination (isTrimmed() -- meaning killed while
306		* suspended) to correctly maintain pool counts.
307		*
308		* 6. Deciding when to create new workers. The main dynamic
309		* control in this class is deciding when to create extra threads,
310		* in methods awaitJoin and awaitBlocker. We always need to create
311	<	* one when the number of running threads becomes zero. But
312	<	* because blocked joins are typically dependent, we don't
313	<	* necessarily need or want one-to-one replacement. Instead, we
314	<	* use a combination of heuristics that adds threads only when the
315	<	* pool appears to be approaching starvation.  These effectively
316	<	* reduce churn at the price of systematically undershooting
317	<	* target parallelism when many threads are blocked.  However,
318	<	* biasing toward undeshooting partially compensates for the above
319	<	* mechanics to suspend extra threads, that normally lead to
320	<	* overshoot because we can only suspend workers in-between
321	<	* top-level actions. It also better copes with the fact that some
322	<	* of the methods in this class tend to never become compiled (but
323	<	* are interpreted), so some components of the entire set of
324	<	* controls might execute many times faster than others. And
325	<	* similarly for cases where the apparent lack of work is just due
298	<	* to GC stalls and other transient system activity.
311	>	* one when the number of running threads would become zero and
312	>	* all workers are busy. However, this is not easy to detect
313	>	* reliably in the presence of transients so we use retries and
314	>	* allow slack (in tryAwaitJoin) to reduce false alarms.  These
315	>	* effectively reduce churn at the price of systematically
316	>	* undershooting target parallelism when many threads are blocked.
317	>	* However, biasing toward undeshooting partially compensates for
318	>	* the above mechanics to suspend extra threads, that normally
319	>	* lead to overshoot because we can only suspend workers
320	>	* in-between top-level actions. It also better copes with the
321	>	* fact that some of the methods in this class tend to never
322	>	* become compiled (but are interpreted), so some components of
323	>	* the entire set of controls might execute many times faster than
324	>	* others. And similarly for cases where the apparent lack of work
325	>	* is just due to GC stalls and other transient system activity.
326		*
327		* Beware that there is a lot of representation-level coupling
328		* among classes ForkJoinPool, ForkJoinWorkerThread, and
#	Line 310 | Line 337 | public class ForkJoinPool extends Abstra
337		* "while ((local = field) != 0)") which are usually the simplest
338		* way to ensure read orderings. Also several occurrences of the
339		* unusual "do {} while(!cas...)" which is the simplest way to
340	<	* force an update of a CAS'ed variable. There are also a few
341	<	* other coding oddities that help some methods perform reasonably
342	<	* even when interpreted (not compiled).
340	>	* force an update of a CAS'ed variable. There are also other
341	>	* coding oddities that help some methods perform reasonably even
342	>	* when interpreted (not compiled), at the expense of messiness.
343		*
344		* The order of declarations in this file is: (1) statics (2)
345		* fields (along with constants used when unpacking some of them)
#	Line 431 | Line 458 | public class ForkJoinPool extends Abstra
458		private volatile long eventWaiters;
459
460		private static final int  EVENT_COUNT_SHIFT = 32;
461	<	private static final long WAITER_INDEX_MASK = (1L << EVENT_COUNT_SHIFT)-1L;
461	>	private static final long WAITER_ID_MASK = (1L << EVENT_COUNT_SHIFT)-1L;
462
463		/**
464		* A counter for events that may wake up worker threads:
#	Line 503 | Line 530 | public class ForkJoinPool extends Abstra
530		*/
531		private final int poolNumber;
532
533	<	// utilities for updating fields
533	>	// Utilities for CASing fields. Note that several of these
534	>	// are manually inlined by callers
535
536		/**
537		* Increments running count.  Also used by ForkJoinTask.
#	Line 511 | Line 539 | public class ForkJoinPool extends Abstra
539		final void incrementRunningCount() {
540		int c;
541		do {} while (!UNSAFE.compareAndSwapInt(this, workerCountsOffset,
542	<	c = workerCounts,
542	>	c = workerCounts,
543		c + ONE_RUNNING));
544		}
545	<
545	>
546		/**
547		* Tries to decrement running count unless already zero
548		*/
#	Line 527 | Line 555 | public class ForkJoinPool extends Abstra
555		}
556
557		/**
558	+	* Tries to increment running count
559	+	*/
560	+	final boolean tryIncrementRunningCount() {
561	+	int wc;
562	+	return UNSAFE.compareAndSwapInt(this, workerCountsOffset,
563	+	wc = workerCounts, wc + ONE_RUNNING);
564	+	}
565	+
566	+	/**
567		* Tries incrementing active count; fails on contention.
568		* Called by workers before executing tasks.
569		*
#	Line 635 | Line 672 | public class ForkJoinPool extends Abstra
672		private void onWorkerCreationFailure() {
673		for (;;) {
674		int wc = workerCounts;
675	<	if ((wc >>> TOTAL_COUNT_SHIFT) > 0 &&
676	<	UNSAFE.compareAndSwapInt(this, workerCountsOffset,
677	<	wc, wc - (ONE_RUNNING|ONE_TOTAL)))
675	>	if ((wc >>> TOTAL_COUNT_SHIFT) == 0)
676	>	Thread.yield(); // wait for other counts to settle
677	>	else if (UNSAFE.compareAndSwapInt(this, workerCountsOffset, wc,
678	>	wc - (ONE_RUNNING|ONE_TOTAL)))
679		break;
680		}
681		tryTerminate(false); // in case of failure during shutdown
682		}
683
684		/**
685	<	* Create enough total workers to establish target parallelism,
686	<	* giving up if terminating or addWorker fails
685	>	* Creates and/or resumes enough workers to establish target
686	>	* parallelism, giving up if terminating or addWorker fails
687	>	*
688	>	* TODO: recast this to support lazier creation and automated
689	>	* parallelism maintenance
690		*/
691	<	private void ensureEnoughTotalWorkers() {
692	<	int wc;
693	<	while (((wc = workerCounts) >>> TOTAL_COUNT_SHIFT) < parallelism &&
694	<	runState < TERMINATING) {
695	<	if ((UNSAFE.compareAndSwapInt(this, workerCountsOffset,
696	<	wc, wc + (ONE_RUNNING|ONE_TOTAL)) &&
697	<	addWorker() == null))
691	>	private void ensureEnoughWorkers() {
692	>	while ((runState & TERMINATING) == 0) {
693	>	int pc = parallelism;
694	>	int wc = workerCounts;
695	>	int rc = wc & RUNNING_COUNT_MASK;
696	>	int tc = wc >>> TOTAL_COUNT_SHIFT;
697	>	if (tc < pc) {
698	>	if (UNSAFE.compareAndSwapInt
699	>	(this, workerCountsOffset,
700	>	wc, wc + (ONE_RUNNING|ONE_TOTAL)) &&
701	>	addWorker() == null)
702	>	break;
703	>	}
704	>	else if (tc > pc && rc < pc &&
705	>	tc > (runState & ACTIVE_COUNT_MASK)) {
706	>	ForkJoinWorkerThread spare = null;
707	>	ForkJoinWorkerThread[] ws = workers;
708	>	int nws = ws.length;
709	>	for (int i = 0; i < nws; ++i) {
710	>	ForkJoinWorkerThread w = ws[i];
711	>	if (w != null && w.isSuspended()) {
712	>	if ((workerCounts & RUNNING_COUNT_MASK) > pc)
713	>	return;
714	>	if (w.tryResumeSpare())
715	>	incrementRunningCount();
716	>	break;
717	>	}
718	>	}
719	>	}
720	>	else
721		break;
722		}
723		}
#	Line 689 | Line 753 | public class ForkJoinPool extends Abstra
753
754		accumulateStealCount(w); // collect final count
755		if (!tryTerminate(false))
756	<	ensureEnoughTotalWorkers();
756	>	ensureEnoughWorkers();
757		}
758
759		// Waiting for and signalling events
760
761		/**
762		* Releases workers blocked on a count not equal to current count.
763	+	* @return true if any released
764		*/
765		private void releaseWaiters() {
766		long top;
767	<	int id;
703	<	while ((id = (int)((top = eventWaiters) & WAITER_INDEX_MASK)) > 0 &&
704	<	(int)(top >>> EVENT_COUNT_SHIFT) != eventCount) {
767	>	while ((top = eventWaiters) != 0L) {
768		ForkJoinWorkerThread[] ws = workers;
769	<	ForkJoinWorkerThread w;
770	<	if (ws.length >= id && (w = ws[id - 1]) != null &&
771	<	UNSAFE.compareAndSwapLong(this, eventWaitersOffset,
772	<	top, w.nextWaiter))
773	<	LockSupport.unpark(w);
769	>	int n = ws.length;
770	>	for (;;) {
771	>	int i = ((int)(top & WAITER_ID_MASK)) - 1;
772	>	if (i < 0 || (int)(top >>> EVENT_COUNT_SHIFT) == eventCount)
773	>	return;
774	>	ForkJoinWorkerThread w;
775	>	if (i < n && (w = ws[i]) != null &&
776	>	UNSAFE.compareAndSwapLong(this, eventWaitersOffset,
777	>	top, w.nextWaiter)) {
778	>	LockSupport.unpark(w);
779	>	top = eventWaiters;
780	>	}
781	>	else
782	>	break;      // possibly stale; reread
783	>	}
784		}
785		}
786
#	Line 717 | Line 790 | public class ForkJoinPool extends Abstra
790		*/
791		private void signalEvent() {
792		int c;
793	<	do {} while (!UNSAFE.compareAndSwapInt(this, eventCountOffset,
793	>	do {} while (!UNSAFE.compareAndSwapInt(this, eventCountOffset,
794		c = eventCount, c+1));
795		releaseWaiters();
796		}
#	Line 727 | Line 800 | public class ForkJoinPool extends Abstra
800		* other releasing threads is detected.
801		*/
802		final void signalWork() {
803	<	// EventCount CAS failures are OK -- any change in count suffices.
804	<	int ec;
805	<	UNSAFE.compareAndSwapInt(this, eventCountOffset, ec=eventCount, ec+1);
806	<	outer:for (;;) {
807	<	long top = eventWaiters;
808	<	ec = eventCount;
803	>	int c;
804	>	UNSAFE.compareAndSwapInt(this, eventCountOffset, c=eventCount, c+1);
805	>	long top;
806	>	while ((top = eventWaiters) != 0L) {
807	>	int ec = eventCount;
808	>	ForkJoinWorkerThread[] ws = workers;
809	>	int n = ws.length;
810		for (;;) {
811	<	ForkJoinWorkerThread[] ws; ForkJoinWorkerThread w;
812	<	int id = (int)(top & WAITER_INDEX_MASK);
739	<	if (id <= 0 || (int)(top >>> EVENT_COUNT_SHIFT) == ec)
740	<	return;
741	<	if ((ws = workers).length < id || (w = ws[id - 1]) == null ||
742	<	!UNSAFE.compareAndSwapLong(this, eventWaitersOffset,
743	<	top, top = w.nextWaiter))
744	<	continue outer;      // possibly stale; reread
745	<	LockSupport.unpark(w);
746	<	if (top != eventWaiters) // let someone else take over
811	>	int i = ((int)(top & WAITER_ID_MASK)) - 1;
812	>	if (i < 0 || (int)(top >>> EVENT_COUNT_SHIFT) == ec)
813		return;
814	+	ForkJoinWorkerThread w;
815	+	if (i < n && (w = ws[i]) != null &&
816	+	UNSAFE.compareAndSwapLong(this, eventWaitersOffset,
817	+	top, top = w.nextWaiter)) {
818	+	LockSupport.unpark(w);
819	+	if (top != eventWaiters) // let someone else take over
820	+	return;
821	+	}
822	+	else
823	+	break;      // possibly stale; reread
824		}
825		}
826		}
#	Line 755 | Line 831 | public class ForkJoinPool extends Abstra
831		* release others.
832		*
833		* @param w the calling worker thread
834	+	* @param retries the number of scans by caller failing to find work
835	+	* @return false if now too many threads running
836		*/
837	<	private void eventSync(ForkJoinWorkerThread w) {
838	<	if (!w.active) {
839	<	int prev = w.lastEventCount;
840	<	long nextTop = (((long)prev << EVENT_COUNT_SHIFT) |
837	>	private boolean eventSync(ForkJoinWorkerThread w, int retries) {
838	>	int wec = w.lastEventCount;
839	>	if (retries > 1) { // can only block after 2nd miss
840	>	long nextTop = (((long)wec << EVENT_COUNT_SHIFT) |
841		((long)(w.poolIndex + 1)));
842		long top;
843		while ((runState < SHUTDOWN || !tryTerminate(false)) &&
844	<	(((int)(top = eventWaiters) & WAITER_INDEX_MASK) == 0 ||
845	<	(int)(top >>> EVENT_COUNT_SHIFT) == prev) &&
846	<	eventCount == prev) {
844	>	(((int)(top = eventWaiters) & WAITER_ID_MASK) == 0 ||
845	>	(int)(top >>> EVENT_COUNT_SHIFT) == wec) &&
846	>	eventCount == wec) {
847		if (UNSAFE.compareAndSwapLong(this, eventWaitersOffset,
848		w.nextWaiter = top, nextTop)) {
849		accumulateStealCount(w); // transfer steals while idle
850		Thread.interrupted();    // clear/ignore interrupt
851	<	while (eventCount == prev)
851	>	while (eventCount == wec)
852		w.doPark();
853		break;
854		}
855		}
856	<	w.lastEventCount = eventCount;
856	>	wec = eventCount;
857		}
858		releaseWaiters();
859	+	int wc = workerCounts;
860	+	if ((wc & RUNNING_COUNT_MASK) <= parallelism) {
861	+	w.lastEventCount = wec;
862	+	return true;
863	+	}
864	+	if (wec != w.lastEventCount) // back up if may re-wait
865	+	w.lastEventCount = wec - (wc >>> TOTAL_COUNT_SHIFT);
866	+	return false;
867		}
868
869		/**
#	Line 798 | Line 884 | public class ForkJoinPool extends Abstra
884		* upon resume it rechecks to make sure that it is still needed.
885		*
886		* @param w the worker
887	<	* @param worked false if the worker scanned for work but didn't
887	>	* @param retries the number of scans by caller failing to find work
888		* find any (in which case it may block waiting for work).
889		*/
890	<	final void preStep(ForkJoinWorkerThread w, boolean worked) {
890	>	final void preStep(ForkJoinWorkerThread w, int retries) {
891		boolean active = w.active;
892	<	boolean inactivate = !worked & active;
892	>	boolean inactivate = active && retries != 0;
893		for (;;) {
894	<	if (inactivate) {
895	<	int rs = runState;
896	<	if (UNSAFE.compareAndSwapInt(this, runStateOffset,
897	<	rs, rs - ONE_ACTIVE))
898	<	inactivate = active = w.active = false;
899	<	}
900	<	int wc = workerCounts;
901	<	if ((wc & RUNNING_COUNT_MASK) <= parallelism) {
816	<	if (!worked)
817	<	eventSync(w);
818	<	return;
894	>	int rs, wc;
895	>	if (inactivate &&
896	>	UNSAFE.compareAndSwapInt(this, runStateOffset,
897	>	rs = runState, rs - ONE_ACTIVE))
898	>	inactivate = active = w.active = false;
899	>	if (((wc = workerCounts) & RUNNING_COUNT_MASK) <= parallelism) {
900	>	if (active || eventSync(w, retries))
901	>	break;
902		}
903	<	if (!(inactivate |= active) &&  // must inactivate to suspend
903	>	else if (!(inactivate |= active) &&  // must inactivate to suspend
904		UNSAFE.compareAndSwapInt(this, workerCountsOffset,
905		wc, wc - ONE_RUNNING) &&
906	<	!w.suspendAsSpare())        // false if trimmed
907	<	return;
906	>	!w.suspendAsSpare())             // false if trimmed
907	>	break;
908		}
909		}
910
911		/**
912	<	* Tries to decrement running count, and if so, possibly creates
913	<	* or resumes compensating threads before blocking on task joinMe.
914	<	* This code is sprawled out with manual inlining to evade some
915	<	* JIT oddities.
912	>	* Awaits join of the given task if enough threads, or can resume
913	>	* or create a spare. Fails (in which case the given task might
914	>	* not be done) upon contention or lack of decision about
915	>	* blocking. Returns void because caller must check
916	>	* task status on return anyway.
917	>	*
918	>	* We allow blocking if:
919	>	*
920	>	* 1. There would still be at least as many running threads as
921	>	*    parallelism level if this thread blocks.
922	>	*
923	>	* 2. A spare is resumed to replace this worker. We tolerate
924	>	*    slop in the decision to replace if a spare is found without
925	>	*    first decrementing run count.  This may release too many,
926	>	*    but if so, the superfluous ones will re-suspend via
927	>	*    preStep().
928	>	*
929	>	* 3. After #spares repeated checks, there are no fewer than #spare
930	>	*    threads not running. We allow this slack to avoid hysteresis
931	>	*    and as a hedge against lag/uncertainty of running count
932	>	*    estimates when signalling or unblocking stalls.
933	>	*
934	>	* 4. All existing workers are busy (as rechecked via repeated
935	>	*    retries by caller) and a new spare is created.
936	>	*
937	>	* If none of the above hold, we try to escape out by
938	>	* re-incrementing count and returning to caller, which can retry
939	>	* later.
940		*
941		* @param joinMe the task to join
942	<	* @return task status on exit
942	>	* @param retries if negative, then serve only as a precheck
943	>	*   that the thread can be replaced by a spare. Otherwise,
944	>	*   the number of repeated calls to this method returning busy
945	>	* @return true if the call must be retried because there
946	>	*   none of the blocking checks hold
947		*/
948	<	final int tryAwaitJoin(ForkJoinTask<?> joinMe) {
949	<	int cw = workerCounts; // read now to spoil CAS if counts change as ...
950	<	releaseWaiters();      // ... a byproduct of releaseWaiters
951	<	int stat = joinMe.status;
952	<	if (stat >= 0 && // inline variant of tryDecrementRunningCount
953	<	(cw & RUNNING_COUNT_MASK) > 0 &&
954	<	UNSAFE.compareAndSwapInt(this, workerCountsOffset,
955	<	cw, cw - ONE_RUNNING)) {
956	<	int pc = parallelism;
957	<	int scans = 0;  // to require confirming passes to add threads
958	<	outer: while ((workerCounts & RUNNING_COUNT_MASK) < pc) {
959	<	if ((stat = joinMe.status) < 0)
960	<	break;
961	<	ForkJoinWorkerThread spare = null;
962	<	ForkJoinWorkerThread[] ws = workers;
963	<	int nws = ws.length;
964	<	for (int i = 0; i < nws; ++i) {
965	<	ForkJoinWorkerThread w = ws[i];
966	<	if (w != null && w.isSuspended()) {
967	<	spare = w;
968	<	break;
948	>	final boolean tryAwaitJoin(ForkJoinTask<?> joinMe, int retries) {
949	>	if (joinMe.status < 0) // precheck for cancellation
950	>	return false;
951	>	if ((runState & TERMINATING) != 0) { // shutting down
952	>	joinMe.cancelIgnoringExceptions();
953	>	return false;
954	>	}
955	>
956	>	int pc = parallelism;
957	>	boolean running = true; // false when running count decremented
958	>	outer:for (;;) {
959	>	int wc = workerCounts;
960	>	int rc = wc & RUNNING_COUNT_MASK;
961	>	int tc = wc >>> TOTAL_COUNT_SHIFT;
962	>	if (running) { // replace with spare or decrement count
963	>	if (rc <= pc && tc > pc &&
964	>	(retries > 0 || tc > (runState & ACTIVE_COUNT_MASK))) {
965	>	ForkJoinWorkerThread[] ws = workers;
966	>	int nws = ws.length;
967	>	for (int i = 0; i < nws; ++i) { // search for spare
968	>	ForkJoinWorkerThread w = ws[i];
969	>	if (w != null) {
970	>	if (joinMe.status < 0)
971	>	return false;
972	>	if (w.isSuspended()) {
973	>	if ((workerCounts & RUNNING_COUNT_MASK)>=pc &&
974	>	w.tryResumeSpare()) {
975	>	running = false;
976	>	break outer;
977	>	}
978	>	continue outer; // rescan
979	>	}
980	>	}
981		}
982		}
983	<	if ((stat = joinMe.status) < 0) // recheck to narrow race
983	>	if (retries < 0 || // < 0 means replacement check only
984	>	rc == 0 || joinMe.status < 0 || workerCounts != wc ||
985	>	!UNSAFE.compareAndSwapInt(this, workerCountsOffset,
986	>	wc, wc - ONE_RUNNING))
987	>	return false; // done or inconsistent or contended
988	>	running = false;
989	>	if (rc > pc)
990		break;
991	<	int wc = workerCounts;
992	<	int rc = wc & RUNNING_COUNT_MASK;
993	<	if (rc >= pc)
991	>	}
992	>	else { // allow blocking if enough threads
993	>	if (rc >= pc || joinMe.status < 0)
994		break;
995	<	if (spare != null) {
996	<	if (spare.tryUnsuspend()) {
997	<	int c; // inline incrementRunningCount
998	<	do {} while (!UNSAFE.compareAndSwapInt
999	<	(this, workerCountsOffset,
1000	<	c = workerCounts, c + ONE_RUNNING));
1001	<	LockSupport.unpark(spare);
995	>	int sc = tc - pc + 1; // = spare threads, plus the one to add
996	>	if (retries > sc) {
997	>	if (rc > 0 && rc >= pc - sc) // allow slack
998	>	break;
999	>	if (tc < MAX_THREADS &&
1000	>	tc == (runState & ACTIVE_COUNT_MASK) &&
1001	>	workerCounts == wc &&
1002	>	UNSAFE.compareAndSwapInt(this, workerCountsOffset, wc,
1003	>	wc+(ONE_RUNNING|ONE_TOTAL))) {
1004	>	addWorker();
1005		break;
874	–	}
875	–	continue;
876	–	}
877	–	int tc = wc >>> TOTAL_COUNT_SHIFT;
878	–	int sc = tc - pc;
879	–	if (rc > 0) {
880	–	int p = pc;
881	–	int s = sc;
882	–	while (s-- >= 0) { // try keeping 3/4 live
883	–	if (rc > (p -= (p >>> 2) + 1))
884	–	break outer;
1006		}
1007		}
1008	<	if (scans++ > sc && tc < MAX_THREADS &&
1009	<	UNSAFE.compareAndSwapInt(this, workerCountsOffset, wc,
1010	<	wc + (ONE_RUNNING|ONE_TOTAL))) {
1011	<	addWorker();
1012	<	break;
1008	>	if (workerCounts == wc &&        // back out to allow rescan
1009	>	UNSAFE.compareAndSwapInt (this, workerCountsOffset,
1010	>	wc, wc + ONE_RUNNING)) {
1011	>	releaseWaiters();            // help others progress
1012	>	return true;                 // let caller retry
1013		}
1014		}
894	–	if (stat >= 0)
895	–	stat = joinMe.internalAwaitDone();
896	–	int c; // inline incrementRunningCount
897	–	do {} while (!UNSAFE.compareAndSwapInt
898	–	(this, workerCountsOffset,
899	–	c = workerCounts, c + ONE_RUNNING));
1015		}
1016	<	return stat;
1016	>	// arrive here if can block
1017	>	joinMe.internalAwaitDone();
1018	>	int c;                      // to inline incrementRunningCount
1019	>	do {} while (!UNSAFE.compareAndSwapInt
1020	>	(this, workerCountsOffset,
1021	>	c = workerCounts, c + ONE_RUNNING));
1022	>	return false;
1023		}
1024
1025		/**
1026	<	* Same idea as (and mostly pasted from) tryAwaitJoin, but
1027	<	* self-contained
1026	>	* Same idea as (and shares many code snippets with) tryAwaitJoin,
1027	>	* but self-contained because there are no caller retries.
1028	>	* TODO: Rework to use simpler API.
1029		*/
1030		final void awaitBlocker(ManagedBlocker blocker)
1031		throws InterruptedException {
1032	<	for (;;) {
1033	<	if (blocker.isReleasable())
1034	<	return;
913	<	int cw = workerCounts;
914	<	releaseWaiters();
915	<	if ((cw & RUNNING_COUNT_MASK) > 0 &&
916	<	UNSAFE.compareAndSwapInt(this, workerCountsOffset,
917	<	cw, cw - ONE_RUNNING))
918	<	break;
919	<	}
920	<	boolean done = false;
1032	>	boolean done;
1033	>	if (done = blocker.isReleasable())
1034	>	return;
1035		int pc = parallelism;
1036	<	int scans = 0;
1037	<	outer: while ((workerCounts & RUNNING_COUNT_MASK) < pc) {
1038	<	if (done = blocker.isReleasable())
925	<	break;
926	<	ForkJoinWorkerThread spare = null;
927	<	ForkJoinWorkerThread[] ws = workers;
928	<	int nws = ws.length;
929	<	for (int i = 0; i < nws; ++i) {
930	<	ForkJoinWorkerThread w = ws[i];
931	<	if (w != null && w.isSuspended()) {
932	<	spare = w;
933	<	break;
934	<	}
935	<	}
936	<	if (done = blocker.isReleasable())
937	<	break;
1036	>	int retries = 0;
1037	>	boolean running = true; // false when running count decremented
1038	>	outer:for (;;) {
1039		int wc = workerCounts;
1040		int rc = wc & RUNNING_COUNT_MASK;
940	–	if (rc >= pc)
941	–	break;
942	–	if (spare != null) {
943	–	if (spare.tryUnsuspend()) {
944	–	int c;
945	–	do {} while (!UNSAFE.compareAndSwapInt
946	–	(this, workerCountsOffset,
947	–	c = workerCounts, c + ONE_RUNNING));
948	–	LockSupport.unpark(spare);
949	–	break;
950	–	}
951	–	continue;
952	–	}
1041		int tc = wc >>> TOTAL_COUNT_SHIFT;
1042	<	int sc = tc - pc;
1043	<	if (rc > 0) {
1044	<	int p = pc;
1045	<	int s = sc;
1046	<	while (s-- >= 0) {
1047	<	if (rc > (p -= (p >>> 2) + 1))
1048	<	break outer;
1042	>	if (running) {
1043	>	if (rc <= pc && tc > pc &&
1044	>	(retries > 0 || tc > (runState & ACTIVE_COUNT_MASK))) {
1045	>	ForkJoinWorkerThread[] ws = workers;
1046	>	int nws = ws.length;
1047	>	for (int i = 0; i < nws; ++i) {
1048	>	ForkJoinWorkerThread w = ws[i];
1049	>	if (w != null) {
1050	>	if (done = blocker.isReleasable())
1051	>	return;
1052	>	if (w.isSuspended()) {
1053	>	if ((workerCounts & RUNNING_COUNT_MASK)>=pc &&
1054	>	w.tryResumeSpare()) {
1055	>	running = false;
1056	>	break outer;
1057	>	}
1058	>	continue outer; // rescan
1059	>	}
1060	>	}
1061	>	}
1062		}
1063	+	if (done = blocker.isReleasable())
1064	+	return;
1065	+	if (rc == 0 || workerCounts != wc ||
1066	+	!UNSAFE.compareAndSwapInt(this, workerCountsOffset,
1067	+	wc, wc - ONE_RUNNING))
1068	+	continue;
1069	+	running = false;
1070	+	if (rc > pc)
1071	+	break;
1072		}
1073	<	if (scans++ > sc && tc < MAX_THREADS &&
1074	<	UNSAFE.compareAndSwapInt(this, workerCountsOffset, wc,
1075	<	wc + (ONE_RUNNING|ONE_TOTAL))) {
1076	<	addWorker();
1077	<	break;
1073	>	else {
1074	>	if (rc >= pc || (done = blocker.isReleasable()))
1075	>	break;
1076	>	int sc = tc - pc + 1;
1077	>	if (retries++ > sc) {
1078	>	if (rc > 0 && rc >= pc - sc)
1079	>	break;
1080	>	if (tc < MAX_THREADS &&
1081	>	tc == (runState & ACTIVE_COUNT_MASK) &&
1082	>	workerCounts == wc &&
1083	>	UNSAFE.compareAndSwapInt(this, workerCountsOffset, wc,
1084	>	wc+(ONE_RUNNING|ONE_TOTAL))) {
1085	>	addWorker();
1086	>	break;
1087	>	}
1088	>	}
1089	>	Thread.yield();
1090		}
1091		}
1092	+
1093		try {
1094		if (!done)
1095	<	do {} while (!blocker.isReleasable() &&
973	<	!blocker.block());
1095	>	do {} while (!blocker.isReleasable() && !blocker.block());
1096		} finally {
1097	<	int c;
1098	<	do {} while (!UNSAFE.compareAndSwapInt
1099	<	(this, workerCountsOffset,
1100	<	c = workerCounts, c + ONE_RUNNING));
1097	>	if (!running) {
1098	>	int c;
1099	>	do {} while (!UNSAFE.compareAndSwapInt
1100	>	(this, workerCountsOffset,
1101	>	c = workerCounts, c + ONE_RUNNING));
1102	>	}
1103		}
1104	<	}
1104	>	}
1105
1106		/**
1107		* Possibly initiates and/or completes termination.
#	Line 1103 | Line 1227 | public class ForkJoinPool extends Abstra
1227		* active thread.
1228		*/
1229		final int idlePerActive() {
1230	<	int pc = parallelism; // use targeted parallelism, not rc
1230	>	int pc = parallelism; // use parallelism, not rc
1231		int ac = runState;    // no mask -- artifically boosts during shutdown
1232		// Use exact results for small values, saturate past 4
1233		return pc <= ac? 0 : pc >>> 1 <= ac? 1 : pc >>> 2 <= ac? 3 : pc >>> 3;
#	Line 1154 | Line 1278 | public class ForkJoinPool extends Abstra
1278		* use {@link java.lang.Runtime#availableProcessors}.
1279		* @param factory the factory for creating new threads. For default value,
1280		* use {@link #defaultForkJoinWorkerThreadFactory}.
1281	<	* @param handler the handler for internal worker threads that
1282	<	* terminate due to unrecoverable errors encountered while executing
1281	>	* @param handler the handler for internal worker threads that
1282	>	* terminate due to unrecoverable errors encountered while executing
1283		* tasks. For default value, use <code>null</code>.
1284	<	* @param asyncMode if true,
1284	>	* @param asyncMode if true,
1285		* establishes local first-in-first-out scheduling mode for forked
1286		* tasks that are never joined. This mode may be more appropriate
1287		* than default locally stack-based mode in applications in which
#	Line 1171 | Line 1295 | public class ForkJoinPool extends Abstra
1295		*         because it does not hold {@link
1296		*         java.lang.RuntimePermission}{@code ("modifyThread")}
1297		*/
1298	<	public ForkJoinPool(int parallelism,
1298	>	public ForkJoinPool(int parallelism,
1299		ForkJoinWorkerThreadFactory factory,
1300		Thread.UncaughtExceptionHandler handler,
1301		boolean asyncMode) {
#	Line 1216 | Line 1340 | public class ForkJoinPool extends Abstra
1340		throw new NullPointerException();
1341		if (runState >= SHUTDOWN)
1342		throw new RejectedExecutionException();
1343	<	// Convert submissions to current pool into forks
1344	<	Thread t = Thread.currentThread();
1345	<	ForkJoinWorkerThread w;
1222	<	if ((t instanceof ForkJoinWorkerThread) &&
1223	<	(w = (ForkJoinWorkerThread) t).pool == this)
1224	<	w.pushTask(task);
1225	<	else {
1226	<	submissionQueue.offer(task);
1227	<	signalEvent();
1228	<	ensureEnoughTotalWorkers();
1229	<	}
1343	>	submissionQueue.offer(task);
1344	>	signalEvent();
1345	>	ensureEnoughWorkers();
1346		}
1347
1348		/**
1349		* Performs the given task, returning its result upon completion.
1350		* If the caller is already engaged in a fork/join computation in
1351	<	* the current pool, this method is equivalent in effect to
1351	>	* the current pool, this method is equivalent in effect to
1352		* {@link ForkJoinTask#invoke}.
1353		*
1354		* @param task the task
#	Line 1249 | Line 1365 | public class ForkJoinPool extends Abstra
1365		/**
1366		* Arranges for (asynchronous) execution of the given task.
1367		* If the caller is already engaged in a fork/join computation in
1368	<	* the current pool, this method is equivalent in effect to
1368	>	* the current pool, this method is equivalent in effect to
1369		* {@link ForkJoinTask#fork}.
1370		*
1371		* @param task the task
#	Line 1280 | Line 1396 | public class ForkJoinPool extends Abstra
1396		/**
1397		* Submits a ForkJoinTask for execution.
1398		* If the caller is already engaged in a fork/join computation in
1399	<	* the current pool, this method is equivalent in effect to
1399	>	* the current pool, this method is equivalent in effect to
1400		* {@link ForkJoinTask#fork}.
1401		*
1402		* @param task the task to submit

Diff Legend

–	Removed lines
+	Added lines
<	Changed lines
>	Changed lines