Java Concurrency in Practice - Summary - Part 5

This is part 5 of  my notes from reading Java Concurrency in Practice.

NOTE: These summaries are NOT meant to replace the book.  I highly recommend buying your own copy of the book if you haven't already read it. 

Chapter 7 - Cancellation & Shutdown

1. Java does not provide any mechanism for safely forcing a thread to stop what it is doing.  Instead, we need to rely on interruption where a thread can request another thread to stop.  The thread to be stopped may choose to ignore the request or can terminate after optionally performing cleanup operations.
2. Don't use the deprecated Thread.stop() and suspend() methods.
3. A task is cancelable if some external code can move it to a completed state before its normal completion. A task that supports cancellation must specify its cancellation policy:
1. how external code can request cancellation
2. responsiveness guarantees to cancellation requests
3. what happens on cancellation (such as cleanup operations)
4. One cooperative mechanism for terminating a task is to use a cancellation requested flag which the task periodically checks.  If the flag is set by some external code, then the task terminates early.  Remember to make the cancellation requested flag volatile.  Otherwise changes made by the external code may never become visible to the task.
1. The thread will exit only when it checks the cancellation flag.  Hence there is no guarantee on if and when the check will be made.
2. Can take a very long time to take effect (if at all) if the thread to be cancelled is stuck at a blocking operation.
5. Interruption Is usually the most sensible way to implement cancellation
6. Each thread has a boolean interrupted status, which is set to true when a thread is interrupted.
1. interrupt() interrupts the target thread
2. isInterrupted() returns the interrupted status of the target thread
3. interrupted() clears the interrupted status of the current thread and returns its previous value.  This is the only way to clear the interrupted status of a thread.  Poor choice of function name.
7. Blocking library calls try to detect when a thread has been interrupted and return early.  They clear the interrupted status and throw an InterruptedException.  There is no guarantee about how quickly a blocking method will detect interruption.  In practice, it happens fairly quickly.  When a thread that is not blocked is interrupted, its interrupted status is set.  It is upto the activity being cancelled to poll the interrupted status and respond appropriately. 
1.  A task does not need to immediately stop on detecting the interrupted status.  It can postpone acting on the interruption till a more opportune moment.  This can prevent internal data structures from being corrupted when interrupted in the middle of some critical operations.
8. There is a distinction between how tasks and threads react to interruption.  An interrupt on a worker thread in a thread pool can cancel the current task as well as shut down the worker thread. Hence, guesT code that doesn't own a thread must preserve the interrupted status of the thread after acting on the interrupt, so that the owner can appropriately deal with it later.
9. A thread should be interrupted only by its owner.  Because each thread has its own interruption policy, you should not interrupt a thread unless you know what interruption means to that thread.
10. Responding to interruption
1. Propagate the interruptedException, OR
2. Restore the interrupted status by calling Thread.currentThread.interrupt().  This is the only feasible solution in cases like Runnable.run() which does not allow exceptions to be thrown.
3. Only code that implements a thread's interruption policy may swallow an interruption request.  General purpose task and library code must never swallow interruption requests.
11. Activities that do not support cancellation but still call interruptible blocking methods must call them in a loop, retrying when interruption is detected.  The interruption status should be saved locally and restored before returning. Restoring the interrupted status immediately can result in an infinite loop.
12. Cancellation via Future
1. Future.cancel(boolean mayInterruptIfRunning)  - if mayInterruptIfRunning is true and the task is running in some thread, then that thread is interrupted.  If false, cancel() only means that don't run this task if it hasn't started yet.
2. Standard Executor implementation implement a thread interruption policy that allows tasks to be canceled through interruption.  Hence, it is ok to call Future.cancel(true) which interrupts the thread.
3. You should not interrupt a pool thread directly when attempting to cancel a task because you won't know what task is running at the time the interrupt is delivered. Cancel only through the task's Future.
4. When Future.get() throws an InterruptedException or TimeoutException and you know that the result is no longer required, cancel the task by calling Future.cancel().
13. Dealing with non-interruptile blocking
1. synchronous socket I/O in java.io - read/write in InputStream and OutputStream are not responsive to interruption, but closing the underlying socket makes any threads blocked in read/write to throw a SocketException.
2. Synchronous I/IO in java.nio - Interrupting a thread waiting on an InterruptibleChannel causes it to throw ClosedByInterruptionException and close the channel.  Closing an InterruptibleChannel cause threads blocked on channel operations to throw AsynchronousCloseException.
3. Asynchronous IO with Selector - A thread blocked in Selector.select() returns prematurely if close() or wakeup() is called.
4. Lock acquisition - A thread waiting for an intrinsic lock cannot be interrupted.  Explicit Lock class offers the lockInterruptibly method.
5. To perform non standard cancellation tasks (like closing a socket), override newTaskFor() in ThreadPoolExecutor to return a CancellableTask. CancellableTask extends Callable and overrides the FutureTask.cancel() method to close socket or perform any other nonstandard cancellation tasks.
14. Stopping a thread-based service
1. A thread pool owns the worker threads, and should be responsible for stopping them.  It should provide lifecycle methods that can be used by the application to shut down the pool, which in turn shuts down the worker threads.
15. ExecutorService provides shutdown() and shutdownNow(). shutdownNow() returns the list of tasks that had not started, so they can be logged or saved for future processing.  The returned Runnable objects may not be the same as what was submitted - they may be wrapped.
1. shutdownNow() provides no way of knowing the state of tasks in progress at shutdown time, unless the tasks themselves do checkpointing. Another option is to override execute() of AbstractExecutorService and pass in a wrapper Runnable that records tasks cancelled at shutdown.  There is a race condition that may cause a completed task to be marked as cancelled.  So tasks must be idempotent.
16. Poison Pill - a special object placed on the work queue is another way to convince a producer-consumer service to shut down.  Applicable only when the number of consumers and producers is known, and when the queue is unbounded.
17. Handling abnormal thread termination
1. Leading cause of premature thread death is RuntimeException.  If nothing special is done, the exception bubbles all the way up the stack and the thread is killed after printing a stacktrace to the console (which no one may be watching for)
2. If you are writing a worker thread class for a thread pool or executor service, be sure to catch Throwable and then notify the executor service of premature thread death.
3. Thread API provides an UncaughtExceptionHandler facility - when a thread exits due to an uncaught exception, the JVM reports this to an application provided UncaughtExceptionHandler.  Use Thread.setUncaughtExceptionHandler() to set the handler for the current thread or Thread.setDefaultUncaughtExceptionHandler to set it for all threads.
4. In long-running applications, always use uncaught exception handlers for all threads that at least log the exception.
5. To set an UncaughtExceptionHandler for pool threads, provide a ThreadFactory to the ThreadPoolExecutor.  Exceptions thrown from tasks make it to the UncaughtExceptionHandler only for tasks submitted with execute().  For those submitted with submit(), the exceptions are rethrown when calling Future.get()
18. JVM Shutdown
1. orderly shutdown - when the last nondaemon thread exits, System.exit(), Ctrl-C
2. abrupt shutdown - Runtime.halt(), SIGKILL the JVM
3. In orderly shutdown, JVM starts all shutdown hooks registered while Runtime.addShutdownHook(). Order of shutdown hook execution is not guaranteed.
4. After all shutdown hooks have completed, JVM may choose to run finalizers if runFinalizersOnExit is true,
5. JVM makes no attempt to stop or interrupt any application threads that are still running.
6. Shut-down hooks can run concurrently with other application threads.  So, they must be thread-safe.  Since JVM is shutting down, the application state may be messy. Hence the shut-down hooks must be coded extremely defensively.
7. Shut-down hooks should not use services that can be shutdown by the application or by other shutdown hooks.  One option is to use a single shutdown hook per application that executes various shutdown operations in sequence.
19. Daemon threads - existence of a Daemon thread does not prevent JVM from shutting down.  Internal JVM threads like GC thread are daemon threads.  When JVM exits, finally blocks of any existing daemon threads are not run.
1. Should be used sparingly, for activities that can be safely abandoned at any time without any cleanup.  
2. Do not use daemon threads for any tasks that perform I/O.
3. Generally used for housekeeping tasks like a background thread to remove expired cache entries.
20. Finalizers
1. GC treats objects that have a non-trivial finalize() method specially. finalize() is called after the memory is reclaimed.
2. Finalizers can run concurrently with application threads.  Hence, they must be thread-safe.
3. No guarantee about if or when they will run.
4. HIgh performance cost.
5. Usually, finally blocks and explicit close statements are sufficient to release resources, instead of using finalize().  
6. finalizers may be needed for objects that hold resources acquired by native methods.
7. Avoid finalizers

Java Concurrency in Practice - Summary - Part 4

This is part 4 of  my notes from reading Java Concurrency in Practice.

NOTE: These summaries are NOT meant to replace the book.  I highly recommend buying your own copy of the book if you haven't already read it. 

Chapter 6 - Task Execution

1. Individual client requests are a natural task boundary choice for server applications.
2. Creating a new thread per task is usually NOT a good idea.
1. Overheads of thread creation and teardown can add up to be quite significant.
2. Active threads consume system resources like memory even when they are idle, and also increase CPU contention.
3. Creating too many threads can result in an OutOfMemoryError.
3. Primary abstraction for task execution in Java is the Executor framework, not Threads.
1. Executor interface has a void execute(Runnable) methods.
4. Executors are the easiest way to implement a producer-consumer design.
5. Decouples task submission from execution.  Execution policy is separated from task submission.
6. Always use Executor instead of new Thread(runnable).start()
7. Different types of Thread Pools can be created using static factory methods of the Executors class:
1. newFixedThreadPool - creates new threads upto a maximum specified size.  Tries to keep pool size constant by adding new threads if some die due to  exceptions.
2. newCachedThreadPool - No bounds on number of threads.  Number of threads increase/decrease based on load.
3. newSingleThreadExecutor - Used to process tasks sequentially in order imposed by task queue (FIFO, LIFO, priority order).  Replaces thread if it dies unexpectedly.
4. newScheduledThreadPool - Fixed size thread pool that supports delayed and periodic task execution.
1. Use instead of Timer.
1. Timer creates only a single thread.  If one task takes too long, it affects the timing accuracy of subsequent TimerTasks.
2. An unchecked exception thrown by a TimerTask terminates the Timer thread.  The Timer thread is not resurrected, and the Timer is simply cancelled.
3.  Scheduled thread pools do not have the above two limitations  However,  Timers can schedule based on absolute time, while scheduled thread pools only support relative time.
8. Executor lifecycle has 3 states - running, shutting down, terminated.
9. ExecutorService interface (that extends Executor) offers methods like shutdown(), shutdownNow(), isShutdown(), isTerminated(), awaitTermination() to control Executor life cycle.
1. shutdown() - graceful shutdown.  Allow all running and previously submitted tasks to complete.  No new tasks are accepted.
2. shutdownNow() - cancel running tasks, and ignores any queued tasks.
3. Tasks submitted to executor after shutdown are passed to a rejection handler, which may silently drop the task or throw a RejectedExecutionException.
4. awaitTermination() is usually called immediately after calling shutdown().
10. ExecutorService.submit(Callable) returns a Future. A  Future represents the lifecycle of a task, and provides methods to monitor/control it.
11. CompletionService combines the functionality of an Executor and a BlockingQueue.  Submit a bunch of Callables to the Executor, and then wait for the results to be available using take() and poll().
1. An ExecutorCompletionService is a wrapper around an Executor - new ExecutorCompletionService(executor).
2. Tasks are submitted to the completion service, and not directly to the Executor.
3. Multiple completion services can share an Executor.
4. Keep track of the number of tasks submitted in order to determine the number of times to call take().
12. Future.get() supports a version that throws a TimeoutException if the result is not available within the specified time delay.
1. If a TimeoutException happens, then call cancel on the Future to cancel the task.  If the task is written to be cancelable, then it can be terminated to avoid consuming unnecessary resources.
13. ExecutorService.invokeAll - takes a collection of tasks and returns a collection of Futures.  Timed version of invokeAll returns when all tasks have completed, the calling thread is interrupted or if the timeout expires.  Use Future.isCancelled() to determine if a particular task completed or was interrupted/cancelled.

Java Concurrency in Practice - Summary - Part 3

This is part 3 of  my notes from reading Java Concurrency in Practice.

NOTE: These summaries are NOT meant to replace the book.  I highly recommend buying your own copy of the book if you haven't already read it. 

Chapter 4 - Composing objects

1. We often create thread-safe objects by composing together other thread-safe objects. In order to design a thread-safe class, we need to identify the object's state variables, establish invariants that constraint them and the post conditions associated with its method,  and then establish a policy for managing concurrent access to them.
1.  An object's state includes the state of other objects referenced from its state variables.  For eg: a LinkedList's state includes all the link node objects.
2. Synchronization policy - how an object uses immutability, thread-confinement, locking (how and which variables are guarded by which locks) to coordinate access to its state variable so that the invariants or post conditions are not violated.
3. Encapsulation enables us to determine that a class is thread-safe without having to examine the entire program, because all paths that use the data can be identified and made thread-safe.
1. Collection wrappers like synchronizedList make the underlying non-thread-safe ArrayList thread-safe by using encapsulation.
4. Java monitor pattern - object encapsulates all state and guards it with its own intrinsic lock.
1. Pro: Simplicity.
2. Con: External code can lock on the object's intrinsic lock and cause liveness problems that may require examining the whole program to debug.  This problem is avoided if a separate private object is used as the lock.
5. A composite object made of thread-safe components need not be thread-safe.  This is usually the case when there are constraints on the state of the components.
6. A state variable can be published if it is thread-safe, does not participate in any invariants that constrain its value and has no prohibited state transitions for any of its operations.
7. Re-use existing java thread-safe libraries whenever possible.  To add-functionality to an existing thread-safe class: 
1. The best option is to modify the source code of the class if available.  
1. Pro: details about synchronization policy are confined to one file, and is thus easier to maintain.
2. If modifying source code is not possible, the next best option is to extend the class, assuming it was designed for extension. 
1. Con: Fragile.  If base class changes synchronization policy (say which locks are used), then the extended class will silently fail.
3. Extension or source modification is not possible for collections wrapped in Collections.synchronized* wrappers, since the underlying wrapped class is unknown.  The solution is to use client-side locking - guard client code with the lock specified by the object's synchronization policy.  For Vector, the lock is the object itself.
4. Another option is Composition.  In a wrapper object, maintain a private internal copy of the object (say List) whose functionality we wish to extend.   Add the new functionality as a synchronized method of the wrapper.  Add synchronized methods for existing functionality of the wrapped object that simply delegate to the underlying object.
1. Pro: Less fragile
2. Con: Minor overhead due to double locking.
8. Document a class's thread-safety guarantees for users of the class; Document its synchronization policy for the class's maintainers.
1. Use @GuardedBy annotations to document the locks used to guard different state variables.
2. Since documentation for commonly used Java libraries is vague, we often have to guess whether a class is thread-safe or not.  For example, since a JDBC DataSource represents a pool of reusable database connections to be shared across multiple threads, we can assume that it is thread-safe.  However, individual JDBC Connection objects are intended to be used by a single thread, and are most likely not thread-safe.

Chapter 5: Building Blocks

1. Java offers a wide variety of thread-safe classes that can serve as building blocks for large concurrent programs.  
2. External locking is still required for thread-safe classes in order to provide a 'reasonable' behavior.  
3. Unreliable Iteration is one example where additional synchronization is required.  For example, using getLast() and deleteLast() on a thread-safe List requires additional synchronization.  If getLast() determines the index of the last element to be L and the element at L is deleted by deleteLast() before getLast() accesses it, an ArrayOutOfBoundsException will be thrown.   Otherwise, an ArrayOutOfBounds exception may be thrown if the last element of the list is deleted after getLast() determined the index of the last element to be returned.  Note that the data in the List is never corrupted, we just get unexpected behavior.
1. Java Iterators throw an unchecked ConcurrentModificationException if it detects that the underlying collection has changed during iteration. This is not reliable as the counters used to track whether the underlying collection has changed are not thread-safe.
2. Iterators can sometimes be hidden - For eg: an iterator is used if a collection object is passed to a log statement which tries to get its string representation.
3. Unreliable iteration can be solved by client-side locking.  When using the synchronized wrappers provided by Java, we can use the underlying collection as the lock for composite actions. Con: Decreases scalability.
4. Another option to avoid concurrent modification exception is to clone the collection as a local copy.  The lock on the collection must be held while cloning.  Con: Can be very expensive to clone large collections.
5. To avoid client-side locking during iteration, use the Concurrent collections offered by Java.  This improves scalability as multiple threads can now access the collections simultaneously without blocking
4. ConcurrentHashMap
1. HashMap uses a single lock to synchronize all its operations.
2. ConcurrentHashMap uses lock-striping.  Supports concurrent non-blocking access by infinite number of readers, and a limited number of writers.
3. Iterators do not throw ConcurrentModificationException.
4. size() and isEmpty() are approximate.  These methods are not useful in concurrent environments anyway.
5. Cannot lock the entire map for synchronized access, needed in rare cases where multiple map entries need to be added atomically.
6. Cannot use client-side mapping while adding new atomic operations.  If these are needed, you most likely need ConcurrentMap instead of ConcurrentHashMap.
5. CopyOnWriteArrayList 
1. Thread-safety derived from the immutability of underlying list.  Mutability is provided by creating and republishing a new copy of the list on every change.  Iterators point to the list at the time the iterator was created.
2. Copying large lists can be expensive.  Hence mainly useful when iteration is the more common operation rather than addition - for eg: when using a list of event listeners.
6. Some more concurrent collections - CopyOnWriteArraySet, ConcurrentLinkedQueue, ConcurrentSkipListMap - concurrent replacement for synchronized SortedMap, ConcurrentSkipListSet - concurrent replacement for synchronized SortedSets
7. Java offers multiple Queue implementations (esp. BlockingQueue) that can be very useful to implement producer-consumer designs.  Queues can be blocking or non-blocking.  For non-blocking queues, .retrieval operations return null if queue is empty.
8. BlockingQueue
1. blocking methods : take, put (blocking happens only if queue is bounded)
2. non-blocking methods: offer, poll
3. Use bounded blocking queues for reliable resource management.  Otherwise, if consumers are slow, producers can keeping adding to the queue till the JVM runs out of heap space.  Do this early in design; hard to retrofit later.
4. We can use offer() to check if the item will be accepted by the queue.  If the queue is full, the item can be discarded in application specific ways (for eg: simply drop it or save it to local disk for later usage)
5. BlockingQueue implementations - contain sufficient internal synchronization to safely publish objects from a producer thread to a consumer thread
1. LinkedBlockingQueue
2. ArrayBlockingQueue
3. PriorityBlockingQueue
4. SynchronousQueue - Not really a queue as it does not maintain storage space for elements.  Just maintains a list of queued threads waiting to enqueue or dequeue an element.  Directly transfers item from producer to consumer - more efficient.  Direct handoff also informs producer that consumer has taken responsibility for the item.  take() and put() will block if no thread is waiting to participate in the handoff.  Mainly used when there are always enough consumer threads.
9. Deque and BlockingDeque - allows efficient insertion and removal from head and tail.
1. Enables Work Stealing designs - In producer-consumer design, there is a single queue that is shared across all threads.  This causes lots of contention.  In work stealing,  each consumer has its own deque, from the head of which it consumes items. If its deque is empty, it can steal objects from the tail of some other consumer's deque. Most of the time, a consumer takes objects from the head of its own deque, thereby avoiding contention.  Even when it steals, there is little contention as it steals from the tail rather than the head.  Work stealing is well-suited for applications where producers are also consumers.
10. Interruption
1. Thread.interrupt() interrupts a thread.
2. Interruption is a cooperative mechanism, i.e.,  One thread cannot force another to stop what it is doing.  Thread.interrupt() merely requests a thread to stop at a convenient stopping point.
3. If a method is marked to throw an InterruptedException, it means that the method is blocking and that it will attempt to stop blocking early if interrupted.
4. No language specification about how to deal with interrupts.  Most natural option is to cancel whatever the thread is currently doing. Blocking methods that are interruptible make it easy to cancel long-running tasks when necessary.
5. One common option to handle the InterruptedException is to propagate it to your caller.  This can be done by not catching it at all, or by catching and rethrowing it after performing some local cleanup.
6. In cases where you cannot throw an InterruptedException (for eg: inside Runnabe.run()), you must catch the InterruptedException and restore the interrupted status by calling Thread.currentThread().interrupt() on the current thread.  This allows  code higher up in the call stack to see that the thread was interrupted.
7. Never catch an InterruptedException and ignore it - except when extending Thread (and therefore controlling all code higher up in the call stack).
11. Synchronizers - an object that coordinates the control flow of threads based on its state.  BlockingQueue is one example of a synchronizer.  Latch, FutureTask, Semaphores and Barriers are other examples of synchronizers.
1. Latch - a synchronizer that can block threads until it reaches its terminal state.  A latch acts as a gate.  Once open, it remains open forever.
1. Eg usage: Ensure that a computation cannot proceed until the resources needed by it have been initialized, Wait until all players in a multi-player game have finished their moves.
2. CountDownLatch - initialized with positive integer.Threads call await(), which blocks till counter becomes 0.  Other threads call countDown() which decreases the count.
2. FutureTask - mainly used to represent long running or async computation (for eg: by the Executor framework)
1. The computation is encapsulated in a Callable (result-bearing equivalent of Runnable).
2. FutureTask.get() returns result immediately if computation is done, or if exception is thrown or if cancelled; otherwise blocks till done.  Result obtained from get() is safely published.
3. Once complete, it stays in completed state forever.
4. Future.get() can throw an ExecutionException if the Callable.run() throws one. Check all known exceptions when calling get().  Other exceptions are generally rethrown.
3. Semaphores
1. Counting semaphores are used to control the number of threads that can simultaneously access a resource.  A thread wishing to use the resource must acquire() a virtual permit and release() it when done.  acquire() blocks if no permits are available.
2. A binary semaphore is a mutex with non-reentrant locking, unlike the intrinsic java object lock which is reentrant.
3. Can be used to turn any collection into a bounded blocking collection.
4. Barriers
1. Similar to latches, but all threads must come together at the barrier point at the same time in order to proceed.
2. CyclicBarrier allows a fixed number of threads to rendezvous repeatedly.  Useful in parallel iterative algorithms.
3. If a thread blocked on await() is interrupted or an await() times out, then BrokenBarrierException is thrown.
4. When barrier is successfully passed, await() returns with a  unique arrival index per thread, which can be used for leader election amongst the threads.
5. Also supports barrier action - a Runnable to be executed when barrier is successfully passed but before threads are released.
12. For building an efficient scalable result cache, use a ConcurrentHashMap> putIfAbsent()