R Programming Training in Kalyan Nagar

Future Scope Of R Programming

• The purpose of the future package is to provide a very simple and uniform way of evaluating R expressions asynchronously using various resources available to the user.
• In programming, a future is an abstraction for a value that may be available at some point in the future. The state of a future can either be unresolved or resolved. As soon as it is resolved, the value is available instantaneously.
• If the value is queried while the future is still unresolved, the current process is blocked until the future is resolved. It is possible to check whether a future is resolved or not without blocking.
• Exactly how and when futures are resolved depends on what strategy is used to evaluate them. For instance, a future can be resolved using a sequential strategy, which means it is resolved in the current R session.
• Other strategies may be to resolve futures asynchronously, for instance, by evaluating expressions in parallel on the current machine or concurrently on a compute cluster.
• With asynchronous futures, the current/main R process does not block, which means it is available for further processing while the futures are being resolved in separate processes running in the background.
• In other words, futures provide a simple but yet powerful construct for parallel and / or distributed processing in R.
• Now, if you cannot be bothered to read all the nitty-gritty details about futures, but just want to try them out, then skip to the end to play with the Mandelbrot demo using both parallel and non-parallel evaluation.
• he future package is designed such that support for additional strategies can be implemented as well.
• For instance, the future callr package provides future backends that evaluates futures in a background R process utilizing the callr package - they work similarly to multisession futures but has a few advantages.
• Continuing, the future batchtools package provides futures for all types of cluster functions ("backends") that the batchtools package supports.
• Specifically, futures for evaluating R expressions via job schedulers such as Slurm, TORQUE/PBS, Oracle/Sun Grid Engine (SGE) and Load Sharing Facility (LSF) are also available. (Comment: The future batchjobs package provides analogue backends based on the batchjobs package; however the BatchJobs developers have deprecated it in favor of batchtools.)
• By default, future expressions are evaluated eagerly (= instantaneously) and synchronously (in the current R session).
• This evaluation strategy is referred to as "sequential". In this section, we will go through each of these strategies and discuss what they have in common and how they differ.

Consistent Behavior Across Futures

Before going through each of the different future strategies, it is probably helpful to clarify the objectives of the Future API (as defined by the future package). When programming with futures, it should not really matter what future strategy is used for executing code. This is because we cannot really know what computational resources the user has access to so the choice of evaluation strategy should be in the hands of the user and not the developer. In other words, the code should not make any assumptions on the type of futures used, e.g. synchronous or asynchronous.

One of the designs of the Future API was to encapsulate any differences such that all types of futures will appear to work the same. This despite expressions may be evaluated locally in the current R session or across the world in remote R sessions. Another obvious advantage of having a consistent API and behavior among different types of futures is that it helps while prototyping. Typically one would use sequential evaluation while building up a script and, later, when the script is fully developed, one may turn on asynchronous processing.

Because of this, the defaults of the different strategies are such that the results and side effects of evaluating a future expression are as similar as possible. More specifically, the following is true for all futures:

• All evaluation is done in a local environment (i.e. local({ expr })) so that assignments do not affect the calling environment. This is natural when evaluating in an external R process, but is also enforced when evaluating in the current R session.

• When a future is constructed, global variables are identified. For asynchronous evaluation, globals are exported to the R process/session that will be evaluating the future expression. For sequential futures with lazy evaluation (lazy = TRUE), globals are "frozen" (cloned to a local environment of the future). Also, in order to protect against exporting too large objects by mistake, there is a built-in assertion that the total size of all globals is less than a given threshold (controllable via an option, cf. help("future.options")). If the threshold is exceeded, an informative error is thrown.

• Future expressions are only evaluated once. As soon as the value (or an error) has been collected it will be available for all succeeding requests.

Synchronous futures are resolved one after another and most commonly by the R process that creates them. When a synchronous future is being resolved it blocks the main process until resolved. There are two types of synchronous futures in the future package, sequential and transparent.

Sequential Futures

• Sequential futures are the default unless otherwise specified.
• They were designed to behave as similar as possible to regular R evaluation while still fulfilling the Future API and its behaviors.

Transparent Futures

• For troubleshooting, transparent futures can be used by specifying plan(transparent).
• A transparent future is technically a sequential future with instant signaling of conditions (including errors and warnings) and where evaluation, and therefore also assignments, take place in the calling environment.
• Transparent futures are particularly useful for troubleshooting errors that are otherwise hard to narrow down.

Asynchronous Futures

• Next, we will turn to asynchronous futures, which are futures that are resolved in the background.
• By design, these futures are non-blocking, that is, after being created the calling process is available for other tasks including creating additional futures.
• It is only when the calling process tries to access the value of a future that is not yet resolved, or trying to create another asynchronous future when all available R processes are busy serving other futures, that it blocks.