Create an AWS Batch launcher object.

Create an R6 AWS Batch launcher object.

Usage

crew_launcher_aws_batch(
  name = NULL,
  seconds_interval = 0.5,
  seconds_timeout = 60,
  seconds_launch = 1800,
  seconds_idle = 300,
  seconds_wall = Inf,
  tasks_max = Inf,
  tasks_timers = 0L,
  reset_globals = TRUE,
  reset_packages = FALSE,
  reset_options = FALSE,
  garbage_collection = FALSE,
  crashes_error = 5L,
  tls = crew::crew_tls(mode = "automatic"),
  processes = NULL,
  r_arguments = c("--no-save", "--no-restore"),
  options_metrics = crew::crew_options_metrics(),
  options_aws_batch = crew.aws.batch::crew_options_aws_batch(),
  aws_batch_config = NULL,
  aws_batch_credentials = NULL,
  aws_batch_endpoint = NULL,
  aws_batch_region = NULL,
  aws_batch_job_definition = NULL,
  aws_batch_job_queue = NULL,
  aws_batch_share_identifier = NULL,
  aws_batch_scheduling_priority_override = NULL,
  aws_batch_parameters = NULL,
  aws_batch_container_overrides = NULL,
  aws_batch_node_overrides = NULL,
  aws_batch_retry_strategy = NULL,
  aws_batch_propagate_tags = NULL,
  aws_batch_timeout = NULL,
  aws_batch_tags = NULL,
  aws_batch_eks_properties_override = NULL
)

Arguments

name: Name of the launcher.
seconds_interval: Number of seconds between polling intervals waiting for certain internal synchronous operations to complete, such as checking mirai::status().
seconds_timeout: Number of seconds until timing out while waiting for certain synchronous operations to complete, such as checking mirai::status().
seconds_launch: Seconds of startup time to allow. A worker is unconditionally assumed to be alive from the moment of its launch until seconds_launch seconds later. After seconds_launch seconds, the worker is only considered alive if it is actively connected to its assign websocket.
seconds_idle: Maximum number of seconds that a worker can idle since the completion of the last task. If exceeded, the worker exits. But the timer does not launch until tasks_timers tasks have completed. See the idletime argument of mirai::daemon(). crew does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, so please allow enough idle time for a new worker to be delegated a new task.
seconds_wall: Soft wall time in seconds. The timer does not launch until tasks_timers tasks have completed. See the walltime argument of mirai::daemon().
tasks_max: Maximum number of tasks that a worker will do before exiting. See the maxtasks argument of mirai::daemon(). crew does not excel with perfectly transient workers because it does not micromanage the assignment of tasks to workers, it is recommended to set tasks_max to a value greater than 1.
tasks_timers: Number of tasks to do before activating the timers for seconds_idle and seconds_wall. See the timerstart argument of mirai::daemon().
reset_globals: TRUE to reset global environment variables between tasks, FALSE to leave them alone.
reset_packages: TRUE to unload any packages loaded during a task (runs between each task), FALSE to leave packages alone.
reset_options: TRUE to reset global options to their original state between each task, FALSE otherwise. It is recommended to only set reset_options = TRUE if reset_packages is also TRUE because packages sometimes rely on options they set at loading time.
garbage_collection: TRUE to run garbage collection between tasks, FALSE to skip.
crashes_error: Positive integer scalar. If a worker exits crashes_error times in a row without completing all its assigned tasks, then the launcher throws an informative error. The reason for crashes_error is to avoid an infinite loop where a task crashes a worker (through a segfault, maxing out memory, etc) but the worker always relaunches. To monitor the resources of crew workers, please see https://wlandau.github.io/crew/articles/logging.html.
tls: A TLS configuration object from crew_tls().
processes: NULL or positive integer of length 1, number of local processes to launch to allow worker launches to happen asynchronously. If NULL, then no local processes are launched. If 1 or greater, then the launcher starts the processes on start() and ends them on terminate(). Plugins that may use these processes should run asynchronous calls using launcher$async$eval() and expect a mirai task object as the return value.
r_arguments: Optional character vector of command line arguments to pass to Rscript (non-Windows) or Rscript.exe (Windows) when starting a worker. Example: r_arguments = c("--vanilla", "--max-connections=32").
options_metrics: Either NULL to opt out of resource metric logging for workers, or an object from crew_options_metrics() to enable and configure resource metric logging for workers. For resource logging to run, the autometric R package version 0.1.0 or higher must be installed.
options_aws_batch: List of options from crew_options_aws_batch(). The job definition and job queue must be specified in crew_options_aws_batch(). crew_options_aws_batch() also allows you to request vCPUs, GPUs, and memory for the jobs.
aws_batch_config: Deprecated. Use options_aws_batch instead.
aws_batch_credentials: Deprecated. Use options_aws_batch instead.
aws_batch_endpoint: Deprecated. Use options_aws_batch instead.
aws_batch_region: Deprecated. Use options_aws_batch instead.
aws_batch_job_definition: Deprecated. Use options_aws_batch instead.
aws_batch_job_queue: Deprecated. Use options_aws_batch instead.
aws_batch_share_identifier: Deprecated. Use options_aws_batch instead.
aws_batch_scheduling_priority_override: Deprecated. Use options_aws_batch instead.
aws_batch_parameters: Deprecated. Use options_aws_batch instead.
aws_batch_container_overrides: Deprecated. Use options_aws_batch instead.
aws_batch_node_overrides: Deprecated. Use options_aws_batch instead.
aws_batch_retry_strategy: Deprecated. Use options_aws_batch instead.
aws_batch_propagate_tags: Deprecated. Use options_aws_batch instead.
aws_batch_timeout: Deprecated. Use options_aws_batch instead.
aws_batch_tags: Deprecated. Use options_aws_batch instead.
aws_batch_eks_properties_override: Deprecated. Use options_aws_batch instead.

Value

An R6 AWS Batch launcher object.

IAM policies

In order for the AWS Batch crew plugin to function properly, your IAM policy needs permission to perform the SubmitJob and TerminateJob AWS Batch API calls. For more information on AWS policies and permissions, please visit https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html.

AWS arguments

The AWS Batch controller and launcher accept many arguments which start with "aws_batch_". These arguments are AWS-Batch-specific parameters forwarded directly to the submit_job() method for the Batch client in the paws.compute R package

For a full description of each argument, including its meaning and format, please visit https://www.paws-r-sdk.com/docs/batch_submit_job/. The upstream API documentation is at https://docs.aws.amazon.com/batch/latest/APIReference/API_SubmitJob.html and the analogous CLI documentation is at https://docs.aws.amazon.com/cli/latest/reference/batch/submit-job.html.

The actual argument names may vary slightly, depending on which : for example, the aws_batch_job_definition argument of the crew AWS Batch launcher/controller corresponds to the jobDefinition argument of the web API and paws.compute::batch()$submit_job(), and both correspond to the --job-definition argument of the CLI.

Verbosity

Control verbosity with the paws.log_level global option in R. Set to 0 for minimum verbosity and 3 for maximum verbosity.