Skip to contents

Changelog

Source: NEWS.md

crew 0.9.2

  • Use .args rather than ... in mirai::mirai() to make sure arguments continue to be passed as local variables in mirai >= 0.13.1.9012.
  • Add new controller methods autoscale(), descale(), and started() to facilitate different kinds of Shiny apps.
  • Deprecate the scale and throttle methods of controller$promise(). promise() now always calls autoscale() to make sure one and only one auto-scaling loop is running asynchronously. Auto-scaling thus continues even after the promise resolves.
  • Add a second example vignette that simulates coin flips.

crew 0.9.1

CRAN release: 2024-03-25

  • Rewrite the async Shiny vignette with crew promises and Shiny extended tasks (#157, @jcheng5).
  • Clarify the intent of controller$promise(mode = "one") in the vignette on promises (@jcheng5).
  • Implement an error argument in pop() which may help with integration with ExtendedTask (@jcheng5).
  • Handle task errors in the Shiny vignette (@jcheng5).

crew 0.9.0

CRAN release: 2024-02-08

  • Require nanonext >= 0.12.0 and mirai >= 0.12.0.
  • Return to always re-launching backlogged inactive workers (#79, https://github.com/shikokuchuo/mirai/discussions/95).
  • Implement push_backlog() and pop_backlog() to manage cases when it is not desirable to push to saturated controllers (https://github.com/ropensci/targets/issues/1220).
  • Invisibly return the mirai object of a task pushed to the controller. This allows users to interact with the task directly, e.g. to create a promise object with promises::as.promise() (#146, @jcheng5).
  • Add a new walk() method for controllers and controller groups to submit tasks in batch and return control immediately without waiting for any task to complete (#148, @jcheng5).
  • Revive the collect() method for popping multiple tasks at once (#148, @jcheng5).
  • Add controller group methods nonempty(), resolved(), unresolved(), and unpopped() to help with #148.
  • Make the mirai dispatcher error message extremely verbose.

crew 0.8.0

CRAN release: 2024-01-09

crew 0.7.0

CRAN release: 2023-12-11

  • Migrate from asyncdial to autoexit.
  • Use Sys.info()[["user"]] to get the user in crew_clean() (#135, @luwidmer).
  • Use condition variables to wait for tasks more efficiently (#108).
  • Because of #108, controller$map() can no longer be used if there are tasks waiting to be popped.
  • Use a cli progress bar in map().
  • Encapsulate non-function R6 members in the private list and add active bindings where interfaces are necessary (#137). Ad hoc tests that absolutely need to modify private objects can use object$.__enclos_env__$private trick, but these tests should be skipped on CRAN in case there is a change to R6 that breaks this.
  • Drop .signal from mirai::mirai() since all mirai tasks signal as of mirai version 0.11.2.9025.
  • Implement crew_throttle(), a decoupled mechanism for throttling that can be applied to scaling.
  • Bring back the throttle argument, powered by crew_throttle().
  • Retry mirai::status() again in daemons_info() and make it configurable using seconds_interval and seconds_timeout in both the client and the launcher (#128).

crew 0.6.0

CRAN release: 2023-10-12

  • Migrate checks to enforce features in version 0.5.0 for reverse dependencies.
  • Drop check of backlogged workers, c.f. https://github.com/shikokuchuo/mirai/discussions/63#discussioncomment-7051889 (#79, #124, @shikokuchuo).
  • Deprecate seconds_exit because exitlinger in mirai is now obsolete (#125, @shikokuchuo).
  • Use mirai::nextget("cv") to count unresolved tasks instead of looping through all the task objects (#131).
  • Remove throttling and collect() in auto-scaling. Simplifies much of the code. Made possible by the efficiency gains in #131.
  • Simplify wait().
  • seconds_interval in map() no longer defaults to controller$client$seconds_interval.
  • launcher$terminate_workers() terminates one or more workers, and launcher$terminate terminates the whole launcher.
  • Add infrastructure to let custom launcher plugins launch and terminate workers asynchronously (#133). Launchers can set a positive number in the processes field to set the number of local mirai daemons for asynchronous requests to launch and terminate the serious workers. Then, launch_worker() and terminate_worker() can optionally make use of launcher$async$eval() to send these asynchronous calls.

crew 0.5.0

CRAN release: 2023-09-18

  • Suppress interactive browser on Windows which launched on each worker previously (@psychelzh).
  • Migrate to the new host/daemon nomenclature in mirai 0.9.1 (#96).
  • Suppress status() retries (@shikokuchuo, #100).
  • Implement launch_max to error out if workers repeatedly launch without completing any tasks (#101, @shikokuchuo, @multimeric).
  • Detect discovered workers more robustly in launcher$done().
  • Add a new algorithm argument to the push(), shove(), and map() methods of controllers and controller groups, as well as arguments/fields in crew_eval() and its return value (#113, @shikokuchuo).
  • As a default for pseudo-random number generation, leverage widely-spaced L’Ecuyer streams as supported by mirai::nextstream() (#115, @shikokuchuo).
  • Move README documentation to vignettes.
  • Add a new crew_tls() function for TLS configuration (#120).
  • Deprecate the tls_enable and tls_config arguments of crew_client() etc. in favor of a tls argument which accepts crew_tls() objects (#120).

crew 0.4.0

CRAN release: 2023-07-10

Highlights

  • Use the TLS capabilities of nanonext and mirai.
  • Add a map() method to the controller class for functional programming (#93).
  • Fix bug in scale() which launched too many workers when active - demand was less than 0.
  • Add crew_clean() to help clean up any lingering dispatchers and workers from previous sessions.

Other improvements

  • Remove temporary controller summary workaround that ensured CRAN compatibility with targets 1.1.3.
  • Add guardrails to make sure the controller is started for methods that require this. Part of this involved improving the error messages.
  • Retry daemons() in daemons_info().

crew 0.3.0

CRAN release: 2023-06-26

  • Track warnings properly (#81, @brendanf).
  • Optimize heavily (#81, #83, @shikokuchuo, @brendanf).
  • Use CRAN nanonext 0.9.0.
  • Delegate the task pushing and collection logic to a new R6 schedule class (#84).
  • Delegate auto-scaling to the launcher (#84).
  • Drastically simplify internal logic (#84).
  • Rename “router” to “client” and change the function signature of launch_worker() (#84). Unfortunately neither of these changes is back-compatible for third-party launcher plugins. However, the controller interface is still back-compatible, so targets remains unaffected.
  • Terminate “lost” workers on websocket rotation.
  • Allow NULL result in pop()$result[[1L]].
  • Delegate summaries among the controller, schedule, launcher, and client.
  • Default to the loopback address for the local controller.
  • Use already cumulative assigned and complete stats from mirai 0.8.7.9029.
  • Reorganize pkgdown reference and Rd families to clarify which parts are for end-users and which parts are for launcher plugin developers.
  • Add controller$shove() and schedule$list() to support fast extensions for functional programming.
  • Depend on mirai 0.9.0 or greater.

crew 0.2.1

CRAN release: 2023-05-22

crew 0.2.0

  • Use .args instead of ... in mirai::mirai().
  • Use set.seed() instead of withr::local_seed() (#67, @shikokuchuo).
  • Add a vignette to showcase integration with Shiny, co-authored with @dwoodie and with helpful suggestions by @shikokuchuo.
  • Drop rlang::duplicate() (#72, @shikokuchuo).
  • Do not poll dispatcher availability when starting router objects (@shikokuchuo).
  • Implement a saturated() controller method to support targets.
  • Add a worker_index column to controller summaries.
  • When relaunching workers, prioritize those with unresolved tasks stuck at the NNG level (#75, @shikokuchuo).
  • Always relaunch backlogged inactive workers (#79).
  • Deprecate the auto_scale argument/field of controllers in favor of the scale argument of push(), pop(), and wait().
  • Throttle auto-scaling with interval self$router$seconds_interval (#76).
  • Remove clean() and instead terminate lost workers on launch.
  • Fix examples.
  • Add a collect argument to pop().

crew 0.1.1

CRAN release: 2023-04-21

  • Ensure checks pass on CRAN.

crew 0.1.0

CRAN release: 2023-04-19

  • Remove asyncdial from call to mirai::daemons().
  • Explain vision in the launcher plugin vignette.
  • Remove the superfluous clean() controller method.
  • Clean up logic for scale() and launch() controller methods.
  • Add a new inactive() launcher method to help with the above.
  • Eliminate superfluous worker launches: clean up the controller logic, and check worker discoveries before worker connections (#51).
  • Simplify the launcher plugin interface (#54).
  • Commit to using the fast nanonext::.unresolved() (#49).
  • Rename the callr launcher to the local process launcher (#56).
  • Switch from SHA224 to SHA1 as the hash for random tokens and names (#59, @shikokuchuo).
  • Open and confirm bus socket listeners asynchronously in the launcher abstract class (#58).
  • Use condition variables to test the connection status of worker instances (#57, @shikokuchuo).
  • Allow the dispatcher process to not exist so that the router will keep working if mirai implements the dispatcher another way (#60).
  • Add a substitute argument to push() (#63).
  • Remove dependency on custom NNG bus sockets, which significantly simplifies the internals, improves robustness, and smooths the integration with mirai (#61).
  • Use processx instead of callr for the local process launcher (#64”).
  • Speed up crew_eval() by e.g. relying on nanonext::mclock() for the time.
  • Add an empty() method to check if the controller or controller group is empty.
  • Rely on mirai for globals cleanup, garbage collection, and global option resets (#65).
  • Expose the package unloading capabilities of mirai (#65).
  • Make monads classed environments and leave tibbles to the last minute in pop().
  • Rename crew_wait() to crew_retry().
  • Export crew_assert() and crew_random_name() to help launcher plugins.
  • Rename the listen() method to start() in the router class.
  • The poll() method now retries mirai::daemons() for robustness.
  • crew_retry() gains a max_tries argument.
  • Bug fix: stop setting router$daemons to NULL in poll() when the connection has a momentary problem.
  • Simplify listening().

crew 0.0.5

CRAN release: 2023-03-23

  • Write a call() method to make it easier to write custom launcher plugins.
  • Add robust documentation and tests for custom launcher plugins.
  • Use as.character() to make sure the list argument supplied to rm() is safe. Fixes a test that was failing on R-devel with crew version 0.0.4.
  • Remove the seconds_poll_high and seconds_poll_low fields and arguments for compatibility with mirai development and release versions.
  • Hard-code the async_dial field in the router for compatibility with mirai development and release versions.

crew 0.0.4

CRAN release: 2023-03-21

  • Adjust references and tests as requested by CRAN.

crew 0.0.3

  • First version.