How to use box modules and mock dependencies when unit testing using testthat?

[dereckmezquita]

Hello,

I’m encountering difficulties when attempting to unit test functions defined in a box module. My module defines a function that, in turn, imports several helper functions (e.g. build_headers, process_kucoin_response, etc.). I then import that function into my test file using box. However, when I try to mock these internal dependencies using testthat’s mocking facilities (such as local_mocked_bindings()), I run into errors.

For example, one of the errors I receive is:

Error: Can't find binding for `build_headers`

This error occurs when I attempt something like:

box::use(
    impl = ../../R/impl_account_and_funding
)

local_mocked_bindings({
    impl$get_account_summary_info_impl()$
        then(function(result) { 
            # assertions here 
        })$
        catch(function(e) { 
            fail(paste("Promise rejected with error:", conditionMessage(e)))
        })
    while (!loop_empty()) run_now(timeoutSecs = 0.1)
}, build_headers = fake_build_headers, .env = environment(impl$get_account_summary_info_impl))

It appears that the functions imported into my module via box are not available as modifiable bindings in the module’s environment. I suspect that it is because box locks the environment or restructures the namespace such that these imported functions cannot be overridden by testthat’s mock tools.

I suspect the issue stems from how box handles module environments and imports, possibly locking them or not exposing imported functions for modification.

Has anyone successfully implemented unit tests with mocking in a box based project using testthat? Are there any recommended strategies or workarounds to allow for dependency mocking in box modules?

[dereckmezquita]

I've been having a long think about how I can get what I want to work and just to better understand the issue at hand.

I think what I want is I want to be ablet to manipulate the search path for box.

This is my setup:

.
├── R
│   ├── hello_world.R
│   └── print_message.R
├── index.R
├── stubs
│   └── print_message.R
└── test.R

Where:

index.R

box::use(./R/hello_world[ hello_world ])

hello_world()

test.R

# somehow set the path so it will get the print_message function from the stub folder instead
options(box.path = c("stubs", "R"))
box::use(./R/hello_world[ hello_world ])

hello_world()

R/hello_world.R

box::use(print_message[ print_message ])

#' @export
hello_world <- function() {
    print_message()
}

R/print_message.R

#' @export
print_message <- function() {
    message("Hello, world!")
}

stubs/print_message.R

#' @export
print_message <- function() {
    message("Goodbye, world!")
}

When I run this I get:

(base) work@Derecks-MacBook-Air research % Rscript test.R
Loading .Rprofile.
Setting conda environment to base.
Multi-threading for data.table using: 7 cores.
Error in box::use(./R/hello_world[hello_world]) : 
  there is no package called ‘print_message’
Calls: <Anonymous> ... tryCatchList -> tryCatchOne -> <Anonymous> -> rethrow
Execution halted

So what I want is:

• When I run index.R the code will get the print_message function from R/ but when I run test.R it will get the print_message function from the stubs/ directory.

I understood that if I use ./ it will always use a relative search, but I just can't get it to work and import without the ./ I keep getting error of "Cannot find package".

The reason I want this is because I want to be able to "mock" my own functions. I am writing unit tests for a package that interacts with an API and I am having a lot of trouble with this.

https://github.com/dereckmezquita/kucoin/tree/TRADE-2-kucoin2

Any help would be greatly appreciated!

[klmr]

See my top-level reply for my thoughts on mocking modules. However, modifying the module search path to replace entire modules is an interesting complementary approach (though I think it would be limited to a handful of scenarios).

The reason why your approach isn’t working is because ‘box’ always interprets names without slashes as package names. So box::use(print_message[ print_message ]) searches for a package named print_message. It will never search for a module of that name. To make ‘box’ interpret the name as a module name, it needs to contain a slash.

And there are two ways of specifying module paths, and they cause modules to be searched in different ways:

1. Local module paths (aka. relative module paths), which start with ./ or ../. These are looked up relative to the location of the calling code; they never use the box.path option.
2. Global module paths (aka. absolute module paths): all other module paths. Those are looked up in the box.path search path (and, as a fallback, relative to the location of the calling code).

So, as you correctly noticed, you can’t use box::use(./print_message[ print_message ]) inside R/hello_world.R if you want to be able to manipulate the path lookup. But you also can’t omit the ./.

The proper way to solve this would require writing proper support for mocking in ‘box’ (see other reply). But a quick and dirty hack would allow your code to work (kind of… it’s probably very limited, and I have not tested extensively). Replace the test.R code with

add_module_mock_path = function (mock_path) {
  trace(
    'module_path',
    bquote(path <- c(.(normalizePath(mock_path)), path)),
    at = list(c(2L, 4L, 3L, 3L, 2L)),
    print = FALSE,
    where = asNamespace('box')
  )
}

set_module_mock_path('stubs')
box::use(./R/hello_world[hello_world])

hello_world()

(And change the use-declaration to box::use(./print_message[ print_message ]) inside R/hello_world.R.)

By way of (a very brief) explanation, box:::module_path is the internal work horse that finds the base path of a given module from which box::use() is called, and which should be used as the base to resolve relative imports. We use trace() to inject our mock stub path into that function’s return value just before return() is called (at specifies the source code location inside the function’s syntax tree, and path is the name of the local variable that holds the return value).

[klmr]

I believe that the mocking support in ‘testthat’ only supports packages, no other scenario whatsoever. That’s a shame but it’s a built-in hard requirement that we can’t easily circumvent. In particular, the env argument of local_mocked_bindings doesn’t work the way you are attempting to use it.

However, even if ‘testthat’ wasn’t hard-wired to expect packages, it would presumably only support mocking “bare” names, not fully-qualified module names, nor dynamically loaded names.

That is, a generic mocking library could (at least conceptually) easily support mocking bar in the following module:

box::use(./nested[bar])

#' @export
foo = function () {
  bar()
}

But there’s basically no reasonable chance to support the following scenarios without having dedicated ‘box’ mocking support:

box::use(./nested)

#' @export
foo1 = function () {
  nested$bar()
}

#' @export
foo2 = function () {
  box::use(./nested[bar])
  bar()
}

And the reasons are different for foo1 and foo2:

• foo1 does not call bar() at all. — It calls nested$bar(). The lookup for these two names happens in completely different ways, and if we injected bar into the scope of foo1 that wouldn’t help. Instead, we’d have to mock all of nested, and intercept its subset operator ($). This could be done, but it would then only work for the narrow case in which foo1 calls exactly nested$bar(), not in any other cases (e.g. when foo1 imports the module with an alias, i.e. via box::use(n = ./nested)). A better approach would therefore be to have dedicated ‘box’ modules support built into the mocking code.

• foo2 calls bar, but it calls a locally defined function. A generic mocking function would inject the mocked bar into the parent scope of foo2, but the locally-defined bar takes precedence over that injected function. Again this would be solvable most easily via a mocking function that is specifically designed for ‘box’.

I definitely see the need for this functionality. Unfortunately I am currently strapped for time and I can’t work on ‘box’ very much (that’s also why I have removed the sponsorship button). And adding this functionality would not be trivial (and probably doesn’t belong into ‘box’ itself but rather a dedicated testing/mocking package).

---

Before ‘testthat’ shipped with built-in mocking support, I built my own mock helper which I use in a number of package tests. It does not support ‘box’ modules either but I think its general approach could be extended to support modules.

Currently, it supports overriding functions that are either findable in the current environment (including in parent environments), or by specifying an explicit package name (it does not currently support overriding lazy data though, since that uses a completely different mechanism):

my_sum = function (...) sum(...)

with_mock(
  sum := \(a, b) a * b,
  utils::checkHT := \(...) stop('intercepted!'),  # called inside `utils:::head.function`
  {
    message(my_sum(2, 3))  # prints 6
    head(my_sum)  # Error: intercepted!
  }
)

This syntax could be extended to support mocking functions from modules as follows:

with_mock(
  ./mod/a$bar := \() 'mocked bar',
  {
    box::use(alias = ./mod/a)
    message(alias$foo())
  }
)

The idea here is that the mock setup needs to specify the target module in the same way as box::use() to identify it unambiguously. That way, regardless of how the module is later loaded and how functions inside it are referred to, the mocking code could intercept the desired target.

For completeness, here’s my implementation of the with_mock() function. As a reminder, it doesn’t support module interception; but it might be used as a stepping stone to build that support.

Implementation of with_mock()

with_mock = function (..., envir = parent.frame()) {
  dots = match.call(expand.dots = FALSE)$...

  expr = dots[[length(dots)]]
  stubs = dots[-length(dots)]
  self = environment()

  for (stub in stubs) {
    stopifnot(identical(stub[[1L]], quote(`:=`)))
    qual_name = stub[[2L]]
    value = eval.parent(stub[[3L]])

    if (is.name(qual_name)) {
      name = as.character(qual_name)
      env = find_in_parent_envs(name)
      local_assign(name, value, env)
    } else if (identical(qual_name[[1L]], quote(`::`))) {
      pkg_name = as.character(qual_name[[2L]])
      name = as.character(qual_name[[3L]])

      if (pkg_name %in% loadedNamespaces()) {
        env = asNamespace(qual_name[[2L]])
        local_assign(name, value, env)

        attached_env = find_in_parent_envs(name)
        if (! is.null(attached_env)) {
          local_assign(name, value, attached_env)
        }
      } else {
        # Defer mocking the name until package is loaded, since some tests may rely on deterministic package loading. If multiple hooks are registered we furthermore need to capture the name and value locally, otherwise only the last hook is registered.
        load_hook = local({
          name = name
          value = value

          \(pkg, lib) local_assign(name, value, asNamespace(pkg), scope = self)
        })

        attach_hook = local({
          name = name
          value = value

          \(...) {
            attached_env = find_in_parent_envs(name, envir = self)
            if (! is.null(attached_env)) {
              local_assign(name, value, attached_env, scope = self)
            }
          }
        })

        setHook(packageEvent(pkg_name, 'onLoad'), load_hook)
        setHook(packageEvent(pkg_name, 'attach'), attach_hook)

        on_exit_q(bquote({
          remove_hook(packageEvent(.(pkg_name), 'onLoad'), .(load_hook))
          remove_hook(packageEvent(.(pkg_name), 'attach'), .(attach_hook))
        }))
      }
    } else {
      stop('Expected name or qualified name for mock stub')
    }
  }

  eval(expr, envir = envir)
}

remove_hook = function (hook_name, hook) {
  other_hooks = Filter(\(f) ! identical(f, hook), getHook(hook_name))
  setHook(hook_name, other_hooks, action = 'replace')
}

local_assign = function (name, value, env, scope = parent.frame()) {
  binding_locked = bindingIsLocked(name, env)

  if (binding_locked) unlockBinding(name, env)

  old_value = get(name, envir = env, inherits = FALSE)
  assign(name, value, envir = env)

  restore_expr = substitute(
    {
      # Necessary (for some reason!) when called from within a package event hook.
      if (bindingIsLocked(name, env)) unlockBinding(name, env)
      assign(name, old_value, envir = env)
      if (binding_locked) lockBinding(name, env)
    },
    as.list(environment())  # Needs `as.list()` to prevent inheriting names.
  )
  on_exit_q(restore_expr, scope)
}

on_exit_q = function (expr, scope = parent.frame()) {
  do.call(on.exit, list(expr, add = TRUE), envir = scope)
}

find_in_parent_envs = function (name, envir = parent.frame()) {
  # ‘testthat’ copies the contents of the package to be tested into a custom environment, which is not searched by `utils::find()`. Consequently, we cannot use the latter to replace the name with the mock stub, because this is not the value found when using the name directly inside the test. Instead, this function searches *all* parents of the current frame, not just the search path.

  while (! identical(envir, emptyenv())) {
    if (exists(name, envir = envir, inherits = FALSE)) {
      return(envir)
    }
    envir = parent.env(envir)
  }

  NULL
}