[R-package] Add support for R 4.0 (fixes #3064, fixes #3024) #3065
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,14 +20,56 @@ Note: 32-bit (i386) R/Rtools is currently not supported. | |
|
|
||
| #### Windows Preparation | ||
|
|
||
| Installing a 64-bit version of [Rtools](https://cran.r-project.org/bin/windows/Rtools/) is mandatory. | ||
|
|
||
| After installing `Rtools` and `CMake`, be sure the following paths are added to the environment variable `PATH`. These may have been automatically added when installing other software. | ||
|
|
||
| * `Rtools` | ||
| - If you have `Rtools` 3.x, example: | ||
| - `C:\Rtools\mingw_64\bin` | ||
| - If you have `Rtools` 4.0, example: | ||
| - `C:\rtools40\mingw64\bin` | ||
| - `C:\rtools40\usr\bin` | ||
| * `CMake` | ||
| - example: `C:\Program Files\CMake\bin` | ||
| * `R` | ||
| - example: `C:\Program Files\R\R-3.6.1\bin` | ||
|
|
||
| As of `Rtools` 4.0, some common paths changed and software was removed from `Rtools`. If you are using `R` 4.0 or later and the corresponding `Rtools`, you need to add one additional path to `PATH`. | ||
|
|
||
| * `Rtools` usr bin: | ||
| - example: `C:\Rtools\usr\bin` | ||
StrikerRUS marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Why not just add this path to the case There was a problem hiding this comment. huh I thought I did that already! sorry, will do There was a problem hiding this comment. oh! now I remember. That section you're referring to is documenting how the R package will fall back to msys2 if you don't have Visual Studio. Adding this There was a problem hiding this comment. Sorry, I didn't get it. I'm referring the following section:
Fallbacks are documented much later in the README. There was a problem hiding this comment. ooooooooh I misunderstood! Yes you're right, ok those two are definitely duplicates There was a problem hiding this comment. almost missed this one, sorry! Just fixed this in d0e995f. I also added one more fix in that commit noticed during testing...if using |
||
|
|
||
| #### Windows Toolchain Options | ||
|
|
||
| A "toolchain" refers to the collection of software used to build the library. The R package can be built with three different toolchains. | ||
|
|
||
| **Warning for Windows users**: it is recommended to use *Visual Studio* for its better multi-threading efficiency in Windows for many core systems. For very simple systems (dual core computers or worse), MinGW64 is recommended for maximum performance. If you do not know what to choose, it is recommended to use [Visual Studio](https://visualstudio.microsoft.com/downloads/), the default compiler. **Do not try using MinGW in Windows on many core systems. It may result in 10x slower results than Visual Studio.** | ||
|
|
||
| **Visual Studio (default)** | ||
|
|
||
| By default, the package will be built with [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/). | ||
|
|
||
| **MinGW (R 3.x)** | ||
|
|
||
| If you are using R 3.x and installation fails with Visual Studio, `LightGBM` will fall back to using [MinGW](http://mingw-w64.org/doku.php) bundled with `Rtools`. | ||
|
|
||
| If you want to force `LightGBM` to use MinGW (for any R version), open `R-package/src/install.libs.R` and change `use_mingw`: | ||
|
|
||
| ```r | ||
| use_mingw <- TRUE | ||
| ``` | ||
|
|
||
| **MSYS2 (R 4.x)** | ||
|
|
||
| If you are using R 4.x and installation fails with Visual Studio, `LightGBM` will fall back to using [MSYS2](https://www.msys2.org/). This should work with the tools already bundled in `Rtools` 4.0. | ||
|
|
||
| If you want to force `LightGBM` to use MSYS2 (for any R version), open `R-package/src/install.libs.R` and change `use_msys2`: | ||
|
|
||
| ```r | ||
| use_msys2 <- TRUE | ||
| ``` | ||
|
|
||
| #### Mac OS Preparation | ||
|
|
||
| You can perform installation either with **Apple Clang** or **gcc**. In case you prefer **Apple Clang**, you should install **OpenMP** (details for installation can be found in [Installation Guide](https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#apple-clang)) first and **CMake** version 3.16 or higher is required. In case you prefer **gcc**, you need to install it (details for installation can be found in [Installation Guide](https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#gcc)) and set some environment variables to tell R to use `gcc` and `g++`. If you install these from Homebrew, your versions of `g++` and `gcc` are most likely in `/usr/local/bin`, as shown below. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # [description] | ||
| # Create a definition file (.def) from a .dll file, using objdump. | ||
| # | ||
| # [usage] | ||
| # | ||
| # Rscript make-r-def.R something.dll something.def | ||
| # | ||
| # [references] | ||
| # * https://www.cs.colorado.edu/~main/cs1300/doc/mingwfaq.html | ||
|
|
||
| args <- commandArgs(trailingOnly = TRUE) | ||
|
|
||
| IN_DLL_FILE <- args[[1L]] | ||
| OUT_DEF_FILE <- args[[2L]] | ||
| DLL_BASE_NAME <- basename(IN_DLL_FILE) | ||
|
|
||
| message(sprintf("Creating '%s' from '%s'", OUT_DEF_FILE, IN_DLL_FILE)) | ||
|
|
||
| # system() will not raise an R exception if the process called | ||
| # fails. Wrapping it here to get that behavior. | ||
| # | ||
| # system() introduces a lot of overhead, at least on Windows, | ||
| # so trying processx if it is available | ||
| .pipe_shell_command_to_stdout <- function(command, args, out_file) { | ||
| has_processx <- suppressMessages({ | ||
| suppressWarnings({ | ||
| require("processx") # nolint | ||
| }) | ||
| }) | ||
| if (has_processx) { | ||
| p <- processx::process$new( | ||
| command = command | ||
| , args = args | ||
| , stdout = out_file | ||
| , windows_verbatim_args = FALSE | ||
| ) | ||
| invisible(p$wait()) | ||
StrikerRUS marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } else { | ||
| message(paste0( | ||
| "Using system2() to run shell commands. Installing " | ||
| , "'processx' with install.packages('processx') might " | ||
| , "make this faster." | ||
| )) | ||
| exit_code <- system2( | ||
| command = command | ||
| , args = args | ||
| , stdout = out_file | ||
| ) | ||
| if (exit_code != 0L) { | ||
| stop(paste0("Command failed with exit code: ", exit_code)) | ||
| } | ||
| } | ||
| return(invisible(NULL)) | ||
| } | ||
|
|
||
| # use objdump to dump all the symbols | ||
| OBJDUMP_FILE <- "objdump-out.txt" | ||
| .pipe_shell_command_to_stdout( | ||
| command = "objdump" | ||
| , args = c("-p", IN_DLL_FILE) | ||
| , out_file = OBJDUMP_FILE | ||
| ) | ||
jameslamb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| objdump_results <- readLines(OBJDUMP_FILE) | ||
| invisible(file.remove(OBJDUMP_FILE)) | ||
|
|
||
| # Only one table in the objdump results matters for our purposes, | ||
| # see https://www.cs.colorado.edu/~main/cs1300/doc/mingwfaq.html | ||
| start_index <- which( | ||
| grepl( | ||
| pattern = "[Ordinal/Name Pointer] Table" | ||
| , x = objdump_results | ||
| , fixed = TRUE | ||
| ) | ||
| ) | ||
| empty_lines <- which(objdump_results == "") | ||
| end_of_table <- empty_lines[empty_lines > start_index][1L] | ||
|
|
||
| # Read the contents of the table | ||
| exported_symbols <- objdump_results[(start_index + 1L):end_of_table] | ||
| exported_symbols <- gsub("\t", "", exported_symbols) | ||
| exported_symbols <- gsub(".*\\] ", "", exported_symbols) | ||
| exported_symbols <- gsub(" ", "", exported_symbols) | ||
|
|
||
| # Write R.def file | ||
| writeLines( | ||
| text = c( | ||
| paste0("LIBRARY \"", DLL_BASE_NAME, "\"") | ||
| , "EXPORTS" | ||
| , exported_symbols | ||
| ) | ||
| , con = OUT_DEF_FILE | ||
| , sep = "\n" | ||
| ) | ||
| message(sprintf("Successfully created '%s'", OUT_DEF_FILE)) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.