Child Row & List Structure Error after updating to DT 0.4 #525
Comments
|
it will be better if you can minimize the example code... |
|
@shrektan sorry, I tried to keep it minimal and reproducible, but here's another example that's been used before in the linked issue. It uses mtcars, but I think the code is a bit long by nature of what's done to create the nested table. The recent reproduction issue was raised by @Plum57 on Jan 31st. |
|
Thanks for the following up.
|
|
I've created a worked example. What's wrong with your code is, probably due to the upgrading of the underlying
Check https://datatables.net/manual/data/ for more info. Moreover, it's an illustration of what minimal really means. It's not about the absolute of the code length but about removing all the unnecessary code. For instance, you don't need to provide such complex data. The style of the table can be more concise as well. So that we can save the time to ensure the issue is not caused by those codes. Minimal Examplelibrary(data.table)
library(shiny)
library(DT)
dat <- local({
n <- 6
DT <- data.table(
HEADER = '►',
A = rep_len(LETTERS, n),
B = rep_len(1:10, n),
C = rep_len(c("GRP1", "GRP2"), n)
)
DT[, .(D=.(.SD)), by = .(HEADER, C)]
})
callback_js <-
JS("
var init_tbl = function(id) {
return '<table id=\"' + id + '\" class=\"display\" width=\"100%\"></table>';
}
// you need this converter ==========================
var converter = function(x) {
var res = [];
for (var i = 0; i < x[Object.keys(x)[0]].length; ++i) res.push({});
for (var key in x) {
var elem = x[key];
for (var i = 0; i < elem.length; ++ i) {
res[i][key] = elem[i];
}
}
return res;
}
var init_datatable = function(id, d) {
d = converter(d);
id = '#'+id;
$(id).DataTable({
data: d,
// you need set the columns ==========================
columns: [
{ data: 'A' },
{ data: 'B' }
]
}).draw();
}
var sub_tbl_id = 0;
table.on('click', 'td.details-control', function() {
var table = $(this).closest('table');
var td = $(this);
var row = $(table).DataTable().row(td.closest('tr'));
if (row.child.isShown()) {
row.child.hide();
td.html('►');
} else {
sub_tbl_id = sub_tbl_id + 1;
var id = 'sub_tbl_id_' + sub_tbl_id;
row.child(init_tbl(id)).show();
init_datatable(id, row.data()[3]);
td.html('▼');
}
})
")
shinyApp(
ui = fluidPage(DT::DTOutput("tbl")),
server = function(input, output) {
output$tbl <- DT::renderDT({
datatable(
dat,
escape = FALSE,
options = list(
columnDefs = list(
list(orderable = FALSE, className = 'details-control', targets = 1),
list(visible = FALSE, targets = -1)
)
),
callback = callback_js
)
}, server = FALSE) # you need set the server to FALSE
},
options = list(port = 33333)
)
You original example (Modified. Work now.)# dt_url <- "http://cran.r-project.org/src/contrib/Archive/DT/DT_0.2.tar.gz"
# install.packages(dt_url, repos=NULL, type="source")
library(data.table)
library(lubridate)
library(shiny)
library(DT)
ui <- fluidPage(
tags$head(
tags$style(HTML("@import url('https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css');"))
),
fluidRow(DT::dataTableOutput(width = "100%", "table"))
)
dat <- local({
# ---- Sample Set ----
rn <- 1000
DT <- data.table(
ID = seq(1, rn, by=1),
Date = sample(seq(as.Date('2018/01/01'), as.Date('2018/04/01'), by="day"), rn, replace = TRUE),
Customer = sample(LETTERS[1:5], rn, replace = TRUE, prob = runif(5)),
Location = c(sample(LETTERS[10:26], floor(runif(rn, min=0, max=101/2)), replace = TRUE), sample(LETTERS[10:26], floor(runif(rn, min=0, max=101/2)), replace = TRUE)),
Price = round(runif(rn, min=50, max=100), 2),
Cost = round(runif(rn, min=20, max=80), 2),
VIP = sample(c("VIP", "REG"), rn, replace = TRUE)
)
# ---- Creating Parent Table w/ Children ----
loc_ref <- unique(DT[,.(Location)])
children <- DT[, .(Details=.(.SD)), by = Location]
parent <- DT[
, .(
"Dates" = paste0(min(format(Date, "%D")), " - ", max(format(Date, "%D")))
, "Customers" = uniqueN(Customer)
, "Total Volume" = .N
, "Total Margin" = sum(Price - Cost, na.rm = TRUE)
, "Average Margin" = round(((mean(Price, na.rm = TRUE) - mean(Cost, na.rm = TRUE))/mean(Price, na.rm = TRUE))*100, 2)
, "Average Cost" = mean(Cost, na.rm = TRUE)
, "Average Price" = mean(Price, na.rm = TRUE)
, "VIP" = paste0(round((sum(grepl("VIP", VIP, fixed = TRUE, useBytes = TRUE)) / .N)*100, 1),"%")
)
, by = Location]
volume <- DT[, .(Orders = .N), by = .(Location, Week = floor_date(Date, "week"))]
volume <- dcast.data.table(volume, Location ~ Week, fun.aggregate = cumsum, fill = "0", value.var = "Orders")
if (ncol(volume) > 4) {
volume <- volume[, .("WoW Volume" = .(paste0(.SD))), .SDcols=c(c(ncol(volume)-3):c(ncol(volume))), by = Location]
volume$`WoW Volume` <- lapply(volume[,`WoW Volume`], as.numeric)
} else {
volume <- volume[, .("WoW Volume" = .(paste0(.SD))), .SDcols=c(2:c(ncol(volume))), by = Location]
volume$`WoW Volume` <- lapply(volume[,`WoW Volume`], as.numeric)
}
parent <- parent[volume, on = "Location"]
setcolorder(parent, c(1:4,10,5:9))
loc_ref <- loc_ref[parent, on = "Location"]
loc_ref <- loc_ref[children, on = "Location"]
loc_ref[, ' ' := ' ']
setcolorder(loc_ref, c(length(loc_ref),c(1:(length(loc_ref) - 1))))
loc_ref <- loc_ref[order(-`Total Volume`)]
})
server <- function(input, output, session) {
output$table <- DT::renderDataTable({
DT::datatable(
data = dat,
rownames = FALSE,
escape = FALSE,
extensions = c('Scroller'),
options = list(
server = TRUE,
searching = FALSE,
order = list(list(4 , 'desc')),
dom = 'tfrilp',
autoWidth = TRUE,
stripeClasses = list(),
deferRender = TRUE,
scrollX = TRUE,
lengthChange = FALSE,
scrollY = "51vh",
scroller = TRUE,
scollCollapse = TRUE,
columnDefs = list(
list(visible = FALSE, targets = -1 ),
list(orderable = FALSE, className = 'details-control', targets = 0, render = JS("function(data, type, full){ return '<i class=\"fa fa-plus\" aria-hidden=\"true\">' + data + '</i>' }") ),
list(targets = 5, render = JS("function(data, type, full){ return '<span class=spark>' + data + '</span>' }")),
list(targets = 7, render = JS("function(data, type, full){ return '<span class=sparkSamples>' + data + '</span>' }"))
)
),
callback = JS("
table.column(1).nodes().to$().css({cursor: 'pointer'})
// Generate the child table layout
var generateChildTableLayout = function(parentRowData) {
if (parentRowData != null){
var result = ('<table id=\"' + parentRowData[1] + '\"><thead><tr>')
for (var col in parentRowData[parentRowData.length - 1][0]){
result += '<th>' + col + '</th>'
}
result += '</tr></thead></table>'
return result
} else {
return ''
}
}
var converter = function(x) {
var res = [];
for (var i = 0; i < x[Object.keys(x)[0]].length; ++i) res.push({});
for (var key in x) {
var elem = x[key];
for (var i = 0; i < elem.length; ++ i) {
res[i][key] = elem[i];
}
}
return res;
}
var cols = function(x) {
var res = [];
for (var key in x) {
res.push({'data': key});
}
return res;
}
// Generate the child table content
var generateChildTableContent = function(parentRowData) {
var childRowData = converter(parentRowData[parentRowData.length - 1]);
var childColumns = cols(parentRowData[parentRowData.length - 1]);
var subtable = $('table#' + parentRowData[1]).DataTable({
'data': childRowData,
'columns': childColumns,
'autoWidth': false,
'deferRender': true,
'class': 'stripe',
'info': false,
'lengthChange': false,
'ordering': false,
'paging': false,
'scrollX': false,
'scrollY': false,
'searching': false
}).draw()
}
table.on('click', 'td.details-control', function() {
var table = $(this).closest('table')
var tr = $(this).closest('tr');
var td = $(this)
var tdi = tr.find('i.fa').first();
var row = $(table).DataTable().row(td.closest('tr'))
if (row.child.isShown()) {
row.child.hide()
tdi.removeClass('fa-minus');
tdi.addClass('fa-plus');
tdi.innerText = ' ';
} else {
var parentRowData = row.data();
row.child(generateChildTableLayout(parentRowData)).show()
generateChildTableContent(parentRowData)
tdi.removeClass('fa-plus');
tdi.addClass('fa-minus');
tdi.innerText = ' ';
}
})
") ) %>% formatCurrency(c(7, 8, 9, 10), currency = "$")
}, server = FALSE)
}
shinyApp(ui = ui, server = server) |
|
@shrektan Thank you for this explanation and example. I see what you mean in your example, but wanted to provide a dynamic example that could better represent the real-world use case. Of course, I see the merit in minimalism, too. Now concerning the response, I applied the converter, and the solution worked, but requiring client-side processing is a serious limitation. The dataset I'm working with has over 120k rows and I'm afraid it will struggle without using server processing. Since the server-side processing used to be an option before 0.4, do you believe that this may return as a possibility soon? I'm not sure if it's better to stay in 0.2 with server processing or upgrade to 0.4 with client processing. EDIT: Upon profiling, I see that the difference is too large to use client-side processing. My datatable render function increased from 1440ms to 11240ms. |
|
@amjadtalib Understood. I'll take a deeper look later to see if it's easy to make the server-side mode processing possible for your case. |
|
@amjadtalib I've filed a PR #530 which should fix this issue. However, in the server-side processing mode (after this PR gets merged), your original example code could be able to work without any modifications (you don't need that It seems like the JSON objects sent to the browser are different for the client mode and the server-side processing mode, regarding this kind of complex data (with sub data.frame in the column). It means that you need to different callback codes for different modes. I didn't look into the root cause but if there's a real-world use case for this, please let me know. Thanks. |
|
@shrektan this is awesome, thank you for taking care of this so promptly! I installed your branch and it works great. I guess it's good to know about the converter requirements if using client-side processing. It may be worth updating this page to reflect that information: https://rstudio.github.io/DT/002-rowdetails.html I don't see any need for client-side on my end. Thank you again for the great work. It looks like @yihui merged this with the master. Does this make the recent code immediately available if downloading from github? Also, is there an expected timeline for 0.5 for be available through CRAN? |
|
You're welcome. Yes, it means you are able to install the latest version of DT via About the expected timeline on CRAN, @yihui can answer this. |
|
V0.5 can be released to CRAN any time if you have difficulties with installing from Github. |
|
@yihui @shrektan i used a piece of code that davlee1972 had written in a previous post for creating child rows. this worked example was done on the mtcars file within R. I have no background in javascript and therefore reversed engineered the piece of code to try and fit it to my data. however, ive been struggling with it as the parent/child layers present themselves in chrome, however the data is absent. the piece of code im using is the following - library(data.table) ui <- fluidPage(fluidRow(DT::dataTableOutput(width = "100%", "table"))) server <- function(input, output) { output$table = DT::renderDataTable({ }) shinyApp (ui = ui, server = server) however, the output looks something like the below - there should be groups and the corresponding amount next to it in the "AccountReffulname" and "Fullamount" columns, however doesnt show up. The same piece of code, if the mtcars file is used produces the desired output as per the following post -> rstudio/shiny-examples#9 i even tried renaming the fields of my dataset to match the code devlee1972 wrote so that the child/parent relationships would be the same, however finally concluded that it is not this but something else. it would be much appreciated if you guys could shed some light as to why this may be the case. Thanks |
|
@piyuw I received your email. First of all, please make sure that you're using the dev version of DT (otherwise, install the dev version by If the dev version still fails, please take some time and provide a minimal reproducible example for us. Thanks. |
|
Hi Shrektan! thank you so much for getting back to me, you are a life saver! I've already gone through all the previous posts that discussed the issues of the current CRAN DT package and what has been update within github. therefore, i've already carried out the first step you recommended, however, unfortunately this has been unsuccessful. I've create a sample set of data and tried to best replicate the current data structure. As such the code : ` #Create data frame that acts as the foundation for the Javascript Callback function. child_2lvl = data.table(child_2lvl[,-2]) child_1lvl = merge(child_1lvl,child_2lvl,all.x = T, by = c("Summary_date", "V1","V2")) Run Applibrary(data.table) ui <- fluidPage(fluidRow(DT::dataTableOutput(width = "100%", "table"))) server <- function(input, output) { output$table = DT::renderDataTable({ },server = TRUE) shinyApp (ui = ui, server = server) ` what youll notice is the following output :
In this output, you'll notice that whilst the parent child relationship works for the "Summary Date" and "V1", and "V2" columns, it fails for the "AccountRefFullName" and "Full Amount" Columns where the expectation is that more data should exist as a parent/child relationship. my understanding is that (happy to be corrected) : that the current java script is equipped to automatically find the parent child relationship in the way the data is structured and then print it on to web page. if this is not the case, please let me know where in the java script i need to specify the different parent/child relationship - I dont have much knowledge of Java unfortunately :( The commented out code of the MTCars (default with R) file is what came with the orignal script, however, works for this but not for my data. My data has the same level of parent/child relationships as per the original MTcars data had. The only different is the data in my view (happy to be corrected) :). your help would be much appreciated! if i havent been clear enough, please let me know thanks in advance |
|
@piyuw My eyes really hurt 🤣 Use three backticks (i.e., ```) will enable you to include a multiple lines code example. Moreover, would you please You example shows that you are trying to implement a double nested datatable feature and the second datatable can't be displayed properly, right? The thing is that you only claimed to build one nested datatable in your example's JS code. I will try to give you a workable double nested example later when I have time... |
|
@piyuw Sorry, I tried but failed... I don't think I can figure it out in an hour so I choose to give up. However, if you insist, you may take the chance on StackOverflow... |
|
@shrektan... Hi, apologies for the code submission! I followed the ' ' fornat but it clearly hasn't worked 🤔😓. Yes it is a double nested table, but moreover I want to understand where on the JS do I need to change for it to automatically identify the number of nested layers? I can do the trial and error part, I just need some sample JS code that picks up how many layers of nests exists from the last column in the data table. I tried Stackoverflow but so far haven't had any luck. @yihui any chance you might be able to assist in how to create the JS with some sample data for having a double or 'n' layers of nested table? Please let me know your thoughts. Thanks in advance! |
|
@shrektan Been a while since my last post, i was wondering if you figured out how to do a double nested datatable feature or a # nested datatable feature with the code i provided you in july ? or why the prescribed code wasnt resulting in the expected result. i was wondering if you had a chance to speak to @yihui as well regarding the matter ? your thoughts would be much appreciated! cheers |
Uh oh!
There was an error while loading. Please reload this page.
Hi, I'd like to echo a bug that's been surfaced in a closed issue and is causing problems only after updating to the latest versions (0.3 & 0.4) of DT.
The recent bug has been mentioned in the posts from this year on issue 9 from the rstudio/shiny-examples repo by @davlee1972 @Plum57. One of the solutions proposed by converting nested data.tables to JSON strings is not effective for large datasets.
The code I'm using is as follows, substituted with sample data:
With DT version 0.2, the app renders fine and the rows work as needed:
After updating to version 0.4, the following error appears:
The list replacement values vary by what is actually in the tables. I've noticed that this issue is for lists as well as data.tables in a given row. This image shows that the error appears for the "WoW Volume" Column that is a list of integer values and is not nested as a child row. The reason for the list structure is to use the sparkline package.
@yihui @jcheng5 Thank you for your work in making this package possible. I hope this post contains all the relevant information required to diagnose the issue. Please let me know if there's something else I can provide.
The text was updated successfully, but these errors were encountered: