Example runs slowly

[Beliavsky]

Downloading the package and running the example script (saved as xloess.jl) in the Readme.md on Windows 10 with Julia 1.6.1, using julia xloess.jl takes 36 s of wall time. This is a long time for a small data set of 100 points. A similar code in R or Fortran runs instantly.

[pnavaro]

Yes, it is true, it is due to the compilation time. With Julia you need to adapt your workflow. You can't use it like R or Fortran.

julia> @time include("xloess.jl")
 58.854341 seconds (100.47 M allocations: 6.107 GiB, 4.95% gc time, 0.01% compilation time)
false

julia> @time include("xloess.jl")
  0.729143 seconds (1.10 M allocations: 57.652 MiB, 3.88% gc time, 59.25% compilation time)
false

The second time you include the program, it runs instantly.

[lmiq]

Probably worth mentioning that what takes time there is the loading of the Gadfly plotting library. Commenting the plotting part we get:

julia> @time include("./test.jl")
2.449248 seconds (6.63 M allocations: 379.517 MiB, 3.65% gc time, 0.21% compilation time)

and for the second run:

julia> @time include("./runtests.jl")
0.003383 seconds (14.14 k allocations: 3.003 MiB)

[Beliavsky]

Thanks @lmiq. Is there a Julia plotting library that loads faster than Gadfly?

[lmiq]

Gadfly seems particularly slow. Using Plots with the standard backend (GR), takes 10s here compared to 24s of Gadfly, and using GR directly takes ~5s.

time julia -e "using Gadfly; p=plot(rand(10)); draw(SVG("test.svg", 6inch, 3inch), p)"
real 0m24,984s

time julia -e "using Plots; plot(rand(10)); savefig("test.svg")"
real 0m10,216s

time julia -e "using GR; plot(rand(10)); savefig("test.svg")"
real 0m5,529s

But in any case, if the goal is to use Julia as a script (and speed is necessary), probably something like https://github.com/dmolina/DaemonMode.jl is interesting.

[ayushpatnaikgit]

Hi,
I am switching my code from R to Julia (primarily for speed). I have translated most of my code and in some places, I've used RCall. My code uses RCall for Loess, but I am hoping to switch to native Julia using Loess.jl. When I try the example (without Gadly), it seems RCalling loess is much faster than Loess.jl and doesn't use even a fraction of the memory. Here is my simple code for benchmarking:

function LoessCall() # Function to call the example 
    xs = 10 .* rand(100)
    ys = sin.(xs) .+ 0.5 * rand(100)
    model = loess(xs, ys)
    return predict(model, xs)
end

LoessCall() # Calling it to compiled it. 


using RCall

R"""
RLoess <- function(x, y) 
{
    m.loess      <- loess(y ~ x)
    return <- predict(m.loess, x)
}
"""

function RLoessCall() # Function to call the example in R
    xs = 10 .* rand(100)
    ys = sin.(xs) .+ 0.5 * rand(100)
    return rcall(:RLoess, xs, ys)
end

RLoessCall() # Calling it to compiled it. 

@time for i in 1:10000 
    RLoessCall()
 end

9.363060 seconds (310.00 k allocations: 49.744 MiB, 0.08% gc time)

@time for i in 1:10000 
    LoessCall()
end

 12.357978 seconds (101.69 M allocations: 27.990 GiB, 19.11% gc time)

[pnavaro]

Dear Ayush

The loess function in R is written in C language and compiled so it is not surprising . Performance is nice but you can choose Julia for other reason like multiple dispatch and the syntax. There is a small mistake in your R code

RLoess <- function(x, y) 
{
    m.loess      <- loess(y ~ x)
    return(predict(m.loess, x))
}

[ayushpatnaikgit]

Hi @pnavaro

I forgot to mention that I did run LoessCall() to compile it. I will edit my comment and mention it.

Thanks for pointing out the correction in the R code. It's a different style, but both commands work (at least in R 3.6 upwards).

[pnavaro]

The main error was here loess(x ~ y) . I think you mean loess(y ~ x)

[ayushpatnaikgit]

Oh.. yes, you are right. I will edit the comment. It doesn't change the time though.

[nalimilan]

AFAICT none of the performance issues mentioned here are related to Loess.jl. They are rather due to the plotting. So I'm closing this, feel free to reopen if you see slowdowns without calling using Plots.

[nalimilan]

Ah sorry I missed #47 (comment). Reopening then, though it would be clearer to file a separate issue as it's completely distinct from the original description.

[ayushpatnaikgit]

@carljv you've mentioned a trick to speed up loess: http://slendermeans.org/lowess-speed.html

I think the same is applicable here as well. What's your opinion?