Rethinking `eltype` conversions in Adaptors

[avik-pal]

Currently, we forward calls to cu, mtl, etc., from the device adaptors. This is well-motivated because users often don't want to do FP64 ops on GPUs.

But there is a clear design problem. Let's say we want to convert to FP64 GPUArray, the current way is Array FP64 --> CuArray FP32 --> CuArray FP64.

We can solve this by associating an eltype with the adaptor:

1. Default for CPUAdaptor: nothing, i.e., preserve eltype while converting
2. For all other GPU Backends: FP32, i.e. convert to FP32.

[agdestein]

It would be useful to have the option to preserve the type. Currently, this does not seem to be possible with gpu_device:

julia> x = randn(5)
5-element Vector{Float64}:
 -0.5743739682003972
  0.6633998724244564
  1.1789368640479356
  0.8608185434207882
  0.12467014684547488

julia> x |> gpu_device()
5-element CuArray{Float32, 1, CUDA.DeviceMemory}:
 -0.57437396
  0.6633999
  1.1789368
  0.86081856
  0.12467015

which converts to Float32. I currently do something like

julia> using Adapt

julia> ArrayType = CuArray  # for cpu: Array
CuArray

julia> adapt(ArrayType, x)
5-element CuArray{Float64, 1, CUDA.DeviceMemory}:
 -0.5743739682003972
  0.6633998724244564
  1.1789368640479356
  0.8608185434207882
  0.12467014684547488

which requires the user to manually choose the correct array type.