[wallnuss]

Take a look at CUDAnative.jl (https://github.com/JuliaGPU/CUDAnative.jl) which uses the NVPTX LLVM backend to compile Julia code to the GPU. What you described sounds very similar to that and could definitely made to work.