| Wow, that G-GLN paper is really new. Thanks for sharing it :) My first impression is that this will be very challenging to use for many of the people currently using AI in practice, because of the requirement to have a convex and gaussian-distributed result. For example, I currently work on optical flow and the loss functions are usually very jumpy and usually only convex within a few pixels around the correct result. I have seen plenty of modeling errors in optical flow SOTA papers, for example casting a boolean occlusion term so that it can be added to the loss (Won't work, no gradient). I have also seen how strongly authors struggle with the irregularities of their loss function and issues with convergence, for example by fixing all random initialization to a seed and then parameter-scanning on that value (Very expensive). Notable examples of the difficulties faced would be https://arxiv.org/abs/1904.04998 which I have never seen converge from random initialization or https://arxiv.org/abs/1711.07837 which diverges without supervised pre-training. Also, while many people use the gaussian-distributed euclidean norm of the difference between prediction and groundtruth as their main loss term, there has been a lot of discussions recently if that is a good idea, because it forces the neural network to represent uncertainty with blur. But optical flow tends to have sharp edges where objects end. Combined, that means the problems that I work on usually do not have a gaussian-distributed loss and usually are irregular and never convex, so I'm not sure if I could use G-GLN. But the application to contextual bandits looks VERY interesting to me :) I see great potential in using conv layers as pre-compression and then applying decision trees on the resulting intermediate representation. What I previously did for object segmentation was to sample a random but fixed set of feature pair differences and use the signs as bit flags. I then trained the decision trees on those bits to predict the boundary shape around that pixel. I got the general idea from this paper: https://arxiv.org/abs/1406.5549 But it sounds like moving from difference bits to halfspaces and from linear regression to GLN, your paper "Online Learning in Contextual Bandits using Gated Linear Networks" could greatly improve on that. I'd be curious to see how those bandits do on BSDS500. BTW, are you aware of any discussion groups for AI image processing that are open to members of the public? https://www.reddit.com/r/deeplearning/ seems to be mostly people that just took a Udemy / Coursera course, so it's usually about re-using existing models and almost no talk about research. |
r/machinelearning is the most suitable subreddit for this, since it is actively frequented by researchers and the quality of discussion is much higher than in r/deeplearning