[eihli]

Hey! Thanks for all of the work you do. I love reading your stuff.

Lately, I've been playing a lot with the code from Neural Guided Constraint Logic Programming for Program Synthesis [1]. I want to apply it to searching for neural network architectures. Something like:

    (define-relation (layero layer in out) ;; [2]
      (conde
      ((== layer `(Linear ,in ,out)))
      ((== layer `(Relu ,in ,out))
        (== in out))
      ((fresh (percent)
          (== layer `(Dropout ,in ,out ,percent))
          (== in out)
          (countero percent)
          (<o '(1) percent)
          (<o percent '(0 0 1 0 0 1 1))))))

And a `layerso` that is just a sequence of `layers`. And then use some metric that captures how effective the training run was and use that metric for teaching the search network.

I'm sure there's better ways to do it than the way I'm going about it. But it's a fun way to learn a bunch of cool new stuff.

Thanks for getting me started down this path!

[1] https://arxiv.org/abs/1809.02840

[2] https://github.com/eihli/reason/blob/523920773b7040325c8098a...