[_9jgl]

This looks incredibly cool; I'm wowed by the fact that the model has learnt to negate words in if-else statements, though I struggle to think of a case where that particular completion would have been useful.

At the same time, I'm less excited about the fact that the model is cloud-only, both for security/privacy reasons and because I spend a not-insignificant amount of my time on limited-bandwidth/high-latency internet connections.

I'm also curious as to why the survey didn't ask about GPU specifications; most of the time I use my laptop to code whilst plugged in, and I'd happily use only LSP completions when on battery, so power consumption wouldn't be an issue (though fan noise might), and allegedly my GPU (a GTX 1050) can pull off almost 2 TFLOPs, which is well over the "10 billion floating point operations" mentioned in the post.

[zawerf]

> I'm wowed by the fact that the model has learnt to negate words in if-else statements

I know it learned natural languages from using GPT-2, but I am surprised it didn't get "confused" since words are used in such a different way in programming.

For example strong appears as the html tag <strong> with no corresponding <weak> tag. And weak appears in weak_ptr in C++ and there's no such thing as a strong_ptr.

[Felz]

Vanilla GPT2 actually does a half decent job of emulating the style of programming.

Try entering "public static int main() {" into https://talktotransformer.com/

[0b01]

I think the tokens are simply one hot encoded. Though some sort of word2vec embedding is also situationally plausible

[jacob-jackson]

Good point, I added it to the survey.

[frou_dh]

Since it's asking for FLOPS, are you ready for people typing in 14-digit numbers? ;)

[jacob-jackson]

I'm looking forward to some data cleaning :)

[Mil0dV]

So for reference (and checking) I entered 1862000000000 for my 1050 ti.