[terracatta]

Using it in practice, the sheer quantity of suggestions (often one for every line) is fatiguing especially when 99% of the time they seem fine.

I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion.

This risk to me is inherently different than the risk accepted that engineers will use bad code from Stack Overflow. Even Stack Overflow has social signals (upvotes, comments) that allow even an inexperienced engineer to quickly estimate quality. The amount of code used by engineers from Stack Overflow or blogs etc, is much smaller.

Github Copilot is constantly recommending things and does not gives you any social signals lower experienced engineers can use to discern quality or correctness. Even worse, these are suggestions that are written by an AI that does not have any self-preserving motivations.

[lolinder]

Copilot's default behavior is stupid. You can turn off auto-suggest so that it only recommends something when you prompt it to, and that should really be the default behavior. This would encourage more thoughtful use of the tool, and solve the fatigue problem completely.

In IntelliJ, disabling auto complete just requires clicking on the Copilot icon in the bottom and disabling it. Alt+\ will then trigger a prompt. I know there's a way to do this in VSCode as well, but I don't know how.

[joenot443]

> I know there's a way to do this in VSCode as well, but I don't know how.

I dug into this a bit, since I want the same functionality, I found I needed an extension called settings-cycler (https://marketplace.visualstudio.com/items?itemName=hoovercj...) which lets one flip the 'github.copilot.inlineSuggest.enable' setting on and off with a keybind.

Not sure who's in charge of the Copilot extension for VS Code, but if you're out there reading this, the people definitely want this :) Otherwise of course, your tool rocks!

[nprateem]

I switched it off and never remember to bother using it. It's obvious why it's enabled by default.

[bGl2YW5j]

"...does not gives you any social signals lower experienced engineers can use to discern quality or correctness" is very astute.

I experienced this in practice. I was pairing with an inexperienced engineer who was using Copilot. He was blindly accepting every Copilot suggestion that came up.

When I expressed doubt in the generated code (incorrect logic + unnecessarily complex syntax), he didn't believe me and instead trusted that the AI was right.

[throwaway675309]

I would argue that this kind of problem is going to become less of an issue overtime, since they're going to have to also solve the issue of suggesting code samples from deprecated API versions - it's likely that eventually they'll figure out a similar way to promote more secure types of code in the suggestions based on Stack overflow or other types of ranking systems.

[visarga]

Yes, the will surely improve a lot and also train users to write better prompts and comments. With millions of users accepting suggestions, then fixing them, they get tons of free labeling. If they monitor the execution errors they got another useful signal. If they use an execution environment they could use reinforcement learning, like AlphaGo, to generate more training data.

[visarga]

As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times. It also makes developers happier, reduces the need to context switch, increases speed and reduces frustration.

> Github Copilot is constantly recommending things

It's only a momentary problem, will be fixed or worked around. And is this a bad thing to get as many suggestions as you could? I think it's ok as long as you can control its verbiage.

> does not gives you any social signals

I don't see any reason it could not report on the number of stars and votes the code has received. It's a problem of similarity search between the generated code and the training set, thus finding attribution and having the ability to check votes and even the license. All doable.

> an AI that does not have any self-preserving motivations

Why touch on that, people have bodies and AIs like Copilot have only training sets. We can explore and do new things, AIs have to watch and learn but never make a move of their own.

[thesuperbigfrog]

>> As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times.

That's what libraries are for.

Copilot is just copy / paste of the code it was trained on.

When the code it was trained on is later discovered to have CVEs, will it automatically patch the pasted code?

With a library, you can update to the patched version. Copilot has no such feature.

[lolinder]

> Copilot is just copy / paste of the code it was trained on.

Every time I hear someone say this, I hear "I've never really tried Copilot, but I have an opinion because I saw something on Twitter."

Given the function name for a test and 1-2 examples of tests you've written, Copilot will write the complete test for you, including building complex data structures for the expected value. It correctly uses complex internal APIs that aren't even hosted on GitHub, much less publicly.

Given nothing but an `@Test` annotation, it will actually generate complete tests that cover cases you haven't yet covered.

There are all kinds of possible attacks on Copilot. If you had said it can copy/paste its training data I wouldn't have argued, but "it just copy/pastes the code it was trained on" is demonstrably false, and anyone who's really tried it will tell you the same thing.

EDIT: There's also this fun Copilot use I stumbled across, which I dare you to find in the training data:

    /**
    Given this text:
 
    Call me Ishmael. Some years ago - never mind how long precisely - having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world.

    Fill in a JSON structure with my name, how much money I had, and where I'm going:
    */

    {
        "name": "Ishmael",
        "money": 0,
        "destination": "the watery part of the world"
    }

[visarga]

It can even read an invoice, you can ask it "what is the due date?" It's a system that solves due date and Ishmael questions out of the box. And everything in-between.

[thesuperbigfrog]

>> It can even read an invoice, you can ask it "what is the due date?" It's a system that solves due date and Ishmael questions out of the box. And everything in-between.

That's cool.

But emitting copyrighted code without attribution and in violation of the code's license is still copyright infringement.

If I created a robot assistant that cleans your house, does the shopping, and occasionally stole things from the store, it would still be breaking the law.

[visarga]

> occasionally stole things from the store

It's fascinating to see how stretchy the word "steals" is nowadays. You can make anything be theft - copying open online content and sharing? theft, learning from data and generating - also theft. Stealing from a physical store - you guessed it.

[throwaway675309]

While I do enjoy everybody acting as armchair lawyers.... until we get an actual legal ruling, the general consensus seems to be that it is sufficiently transformative as to be considered fair use.

[thesuperbigfrog]

>> If you had said it can copy/paste its training data I wouldn't have argued, but "it just copy/pastes the code it was trained on" is demonstrably false, and anyone who's really tried it will tell you the same thing.

So if "it could commit copyright infringement, but does not always do so" is good enough for your company's legal review team, then go for it.

[visarga]

Has anyone tried to see how similar is their manually written code to other codes out there? I bet small snippets 1-2 lines long are easy to find. It would be funny to realise that we're more "regurgitative" than Copilot by mere happenstance.

[thesuperbigfrog]

Will the court believe that Copilot created an exact copy of Tim Davis's code "by mere happenstance"?

https://twitter.com/DocSparse/status/1581461734665367554

[visarga]

> I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion.

AI can also do code review and documentation helping us reduce the number of bugs. Overall it might actually help.

[tevon]

This is a very solid argument. How do we fix that?

THIS is the article I want to read!

[redleggedfrog]

"I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion."

I'll go one further with the "Co-pilot is stupid."

It's supposed to be artificial intelligence. Why in the eff is it suggesting code with a bug or security issue? Isn't the whole point that it can use that fancy AI to analyze the code and check for those kind of things on top of suggesting code?

Half-baked.

[visarga]

Ah yes, humans are perfect, never make any mistakes. That's why only AI write bugs.