The exact answers to leetcode questions are probably included many times in the Copilot dataset, in many programming languages. Fresh Advent of code problems are much more interesting.
Copilot is trained on examples of programs, so it will not be able to produce
programs unlike any it has seen before.
This will not be easy to see in any kind of coding exercise. It is very hard to
come up with coding problems with solutions that have never been written before,
at least for a coding competition like Advent of Code. For example, we don't
know of polynomial time solutions to the Travelling Salesman problem but it
wouldn't make sense to have that as a coding challenge in Advent of Code, or
Leetcode, or anywhere else.
Since nobody has ever produced a program solving such a problem, Copilot will
not be able to produce a solution either, but it's not designed to do that
anyway.
This will not be easy to see in any kind of coding exercise. It is very hard to come up with coding problems with solutions that have never been written before, at least for a coding competition like Advent of Code. For example, we don't know of polynomial time solutions to the Travelling Salesman problem but it wouldn't make sense to have that as a coding challenge in Advent of Code, or Leetcode, or anywhere else.
Since nobody has ever produced a program solving such a problem, Copilot will not be able to produce a solution either, but it's not designed to do that anyway.