[chse_cake]

Not to gripe on your work, But I am extremely skeptical about tools like these. How do you ensure the equivalence of the computation ? What are the SLA's on equivalence and how do you even verify if the programs are equivalent?

The solution to bad code written by data scientists is not more AI tools that write non-verifiable code. Its ergonomic API frameworks which can relieve the pain-points / bad practices via throughly testable / verifiable / deterministic code.

The site even promises test-case generation in the future. This seems very flaky at best.

[oshams]

I'm glad you asked. You are right to suggest that proving the equivalence of two programs is in general impossible (equivalent to the halting problem). However, if you start with one program with a known special structure and apply a known manipulation, it becomes possible in special cases to guarantee equivalence with some reasonable assumptions about how the original program is used (e.g. to give a trivial example, if one of your tests is to hash the source code of the new program against a specific value it will fail that test after refactoring -- macros are not extremely far from this contrived example).

There is no replacing the developer (anytime soon) that's why we don't overwrite your code. You still have to review the suggested changes yourself. I personally don't want to spend more time making changes an AI can guess for me to improve my code and even if I have to review every change that still saves a lot of time.

Also I agree, test cases are also hard because while it's easy to generate some tests that verify some trivial behavior, most of the meaningful behavior of programs requires knowing more than just the types of the inputs & outputs. But I believe AI can help with even this problem.