Hacker News new | ask | show | jobs
by chasd00 26 days ago
> ..you’ve run into a problem that you’ve spent days trying to get Opus to solve

do you have an example of this? If i can't get an agent to do something in a couple hours i do it myself.

7 comments

I had a 300k LOC game that needed refactored and segmented into DAG assemblies for a composable engine to reuse in a few other similar games our studio is prepping to build.

I tried holding Opus's hand over a week trying to get it done with a ton of in-depth planning and manual course-correction, but it never got it to a state where it was "done" (kept struggling to differentiate between game content specific to that game versus game systems we'd want to reuse, and how to cleanly separate systems vs content when extracting).

Fable needed a little bit of hand holding but got it done in less than a day.

Many, but I think that the model delta is only meaningfully convincing when experienced firsthand.

Here’s an example of improvement when trying to solve the label placement problem (NP-hard):

https://imgur.com/a/kCUZxPi

It’s also an example of something that I could not (and would not bother trying to) code up a solution / heuristic for.

Not the OP, but I had Fable orchestrate this project.

https://github.com/ByteTerrace/Puck

It has required constant hand holding, and there was the outage to deal with, but I can't argue with the end results. A fully deterministic recursive engine within an engine framework that includes a rendering VM, emulators, custom ROMs, and an in-game editor? Insane. Sure, it's nowhere near primetime but this kind of thing was unimaginable just a year ago.

Debugging. Memory leak hunting, figuring out latency issues

For things where I'm not familiar with the code base, it can take days or weeks to become familiar with structure and flows.

If I can get an agent to help visualize and analyze the architecture and zero in on a subset of code, that can be a huge win.

For instance, we had some archaic "cache" that just dumped things into a static/module-level map. I tried a few different things to try to find it over the course of a few days and eventually gave Claude a Python REPL into a running process with pyrasite after some memory leaked and it traced through heap allocations and references to find the referent. It would have taken me probably 2-4+ weeks of just learning about Python heap to figure that out of my own.

Here's one:

I was working on an SDF-based CAD tool. One of the things I want to be able to do is select a pair of surfaces (which are identified by a "surface id" propagated up the expression tree) and add a blend between those two surfaces (e.g. a fillet or chamfer).

Here is a video demo of how far I got by doing it myself and using o3 (I think?) to help: https://www.youtube.com/watch?v=LOvqdlDbkBs

The video is a bit confusing because there was some screen-recording lag so it sometimes looks like I clicked on something other than what I clicked on.

You can see that the strategy I have implemented there works most of the time but at the end it fails to apply the blend.

That strategy is to rewrite the expression tree using distributivity so that blend arguments are siblings, and then apply the blend at the union/intersection (min/max) that is their parent.

But this fails when you need conflicting pairs of blends.

The problem is: given an expression tree describing an SDF, (but where the value passed up the tree is a tuple `(distance, surface_id)` rather than just distance), and given a set of fillets of the form `(surface_id_1, surface_id_2, radius)`, produce a new expression tree which fillets all of the places where those surfaces join.

In ambiguous situations, for example the 2 surfaces come together at an edge, and then that edge runs into a 3rd surface, I don't mind how you resolve the region near the 3rd surface as long as it is intuitive and predictable for the end user.

I spent quite some days working with various agents to come up with a solution to this and still haven't managed to find one.

Maybe you could do it in a couple of hours yourself?

Reverse engineering/decompilation of game binary (Tears of the Kingdom) using Ghidra for modding spanning Java (ghidra), C++ (mods), Python (scripting) and PowerShell (scripting for builds/deploys/etc.).

Fable succeeded in cases where Opus 4.8 consistently marked situations as walled/impossible.

Build a distributed system using raft groups. You will see Opus fail.