| Thank you for tackling this critical problem for logic designiners. I think the tools available are much too old for fast paced workflows. From my experience attempting to get a similar workflow down for my company: I tried to use verilator a while back but ultimately I couldn't because it didn't have same constraints in the verilog language features that I was going to use in production. It doesn't even matter who was missing a feature, verilator or the proprietary tool, it was just about getting them to be same that caused the cognitive dissonance that I didn't want to deal with. I ultimately decided to move away from verilator and use the clunky proprietary tools since it was what would be used in production. Getting "verilator compatibility" seemed like a "nice to have". Second, the a winning local-first framework of verilator wasn't really established. You show in your example running a test from the yaml file using what looks like a bash script. Even as an experienced programmer who knows bash and sh well, I still find it very hard to write complex thoughts in it. The last high level attempt I found to bridge this gap is likely https://www.myhdl.org/ I don't know them personally, but it seemed like they had some very good thoughts on what makes writing good hardware level tests good. I think it would be worth reaching out to them if you haven't already. The one thing that even more critical was a way to run our tests locally. The 10-20 seconds it takes to start a docker image (best case) in the cloud is really frustrating when you are "so close to finding a bug" and you "just want to see if this one line change is going to fix it". Once we got our whole pipeline going, it would take 1-6 minutes to "start a run" since it often had to rebuild previous steps that cache large parts of the design. So I think you will want to see how you can help bring people's "local's first" workflows slowly into the cloud. Some tools (or just tutorials) that help you take a failing test, and run it locally and on the cloud will be really good especially as you get people to transition! |
Bash is just an example; most people should make the test rule call the simulator executable directly or via a thin wrapper script. MyHDL is interesting too. Admittedly this is the first we've heard of it but we'll take a look.
We are working on supporting additional simulators in addition to Verilator.
Also, we are working on an API and command line tool so you can kick off test runs and view the results from the command line. This CLI tool should also support local test runs at some point.