Hacker News new | ask | show | jobs
by jboy 3303 days ago
Yes, my startup Object AI uses Nim code in production. We have in-house implementations of machine learning, computer vision & image processing code in Nim, using a library called "Nim-Pymod" to integrate with Python's Numpy: https://github.com/jboy/nim-pymod

(As you can see, I was one of the authors of that library in a previous startup. We haven't worked on Nim-Pymod in a while, alas -- I've been focused on the new startup! -- but Nim-Pymod is sufficient for our needs right now.)

Our webserver main-loops are in Python; our number-crunching ML/CV/img-proc code is Python extension modules written in Nim.

As a C++ & Python programmer, I'm a huge fan of Nim, which to me combines the best of both languages (such as Python's clear, concise syntax & built-in collection types, with C++'s powerful generics & zero-cost abstractions), with some treats from other languages mixed in (such as Lisp-like macros and some Ruby-like syntax). I find Nim much more readable than C or C++, especially for Numpy integration. I also find Nim much more efficient to code in than C or C++ (in terms of programmer time).

And Nim is a very extensible language, which enables Nim-Pymod to be more than just a wrapper. For example:

1. Nim-Pymod uses Nim macros (which are like optionally-typed Lisp macros rather than text-munging C preprocessor macros) to auto-generate the C boilerplate functions around our Nim code to create Python extension modules.

2. Nim-Pymod provides statically-typed C++-like iterators to access the Numpy arrays; these iterators include automatic inline checks to catch the usual subtle array-access errors. Nim macros are themselves Nim code, which can be controlled via globals, which in turn can be set by compiler directives; by compiling the Nim code in "production" mode rather than "debug" mode after testing, we can switch off the slowest of these checks to get back to direct-access speed without needing to make any code changes. (And of course Nim's static typing catches type errors at compilation time regardless of the compilation mode.)

3. Nim exceptions have an informative stack trace like Python exceptions do, and Nim-Pymod converts Nim exceptions into Python exceptions at the interface, preserving the stack trace, meaning you have a Python stack trace all the way back to the exact location in your Nim code.

Earlier on in our development of Nim-Pymod, there were some occasional headaches with Nim due to its in-development status. Occasionally the Nim syntax would change slightly and that would break our code (boo). We've also debugged a few problems in the Nim standard library. I suppose these problems are an unfortunate consequence of Nim having a small set of core devs contributing their time (rather than being supported by Microsoft, Sun, Google or Mozilla). Fortunately, these problems seem to have stabilised by now.

The Nim standard library is reasonably large, somewhere between C++ STL (data structures & algos) & Python stdlib (task-specific functionality). I recall that the stdlib could use some standardisation for uniformity, but I haven't been watching it closely for the last year or so.

Third party libraries are not abundant, aside from a handful of prolific Nim community-members who have produced dozens of fantastic libraries (eg, https://github.com/def- , https://github.com/dom96 , https://github.com/fowlmouth , https://github.com/yglukhov ).

I'm happy to answer any other questions about using Nim in production!

1 comments

Being a hosted language (compiles to C/C++/Javascript) which are the biggest pains?

Is there any way to ensure type checking for the exposed interfaces in runtime? Let's say, exposing a function to the host language (C or Javascript) which accepts a String and then passing an integer in the host language.

Did you ever use the javascript backend? How is the experience having a shared code base among very different platforms?

How does interfacing to existing libraries look like? Any examples out there?

Does it really compile to C/C++ or just binary? If it compiles to code, then is it possible to write libraries for mobile platforms as well?

I've never used the JS backend or C++ backend, so I can only talk about the C backend.

With the C backend, the Nim code is transpiled to C code (surprisingly readable C code, as far as auto-generated C goes), which is then compiled to one of: a binary executable, a shared C library, or a static C library: https://nim-lang.org/docs/nimc.html#compiler-usage

In fact, Nim-Pymod relies upon the Nim->C->library compilation, because it uses the Python C API to produce Python extension modules: https://docs.python.org/2/extending/extending.html , https://docs.python.org/2/c-api/

If you want to see a shared C library produced by Nim, have a play with some of the examples provided with Nim-Pymod: https://github.com/jboy/nim-pymod/tree/master/examples It auto-generates the appropriate Python C API boilerplate functions around your Nim functions, then compiles the whole thing as a shared C library, exposing the auto-generated functions in the shared library.

Regarding runtime type-checking: Again, I can only talk about the C backend. The type checking provided by shared or static C libraries will be whatever the C compiler enforces... I haven't tested whether (or how much) this can be violated.

Nim-Pymod does auto-generate the Python->C type-checking code for you, so you can't (for example) pass in a Python string when an int is expected by your Nim function.