Ask HN: Would you be interested in an embeddable, lightweight subset of Python?

[parnor]

There are several implementations of the Python programming language. These include the standard CPython, the fast PyPy, and the microcontroller-focused MicroPython. However, none of these implementations are particularly good for embedding into applications - instead, this space is dominated by Lua.

I have an upcoming project which would benefit from an embedded scripting language. I would personally rather use Python than Lua, and people writing scripts are more likely to be familiar with Python.

I also have an interest in language implementations (interpreters, garbage collectors etc.). I am therefore considering both scratching my own itch and indulging my interest - by writing a new, embeddable implementation of Python.

Like Lua, this implementation would be lightweight - so it could only support a subset of the Python language. I believe a Ruby implementation with similar goals already exists: https://github.com/mruby/mruby

However, I am aware that a new language implementation represents a large amount of work. If I am going to spend a significant amount of time on something like this, I would like the result to be useful to others as well as to myself.

Would you consider using a Python implementation designed along the lines of Lua? If so, are there any particular features which you would like it to have?

[etendue]

I doubt I would, even though I use Python for other purposes.

Lua isn't the only one in this space: I personally use Tcl (or Jim Tcl) for this sort of work, and it (IMHO) excels in this area. What advantages would you see your Python implementation having over Tcl or Lua which were designed with this area in mind?

[parnor]

What advantages would you see your Python implementation having over Tcl or Lua which were designed with this area in mind?

Python is a much more popular language than Tcl or Lua. Embedding Python opens your application to scripting by many more people, because the average person wanting to write a script is more likely to be familiar with it. Even those familiar with multiple languages may prefer to use Python - Tcl and Lua are often considered to be slightly "quirky" languages.

Also, although an embedded Python interpreter is never going to run C libraries like NumPy, there are many pure-Python libraries out there which would potentially be available for scripts to use.

[marktangotango]

> Embedding Python opens your application to scripting by many more people

I don't think this is realistic, anyone who wants to script an application will use what ever is exposed by the developers no matter how bad it is (ie visual basic for applications, and redis with lua).

[robbyt]

Yes, I'm not really a fan of Lua or Forth.

The main feature I would like to see is a commitment for long term support and maintenance. Boring feature, yes- but a technology like this will only gain traction when people know it will still be around in a year or 10...

What does lightweight actually mean?

[parnor]

The main feature I would like to see is a commitment for long term support and maintenance.

How would such a commitment be demonstrated? Which existing projects have done a good job convincing you of their longevity?

What does lightweight actually mean?

I would like to make it possible to embed Python within your application without significantly increasing its size. For example, a standalone implementation of Lua is around 500KB, and of mruby is around 700KB. I would consider both of these to be "lightweight" - they could be embedded just about anywhere.

CPython 3.5, however, is closer to 4MB in size - large enough that embedding it would significantly increase the size of smaller applications. PyPy is much larger still, at around 40MB.

On the other hand, I intend for my implementation to target desktop and mobile applications, so am not too concerned about RAM usage at runtime. This contrasts with MicroPython, which targets microcontrollers with limited memory and is therefore designed "to minimise RAM usage" [1].

[1] http://dpgeorge.net/talks/pycon-au-2016-main.pdf

[marktangotango]

Consider the case of an 'interpreter per thread' in an application server. In lua this is implemented by creating a lua context per thread, which is your memory foot print. The size here limits the number of concurrent requests the server can handle.

So, in addition to my other comment, I think there's a major barrier to wide adoption to any of the 'mini' scripting language implementations, and that is a lot of people who use python would expect your implementation to act like the mainline. Here I'm alluding to the trouble pypy has gaining acceptance, because so many users expect all the c libraries to just work.

I also contemplated going this route, not to get the language I want, but to get the runtime instrumentation I want. The conclusion I came to is that it would almost be better to create a new language than to always be explaining "no, that functionality doesn't work because of these design decisions". The language may in fact be a lot like python, or a subset python as you say, but just don't call it that!

Edit; created a dupe somehow...

[parnor]

An 'interpreter per thread' [...] your memory foot print [...] limits the number of concurrent requests the server can handle

So, in this situation, it would be useful for the interpreter to be lightweight in terms of RAM usage as well as code size. Presumably, though, it is still not necessary to be as frugal as MicroPython, which can get by with as little as 8KB of RAM and is commonly used with 128KB [1]?

A lot of people who use python would expect your implementation to act like the mainline. [...] it would almost be better to create a new language than to always be explaining "no, that functionality doesn't work" [...] just don't call it [Python]

In order to work within tens of kilobytes of RAM, MicroPython does some things differently from CPython [2]. I wonder if its developers spend much time explaining the differences between the two implementations? Or do users expect differences in the language runtimes because of the obvious differences in the hardware on which they run?

On the other hand, I imagine MicroPython would have significantly fewer users if it were advertised as "a Python-like language" rather than as "an implementation of Python".

[1] https://github.com/micropython/micropython/wiki/FAQ

[2] https://github.com/micropython/micropython/wiki/Differences

[marktangotango]

Gave this some thought, pretty sure your conjecture is correct about number of users, and you don't want to waste the effort (it will be large). My only suggestion is to weigh the fact that cpython is embeddable, if a bit large, and consider how that would cannabalize uptake of your effort. Whatever you decide, good luck, and have fun with it!