The source for the primary package is not published, afaik. The primary entrypoint function is implemented in cython, resisting casual analysis. The stated purpose is to pull (unversioned!) code from the internet and run it, but no mechanism to trust or verify that code is even mentioned.
The advisability of the idea aside, the presentation here feels exceedingly and unnecessarily risky
Friend,
(1) there is a versioning mechanism already in place
(2) Cython is used to optimize the speed
(3) FunctorFlow only works on Repl.it. We are in alpha.
It does not work on your machine. Just go and install it. See if it works. It doesn't!
I’m all for improving Python, but as feedback on the minimal website, the impression this gives is you’re working on the npmification of Python libraries. (I don’t mean that as a good thing.)
Please don't give these functions simple names like "f". I often use these names as generic function names locally. I know I can use "import ... as" but doing so shouldn't be necessary and will result in everybody using different names, which will hurt consistency, readability and searchability of code.
I was hoping this would combine the best features of existing package managers, but instead this seems to have been made in a vacuum.
While making a good package manager/community around it is hard, there's plenty of hard earned knowledge that's been completely ignored here.
This seems to make dependency management harder (imports spilled all over the code base, versioning seems non-existent, documentation for this/plugins? Twitter is not documentation).
How does plugin authoring work? How does distribution and trust work?
What problem is this supposed to solve that isn't addressed by other package managers (or what does this do better than said package managers)
Hi friend, This is a humble beginning. Now docs are in YAML-like format and soon they will be published in a readable and interactive format on our new website. For now, I just wanted to share it earlier and to involve other developers in the process.
Hi friend,
This is a humble beginning. Now docs are in YAML-like format and soon they will be published in readable/interactive format on our new website.
For now, I just wanted to share it earlier and to involve other developers in the process.
This is just an awful idea all the way through. I'm baffled by how much effort had been put into this. It's almost as if the author went off grid and made this without ever asking anyone if it was a good idea without a second thought for whether it was needed at all. It's also obvious to anyone that you cannot just magically make version numbers disappear and expect everything to go well.
Looks convenient for notebook/repl style use (assuming the the editing environment still supports autocomplete/correct and other niceties). I might just be stodgy and change averse, but I'd be a bit less willing to try things like this in production.
Landing page or FAQ could possibly benefit from an explanation along these lines.
I get that. I mean even the idea. Say in 5 years this is far more mature and stable, what use cases do you see it filling? Would they include production applications?
Yeah, this is pretty awful. I don't see myself ever using this.
1. What's the difference between this and just packaging a function as a pip package? The only one I see is just shaving off a few seconds and one less command.
2. Where is the code running from for each plugin? Eval'ing random code just seems like a bad idea. You can argue that it's the same as blindly installing a pip package, but this apparently happens each time on runtime and I see no option to check the code beforehand.
3. f and ff are pretty bad function names. I see what you're doing, but why?
4. Dep management issues aren't as common as you think they are.
5. Lots of these plugins are already in the standard library. XML to JSON is already possible quite trivially with a pip package. Not only that, but this seems like a less flexible version of that - and more obscure. (how to map xml to json?)
I'm giving this a hard pass for me, but looking forward to seeing my doubts cleared. Cheers.
I find the naming of this library confusing and almost clickbait. I don't see any kind of Functor (object?) in the API, furthermore what the heck is "flowing"? Seems like a copycat to "TensorFlow".
A better suited name which encodes the core feature, "installationless helper routines", could really help this library to become actually recognized for what it can provide.
Here's another comment about the API: I would prefer a syntax like
f.name_of_function(args)
instead of
f("name-of-function", args)
The syntactic sugar variant is easy to obtain with Python and has the advantage of much better readability.
For those developers who wish to earn an extra income in their free time while helping MOOC students complete their MOOC courses consider registering as an expert tutor here: https://moocteas-stage-web.azurewebsites.net/
Lack of versioning support makes this absolutely unusable for anything but sandboxing. And since the imports are scattered all over the code, it's hard for security scanning tools to check for vulnerabilities and to update libraries regularly. Pip isn't perfect but at least piprot helps me keep libs up to date.
"Lego blocks" "tech toys" "tinkering" "cool kids" "next big-thing" -- perhaps they are your hopes for this project, but these are all vague, filler words that don't tell me what this thing actually does. If I take a look at it what do I see?
It's not until I get to the twitter account that I see that it looks like you're trying to create not just a different python package manager, but a new ecosystem that's not backwards compatible. I don't think this is a good idea. The Python package ecosystem is gigantic and one of the largest, most mature language ecosystems out there. If you're trying to make the way Python apps are developed better, try taking a look at alternate Python package managers like `pipenv`, `poetry` and `anaconda` and figure out what you like and dislike about them. Try making changes to them before you reinvent the wheel.
As it stands right now, this isn't even a wheel. It's an uninspired marketing page on twitter that is not doing a great job of convincing anyone it's better than the alternatives. In fact, it's not even clear that you're even aware of the alternatives.
That diagrams is 90% about different versions of PYTHON and not python modules. So I have no idea what it has to do with the discussion at hand.
Even if it was about modules, if anything FunctorFlow makes it worse because you now have X+1 things creating modules (FunctorFlow has to live somewhere after all so it doesn't remove the previous system).
Not an issue I've dealt with during the last 10 years in production. On OSX, homebrew python + standard virtual environments + pip mitigates that completely. On the Ubuntu side, system python + standard virtual environments + pip work fine as well. Anything else?
Well, this has all the signs of well-meaning but inexperienced developers seeing a market opportunity without appreciating the many difficult problems that will have to be solved to produce something that is secure, reliable and interoperable with current development ecosystems.
Rather than abuse them, we might perhaps use them as an example of the Dunning-Kruger effect.
So, aside from other comments, doesn't this break the already mediocre tab completion side of python? Scrolling through API docs to figure out what exactly that method or argument was called gets really tedious after the 5000th time.
Absolutely terrible. I can see why data science types who use / produce a lot of (typically throwaway) Python code might be enamored with this, but as far as the Python language goes, functorflow will be another nail in its coffin if it ever gains mass-acceptance. It will turn the already messy and inconsistent ecosystem into shambles that no sane person will want to build anything robust on top of.
With the mass migration of Python developers into Go, this is the last thing that Python needs.
Hmmm, that feels more like a jupyter extension than a standalone module. The extension could intercept imports automatically without the need for a prefix. Might require a custom kernel that does some magical per notebook lib mapping. Maybe it installs libs in version specific central folders and then does some symlinks to build a notebook specific lib folder based on the date specified. Feels more seamless but also more complicated.
That'd make sense. You'd need to pin all transitive dependencies as well and cache the venv. Not sure if you'd need to find older versions of new direct dependencies to avoid conflicts. For example, you run the sheet in 2015 and then again in 2019 but with a new import. That new import's latest version has a transitive dependency that you already pinned but has different version requirements than what you pinned. I wonder if you can embed the requirements.txt inside the sheet itself.
Only issue is that it'd be really space inefficient, my DS venvs clock in at 400+mb each so having one per sheet will probably quickly become unusable. Which is why I thought of some sort of smart system wide caching akin to maven/ivy. But I'd forgotten how complicated python dependencies (binaries, c code, etc.) were and how little api support pip had.
> You'd need to pin all transitive dependencies as well and cache the venv.
I think to do this stuff, you need hooks in Jupyter to setup and teardown the venv before it runs the kernel. (And generally Jupyter would want to clean up unused venvs to mitigate the teardown hook not firing.)
> Only issue is that it'd be really space inefficient
A venv is overkill. You can just run the kernel in a regular directory, and `pip install --target kernel_dir foo-bar-pkg` to put packages directly in it. As long as the linker sees it, third-party libraries will work. This technique is used in the serverless project[1] to bundle dependencies for use on AWS Lambda.
> Not sure if you'd need to find older versions of new direct dependencies to avoid conflicts.
Curation is a solution to this. Stackage[1] is popular in the Haskell community; they build a consistent version set of everything every night and curate stable releases periodically.
With curation, a date and a set of top-level packages is enough to pin your dependencies.
Their comment seemed respectful to me. It was pointing out that some users of the platform have priorities other than maintainability and reproducibility.
What? Python is arguably the go-to language in data (and other) science. With a massive ecosystem of mature, powerful libraries like numpy and scipy, you immediately and freely have access to a range of features, managed almost trivially with pip/virtualenv/conda. I don't think any other language has a larger number of performant ML libraries like tensorflow and pytorch, and more importantly, if you want source from the latest ML papers on arxiv, it will almost always be implemented in python.
Then the language itself offers trivial iteration, possibly in real time with tools like jupyter, and requires substantially less bootstrapping knowledge than something like C/C++, which creates a low barrier of entry for non-programmers, who are less likely to shoot themselves in their feet with memory management and such. And if you're concerned about performance, most of the standard data science packages are just wrapped C/C++ anyway.
Sure, it's not perfect, but it's practically the lingua franca of data science right now, and it fits the role quite well.
The advisability of the idea aside, the presentation here feels exceedingly and unnecessarily risky