Hacker News new | ask | show | jobs
by bartvds 4456 days ago
I am a DefinitelyTyped (DT) org member, not just for the definitions but I also do infrastructure work, and as side-project I maintain TSD which is a dedicated package manager for DT.

(thanks @WalkerCodeRanger from reddit to link me here: it seems I'm the only DT org member who is active on reddit and now hackernews.. shit is about to get real :)

I'll do some representin', but keep in mind this is just my personal view as active member and not some sort of 'official' statement (let's be clear there).

--

DT and this type-definition effort is actually a pretty complex with some interesting aspects:

Org:

DefinitelyTyped has mostly grown organically from just one guy starting to share his definitions (borisyankov), then some others joined in (diullei) and then it just grew and grew and we now have > 380 definitions.

The org members are just enthusiast pitching in some time, with like 10 active members handling the PR's and a huge flock of random people sending in files and fixes. It is a mildly chaotic, best effort kind of thing. We have no hard ties to the TypeScript team, although we harass them mercilessly in the typescript-compiler repos with bug reports and feature requests. They are fully aware of DT, they link us and AFAIK they run our test suite with their new compiler versions.

Contribs:

Some definitions are really well maintained and much used; especially jquery and the angularjs defs are very popular, together with mocha/chai/sinon, underscore/lodash, jasmine etc. Others are less active: some have never been touched after a 16 year old enthusiast pushed his proud work for some niche module to us. This is no different then any package manager.

It is all welcome too, because work done is work done: next time somebody might pick up on it and improve, send it back, rinse, repeat.

Keep in mind it can be quite a time-intensive process to write the initial defintion file (and its test): it is easily a few hours burned, likely more for the bigger libraries. Not many people create new ones as it can be a slog sometimes (I actually like to do it: you learn buckets about the library you're modelling, so ask me anything about bluebird, lazy.js or highland API :)

There are some tools available out-there to automate extractiob: some export from JSDoc, C#, or other crazy solutions that cost as much work to use as they save, but even then it is a lot of hassle to do right.

In an ideal world the definitions would be maintained by the library authors, but alas many do not care for TypeScript (or Dart or whatever). Although some authors actively send in updates, mostly it are enthusiasts contributing what they can. Some are JavaScript/node open source veterans (on vim/sublime /webstorm/eclipse), while many other come from the MicroSoft Visual Studio / C# crowd. For many this is their first time using git and github or their first PR ever.

This then means that quality is very varied: we do have some quality rules and a contribution guide and try to be strict but is only as good as people can be pushed without bailing out (again, similar to any package repo). We have a testing procedure (running on Travis) and are actively working on improvements, but again, it is all loosely organised spare-time work with imperfect data.

Consumers:

There are a few main consumers of the repository:

- our NuGet exporter is a PowerShell script that bundles each directory and pushes that as NuGet packages. There are some improvements planned but we need some other things first (who has time? :) - TSD is a dedicated client (command-line and module). This is my main side project, previously by another org member. I'm actively developing this to cover many of the issues we see (it supports stuff like retrieving definitions for semver versions). This v0.5.x was a full re-imagining of a simpler version by @diullei, and v0.6.x is in the works (cleaner data model, betetr code style, more best-practices, more externalised functionality to re-use) - JetBrains IntelliJ (WebStorm etc) also reads from the Github repos: you can use it to augment auto-suggestion (even for plain JS files!) - Some other experimental clients (TPM) - And many people just use a git clone or manually grab definitions.

Version matching:

I see a lot of comments about how to get the correct definition for a particular library version. The repository (and our rules) already facilitate this: you can postfix a semver to the filename. For example: https://github.com/borisyankov/DefinitelyTyped/tree/master/n...

- TSD parses this and can filter it with semver ranges.

- The NuGet exporter currently does not do anything for this; it just bundles everything from each directory and pushes it to NuGet. We have a notion to improve the NuGet exporter to smarter package and annotate versions and other meta data, but we need a few other things before that (some of which I have in TSD but am now extracting to external modules).

But even then, in practice, nobody is creating or maintaining these older versions!

This is odd: for many critics it is a big deal, but for active users comes up only occasionally (it is after all a valid concern). It is mentioned in the contribution guide but almost nobody sends them in. If you'd need one you can look in the commit history and try to find one, then add it with the semver postfix. But then you need to backport any relevant fixes, and you never really know how perfect a definition is at any point in its history. Usually there are imperfections: that is just the nature of how it works with manual descriptions and of course the TypeScript type system can not describe everything.

Future:

We're brooding on a meta-data feature (a small yaml) that describes some properties for each definition, like if it can be found on npm, bower, if it has external module declaration etc. I need this for TSD, and NuGet could use it too. The requirements are a bit vague around the edges and we're weary to commit to something unmaintainable.

Bower / component.io:

It comes up time-and-time again that we should export these to bower. I think this is not a good idea at all:

The problem with bower is that every package needs it's own repository. This would mean that at the current count we'd maintain ~400 repositories, with likely more to come. This is not a lot of fun to maintain. Besides that github will be happy there is the real problem of how we'd ever going to test interdependent type definitions (yes, this is done a lot).

Then there is the thing with the git tags: bower identifies packages by a git tag with a semver. If you give a definition repos tags with semvers of the library it is modelling then it will be very unpleasant to back-port API fixes (as you'd have to update all the tags), or abandon the old ones which kinda defeats the purpose.

We can automate some of this of course, but that brings complexity, risk etc. Some users have started a test on this (dt-bower), but AFAIK it didn't work out (as I'd expect). If you look in the DT's issues and the codeplex forum you'll find plenty of talk about it (lots by me too).

Conclusion:

So yea, it is all a bit messy and organic, but it is very difficult to formalise. As long as libraries authors don't export type information by themselves it will have to be retrofitted by the community, which means human labour which means things get a little messy and less precise. But as we pool effort we all build on each other.

Is this bad?

That depends on what you expect out of it. It seems some of you expect C# level of perfect typing. TypeScript is not C#, TypeScript is power-boosted JavaScript.

You start with what you like about dynamic JavaScript, then add typing and compiler features where possible.

Fact is that even with all the imperfect type definitions this is way more exact then whatever you can do in Vanilla JS and your human meatbrain: instead of trying to remember every detail of the API from the documentation, we now describe it in definition files and have the compiler track it for us. Then we share this information, so other devs can benefit and need to spend less time in API docs themselves. And just like memories these get outdated, then you get back in the doc and update your knowledge (except with DT you can now share your enlightenment).

And don't forget Typescript works fine with definitions: you can do all the dynamic coding you want (implicit, or explicit with the 'any' data-type). Only add typing if you want a little more safety:

I personally use many modules from npm without typing them (I still do lots of regular JavaScript too btw). Before all this I did a lot of ActionScript (1, 2 and 3) and that also hits this sweetspot of dynamic and static in one language (but without the glory of npm).

And to be honest this is a bit of a #firstworldproblem:

IMHO one of the most amazing things in tech is this ongoing explosion of the JavaScript ecosystem: it was fun before, but node + npm made it go apeshit critical. I was all over that before TypeScript came along, and here we have this technology that is a clean, tight super-set that smoothly improves the original with that little bit of extra sanity and syntax power. It is not just a thing on its own but the way it meshes with the whole ecosystem that makes it pop. Dart has nothing on this. I will say nothing about CoffeeScript although it has it's place.

My penny (it got out of hand.. deal with it :)

1 comments

Thanks for the good info about the DT project. I want to say that even though I'm not personally a fan of TypeScript I really appreciate the work of you and everyone else who has contributed to the DT project. I know from my previous work on OSS how much work that can be.

My only experience using DT was through Nuget so I didn't realize you had the semver postfix. I suspect that like me most .NET devs will go to Nuget first, so I personally would place a high priority on getting semver in Nuget, but I understand that there are lots of other important things too.

My suggestion would to make including a semver part of your quality requirements so that at least you have a record of what version they were working against when they made the type definitions.

I hadn't realized the interdependent library testing/validation. You really are in a pickle about git branching/tagging and not being able to have separate repos. Problem is, the side by side definitions for different versions makes it difficult to merge fixes across versions (as compared to say a different branch for each version of the library). I think this is something that needs to be thought about long and hard. Could you use git submodules or subtrees to do what you need? Perhaps have each library in its own repo but then have a repo that pulls them together and has the interdependecy testing? That way most contributors could work in the repo for the project they care about and only a few contributors would worry about the combined one?

Thanks for sharing your experience with TypeScript and what is good about it. It's just not possible for me to address everything bad & good about a language in one blog post, so I have to focus on what's relevant to the point I am trying to make (in this case why I don't personally think TypeScript is the answer to JavaScript).

I absolutely agree that meshing with the JS ecosystem is of critical importance and Dart just doesn't have that the way TypeScript or CoffeeScript does.

I'm on the Dart team and I can honestly say that we're working very hard on this.

We simply have a harder problem to solve than TypeScript (different features and semantics, working with the VM as well as compiling to JavaScript), so it's taking longer, but we're aiming for nothing short of awesome and seamless interop.

We already have dart:js, a low-level JS-interop library that works in dart2js and with the Dart VM. Admittedly it's a bit cumbersome for developers to use, but it provides the building blocks that we can use to make TypeScript-like Dart interfaces for JavaScript libraries, and to export Dart libraries to JavaScript.

It's great to hear about a project like DefinitelyTyped, because once we have the support in place, we're going to need something very much like it in Dart.