This is a significant amount of effort, and has some very good touches. Especially the Loom-Viewer, I really liked the ease with which one's workflow can integrate into the data analysis.
Thanks! :) It's always good to hear that we're on the right track with usability.
It's one of the things that I'm most worried about: I've been working pretty isolated for the last year and a half, and lack a background in biology or bioinformatics, and did not even have webdev experience when I took on this project (plenty of embarrassing proof of that in the code). Kudos to Sten Linnarsson, the PI of the group and my boss, for taking a gamble and hiring me anyway.
> I really liked the ease with which one's workflow can integrate into the data analysis.
Just to make this clear: the file format is a "dumb" data storage, and the viewer a "dumb" plotter of that data. To do more in-depth analysis requires loading the file in python, R, or anything else that might support the files in the future. The idea is to then store the results of this as attributes in the file. For example, the tSNE plot here[0] is just pre-calculated x/y data stored as two attributes.
Currently there is an issue with fully integrating the viewer into such a work-flow: for performance reasons, it caches accessed data from the file. This cache needs to be refreshed manually.
Sten recently added library support for keeping track of file modifications[1]. That enables me to make the viewer automatically update stale cache whenever a file is modified, making it even easier to integrate. Currently working on that.
There's still a ton of polishing and bug-fixing to do. Feedback, suggestions and help are always welcome!
It's one of the things that I'm most worried about: I've been working pretty isolated for the last year and a half, and lack a background in biology or bioinformatics, and did not even have webdev experience when I took on this project (plenty of embarrassing proof of that in the code). Kudos to Sten Linnarsson, the PI of the group and my boss, for taking a gamble and hiring me anyway.
> I really liked the ease with which one's workflow can integrate into the data analysis.
Just to make this clear: the file format is a "dumb" data storage, and the viewer a "dumb" plotter of that data. To do more in-depth analysis requires loading the file in python, R, or anything else that might support the files in the future. The idea is to then store the results of this as attributes in the file. For example, the tSNE plot here[0] is just pre-calculated x/y data stored as two attributes.
Currently there is an issue with fully integrating the viewer into such a work-flow: for performance reasons, it caches accessed data from the file. This cache needs to be refreshed manually.
Sten recently added library support for keeping track of file modifications[1]. That enables me to make the viewer automatically update stale cache whenever a file is modified, making it even easier to integrate. Currently working on that.
There's still a ton of polishing and bug-fixing to do. Feedback, suggestions and help are always welcome!
[0] http://loom.linnarssonlab.org/dataset/cells/Dentate%20gyrus/...
[1] https://github.com/linnarsson-lab/loompy/issues/26