[j2kun]

If you want me (as an academic) to use ScholarlyMarkdown, then you can't force my collaborators to use ScholarlyMarkdown as a consequence.

There should be a nice ScholarlyMarkdown -> LaTeX cross compiler for starting simple documents in TeX and then sharing with collaborators (pick some obvious defaults or allow a config file to get fancy). But more importantly, if I am joining a project that already has a bunch of LaTeX wizardry going on, I should be able to seamlessly and implicitly edit the text parts in ScholarlyMarkdown without my collaborators knowing.

Can ScholarlyMarkdown do this? If not then I'm not really interested.

[timtylin]

I've actually been wondering if this is possible from a theoretical standpoint. I'm thinking you can use Pandoc's LaTeX to MD conversion mode, save changes to a copy, wdiff to get the total change set, then somehow convert that into a LaTeX patch. Whether you can guarantee that this won't break anything is going to be a really challenging problem, as my head is already swimming with edge-cases. I guess we'll know the day that some genius comes up with automatically generated non format-breaking Critic Markup diffs. At least the problem should be easier than LaTeX --- look how long it took for the latexdiff tool to be what it is today, and it still breaks often when you have anything slightly exotic.

I'm currently using a workflow in my thesis where I use ScholarlyMarkdown to write individual chapters for final inclusion into an existing LaTeX book document. I find that ScholarlyMarkdown works quite well this way, and it potentially allows collaborators, since individual parts are isolated.

[j2kun]

In any case, my point is plain and simple: I collaborate in almost everything I do, and my collaborators are my primary concern when I choose a tool. If they use notepad and Dropbox, I need to make sure my tools don't conflict with that. There's no way in hell I would ask a collaborator to learn to use git or a new markup language just to work with me.

I'd love a tool that works with this philosophy, and I feel certain anything like ScholarlyMarkdown won't catch on in my field (theoretical computer science) without such tools.

[timtylin]

As a thought exercise, assuming for simplicity that you can isolate the parts you're authoring, so that your changes only involve additions to other people's work: what kind of additional metainfo would you need for this workflow? All defined labels and macros? Available bib entries? What else?

[j2kun]

I think it depends on how I'm viewing the file. If I'm just editing in SM and building the tex as usual, then I don't need anything extra. If I'm trying to convert to all sorts of other documents (which I haven't ever really done) then labels, theorem environments, one-line macros, and bib entries cover almost every tex-specific thing I use.

Here is an excerpt from a typical paper's macro section [1]. As you can see they're mostly one-liners to remove the need to keep typing textup and mathbb, simple mathoperator definitions and such.

[1]: http://pastebin.com/RL1gejEZ

[timtylin]

I think a good way to build this into SM is to run an existing document thorough latex and looking at the aux file. This approach would be faster and almost certainly more robust, similar to how most latex build scripts look at the fls recorder file for the list of external includes.

The only only thing you can't get will be the user-defined macros (and of course bib entries that doesn't already exist), but there is already a consistent mechanism to define your own macros in SM via the "math_def" block. They "do the right thing" in the sense that if you render to latex snippets then it wouldn't redundantly include these macros in the output.

[arnsholt]

That's probably never going to happen (for any Markdown-based project).

As you probably know, TeX is a full programming language (albeit an odd one), which means that to compile (La)TeX to Markdown, Markdown needs to support all the features of TeX, which means that it has to be Turing-complete. Not to mention the fact that parsing TeX is kind of nuts too.

[chriswarbo]

There's no need to convert all TeX back to markdown; the converter only needs to recognise the sub-set of TeX that it produces.

If anyone edits the TeX to fall outside that sub-set, it's reasonable to just embed those parts verbatim in the resulting Markdown.

[j2kun]

Exactly. And I have a very strong feeling that without the ability to collaborate implicitly (at least on the text/math mode part), no replacement for LaTeX will catch on in my community (mathematics).

[zzleeper]

My guess is that he is building the PDFs through latex/xetex.

I built something similar to that, but instead of forking Pandoc (and investing the time to learn Haskell), I just added filters through Python. It's not at fast, but you can intercept mostly anything.

Thus, the flow ends up as:

paper.md -> Pandoc -> paper.json + metadata -> paper.tex -> paper.pdf

[j2kun]

I want the flow to be

paper.tex -> paper.md + metadata ->(merge) paper.tex -> paper.pdf