Hacker News new | ask | show | jobs
by chrisseaton 2307 days ago
> aren't even the ASCII quote character; you get horrid unicode quotes

Why should they be ASCII, and what’s wrong with Unicode?

3 comments

Because the next step when you face this problem is doing a search-replace of the quotes to remove them. ASCII quotes are on your keyboard, so you can actually type the command to remove them.

Unicode quotes probably aren't, so it's extra annoying to remove them, and they're not even the same character for start and end so you have to do it twice.

The time spent for removal is then vastly higher than with ASCII quotes, which in the context of unwanted characters may well qualify as "horrid" imo.

Yes, this is pretty much what I meant. The "horrid" part is that I would like to do a quick

    /"<CR>xnx
(or similar) in vim to delete the quotes on a multi-line block. Even that is annoying. But with the unicode quotes I have to do a bit more conscious searching, which is a distraction.

Of course, the fact that the text is modified in any way is user-hostile. I don't mind re-typing a code example (since I'm trying to learn it), but I like to copy-paste sometimes (e.g. data literals).

I paid for a DRM-free ebook that's meant to be copy-pasted. We have a universal paradigm for how plaintext copy works. Apple charges a premium for good design. Ebook readers are generally supposed to get out of your way and let you focus on content. There is no reason for Apple Books to have this behavior.

PowerShell acknowledged smart quotes and dashes in its language design and allows proper quotes interchangeably with ASCII straight quotes. I always found that an interesting design choice, although I'm not sure how useful it actually is to allow people to copy/paste code from dubious web pages. On the other hand, they'll do that anyway, so why bother making it harder?
This is going down a rabbit hole. “This” is an English quote, whereas „this“ is a German quote; note that the open quote character in the English quote is equal to the close quote character in the German quote.

Then there is a less-often used style of quoting, similar to the French style, but: «this» is the French quote, whereas »this« is the German version. Yes, the open and close quotes are swapped.

I applaud the idea, but just want to point out that there are dragons lurking in the shadows.

All of “”" parse the same as ", so bracketing doesn't really enter into it. This is just to prevent smart quotes from destroying code, not to have more levels of nesting.
Additionally note that «this» is mostly used in French online and in Switzerland, while « this » is more common in professionally typeset works in France. Those are U+202F NARROW NO-BREAK SPACEs between the guillemets.
Don't forget that the French quotes should have (non-breaking) spaces inside « like this » - only when used with the French language though, other languages usually don't use such spaces.
I've never liked this sort of thing in code, because as things stand they're a bit of a faff to type, and some tools don't support them very well. But it's a shame, because there's loads of types of quote mark and bracket in Unicode, not just the ``...''-type pair commonly used in English. So you could use one type for string delimiters and then use the other types, unescaped, in the string itself.

(The Unicode quotes also mostly come in matching pairs, so, for good or for ill, they'd in principle be nestable.)

isn't that an input problem not an output problem? Is it giving you the quotes directly from the ebook or is it converting them on copy-paste? I'd assume there was a bug on the input side, that some tool wrongly converted `printf("Hello World")` to `printf(“Hello World”)` so what's in the ebook is wrong already. If so the issue is further up the chain.
In this context, the selection within the ebook contains no quotes, whether ASCII or Unicode. The problem is that Apple’s Books app adds quotes and attribution text around the selected text. Books knows that the text it is putting in the clipboard isn’t what you selected, and it doesn’t care.
It's not changing quotes in the code. It is adding quotes. You have to delete them, not search-and-replace them, and it doesn't matter what kind of quotes they are, you have to do the exact same thing.
Search and replace is a pretty reasonable way to delete a quote; lot's of editor UI's make that much more convenient than finding the end of some blob of text you just pasted into some pre-existing context.
Depending on the regex engine, it may not be that much higher.

sed -i '/<open quote>|<close quote>/d'

I think they are talking a about styled quotes, the kinds that point inward, which a REPL won't interpret as an ASCII quote and will throw some type of Syntax error.
Ok but those are standard, valid characters like any others. Why are they ‘horrid’?
Some people hate unicode quotes because lots of software will automatically convert ASCII quotes to them when you copy/paste your code to Slack or email to share with others.
Because they result in a syntax error.
Even though the OP seems offended by the presence of the Unicode quote marks for some reason, they have nothing to do with the issue of Apple Books surrounding the copied text with those quote marks and adding an attribution line. If they'd surrounded the text with ASCII quote marks and added an attribution line, it would have still been a syntax error, wouldn't it.

Personally, I don't see any reason to hate typographically correct quote marks when used correctly, which (obviously) doesn't include code samples.

But they wouldn’t do what you wanted anyway if they were ASCII. What’s the problem specifically with them being Unicode?
The problem specifically is that they are not the ASCII quote character. There is only one ASCII quote character, and that's the one used by programming languages. Any other quote or quote-like character is outside the ASCII range, and must therefore be Unicode (or another non-ASCII code page).
I know they're not compatible with programming languages that you may want to use. But that's the languages' problem. They're perfectly valid, standardised, characters.
They're horrid because you can't easily type them on a keyboard.
Because for some reason terminals are stuck in the 70ies and don't accept those characters as quotes. Anything but ASCII trips them up.

Seems such an obvious interface to innovate, but it seems to run into terminal wizards sense of purity.

The problem, such as it is, is with languages. Terminals (mine at least) handle most of Unicode just fine; admittedly I've seen it choke on emoji, but punctuation, nah.

The vast majority of programming languages are defined in terms of ASCII and only ASCII. I don't care for this, personally.

I've given some thought to how to do quoting right in a programming language, and implemented «guillemets» as an experiment. But it's challenging, you need to decide what to do with all of “”‟„"″ and there aren't obvious pairings, like „this is a sentence” and “this is a sentence” and »this is a sentence» and «this is a sentence» and »this is a sentence«, it ends up feeling like rather a lot of effort for what you get in return.

Oh, one of those characters I typed isn't a quotation mark, did you catch which one? Hacker news won't even let me type two of them!

Having multiple almost identical ways of achieving things is a bug magnet in programming languages. Differing programmers will presumably use different styles (why else even support differences?), and if you mix code like that, bugs ensue. This a hassle not just because autoformatters are liable to make churny changes (which distract from real changes, which makes history harder to understand: bug magnet), but also because people will make mistakes e.g. when find-replacing (another minor bug magnet!). Then there's the fact that some of those quotes aren't symmetric - so you need to think of something to have happen when they're unpaired or used incorrectly, and it wouldn't surprise me if no matter what you did, you surprise somebody (bug magnet!).

Sure: these are all quibbles, and a language wouldn't die from all these minor cuts. But they're definitely downsides, not upsides. So: where is that upside? Why would you ever support something like this? "It looks a little nicer" sounds like a pretty weak argument compared to "it's inconsistent, hard to machine process, and may cause a few bugs"...

Many programming languages, certainly not all, offer single and double-quoted strings already. Which is, granted, a perennial source of annoyance for those of us trying to have a consistent formatting style.

I want «guillemet strings» because they use a matched pair, so you can «quote a string «within a string» without escaping» and I think that's a nice property.

I'm not really interested in supporting all the forms in which they're used in European languages, though, that would be a real hassle. Just the one that looks like the other matched pairs we use in programming.

“smart quotes” have ‟at least two styles” and really „three styles”, and the first two are really easy to confuse with "normal double-quotes". I don't want my users to have to deal with "why doesn't this compile”?, and if I allowed it to compile, now you have to escape all the quote characters inside any string, which is messy.

Raku, as a sibling points out, has bitten down on this bullet, and I respect that. I keep meaning to give it a spin, I'm fond of Parsing Expression Grammars and have good feelings about Perl from the early days.

> I've given some thought to how to do quoting right in a programming language

Already fully designed and implemented in Raku: https://docs.raku.org/language/unicode_entry#Smart_quotes

Test online: https://tio.run/##K0gtyjH7//9Rw7ySjMxiBSBKVChOzStJzUtOfdQw9/...

A language or interface is defined by the set of symbols mutually agreed upon. If you allow Unicode that number simply explodes, thus bloating and complicating the language/interface and its implementation as well. In effect it is no longer the same language but becomes a different, more complex one.

The tradeoff is not worth it for programming languages the same way as learning all the scripts of the world is not worth it for one human being, just to accomplish tasks that don't need all these symbols in the first place.

It is a solved problem however. We chose not tot use it (in terminals).
The terminal can usually handle it, provided it's UTF-8. Most of the editors can handle it. It's very specifically the programming language specifications which don't include them. Even the ones that allow Unicode identifiers.

Also, they're difficult to type - my keyboard has a " key but not a “ or ” character. I had to copy and paste those from the unicode website since I don't have a suitable input method set up.

The idea that "terminal wizards" reject this is rather undermined by the fact that "terminal wizards" made open and close double quotation marks accessible as simply [Group2]+[B04] and [Group2]+[B05] on an ISO 9995-3 conformant keyboard.

Or [Shift]+[Option] [B04] and [Shift]+[Option] [B05] if there's no explicit [Group2] shift.

Open and close single quotation marks use the same keys.

"terminal wizards" have done quite the opposite of rejecting this.

That seems like an obviously bad solution. That is like writing [a(b{c]d}e) except that the quote characters look even more similar.
Its easy (solved) to map Unicode to control characters such as quotes (and when you think about it, quotes are often there to deal with ambiguities stemming from the limited number of allowed characters in ASCII). So to could have a terminal accepting such input, and a few helper function that normalize it into ASCII and so on.

After all, users of non-ascii languages (which is nearly everyone) already know how to deal with it without ambiguity. Its only ambiguous if you don't use encodings, and that should never happen anyway.

> and a few helper function that normalize it into ASCII and so on

If you are going to get rid of it anyways, why bother with it in the first place? Also, you then have the ambiguity/problem whether some symbols are mapped to the same ones or not.

> already know how to deal with it without ambiguity

I wish. Just dealing with ü vs ue vs \"u and ascii, win something, utf8 and utf16 is still annoying in practice. Or latex choking on accents, non-breaking whitespace or similar fun stuff.

Often in code environments, it's hard to tell the difference between " and “.