[benbridle]

I wholeheartedly agree with using "0o" as the octal prefix, I've never been a fan of "0". I've jut implemented this feature and released it in v2.2.0, you can grab it from the project page. Thanks for the suggestion!

I'm not too sure I understand what you're describing with non-Unicode text. Torque doesn't have a built-in concept of bytes, instead each character is treated as an integer with the value being the Unicode code point of that character (using decimal we have 65 for 'A', 955 for 'λ', 129302 for 'robot emoji', etc). It's up to the programmer to choose how to pack the character (integer) into a sequence of bits. Code points are different to encodings like UTF-8 or UTF-16, which define how a code point (integer) is packed down into one or more bytes.

If you want to assemble 7-bit ASCII text, one byte per character, you define a macro that packs each character value into the lower 7 bits of an 8-bit byte. If the string contains a non-ASCII character, the character value will be too large to fit into the field and an error will be displayed.

  %ASCII:c  #0ccc_cccc ;
  ASCII:"This is a string." 

Assembling ISO-8859-1 text would be similar, but would involve remapping the characters above 0x7F like this:

  %BYTE:n  #nnnn_nnnn ;
  %ISO-8859-1:c
    ?[c 0x7F <=]  BYTE:c 
    ?[c '¡'  ==]  BYTE:0xA1 
    ?[c '¢'  ==]  BYTE:0xA2 
    ?[c '£'  ==]  BYTE:0xA3 ;
  ISO-8859-1:"£190.00"

UTF-8 being a variable-width encoding requires a more complicated macro arrangement (you can see an example here [0]). But the key point is that strings aren't treated as byte sequences, they're just character/integer sequences until they get baked down into a byte encoding.

Please let me know if that doesn't answer your suggestion, I'm keen to understand your use-case.

[0] https://benbridle.com/projects/torque/manual-v2.2.0.html#usi...

[zzo38computer]

I meant that the input should not have to be Unicode. The input could be any extended ASCII encoding (supporting ASCII is necessary, in order that the syntax will work; therefore some encodings such as Shift-JIS and UTF-16 are not suitable) which does not have to be mapped to Unicode. (Even if the input actually is UTF-8, you may have a character that is made up of multiple code points. If it is treated as a sequence of bytes then it can be any sequence of bytes, including a combination of multiple UTF-8 code points.)

It is another issue than mapping the character codes for output, which as you say can be unwieldly for large character sets, so another kind of table specification (which might also be useful for stuff other than text), would probably help and be more efficient than using a sequence of conditions in a macro.

About octal, someone said against using "0o"; it is usually not as difficult to distinguish if the "o" is always lowercase. Another alternative would be how it works in PostScript, which uses "8#" prefix for octal and "16#" prefix for hexadecimal. (My opinion is that "0o" is good enough though)

[benbridle]

Oh I think I see what you mean now, you're talking about the encoding of the Torque source code that gets fed into the assembler. To be honest I'd never really considered anything other than UTF-8, the parsing is all implemented in Rust which requires strings to be valid UTF-8 anyway. Are you wanting to write Torque code with a non-Unicode text editor, or are you thinking about how the document encoding affects the handling of string literals inside the assembler?

Some kind of table syntax would be useful for character mappings, but I'm not sure what it'd look like or if it'd be applicable outside of dealing with characters. I'll think more on that.

[vanderZwan]

> I wholeheartedly agree with using "0o" as the octal prefix

Can I give a vote against this? Distinguishing "o" and "0" can be a huge pain when using the wrong font.

Also, why not 0c? 0x already uses the "x" in "hexadecimal", so why not the "c" in "octal". That it also reads a bit like a 13375p34k abbreviation of "octal" is a nice bonus.