display unicode text(by windows api)

[TDarcos]

I have a suggestion I think would work and if I'm wrong someone will point it out.

The problem is the QB64 editor is essentially a DOS program, it is effectively only 7 bit compliant as far as putting text in your program, unless that text exactly matches the character set in the font QB64 has loaded, so if you only display the characters in your existing font, it will work. This allows things like the line drawing characters to be displayed. If your program source is opened by someone with a different code page then the text will get trashed. 

I have a program I use called VBINDIFF which will display a file in hex and show the character in the current code page.  The important point is at about &h0222, which is line 11 of his program listing of 12/6:


As you can see at &h258 to &h25b, the characters are in fact ? symbols (&h3f 4 times), not Chinese text;

Next, using Notepad++ (you'll see why below), I created a 1 line program:



(The Euro symbol is copied from Wikipedia's article about it.) It is found at Unicode U+20AC. Obviously, that's not in the first 256 characters. Here is what the file looks like in hex from VBINDIFF:


The last 3 characters of the top line, E2 82 AC is the UTF-8 encoding of the Euro symbol, &h20ac. To understand this, let's break them into binary bits:
 1110 0010 1000 0010 1010 1100  The bits of the first half of the first byte must be 11 the 10 means it is a 3-character encoding; the next four are the high four bits of the result. The first two bits of both characters must be 10 to indicate it's a continuation character so they are ignored, we only use the last 6 bits of each, giving:
 xxxx 0010 xx00 0010 xx101100, or 0010 0000 1010 1100, or 20AC. And that's how they encode them.

Now what will QB64 do with this? 


And what results from this?


As they say, "garbage in, garbage out." I was wrong, UTF-8 will be retained if present in a QB64 quoted string. You just can't insert it from the QB64 editor. You have to use a text editor that supports UTF-8 encoded text. Notepad and Notepad++ support UTF-8, I'm sure there are others.

Recommendation:
If you're wanting to display something using non-Roman characters (Arabic. Chinese, Hebrew. Japanese, Russian, etc.) find a text editor that will work with creating/editing files using that language, that outputs the text as UTF-8. Insert that into a string in the QB64 Basic program, then when using the MultiByteToWideChar routine, invoke it with the string holding those characters, but set Code Page argument to MultiByteToWideChar. I believe that will work, unfortunately I can't test it to find out

- Paul "TDarcos" Robinson