Tokenizer in QB64

[mnrvovrfc]

For me is main problem null terminated string or chr$(0) or this ""
And such a thing is solved or work in other Basic dialects
so it should be fixed in QB64 to
because produce ILLEGAL FUNCTION CALL runtime error

The ASC("") problem, sadly was created by Microsoft with QuickBASIC and QBasic. QB64 aims always to be compatible with those two products and therefore this won't be changed. QB64 doesn't care about "other" BASIC dialects.

The workaround was presented which was to create a wrapper function that catches the empty string and then the programmer is given the decision to do something about it. It's not any different from dividing by zero with integers, having to create a function for it only to catch zero as divisor.