Tokenizer in QB64

[aurel]

Seems to me the sharing question is first thing to resolve in cleaning up this code.

 what you call sharing ..i called GLOBAL
because whole tokenizer is part of Interpreter  arrays must be GLOBAL
In main part of program tokenizer function is called
when source string is loaded as file or in this TESTING case just as string.
Every char in source must be processed not just string of interest.