REDIM, TYPE, and STRING Woes

[TerryRitchie]

Ok, been pulling my hair out for last 4 hours until I realized what was going on.

I have a TYPE with 3 integers and a string:

TYPE MYTYPE
    INT1 AS INTEGER
    INT2 AS INTEGER
    INT3 AS INTEGER
    INFO AS STRING
END TYPE

Then used it as so:

REDIM MYVARIABLE(1) AS MYTYPE

So far so good. As I start putting data into the array I need to resize it from time to time:

REDIM _PRESERVE MYVARIABLE(UBOUND(MYVARIABLE) + 1) AS MYTYPE

Still good right? Nope. The previous data gets completely garbled. The problem you ask?

INFO AS STRING

Change it to:

INFO AS STRING * 20

and all is good in QB64 land.

It seems variable length strings are a no-no when using REDIM _PRESERVE

If this is already a known issue then somehow I have missed it all these years. If it isn't then this is a warning letting others know this happens.

I was ready to throw my joystick through my screen until it finally dawned on me as to what may have been happening. The integers I was storing ranged from 1 to 6 but somehow they were changing to 16, 57, 3471, etc.. I thought maybe I was going nuts for a while, LOL.

Ugh.

[mnrvovrfc]

It's interesting to notice that QBasic flagged an error when an attempt was made to create an UDT field which was variable-length string. The compiler needed to know what was the size of the UDT. This is everywhere, otherwise must use an interpreter able to trick itself and to "be taught". Need to use _MEM in this case and do the string management yourself.

[bplus]

I've run into trouble using UDT's when I assume after a ReDim _Preserve that the new values are 0 or "" for numbers or strings.

Supposedly the numbers and fixed strings were fixed according to Luke if I recall but variable length strings had to initilized to "" otherwise they pick up garbage from the string pointers to uncleared meomory, something like that.

@TerryRichie could it be possible you were assuming new variable length strings were "" after a ReDim _Preserve ?

Perhaps also, whatever was fixed back when Luke checked it became unfixed with all the updates made since, like with assumed 0 of new integers created with ReDim _preserve?

I will write some test code...

[TerryRitchie]

It's interesting to notice that QBasic flagged an error when an attempt was made to create an UDT field which was variable-length string. The compiler needed to know what was the size of the UDT. This is everywhere, otherwise must use an interpreter able to trick itself and to "be taught". Need to use _MEM in this case and do the string management yourself.

Yes, having fixed-length strings makes sense once I figured out what was going on. I never ran into this during the QuickBasic days so I had no idea fixed-length only allowed in UDTs when using REDIM. The QB64 compiler should warn of this like the QuickBasic compiler did as you point out.

[TerryRitchie]

I've run into trouble using UDT's when I assume after a ReDim _Preserve that the new values are 0 or "" for numbers or strings.

Supposedly the numbers and fixed strings were fixed according to Luke if I recall but variable length strings had to initilized to "" otherwise they pick up garbage from the string pointers to uncleared meomory, something like that.

@TerryRichie could it be possible you were assuming new variable length strings were "" after a ReDim _Preserve ?

Perhaps also, whatever was fixed back when Luke checked it became unfixed with all the updates made since, like with assumed 0 of new integers created with ReDim _preserve?

I will write some test code...

I just posted a controller library in the library section. If you change the "Name AS STRING * 20" in "TYPE TYPE__SLOT" to "Name AS STRING" in the CONTROLLER.BI file you'll see the errors start to happen.

[DSMan195276]

To clarify, this is allowed, it sounds like you're hitting a bug. Variable-length Strings in UDTs is a new feature in QB64, it should work with ReDim and `_Preserve`.

From what bplus said, QB64 is not properly initializing any new memory added from a ReDim if you use `_Preserve` (which wouldn't surprise me...). IMO this should be fixed, but it might be a little annoying to implement due to `_Preserve` being a bit jank. Manually initializing the new entries sounds like it will solve it for the moment (that's what I unintentionally did in some code I wrote a few days ago, it works fine and uses both ReDim and variable-length strings in a UDT).

[mnrvovrfc]

It has to be carefully considered, because people have code with "REDIM _PRESERVE ARRAY(1 TO LIMIT) AS STRING * 1e+06" or something absurd like that. Maybe "reset" each new element created while $CHECKING:ON, and don't do it while $CHECKING:OFF? I understand the code in place was done most of all for speed.

I guess the QB64PE developers are confident enough with coding the variable-length strings despite the overhead and the complications. The programmer is the one who decides to waste memory and give up a bit of performance for convenience! But this is also true with Python and other programming languages with libraries "to make it all easier".

However, allowing variable-length string as UDT member sort of defeats the role of _MEM and that's why some people are still afraid of _MEM and steer from it as much as possible. It would be until QB64(PE) could have some OOP attributes, at least subprograms allowed as members of UDT without the schmaltz.

In the future, with _MEM as UDT member, something like a constructor in OOP-land could become mandatory. Otherwise what is the programmer going to do with it if it's going to occupy some memory over thousands of elements? I don't know, maybe instead go like Pascal and have "New()" and "Dispose()" for a specific datatype and a count of them, a lot like "calloc()" in C. I much preferred the old Turbo Pascal way even though it hindered on the OOP concept, which meant it had to be deprecated by the "Object" of v5.5. Yet the "Class" had to be invented later to keep Pascal/Delphi up with C++.

(According to Free Pascal manual, "Objects" were always allocated from the stack, and "Classes" were less limited in memory requirements according to 16-bit mentality.)

Yes I'm rambling again, but the thing is that the decision is expected, to be made at any cost to initialize any new elements "automatically", by some people, although rather few in number that come over directly from QuickBASIC and QBasic and use UDT's.

[mnrvovrfc]

I just posted a controller library in the library section. If you change the "Name AS STRING * 20" in "TYPE TYPE__SLOT" to "Name AS STRING" in the CONTROLLER.BI file you'll see the errors start to happen.

"NAME" is a reserved word, for sure since QuickBASIC.

https://qb64phoenix.com/qb64wiki/index.php/NAME

But if it does compile, the compiler should instead indicate what is reserved by it.

[bplus]

It is NOT fixed, numbers initialized with ReDim _Preserve do not start at 0 here is code that proves it:

Code: (Select All)

_Title "UDT assignments with ReDim _Preserve Test" ' b+ 2023-04-12
Type MYTYPE
    INT1 As Integer
    INT2 As Integer
    INT3 As Integer
    INFO As String
End Type

ReDim MYVARIABLE(1 To 1) As MYTYPE
Dim As Integer i

For assumeZeroedVariables = 0 To 1
    For i = 1 To 100
        If i > UBound(MYVARIABLE) Then ReDim _Preserve MYVARIABLE(1 To UBound(MYVARIABLE) + 10) As MYTYPE

        If assumeZeroedVariables = 0 Then ' we don't assume new Redim _preserve variables are 0 or ""
            ' direct assignments
            MYVARIABLE(i).INT1 = (i Mod 6) + 1
            MYVARIABLE(i).INT2 = ((i + 1) Mod 6) + 1
            MYVARIABLE(i).INT3 = ((i + 2) Mod 6) + 1
            MYVARIABLE(i).INFO = String$(MYVARIABLE(i).INT1, "X")
            ' works fine!!

        Else ' assume variables are 0 or ""
            ' aha!!! now we get trouble!!! assuming all new variables are 0 or ""
            MYVARIABLE(i).INT1 = MYVARIABLE(i).INT1 + (i Mod 6) + 1
            MYVARIABLE(i).INT2 = MYVARIABLE(i).INT2 + ((i + 1) Mod 6) + 1
            MYVARIABLE(i).INT3 = MYVARIABLE(i).INT3 + ((i + 2) Mod 6) + 1
            MYVARIABLE(i).INFO = MYVARIABLE(i).INFO + String$(MYVARIABLE(i).INT1, "X")
            ' Result is TROUBLE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        End If

    Next
    ReDim _Preserve MYVARIABLE(1 To 100) As MYTYPE

    ' display results
    For i = LBound(MYVARIABLE) To UBound(MYVARIABLE)
        Print i, MYVARIABLE(i).INT1, MYVARIABLE(i).INT2, MYVARIABLE(i).INT3, MYVARIABLE(i).INFO
        If i Mod 20 = 0 Then Print "...zzz": Sleep: Cls
    Next
    ReDim MYVARIABLE(1 To 1) As MYTYPE ' reset for next test
Next

UDT test 1 is run through code where we don't assume variables start at 0 or ""
   
How it's supposed to go nice and regular pattern.

UDT test 2 is run through code where we do assume variables are 0 and add or concat to "initial" value.
   
When variables aren't initialized to 0 or "" get big mess!!!

Surprisingly to me, this test worked fine for just concat new string to "initial" string (after ReDim _Preserve) and didn't have problems until added new integer to "initialized" integer.

So I was mistaken when Luke said it was fixed for numbers and fixed strings or it has since become unfixed.

[TerryRitchie]

Yes, crazy stuff