Bug when redimensioning multi-dimension arrays?

[12centuries]

I have a program that changes the dimensions of a dynamic multidimensional array.

After the `REDIM _PRESERVE`, the array values are scrambled.

Code: (Select All)

Initial Array Values of array Test(5,5):

1          1  2  3  4  5
2          6  7  8  9 10
3         11 12 13 14 15
4         16 17 18 19 20
5         21 22 23 24 25

Array Values after REDIM _PRESERVE Test(10, 10):

1          0 23 20  0  0  0  0  0  0  0
2          2  0 25  0  0  0  0  0  0  0
3          7  4  0  0  0  0  0  0  0  0
4         12  9  0  0  0  0  0  0  0  0
5         17 14  0  0  0  0  0  0  0  0
6         22 19  0  0  0  0  0  0  0  0
7          0 24  0  0  0  0  0  0  0  0
8          3  0  0  0  0  0  0  0  0  0
9          8  5  0  0  0  0  0  0  0  0
10        13 10  0  0  0  0  0  0  0  0

Here is some test code:

Code: (Select All)

REDIM SHARED Grid(5, 5)



' LoadInitialValues

FOR y = 1 TO 5

    FOR x = 1 TO 5

        counter = counter + 1

        Grid(y, x) = counter

    NEXT

NEXT



ShowArrayValues

REDIM _PRESERVE Grid(10, 10)

ShowArrayValues

END







SUB ShowArrayValues

    PRINT "Array Values:": PRINT

    FOR y = 1 TO UBOUND(Grid, 1)

        PRINT USING "Row ##  "; y,

        FOR x = 1 TO UBOUND(Grid, 2)

            PRINT USING " ##"; Grid(y, x);

        NEXT

        PRINT

    NEXT

    PRINT

END SUB

Am I misunderstanding something, or is this a bug? How can I make this work so that the initial values stay in their respective spots?

[bplus]

yeah, don't think redim _preserve works like that

here is sure way

Code: (Select All)

Dim Grid2(1 To 5, 1 To 5)
' LoadInitialValues
For y = 1 To 5
    For x = 1 To 5
        counter = counter + 1
        Grid2(x, y) = counter
    Next
Next

ShowArrayValues Grid2()
ReDim Shared Grid(1 To 10, 1 To 10)
For y = 1 To 5
    For x = 1 To 5
        Grid(x, y) = Grid2(x, y)
    Next
Next

ShowArrayValues Grid()
End



Sub ShowArrayValues (arr())
    Print "Array Values:": Print
    For y = 1 To UBound(arr, 2)
        Print Using "Row ##  "; y,
        For x = 1 To UBound(arr, 1)
            Print Using " ##"; arr(x, y);
        Next
        Print
    Next
    Print
End Sub

[RhoSigma]

_PRESERVE is buggy or better not fully implemented yet, you can only change the last dimension of an array without scrambling it. If any other than the last dimension is changed or even multiple dimensions at once, then it ends up in a mess. To word it a bit different, in such cases it does just preserve the values, but not how they are ordered in the array

[TerryRitchie]

Yep, this a known issue. I'll let Steve explain it in detail though. He explained it to me a few years back when I was wondering the same thing but I can't seem to find that explanation now? It may have been on a previous forum.

[SMcNeill]

Yep, this a known issue. I'll let Steve explain it in detail though. He explained it to me a few years back when I was wondering the same thing but I can't seem to find that explanation now? It may have been on a previous forum.

The details...    ..by Steve(tm)!!  

What's happening here really isn't that hard to understand.  Even though that's a 2-dimensional array, how do you think it's stored in memory?  Is there a 2-dimensional block of memory somewhere?   Does 3-dimensional arrays get stored in 3-dimensional RAM blocks?   Are 4-dimensional arrays only available in quanturm 4-D computers?



Nope.   All of these are stored in a single, contigious block of memory.    You may have a 3x3 array which looks like this:

1, 4, 7
2, 5, 8
3, 6, 9

But, in memory, it's going to look like:  1, 2, 3, 4, 5, 6, 7, 8, 9

Now, how does preserve work??

It copies that continious block of memory and simple moves it over to the new sized array.    Make a 5 x 3 array, and it creates it to look like:

0, 0, 0
0, 0, 0
0, 0, 0
0, 0, 0
0, 0, 0

And then it copies those 9 items from the old over to the start of that new array:

1, 6, 0
2. 7. 0
3, 8, 0
4, 9, 0
5, 0, 0

And you take a look at that and go....  WHUT??   My array is all scrambled now!   Index don't match!!

And they don't, though the data is still in the same order, the index now aren't...

BUT, let's take a look at what happens if you resize on the other index -- making 3 x 5 array:

First, the new array:

0, 0, 0, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0

And then we copy that block of memory over:
1, 4, 7, 0, 0
2, 5, 8, 0, 0
3, 6, 9, 0, 0

Those indexes still match up, and the data is still in the same order!

---

So, when using REDIM _PRESERVE, one can resize on the LAST index only, and have the data stay in place and organized.  Otherwise, the indexes get scrambled (the data still stays in one block, but the indexes to that block no longer match the same format).

To ReDIm an array on 2/3/4/n-dimensions, you'll need to write your own routine for that.

First, create a new array:

REDIM Temp (y_old_size, x_old_size) AS TYPE

Then transfer the values over:

FOR y = 0 to old_y_limit
   FOR x = 0 TO old_x_limit
      Temp(y, x) = Old_Array(y, x)
NEXT x, y

Then resize the old array:

REDIM Old_Array(new_y, new_x) AS TYPE

And transfer the values back over:

FOR y = 0 to old_y_limit
   FOR x = 0 TO old_x_limit
      Old_Array(y, x) = Temp(y, x)
NEXT x, y

And you're done!

(If not in a SUB/Function which cleans up variables at exit, you may want to REDIM Temp(0,0) AS TYPE to reduce memory usage and clean up that temp array to as small a footprint as possible.)

[SMcNeill]

And here's an illustration with your own test code, which helps take a good bit of the mystery out of what's happening:

Code: (Select All)

REDIM SHARED Grid(5, 5)



' LoadInitialValues

FOR y = 0 TO 5

    FOR x = 0 TO 5

        counter = counter + 1

        Grid(y, x) = counter

    NEXT

NEXT



ShowArrayValues

REDIM _PRESERVE Grid(10, 10)

ShowArrayValues

END







SUB ShowArrayValues

    PRINT "Array Values:": PRINT

    FOR y = 0 TO UBOUND(Grid, 1)

        PRINT USING "Row ##  "; y,

        FOR x = 0 TO UBOUND(Grid, 2)

            PRINT USING " ##"; Grid(y, x);

        NEXT

        PRINT

    NEXT

    PRINT

END SUB

The reason things looked so scrambled before, is you were running data from (1 to 5, 1 to 5), in an array dimensioned (0 to 5, 0 to 5), cutting things off and messing up the index where no data was even availabe to look at.

Like this, you can see that the initial data starts out as you suggest:

1, 2, 3, 4, 5, 6
7, 8,......
13, 14,......
19, 20......
25, 26.......
31, 32......

And since data is stored DOWN to RIGHT, when you resize, you see:

1
7
13
19
25
31

(see that pattern continue as the data is moved as you'd expect to be?)  But we now have 5 more rows of data, so we just keep copying:

1
7
13
19
25
31
2
8
14
20
26
32

And then we copy up onto the next column, since we filled the first column now:

1 , 32
7 , 3
13, 9
19 .....
25.
31.
2.
8
14
20
26
32

It's not "scrambling the data", as it appears at first, with your example.  It's just copying it directly in one chunk of memory, and the indexes are aligning and matching up anymore.

The *only* time REDIM _PRESERVE works properly for you us when you REDIM on the *RIGHTMOST* index.  Redimming anywhere else will cause your data to be misaligned with your new index structure. 

---

And if you're curious why your original data looked the way it did, See if this makes sense once you see the whole original array and its resized version:

Code: (Select All)

Array Values:

Row  0    0  0  0  0  0  0
Row  1    0  1  2  3  4  5
Row  2    0  6  7  8  9 10
Row  3    0 11 12 13 14 15
Row  4    0 16 17 18 19 20
Row  5    0 21 22 23 24 25

Code: (Select All)

Array Values:
Row  0    0 21 18 15  0  0  0  0  0  0  0
Row  1    0  0 23 20  0  0  0  0  0  0  0
Row  2    0  2  0 25  0  0  0  0  0  0  0
Row  3    0  7  4  0  0  0  0  0  0  0  0
Row  4    0 12  9  0  0  0  0  0  0  0  0
Row  5    0 17 14  0  0  0  0  0  0  0  0
Row  6    0 22 19  0  0  0  0  0  0  0  0
Row  7    1  0 24  0  0  0  0  0  0  0  0
Row  8    6  3  0  0  0  0  0  0  0  0  0
Row  9  11  8  5  0  0  0  0  0  0  0  0
Row 10  16 13 10  0  0  0  0  0  0  0  0

[Kernelpanic]

The topic of multidimensional arrays has already been discussed in detail here:

Array in an Array
Three-dimensional array

How arrays are stored in memory, I posted a picture about this before, but I can't find it at the moment, so here is the graphical representation again.

[12centuries]

[...] So, when using REDIM _PRESERVE, one can resize on the LAST index only, and have the data stay in place and organized.  Otherwise, the indexes get scrambled (the data still stays in one block, but the indexes to that block no longer match the same format).

Okay, this makes a bit more sense, thanks for taking the time to explain it!

But I believe there is a bit more going on here that you're outlining (maybe you're dumbing it down for me).

So take a look at the array I was getting after I redimmed it:




As you can see, the first "column" values are completely absent from the final array, and many of the "middle" values for each column are missing, as if they're lost in a hidden row just outside the array grid.

So if I'm looking at this as a single block of memory, it initially looks like this:

Code: (Select All)

    Column 2       Column 3       Column 4       Column 5       Column 6
┌──────┴─────┐ ┌──────┴─────┐ ┌──────┴─────┐ ┌──────┴─────┐ ┌──────┴─────┐
01 06 11 16 21 02 07 12 17 22 03 08 13 18 23 04 09 14 19 24 05 10 15 20 25

After redimming it, it looks like this:

Code: (Select All)

??     Column 2   ??   Column 3  ??     Column 4   ??   Column 3     new data
┌┐ ┌──────┴─────┐ ┌┐ ┌────┴────┐ ┌┐ ┌──────┴─────┐ ┌┐ ┌────┴────┐ ┌──────┴─────┐
00 02 07 12 17 22 00 03 08 13 23 00 04 09 14 19 24 00 05 10 20 25 00 00 00 00 00 […]

Some of the values have (apparently) completely disappeared (1, 6, 11, 16, 15, 18 and 21), but they are (evidently) still in memory, because if I redim back to the original size:

Code: (Select All)

REDIM _PRESERVE Grid(5, 5) ' back to the original size

Then I get the original array back, including those "missing" numbers:

Code: (Select All)

1          1  2  3  4  5
2          6  7  8  9 10
3         11 12 13 14 15
4         16 17 18 19 20
5         21 22 23 24 25

I'm wondering where those "missing" numbers went and why they're not accessible to the array after redimming to a larger array, and why they return after redimming to the original size. Also, why there are extra zeros in between the existing data after the initial resize.

I think I may take a look at the source code for this...

[SMcNeill]

I'm wondering where those "missing" numbers went and why they're not accessible to the array after redimming to a larger array, and why they return after redimming to the original size. Also, why there are extra zeros in between the existing data after the initial resize.

I think I may take a look at the source code for this...

As I mentioned, you're not printing your whole array.  Here's your actual first array  (REDIM SHARED Grid(5, 5)):

Code: (Select All)

Array Values:

Row  0    0  0  0  0  0  0
Row  1    0  1  2  3  4  5
Row  2    0  6  7  8  9 10
Row  3    0 11 12 13 14 15
Row  4    0 16 17 18 19 20
Row  5    0 21 22 23 24 25

And here's the resized version of it (REDIM PRESERVE Grid(10, 10)):

Code: (Select All)

Array Values:
Row  0    0 21 18 15  0  0  0  0  0  0  0
Row  1    0  0 23 20  0  0  0  0  0  0  0
Row  2    0  2  0 25  0  0  0  0  0  0  0
Row  3    0  7  4  0  0  0  0  0  0  0  0
Row  4    0 12  9  0  0  0  0  0  0  0  0
Row  5    0 17 14  0  0  0  0  0  0  0  0
Row  6    0 22 19  0  0  0  0  0  0  0  0
Row  7    1  0 24  0  0  0  0  0  0  0  0
Row  8    6  3  0  0  0  0  0  0  0  0  0
Row  9  11  8  5  0  0  0  0  0  0  0  0
Row 10  16 13 10  0  0  0  0  0  0  0  0

Now, look at this resized array, and ONLY print the values from (1 to 5, 1 to 5), and see if it doesn't print the results you're seeing, completely. 

Your arrays are from (0 to 5, 0 to 5), which affects them as above.  You're ust printing them from (1 to 5, 1 to 5), and that makes it seem as if values are totally scrambled or even lost.

Remember:  QB64PE arrays are 0-based, unless you specify otherwise.

[SMcNeill]

For illusttation, notice how things work out when I make both arrays lower limit 1, instead of 0:

Code: (Select All)

REDIM SHARED Grid(1 TO 5, 1 TO 5)



' LoadInitialValues

FOR y = 1 TO 5

    FOR x = 1 TO 5

        counter = counter + 1

        Grid(y, x) = counter

    NEXT

NEXT



ShowArrayValues

REDIM _PRESERVE Grid(1 TO 10, 1 TO 10)

ShowArrayValues

END







SUB ShowArrayValues

    PRINT "Array Values:": PRINT

    FOR y = 1 TO UBOUND(Grid, 1)

        PRINT USING "Row ##  "; y,

        FOR x = 1 TO UBOUND(Grid, 2)

            PRINT USING " ##"; Grid(y, x);

        NEXT

        PRINT

    NEXT

    PRINT

END SUB

REDIM PRESERVE does nothing more than preserve the memory and then move the whole block from the old array to the new array.  If you change more than the RIGHTMOST index, the structure won't read the same and your data will seem as if it's been "scrambled".  Effictively, all you can do is REDIM and PRESERVE that rightmost index -- if you need more, then you'll have to write your own routine to handle that.