What is the maximum size of Bidimensional Array?

[Pedro]

Greetings to all and congratulations for the creators of this forum and qb64Pe programmers.


I am programing something in QB64, and I need to create an array bidimensional of 3^14. I need 13 different values in each cell, so I used _bit*4 to store.


But  the down array is the max lenght array I get
matrix = 3 ^ 10
DIM disaci(matrix, matrix) AS _BIT * 4

If I considerer only one bit for cell to store, then I can up lenght array to 3^11.

The question is: Can I make this array with QB64?
matrix = 3 ^ 14
DIM disaci(matrix, matrix) AS _BIT * 4

or forget it?

Thank you very much for attention

[Pete]

Welcome to the forum.

I'll wait for one of the devs to go into the specifics here. Out of curiosity, did you come from the Quickbasic era, where memory was limited? If so, it's worth noting QB64 was designed to use your system memory, so my guess would be if your system has enough free memory, it may be possible, but this is a very large array size so hopefully you will get a answer from one of the devs soon. Steve has worked with some very large arrays, and Matt may have some additional insights on the internal workings, as well. The only restriction I'm aware of is from the wiki for numbering an array from 1 to something. For that "something" is maxed at: 32767. For using DIM myarray() I see no such restrictions.

Article on QB64 arrays is here: https://qb64phoenix.com/qb64wiki/index.php/Arrays

Pete

[mnrvovrfc]

3^11 = 177417 but [b]that last value[b] squared = 31381059609!

Welcome to the forums Pedro.

Could declare a single string to be that large and then use "_CV()" and "_MK$()" to go back and forth (and critically "MID$()" on LHS) getting and setting values. But that's not for the larger number you're requesting because the large value shown here is close to 29.2GB.

I just tried on Linux setup which does not have QB64PE but Freebasic, only 4GB RAM and 4GB "swap" space. This looks like a job for an associative array.

[Pete]

My kid has a gaming system that could handle 64GB of memory. Me, I'm saddled with the same 4GB max 8GB situation here, so I didn't even give it a try.

Thanks for doing the size calc! +1

Pete

[mnrvovrfc]

Thanks for doing the size calc! +1

Pete

As I've said this is for 3^11 but the OP must want two dimensions of 3^14 which is 22876792454961. LOL had to actually write a Freebasic program to get an unsigned 64-bit number, Galculator gives me "e+13" at the end, sheesh.

[Pedro]

Welcome to the forum.

I'll wait for one of the devs to go into the specifics here. Out of curiosity, did you come from the Quickbasic era, where memory was limited? If so, it's worth noting QB64 was designed to use your system memory, so my guess would be if your system has enough free memory, it may be possible, but this is a very large array size so hopefully you will get a answer from one of the devs soon. Steve has worked with some very large arrays, and Matt may have some additional insights on the internal workings, as well. The only restriction I'm aware of is from the wiki for numbering an array from 1 to something. For that "something" is maxed at: 32767. For using DIM myarray() I see no such restrictions.

Article on QB64 arrays is here: https://qb64phoenix.com/qb64wiki/index.php/Arrays

Pete

Hi Pete, thank you for your answer. I thougth it was a limit QB64 to the arrays. I read the arrays article, but there is nothing about the maximum size length array.

Yes, I come from old Basic compilers, and my first idea it was there is a limit of array. But if you say me, the limit is memory system. It could be the memory of my system. Thank you very much for your answer, I will try to test in other machine.

Pedro

[Jack]

Pedro, what do you need such a huge array for?
if you were to use 1 byte per element it would take about 21 terabytes, you need to come up with different solution to your problem

[mnrvovrfc]

There are a few libraries that do associative arrays and hash tables, but using them could be complicated for the ones who didn't create such libraries. I give away a simple method here.

The trick is to create a single-dimensional array of an UDT type:

TYPE datype
key as STRING * 37
num as _BYTE
END TYPE
REDIM aa(1 to 100) AS datype

The "aa" should be checked for length so it could be expanded with "REDIM _PRESERVE" as needed.

Say the double dimension at 16777216,202020202020 (picking ridiculous numbers on purpose).
Then internally the key should be put down as "00000000016777216,000000202020202020" so both subscripts are 18 figures long, which is about the written extent of a 64-bit integer. This key looks mind-boggling but it would make it easy for sorting and therefore employ a binary search on the keys which is way faster than searching the array sequentially.

Each new element of "aa" should be inserted so the whole array remains sorted by key. This is the only drawback of employing a binary search.

Also note that for a given subscript pair, it must be converted to string to a 37-digit monstrosity LOL as I've indicated above. Again, this makes it easy to sort. Maybe it could be done entirely with numbers but it requires the extra step of using "VAL()" on each subscript, then the additional job of sorting by "major" subscript then by minor subscript. Doing the whole thing as a string, as a fake entry to a real array and padded with zeroes makes it easier to code and to understand.

Unless the OP really requires 3^11 values or more, this associative array method would save a ton of memory, and it could be handled on 32-bit as well as 64-bit.

[Pete]

Pedro, what do you need such a huge array for?
if you were to use 1 byte per element it would take about 21 terabytes, you need to come up with different solution to your problem

Looks like a data storage hash table job at this point. It might be another century or so before we start turning out 128 terabyte memory cards.

Well, hopefully Pedro will clue us in on what he is trying to store in all those arrays.

Pete

[Pedro]

Thank you for all the answers. You are very kind.

I realize the enormous size of memory that would be required: (3^14) * (3^14) * 1 nibble.

I think this array is the maximum I can span with the memory in my computer. I'll give it a try.
-----------------------
matrix = 3 ^ 10
DIM disaci(matrix, matrix) AS _UNSIGNED _BIT * 4

------------------
For Jack, it's a mathematical curiosity about a game. But I have to think of another way to do it.

To mnrvovrfc, thanks for the idea of TYPE and REDIM, I have to think about it.

Regards

Pedro