+- QB64 Phoenix Edition (https://qb64phoenix.com/forum)
+-- Forum: QB64 Rising (https://qb64phoenix.com/forum/forumdisplay.php?fid=1)
+--- Forum: Code and Stuff (https://qb64phoenix.com/forum/forumdisplay.php?fid=3)
+---- Forum: Help Me! (https://qb64phoenix.com/forum/forumdisplay.php?fid=10)
+---- Thread: Question about _MEM blocks and arrays (/showthread.php?tid=3730)
Pages:
1
2
Question about _MEM blocks and arrays - FCS_coder - 06-04-2025
OK, now that I have the 32/64 bit thing handled I have another question I've been chewing on...can I use _MEM blocks to hold dynamic arrays? Looking at the wiki, I see how I can use _OFFSET to manipulate bytes/words in a block of memory, but is there a way to use arrays instead? That would make the access easier: a simple index rather than an opaque location pointer into a block. But it looks like while I could put a string into a _MEM pointer, I can't do the same with an array
RE: Question about _MEM blocks and arrays - ahenry3068 - 06-04-2025
I just use REDIM for dynamic arrays. It's easier and pretty much does the same thing as MEM (reallocates the memory for an array).
Despite the syntax you can also use REDIM to initially allocate an array same as DIM
RE: Question about _MEM blocks and arrays - SMcNeill - 06-04-2025
Mem works with arrays; there's no problem with that. The only issue is the entire array is pointed to by the mem block, so if you want to use individual elements yourself, then you have to calcaulate their position manually and fetch only what you need from that segment of the entire array mem block.
For example
DIM a(10), m AS _MEM
m = _MEM(a()) '<-- to point to the array a() and not some variable a
_MEMPUT m, m.OFFSET, 12.34 AS SINGLE
_MEMPUT m, m.OFFSET + 8, 98.76 AS SINGLE
FOR I = 0 TO 10
PRINT a(i)
NEXT
The above will print:
12.34
0
98.76
(and several more 0s)
Notice how I moved my pointer over 8 spaces to put the data into a(2)? If you're looking to address individual elements of your array, you have to track their position manually like above.
RE: Question about _MEM blocks and arrays - ahenry3068 - 06-04-2025
(06-04-2025, 09:15 AM)SMcNeill Wrote: Mem works with arrays; there's no problem with that. The only issue is the entire array is pointed to by the mem block, so if you want to use individual elements yourself, then you have to calcaulate their position manually and fetch only what you need from that segment of the entire array mem block.Maybe I'm being THICK.
For example
DIM a(10), m AS _MEM
m = _MEM(a()) '<-- to point to the array a() and not some variable a
_MEMPUT m, m.OFFSET, 12.34 AS SINGLE
_MEMPUT m, m.OFFSET + 8, 98.76 AS SINGLE
FOR I = 0 TO 10
PRINT a(i)
NEXT
The above will print:
12.34
0
98.76
(and several more 0s)
Notice how I moved my pointer over 8 spaces to put the data into a(2)? If you're looking to address individual elements of your array, you have to track their position manually like above.
That does Happen
I don't see a real practical use for this outside of the normal Array Indexing notation ???? (Unless MEM pointers can be passed between
different program instances, which I don't think is practically possible with User Space code on most 64 bit OS's)
RE: Question about _MEM blocks and arrays - SMcNeill - 06-04-2025
A practical use for mem?
Anything that requires speed.
Direct memory manipulation is much faster than referenced access to the same memory. Array(index) has to first look up the index, then calculate the offset position of that index, then check the memory location for the data you want, then return it to you. With mem, you specify the offset yourself, the amount of data you want, and presto, retrieve it.
It's also much faster to shift an array in one chuck of movement than it is to shift it in a loop one element at a time, or to copy the whole array at once, and countless other things.
Consider this -- I have an array of 101 elements (DIM array(100) as _BYTE). Now, I want to add an element to this array, but it needs to go in position number 52 and everything else shift down a spot.
We could:
REDIM _PRESERVE array(101) AS _BYTE 'make it an extra byte in size bigger
FOR i = 100 to 52 STEP -1
array(i+1) = array(i) 'shift everything right
NEXT
array(52) = new_data
OR we can:
REDIM _PRESERVE array(101) AS _BYTE
DIM m as _MEM: m = _MEM(array())
_MEMCOPY m, m.offset +52, 48 TO m, m.offset + 53 'shift those elements one byte over
_MEMFREE m
array(52) = new_data
One movement of memory to shift everything an element over, rather than doing it repeatedly in a loop. In a loop of only going from 0 to 100, it's not going to make much difference in speed, but what if we're dealing with an array that holds 100,000,000 records and you need to shift 99,999,987 of them to make room for a new one? Moving it all at once makes a heckuva difference then, compared to shifting it with a loop. Then imagine that this shifting is something you might be doing 500,000 times as you're inserting new records into your existing database and are using a binary insertion sort on them...
The whole point of mem is speed. If that's not something you have to be concerned about, then just use arrays and indexes as they're much easier to track and intuitive to work with. But when it comes time that your program is reaching a bottleneck in performance, you may have resort to using mem to keep things running fast enough to be usable without wasting all your time while it's working in the background on some trivial task with countless looping.
RE: Question about _MEM blocks and arrays - ahenry3068 - 06-04-2025
(06-04-2025, 04:21 PM)SMcNeill Wrote: A practical use for mem?Ok, I was being thick.... That's a very good point. Honestly I don't think of speed often on modern system's (very spoiled that way). Usually things are *Fast Enough* even with inefficient code. But I'll keep that in mind for future use. Actually my X16 Video convertor is at the point where it could use some optimization. Not sure How much speed I can pick up because the whole approach is pretty inefficient as I'm actually shelling ImageMagick for Each and Every Frame of the Video and a HUGE amount of the program's execution is spent in that loop. But my palette shuffling routines also make a difference in the overall execution, perhaps I can shed some execution time there.
Anything that requires speed.
Direct memory manipulation is much faster than referenced access to the same memory. Array(index) has to first look up the index, then calculate the offset position of that index, then check the memory location for the data you want, then return it to you. With mem, you specify the offset yourself, the amount of data you want, and presto, retrieve it.
It's also much faster to shift an array in one chuck of movement than it is to shift it in a loop one element at a time, or to copy the whole array at once, and countless other things.
Consider this -- I have an array of 101 elements (DIM array(100) as _BYTE). Now, I want to add an element to this array, but it needs to go in position number 52 and everything else shift down a spot.
We could:
REDIM _PRESERVE array(101) AS _BYTE 'make it an extra byte in size bigger
FOR i = 100 to 52 STEP -1
array(i+1) = array(i) 'shift everything right
NEXT
array(52) = new_data
OR we can:
REDIM _PRESERVE array(101) AS _BYTE
DIM m as _MEM: m = _MEM(array())
_MEMCOPY m, m.offset +52, 48 TO m, m.offset + 53 'shift those elements one byte over
_MEMFREE m
array(52) = new_data
One movement of memory to shift everything an element over, rather than doing it repeatedly in a loop. In a loop of only going from 0 to 100, it's not going to make much difference in speed, but what if we're dealing with an array that holds 100,000,000 records and you need to shift 99,999,987 of them to make room for a new one? Moving it all at once makes a heckuva difference then, compared to shifting it with a loop. Then imagine that this shifting is something you might be doing 500,000 times as you're inserting new records into your existing database and are using a binary insertion sort on them...
The whole point of mem is speed. If that's not something you have to be concerned about, then just use arrays and indexes as they're much easier to track and intuitive to work with. But when it comes time that your program is reaching a bottleneck in performance, you may have resort to using mem to keep things running fast enough to be usable without wasting all your time while it's working in the background on some trivial task with countless looping.
RE: Question about _MEM blocks and arrays - a740g - 06-04-2025
@ahenry3068, I completely agree. Shelling out to run external programs can be quite slow. Another problem is that it is not portable and introduces dependencies into your program.
I noticed that ImageMagick has the ImageWand and ImageCore C APIs. If you create a QB64 wrapper or use DECLARE LIBRARY directly, you should get improved performance in your code.
RE: Question about _MEM blocks and arrays - ahenry3068 - 06-04-2025
(06-04-2025, 04:56 PM)a740g Wrote: @ahenry3068, I completely agree. Shelling out to run external programs can be quite slow. Another problem is that it is not portable and introduces dependencies into your program.
I noticed that ImageMagick has the ImageWand and ImageCore C APIs. If you create a QB64 wrapper or use DECLARE LIBRARY directly, you should get improved performance in your code.
Actually one reason I used ImageMagick is that it IS PORTABLE at least among Windows & Linux & MACOS. It works almost identically among the 3 major platforms.
In some ways I'm still a neophyte in specific areas in QB64PE. I still have an MS-DOS mind track in some ways. I'm not to familiar with linking in external routines. If
I do I would like it to be portable among the 3 major platforms.
RE: Question about _MEM blocks and arrays - a740g - 06-04-2025
What exactly are you doing with the image frames? If you're dithering or resizing them, you could handle that directly within QB64, eliminating the need for external tools.
RE: Question about _MEM blocks and arrays - ahenry3068 - 06-04-2025
(06-04-2025, 04:56 PM)a740g Wrote: @ahenry3068, I completely agree. Shelling out to run external programs can be quite slow. Another problem is that it is not portable and introduces dependencies into your program.
I noticed that ImageMagick has the ImageWand and ImageCore C APIs. If you create a QB64 wrapper or use DECLARE LIBRARY directly, you should get improved performance in your code.
I need to get from the 24 bit PNG file to an 8 bit pixel array and a 24(32) bit palette I can then transform into Bitmap and palette information for the Commander X16. I also have to be able to arbitrarily resize each image in going from the PNG to the 8 bit array. I'm currently creating that from a Windows BMP rev 3 which I use ImageMagick to produce. Another tack I could take is to have ffmpeg extract BMP files INSTEAD of PNG's but that could take a Huge amount of extra disk space and I don't currently know how to make FFMPEG produce the revision of the Windows BMP standard that my code understands.