Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Question about _MEM blocks and arrays
#11
(06-04-2025, 04:21 PM)SMcNeill Wrote: A practical use for mem?   

Anything that requires speed.  Big Grin

Direct memory manipulation is much faster than referenced access to the same memory.   Array(index) has to first look up the index, then calculate the offset position of that index, then check the memory location for the data you want, then return it to you.  With mem, you specify the offset yourself, the amount of data you want, and presto, retrieve it

My question is, can't we update the compiler to be a little smarter, and generate code that uses these direct memory manipulation tricks, for commands like array(index)?
Reply
#12
a simple example to copy an array
Code: (Select All)

Dim a(10), b(10), m As _MEM
For i = 0 To 10
    b(i) = i + 1
Next

m = _Mem(b()) '<-- to point to the array b()
_MemGet m, m.OFFSET, a() ' copy array b() into array a()
For i = 0 To 10
    Print a(i)
Next
Reply
#13
The motivation for me was the ability to do array processing on dynamicall sized blobs of data.  I wanted to use MEM blocks the same way Pascal lets you allocate arbitrary sized records by using either arrays or pointers to char/byte lists.

I want to have a record/struct  ocontaining a status byte, a size allocated byte, plus the dara.  Thinking about it I realized I can bump the data up by two bytes/words and just offset the data by that much.  That means I can just pass arrays to/from SUBs/PROCs as needed.  I was led to that by the info that you can't (shouldn't?) use arrays in UDTs.

How good is the heap tracking on REDIMed arrays?  Do I nee to worry about fragmentation right away?  Not sure how smart the back-end C compiler's memory anagementt library is.
Reply
#14
You know what would be very helpful, would be a mini-tutorial series showing all the different common tasks in QB64PE that can be done a lot faster with memory operations, with a simple example of the standard QB64PE method of doing the thing, alongside the memory manipulation method, and an explanation. Then we would have a nice and easy reference to work from all in one place...
Reply
#15
it might also be helpful to have side-by-side comparison showing C pointers vs QB64 _Mem methods
Reply
#16
Quote:@FCS_coder - I was led to that by the info that you can't (shouldn't?) use arrays in UDTs.
That is true, but you can use record arrays. Maybe this is a possibility.

PS: Bug fixed. But the loop isn't perfect yet.

Code: (Select All)

'Recordfeld, Hinweis S. 45 und S. 173 - 6. Juni 2025

Option _Explicit

Type Motorrad
  Modell As String * 30
  Kilowatt As Double
  Preis As Double
End Type

Dim MotorradMarken(2) As Motorrad

Dim As Integer anzahl, satzNummer, zeile, spalte

satzNummer = 1
zeile = 3: spalte = 3
Locate zeile, spalte
Do
  Input "Modell    : ", MotorradMarken(satzNummer).Modell
  Locate CsrLin, spalte
  Input "Kilowatt  : ", MotorradMarken(satzNummer).Kilowatt
  Locate CsrLin, spalte
  Input "Preis      : ", MotorradMarken(satzNummer).Preis
  satzNummer = satzNummer + 1
  
Loop Until satzNummer <= 2

Locate CsrLin + 2, spalte
For anzahl = 1 To satzNummer
  'Sa muss ein Semikolon hin, sonst langer Abstand
  Print Tab(4); "Modell    : "; MotorradMarken(anzahl).Modell
  Print Tab(4); Using "Kilowatt  : #####.##"; MotorradMarken(anzahl).Kilowatt
  Print Tab(4); Using "Preis      : #####.##"; MotorradMarken(anzahl).Preis
  Locate CsrLin + 1, spalte
Next

End

The loop doesn't work.  Sad

[Image: Recordarray2025-06-06.jpg]
Reply
#17
(06-04-2025, 07:31 PM)madscijr Wrote:
(06-04-2025, 04:21 PM)SMcNeill Wrote: A practical use for mem?   

Anything that requires speed.  Big Grin

Direct memory manipulation is much faster than referenced access to the same memory.   Array(index) has to first look up the index, then calculate the offset position of that index, then check the memory location for the data you want, then return it to you.  With mem, you specify the offset yourself, the amount of data you want, and presto, retrieve it

My question is, can't we update the compiler to be a little smarter, and generate code that uses these direct memory manipulation tricks, for commands like array(index)?
Steve is overstating the difference between the two Tongue Functionally both do the exact same thing already and will have extremely similar performance. A slight difference is that arrays don't have to start from zero (Ex. `Dim a(20 to 40)`) so a subtraction has to be done to take an index like `a(21)` and make it zero-based (`21 - 20`), but overall that's pretty minor. Normal array access of a single element via `a(index)` vs `_MemGet()` are basically the same.

If you need to move large parts of an array though then `_Mem` is faster, `_MemCopy` uses faster techniques than just doing a loop. In theory the compiler could do optimizations like that, but determining whether a loop in the code can be replaced is complicated and in many cases not possible.

For example, a looping copy of an array to shift elements has pretty specific behavior when there's an error - it will copy every element before it hits the index that is too large. Optimizing the loop while also maintaining the same behavior on errors is pretty messy, `_MemCopy` for example does not have the same behavior (it errors before doing any copying). The alternative of determining at compile-time that the code cannot hit an error is also hard (and in many cases not possible).
Reply
#18
(Yesterday, 04:48 PM)DSMan195276 Wrote:
(06-04-2025, 07:31 PM)madscijr Wrote: My question is, can't we update the compiler to be a little smarter, and generate code that uses these direct memory manipulation tricks, for commands like array(index)?

If you need to move large parts of an array though then `_Mem` is faster, `_MemCopy` uses faster techniques than just doing a loop. In theory the compiler could do optimizations like that, but determining whether a loop in the code can be replaced is complicated and in many cases not possible.

Thanks. That makes sense.
So how about they add a new native QB64PE command called MoveArrayElements(MyArray, SourceStartElementIndex, SourceEndElementIndex, DestIndex) that uses the memory method under the hood? Then the compiler does not need to try and determime what the source code is doing, it's now just a straightforward command.

(Or, I suppose we could create a user-defined function that does the same, with whatever checks to make sure the values are valid. Either way, in the spirt of BASIC, to keep life simple and save the programmer from having to worry about low level headaches like worrying about how many bytes a variable uses, etc. If we don't mind that low-level stuff, we might as well be coding in C!)
Reply
#19
(Yesterday, 05:24 PM)madscijr Wrote: Thanks. That makes sense.
So how about they add a new native QB64PE command called MoveArrayElements(MyArray, SourceStartElementIndex, SourceEndElementIndex, DestIndex) that uses the memory method under the hood? Then the compiler does not need to try and determime what the source code is doing, it's now just a straightforward command.

(Or, I suppose we could create a user-defined function that does the same, with whatever checks to make sure the values are valid. Either way, in the spirt of BASIC, to keep life simple and save the programmer from having to worry about low level headaches like worrying about how many bytes a variable uses, etc. If we don't mind that low-level stuff, we might as well be coding in C!)
That could be reasonable to add, it's actually fairly hard to implement yourself since you can't just take in a generic "array", it has to be of a specific type (of course, internal QB64 commands don't have this problem). That said, for most use-cases just using a regular loop is honestly fine.
Reply




Users browsing this thread: 2 Guest(s)