11 hours ago
For the first, you can always use a fixed length string of large enough size to contain mpos.
DIM sql AS STRING * 1,500,000,000 '1.5 GB string
DIM M AS _MEM: m = _MEM(sql)
mpos tracks the size of the string, so when you finally need to print to disk or whatever, just:
PUT #1, , LEFT$(sql, mpos)
'similar to the below:
Just make certain sql is large enough to hold the data to begin with, rather than variable length in size. (And set a max limit to how much data you're going to process at one time. For example, what would you do with a 100 GB database? You can't load that thing all at once into your memory, so you've got to work with it 1 GB at a time max and process it in chunks.)
As for the second, instead of this:
sql$ = Left$(sql$, mpos - 1) + Mid$(sql$, mpos + 1)
Simply do:
MID$(sql$, mpos) = MID$(sql$, mpos+1)
Now note that this won't reduce the size of your string. You'd have to track how many bytes you shaved off with such a method and then take the needed part later, but this is much more efficient than what you're doing above for processing it. You're not building a new string, adding strings, freeing old strings... All you're doing is writing to an existing string. It'd just be up to you to manually keep track of the true size of that string and then resize it (or only use the valid part of it) once you're finished stripping out parts of it.
DIM sql AS STRING * 1,500,000,000 '1.5 GB string
DIM M AS _MEM: m = _MEM(sql)
mpos tracks the size of the string, so when you finally need to print to disk or whatever, just:
PUT #1, , LEFT$(sql, mpos)
'similar to the below:
Code: (Select All)
Dim sql As String * 10
Dim m As _MEM: m = _Mem(sql)
For i = 0 To 9
_MemPut m, m.OFFSET + i, 65 + i As _BYTE
Next
msize = 5
Print Left$(sql, msize)
Just make certain sql is large enough to hold the data to begin with, rather than variable length in size. (And set a max limit to how much data you're going to process at one time. For example, what would you do with a 100 GB database? You can't load that thing all at once into your memory, so you've got to work with it 1 GB at a time max and process it in chunks.)
As for the second, instead of this:
sql$ = Left$(sql$, mpos - 1) + Mid$(sql$, mpos + 1)
Simply do:
MID$(sql$, mpos) = MID$(sql$, mpos+1)
Now note that this won't reduce the size of your string. You'd have to track how many bytes you shaved off with such a method and then take the needed part later, but this is much more efficient than what you're doing above for processing it. You're not building a new string, adding strings, freeing old strings... All you're doing is writing to an existing string. It'd just be up to you to manually keep track of the true size of that string and then resize it (or only use the valid part of it) once you're finished stripping out parts of it.
Code: (Select All)
Dim sql As String * 10
Dim m As _MEM: m = _Mem(sql)
For i = 0 To 9
_MemPut m, m.OFFSET + i, 65 + i As _BYTE
Next
msize = 5
Print Left$(sql, msize)
Mid$(sql, 3) = Mid$(sql, 5)
Print sql
Print Left$(sql, Len(sql) - 2) ' I removed two bytes above (5 - 3 = 2), so this should look like you'd expect now.