Memory Warning

[Dimster]

I have a vague recollection that a discussion on memory usage was kicked around however I can't seem to find if that discussion specifically dealt with a Warning being displayed in the IDE. 

Every now and then I could use a quick guide to how much memory a particular array that I have just dimmed will take up. I know the act of dimming sets aside memory for the array however the amount of memory set aside is not immediately known. You have to do the math yourself.

As well, how much available memory is also difficult to know at the time of dimming. I don't know if the IDE, in reserving the memory would have access to the total ram in one's computer and therefore, knowing the size of the array being set up in memory, would also be able to calculate the amount of memory left for future arrays.

Would it be possible for the IDE to have a Warning on Memory Available ... or if upon Dimming an array the amount of memory set aside be displayed ... or if the Warning on Unused arrays and variables would display the amount of memory.

Just a thought

[SMcNeill]

For today's 64-bit systems, it's hard to tell how much memory is available for a program.

For starters, there's your RAM.   4GB or 1GB or 25GB or whatever it is...

32-bit OSes had a limit of about 1.5GB of RAM being available for most programs.  (2GB was the actual limit, but then it also has to reserve some space for system junk, so if you ever hit more than the 1.5GB mark, you were pushing the failure limits.)

But as for 64-bit OSes??

The can use the whole system memory....

...and then they start uing the swap file...

...which might have a hard cap to it, or might be "managed by the OS"...

... if it's managed by the OS, then it depends  on how much space the drive has on it, before it ends up completely full...


....  and if you're approaching those type of limits, then you're probably doing something wrong with your code!

---

Modern systems don't tend to get "Out of Memory" errors like they used to.  Instead, they just get slower and slower and slower as they end up swapping more and more data in and out of those files on the hard drive.

Unless some particular error message pops up for you to deal with, personally I wouldn't worry too much about memory and I'd just let the OS handles things as it needs to. 

[Dimster]

I have 32 gb and do not recall ever getting an out of memory error. 

Honestly its more what I'm doing to the speed of the processing of an array. If you Dim an array (0 to 1000) is it bigger (in terms of memory set aside) than an array (250,250,500). Is there a correlation between the speed of processing an array and the amount of memory set aside for it?

[Dav]

I got a memory error in the IDE recently for the first time. When the IDE started it said it was turning syntax highlighting off because of low memory.  This is with the 64-bit version on linux. 12gb of ram.  Nothing else running.  I turned syntax highlighting back on without problems.

- Dav

[SMcNeill]

For your example above, the array (0 to 1000) would have 1001 elements.  It's basically just a simple straight grocery list.  "Honey, go pick up these 1001 and items for me from the grocery store!"

 
The array (250, 250, 500) would have 251 * 251 * 501 elements.  It's basically...   "Honey, we need to go to all 251 malls left in our state, so I can visit 251 shops in each of them, and bring back 501 items from each of those shops!"

Now, which of those is going to be the larger shopping trip? 



If you want a quick little routine to tell you how much mem each is using, you can always try something like this:

Code: (Select All)

DIM m AS _MEM



DIM a(0 TO 1000)

m = _MEM(a())

PRINT MemUsage(m)



DIM b(250, 250, 500)

m = _MEM(b())

PRINT MemUsage(m)



FUNCTION MemUsage~&& (m AS _MEM)

    MemUsage = VAL(STR$(m.SIZE))

END FUNCTION

[RhoSigma]

I have 32 gb and do not recall ever getting an out of memory error. 

Honestly its more what I'm doing to the speed of the processing of an array. If you Dim an array (0 to 1000) is it bigger (in terms of memory set aside) than an array (250,250,500). Is there a correlation between the speed of processing an array and the amount of memory set aside for it?

The correlation is rather on the number of dimensions than on memory usage.
In normal usage each and every access to any array element runs through a function called array_check() for every dimension to verify the given index numbers are in the valid range as dimensioned.
Hence a(x,y,z) = 42 will call that function 3 times while a(x) = 42 will call it only once.

However, once your program runs error free, you may use $CHECKING:OFF to suppress these checks, but hell will open beneth your feet, if you try to access non-existing array elements then.

[Dimster]

Thanks guys, this is helpful.... so if my math is in the ballpark, then that array (20,250,500) takes about .13 gb of memory which should give me about 300 of those (250,250,500) arrays.
That's a handy memory size calculator Steve. Just need an error trap for "Hell Beneath My Feet"

[SMcNeill]

13 MB

MB, KB, Byte

[SMcNeill]

And there's one thing to definitely keep in mind when trying to track memory so closely, @Dimster -- it's impossible once you add this one simple line of code:

Code: (Select All)

INPUT a$

You've now decided to turn control over to the end user and let them enter something into your program.  Do they enter in a 5-byte name?  "STEVE"   Or do they enter in their complete biography?   "Back in 1980, I was first introduced to the World of Steve!!  And let me say, STEVE IS AMAZING!!  Oh my gosh!  There's just not enough amazingly astouding perfect words to describe the awesomeness that is STEVE!!  Wr all wish we could be STEVE!!  Even though I *am* a STEVE, I still *wish* I could be even more a STEVE!  Just call me Steve-THE STEVE-Steven_steve!!!"........

Hard to track memory usage when that user string might be anywhere from 0-bytes (they just hit enter and skipped it), to 2GB worth of garbage.

If "OUT OF MEMORY" is ever a real concern for your stuff, remember to always write your code with a hard limit on how large a string can possibly be. 

[luke]

I got a memory error in the IDE recently for the first time. When the IDE started it said it was turning syntax highlighting off because of low memory.  This is with the 64-bit version on linux. 12gb of ram.  Nothing else running.  I turned syntax highlighting back on without problems.
- Dav

If you mean the "Syntax Highlighter has been disabled to avoid slowing down the IDE" message, that's time-based: the IDE sets a timer and if things get too slow if turns off highlighting.

You don't need _MEM to get the size of an array, just use LEN:

Code: (Select All)

DIM a(250, 250, 500)

bytes&& = LEN(a())

PRINT "Array size in MB:"; bytes&& / 1024 / 1024