Adding memmem() to QB64PE

[Petr]

@Sanmayce
Yeah, I downloaded the whole thing, there's really only an md file in one zip, but the source code in C is actually in chatGPT_vs_GLIBC_vs_musl_vs_Railgun_round4.zip. It was probably expected that the source would be for everything and if it wasn't found in the first ZIP file, then no one tried the next one. If you name a ZIP file as source code and there's only an md file in it, don't be surprised that no one bothered to look further. That'll just be your mess on github. Thanks for sharing.

[SMcNeill]

@Sanmayce
Yeah, I downloaded the whole thing, there's really only an md file in one zip, but the source code in C is actually in chatGPT_vs_GLIBC_vs_musl_vs_Railgun_round4.zip. It was probably expected that the source would be for everything and if it wasn't found in the first ZIP file, then no one tried the next one. If you name a ZIP file as source code and there's only an md file in it, don't be surprised that no one bothered to look further. That'll just be your mess on github. Thanks for sharing.

Aye; that's what I did.  Saw the github was a single MD file.  Downloaded the source file and saw the same singular readme.md file, and then just shrugged it off.  It's not my job to play Sherlock Holmes and go hunt down someone else's code to test and try.  If it's not easily found, then I'm not chasing after it like a bloodhound to try and find where it's hidden.

If you want to share or show-off something, at least make it where folks can find it and oooh and aaah over it easily, without hiding it behind locked doors where they have to sort out a puzzle to get in to view it.

[Sanmayce]

Let's see what the current pseudo-memmem i.e. instr offers, the code below is taken from the QB64PE repository (from the releases section):

Code: (Select All)

int32_t func_instr(int32_t start, qbs *str, qbs *substr, int32_t passed) {
    // QB64 difference: start can be 0 or negative
    // justification-start could be larger than the length of string to search in QBASIC
    static uint8_t *limit, *base;
    static uint8_t firstc;
    if (!passed)
        start = 1;
    if (!str->len)
        return 0;
    if (start < 1) {
        start = 1;
        if (!substr->len)
            return 0;
    }
    if (start > str->len)
        return 0;
    if (!substr->len)
        return start;
    if ((start + substr->len - 1) > str->len)
        return 0;
    limit = str->chr + str->len;
    firstc = substr->chr[0];
    base = str->chr + start - 1;
nextchar:
    base = (uint8_t *)memchr(base, firstc, limit - base);
    if (!base)
        return 0;
    if ((base + substr->len) > limit)
        return 0;
    if (!memcmp(base, substr->chr, substr->len))
        return base - str->chr + 1;
    base++;
    if ((base + substr->len) > limit)
        return 0;
    goto nextchar;
}

An epitomy of basic functionality and the 2GB limitation due to int32_t start (in fact due to using str(ings) instead of mem(ory)).
Not a modern function, no doubt.
The modern and open QB64 should have 64bit functions in its arsenal.

Just tried on my fastest laptop instr vs (Railgun from the GitHub repository):

   

On the screenshot I see 4x speed up, speed is religion, but sharing such a gem seems a mistake, it looks bad (especially when a "maintainer" calling me a spammer and bot) for QB64 community , shame on you @SMcNeill

[SMcNeill]

Oh nos!  I has been shamed on!

I shalt never be the same. 

[Sanmayce]

@Sanmayce
... If you name a ZIP file as source code and there's only an md file in it, don't be surprised that no one bothered to look further. That'll just be your mess on github. ...

You obviously don't know that adding those two files at the bottom of each release are auto-generated, and it is not my mess but the GitHub creators, maybe even Linus' mess.

[Petr]

@Sanmayce Okay, I really didn't know that. In that case, I apologize.

[Sanmayce]

I did quite a thorough benchmarking spanning 18 tables (3 machines, 2 compilers, 3 corpora).
Also, asked 2 AI agents to evaluate all the data:
https://forums.fedoraforum.org/showthrea...ost1891202

Since C sources are kinda scary (but excellent resource for further tweaks/improvements) they have been wrapped into a header file for QB64 use as external functions, I tried on those 3 laptops of mine the vectorized Railgun_Nyotengu against the 32bit-limited instr(), in short, 4x to 5x dominance for the truly 64bit function (actually being 256bit but having 64bit addressable blocks, it could become 128bit just by commenting out 53rd line in manatarka.h and uncommenting 52nd line):

The ARM-like Intel CPU - the lowest wattage (4.5W up to 7W) in action:

   

   

Now, the AMD Zen, (with 15W up to 25W):

   

   

Another problem, a nasty one, stands since time immemorial - Quick lowercasing (either in-place or out-of-place) of the blocks being searched in order to maintain huge speeds by going all vector. In the .h header there are such functions.

Enfun!

[Sanmayce]

And finally,the 3rd laptop, the fastest laptop I got TDP: 15 W (MAX 25 W):

   

   

Simply vectors rule for short and medium needles.