TR’s Un-Linked Lists

This is a library of functions that emulates the linked list data structures Stacks, Queues and Rings using local string variables. The intended audience for this project are intermediate to advanced NWScripters. Due to two functions that are part of both flavours of NwN but not NwN 2, this is NwN only. Please regard this as a “pick & mix” submission. You are encouraged to extract those pieces that you need for your own project.

Should you download it, please read everything starting with the readme and the manual. Also, please read the project page for the reasons behind this project.

Is this of use to you? Maybe. It depends on the project you want to include it in. It is available here -

TR’s Un-Linked Lists

Fully tested and working as long as you follow the manual.

TR

4 Likes

Admittedly, I didn’t read the massive PDF so this might already be covered there, but I did skim the nss and I have a suggestion.

The jist of what you are doing (and please correct me if this is not true) is that you put all the data into a single string, using a special delimiter character/substring, and then implement all operations as string parsing methods on that single string.

This implies a very surprising property of stack/queue structures in that both push() and pop() operations are O(n) - i.e. they get progressively slower the more elements there are in the structure. (Count is O(n^2)) This is not the case in any traditional implementation of these structures, and most problems that are optimally solved with these structures will not be properly applicable to your implementation.

I would suggest you make heavier use of multiple local variables. Getting/setting a local variable is an O(1) operation*, so putting each element in their own local and calculating the name somehow is going to reduce the time complexity by an order of magnitude.

PS: You also make over a dozen string copies per push/pop operation, and over a dozen length checks, all of which are O(n) and get progressively slower the more elements there are.

* In NWN:EE. In 1.69, it is O(N), but still much faster than string parsing.

If you had bothered to read the first paragraph of what I wrote on the project page (never mind the manual), as I suggested in my original post, you would know that code efficiency was never the reason for creating this. Just for you here are the tldr reasons.

If you had bothered to read the first paragraph of what I wrote on the project page (never mind the manual), as I suggested in my original post, you would know that code efficiency was never the reason for creating this. Just for you here are the tldr reasons.

  1. To illustrate the versatility of the string data type beyond just creating relatively simple messages.
  2. To keep the scripts as simple as possible, including eliminating the need for multiple variables with all the attendant house-keeping that involves.
  3. To have readable code so scripters can easily use this system and can see the techniques involved.

In that order.

What it is not.

  1. Absolutely robust - read the manual.
  2. Absolutely the most efficient method that shaves milli-microseconds off each functions runtime. It uses string functions, which are not the most efficient.

Had you read the function names (again never mind the manual), you would know that the terms I use are Push and Pull (same as in 6809/68000 assembly - self taught) which are more correct in UK English as they are opposites.

Now I can’t refute or otherwise your statement on the efficiency of this code. That is because not all of us have been blessed to work in the i.t. industry.

  1. So what does O(n) actually mean?
  2. Also, just how many items do you anticipate needing to store?
  3. What are these traditional problems that won’t be solved by the methods that I used?

I take it by copy you mean passed as parameters. That is a consequence of NWScript only passing by value. To put it bluntly, so what? As already explained this system was never designed for maximum efficiency. What you fail to mention is that the strings concerned (for push) are the new data to be added (could be quite small depending on the type of data to be pushed), the name of the local string (again probably small) and the user choosable delimiter (normally a single character. The actual local string is only “copied” once and that only in the body of the core push function. Anyway, have you never heard the maxim (taught in colleges and universities I believe) - “Speed is a hardware problem. Maintainability is a software concern”.

As to your suggestions for “improvements”, quite frankly I will ignore them. My software works, it does what it was designed to do and, unless any bugs are found in use, I will not change one byte of it. I never made any claims about how efficient (speedwise) this system is. If you require something more efficient, why not create your own and post it on the vault too?

Apologies if you find this reply patronising. That was not my intention. It is fuelled to some extent at the frustration of having to correct your misconceptions that wouldn’t have arisen had you but read what was suggested to you to read.

TR

I’ll just ignore the wounded tone. Big O notation - Wikipedia (I find the explanation in the paperclip’s post clear enough, though he didn’t mention some 1.69 oddities that might be relevant.)

I glanced at also, don’t understand what’s trying to be accomplished that any of the hash/array libraries here can’t do. Aside from using sql or json in ee, since that’s what leapt to mind.

I have to wonder the same thing; it seems extremely bad form to me to hen-peck people’s code over hyper-optimizations, to have it said. In practical terms the actual speed gains from these optimizations will not be noticable on any list that NWN is going to be parsing with anything approaching efficiency to begin with; and at the point where that kind of scale is in play, one is much better served leveraging the sqlite functions.

1 Like

What. In what way is that a valid response (from a moderator, no less) to

[…]and I have a suggestion. […]I would suggest you make heavier use of multiple local variables.

It was a very concrete, actionable suggestion that was delivered in the most neutral way possible. Is this kind of information no longer desirable in this community? Or did I fail to observe some message protocol that would not have triggered this?

Since, last I checked, these forums are meant to be educational for all readers I will elaborate a bit more, then I’m out.

The common mantra that performance in nwscript doesn’t matter is correct, but sometimes misleading. It almost never matters if your code takes 10us or 50us or even 1000us to complete. Also, if you speed up your code by 10x, it is almost never noticeable. What matters is how your code scales. This is what the Big-O notation means. Roughly, for a data structure operation this can be broken down into:

  • O(1) means that regardless of how many elements are already stored, this operation always takes the same amount of time. Adding the 10000th element takes just as long as the first. This can be 1us or 1ms, but it’s always the same.
  • O(n) means that the operation gets progressively slower the more elements (and in this library’s case, the size of the data stored also matters) there are. If the first element takes 1us, the 10000th can take 1ms or 5ms or even 20ms.
  • O(n^2) (or O(n*n)) means that the scaling is quadratic. If the first element takes 1us, then the 100th could take 100ms and 300th could take a whole second.

The last one is particularly problematic because it can creep up on you easily down the line. Maybe you store all the equipped items from a PC (up to 17) and it works fine in testing (17^2 = 289), but then someone plays your module in MP with a party of five and you get (5*17)^2 = 7225 which makes the module unplayable. In particular, a general purpose library meant to be utilized by others should avoid these pitfalls or at very least document them prominently.

O(n) itself is not a huge deal when known, but it is very easy for a program to turn “accidentally quadratic” because it is not clear that a certain operation is O(n). Consider the following code:

for (i = 0; i < GetStringLength(str); i++) {...}

The hidden gotcha here is that GetStringLength() is an O(n) operation - it has to walk the entire string to find the position of the NUL byte to deduce the length. Since we are calling it every iteration - and we have N iterations - the total amount of checks is n^2, meaning that for every extra character in the string, the process takes exponentially longer. Your code might work fine for a string of length 100, but go boom when length is 200.

The gotcha is easily solved by storing the value in a variable like int n = GetStringLength(str); and then checking that variable instead. But the coder has to know about it to do it (or to cache every function call even though 99% of the time it’s not necessary and is exactly the premature optimization we discourage).

Same applies to this library; if you look up any traditional implementation of a stack (technically, a ‘stack’ is an interface, not a data structure. Data structures are e.g. arrays, linked lists, binary trees, etc, all of which can be used to implement a stack), you will see that both Push() and Pop() operations are O(1). Deviation from that is unexpected. All unexpected behavior should ideally be avoided, and failing that, well documented (See “Principle of Least Astonishment”). I have suggested a way to avoid it. You want to avoid it precisely so that the user of a library doesn’t go ‘accidentally quadratic’.

This is all doubly true if this library is meant to serve as some kind of tutorial as seems to be implied. When writing a tutorial you take on additional responsibilities that the knowledge you transfer onto those who would consume it is reasonably correct and complete.


EDIT: Heh, I wanted to quickly compare this approach to the simplest local var implementation below and I accidentally ended locking up my game for over 30 seconds.
image

Although that is with the data stored itself being pretty large. Here’s a run with a more reasonable data size:
image

void push(string data, string stack) {
    int sp = GetLocalInt(GetModule(), "sp:"+stack);
    SetLocalString(GetModule(), IntToString(sp) + stack, data);
    SetLocalInt(GetModule(), "sp:"+stack, sp+1);
}
string pop(string stack) {
    int sp = GetLocalInt(GetModule(), "sp:"+stack) - 1;
    string data = GetLocalString(GetModule(), IntToString(sp)+stack);
    DeleteLocalString(GetModule(), IntToString(sp)+stack);
    SetLocalInt(GetModule(), "sp:"+stack, sp);
    return data;
}
1 Like

Turns out if you’re rude to people you tend to get a rude response. I ready your post as extremely condescending, myself.

I don’t see any rudeness in Sherincall’s reply. It looked valid and made reasonable points. Constructive criticism is not rude. It does look like TR took it as an attack which is a shame. I try (and sometimes fail) to assume positive intent. We are all friends here, which is not the case in other parts of the interwebs…

@Maiyannah I assume you did that on purpose to illustrate the condescension you thought @clippy used? :slight_smile:

The points have been made - let’s move on.

MODERATOR

1 Like