MP Area Load Time Issues (NWN2)

Neverwinter Nights 2 Toolset, Scripting, NWNX
#1

It has been a while since I last tested my module in MP mode, and while I expected a couple of issues, which I have now addressed, there is one issue that has come to light that is worrying.

After playing for a short while and visiting about five areas (2 outdoor, 3 interior), when leaving the fourth area (2nd interior) back to one of the exterior areas, the load time appeared to have slowed down, and by the time we tried to leave the 3rd interior, back to the same outdoor area, the load time became prohibitive - we gave up waiting after around five minutes.

This has never happened before and I can find no reason for the issue. The same interior to exterior load time in a SP took only ten seconds, even with a larger party and longer into the game. In the past, the same transition took only around 20 seconds in previous MP play.

Please can someone offer some advise on what may be happening and how to resolve this. I am going to continue searching. I have tried disabling area entry scripts and heartbeat scripts, but nothing appears to make any difference.

Thanks.

EDIT: During the load screen, the area says “successfully loaded” and the bar travels to around the half way, then stops. The next section, the part that says something like “area 1 update” takes the longest time, and the second “area 2 update” only takes a little time.

Therefore, does anyone know what the “area phase 1 update” is doing, as this may be the key?

#2

I take it this happens in one Module? (ie. not an inter-module transition)

Check if it happens in a different Module also …

The load messages are completely arcane to me.

1 - area loaded successfully
2 - update 1 done
3 - update 2 done

 
but i have no idea :\

2 Likes
#3

@kevL_s

Thanks for commenting … Yes, only one module, not a “mod to mod” at this stage. I was just play testing module one in MP mode again because of the alterations I have made.

As I say, the same module area transition is 10 seconds in a SP and (so far the best achieved) one minute in MP. In one session, the transition did not even finish and we had to alt-tab and force exit.

I am trying to minimise the code around area transition, but even if I remove all scripts to do with area transitions and heartbeats, the time only improves by around 2-3 seconds, so I don’t believe it’s anything to do with that side of things.

At the moment, I edging towards the number of “objects” in the area are the main impact - as there appears to be some correlation - not perfectly so though.

The thing is, I did not have this problem in the past, and so unless I have added more objects since then (which I do not believe I have - especially none to interiors anyway), then we should be around the same.

I am going to keep searching, as it does help me clean up some stuff anyway, but something seems off to me. I may reluctantly end up removing some objects or try adding some “empty” areas to see how they compare.

I’ll keep this post updated. Thanks!

EDIT: I just did a quick test comparing level of objects in an area and it is definitely that! A blank area loads almost instantly, and as I add objects (around 60 on first test), the load slowed down. I imagine “100s” will be comparability slower - and may well be causing the issues I am experiencing. I will keeep testing though just to be sure.

image

EDIT: OK, I am definitely starting to see a pattern here … Simply based on the number of objects, load times in a MP game are 0.04 seconds per object roughly speaking.

However, if anybody wanted to analyse the data further, then certain object types may be weighting the load times.

Here are my results for five areas (2 x outdoor at the top) and three indoor below. I also discovered that there is a VERY HEAVY WEIGHTING on any objects that have been created into an area since baking. E.g. Where I use created placeable for my weather system. My first guess is that leaving them in an area after creation causes significant additional workload for the system in a MP environment. I am going to try to work on a destruction prior leave to see if that helps return to area load times where they would be recreated.

image
image790×580 139 KB

The load times for each area have been added at the bottom of the images. All these times have been calculated with the WEATHER SYSTEM turned off.

Next, I will come back and show comparable load times when playing in SP, but even with the WEATHER SYSTEM on!

OK, so here are the SP loading times … EVEN with the weather system on …

NEShoreline 2s (v10s MP)
New Edgeton 5s (v23s MP)
NESanctuary 3s (v9s MP)
NEGuildhall 4s (v 31 MP)
NEBloated 3s (v 18 MP)

Two more interesting ones … Holy Mountains jumped to 31 seconds with weather units on.

image

And a couple of others …

image

#4

also be wary of (non-environmental) objects that are flagged Dynamic.

And yes it’s a good idea to destroy objects in an area after the PCs leave, if they are going to be re-created the next time PCs enter that area.

1 Like
#5

@kevL_s

Thanks for that tip - I will double check those now … I will also look at implementing a seek and destroy function when leaving. Ironically, I thought it would have been quicker having left them there so as NOT to have to recreate them, not realizing the significant load impact.

BTW, I assume we are talking placeables set as dynamic collisions when they do not need to be.

Assuming (for now) the “most” problematic are to be the NEGuildhall (interior that does not even use my weather objects) and takes 31 seconds to load in a MP game. It only has 2 “dynamic collision” placeable objects. So I would not have thought those two to be an issue …

So, back to the delete step by step in a test area copy to see what makes the most significant impact … My guess at this stage is the “Effects” as they appear quite high … will report back … (EDIT): Although I may test MP load times of a few more areas first to see if anything else stands out …

1 Like
#6

50/50 - you need to measure the performance impact (which you appear to be doing)

creating/destroying/re-creating objects should be kept really tight; a naughty routine could be spawning a bunch of redundant objects … eh :|

yes, placeables with dynamic collision (benefit is no walkmesh cutter is needed, but they impact pathing performance (moreso than area-loading or so id think - unless one of those area updates is ‘optimizing’ the walkmesh for realtime pathing which is quite possible idk.))

( dynamic collisions should only be used on objects that get spawned in/out, when we can’t adapt the walkmesh )

#7

Yes, I only use then for objects that are designed to block until destroyed normally … or add “realism” in some way (if they can be destroyed) to allow path to their spot. (BTW, I will continue to report findings in the previous post to keep all info together.)

#8

k, as you prefer

Also keep an eye on any GetFirst/LastPC() loops that are incorrect … GetFirst/LastEnteringPC() etc etc

#9

so it doesn’t appear to be the weather system … again, Loops onEnter (?)

1 Like
#10

Except that when I turn the system off (and have also tested in blank areas) in a MP environment that adds 65 weather objects between loads, the load time increases significantly. Instead of 2-3 seconds (for extra objects by normal estimated 0.04 calculations), it added about 2-3 times that to around 5-6 seconds. And when testing going from an interior (Guildhall) back to an exterior New Edgeton (with weather objects), the load time doubled from 30 seconds to 60 seconds, compared to around only 5 seconds in a SP game with them employed.

SP (With weather objects): 5 seconds.
MP (No weather objects): 30 seconds.
MP (With weather objects): 60 seconds.

Further testing with a dynamic seek and destroy with full working areas ill provide better data.

1 Like
#11

is that the sort of MP loadtimes that you had before this issue started ? ~30 sec? Is there that much difference between SP (5sec) and MP?

#12

Unfortunately, the last time I was able to test MP fully (ignoring the latest play through) was a long time ago, and was probably even before the weather system and many other additions. However, switching off scripted systems that dis not involved object appears to have negligible effect. Therefore, I cannot recall the exact timings previously, but they were significant enough this time around for me to notice the extra load times.

I think I may have had the longest with the Ancient Crypt (which I just tested again and posted results above) … It’s an interior and took 68 seconds to load (with just the host MP loading). So this had no weather units involved and was all to do with objects - all 1201 of them I guess!

The largest exterior I have is the other one I posted, which took 19 seconds to load without weather units and 31 seconds with.

So, there appears to be some kind of combination of objects that cause the differences, as my timings with a smaller exterior area than this one (and so less weather units) actually had a worse penalty with them than this area.

Actually, I will double check the smaller area and weather unit timings … one minute … Actually the diff between with and without is 23 v 31 by the looks of it … about 8 seconds, instead of 2-3 expected if considering 0.04 per object. The 30 v 60 seconds I recalled was from a previous test that was from an actual MP game my wife and I are testing. (It was the same transition though.) In fact, one time, the same transition never finished, as we gave up waiting after 8 minutes! Something seemed very dodgy at the time. In that game, we had more PCs between us and it involved two computers instead of just the host … so the number of computers involved may make these times even worse. :frowning:

#13

Looking at some of the results, I find this comparison interesting …

image

They are both outdoors, but the larger area loads quicker than the smaller (both without weather objects). The only real significant differences where the smaller area scores higher (ignoring the two lights more) are for the doors and placeables … interestingly, it was the doors that gave occasional issue in the game too, with a slight delay sometimes. Maybe doors are slightly more weighty in the equation of load times, followed by placeables?

With 66 doors and 319 placeables in the ancient crypt interior area … that would explain its 68 s load time! Thankfully, it’s a large area that once inside, you are there for a while.

Checked this by disabling all area enter scripts during initial tests.

#14

have you taken a close look at the Module loadscript? i know it shouldn’t run for area transitions but maybe comment out some code and see what happens

#15

OK, I will take a look at that now … one minute.

RESULT: Unfortunately, no difference … I basically only have some global vars set in there anyway. :frowning:

I have done a lot of tidying in these opening scripts today (fixed some database stuff there too, which had not been checked for some time), and I’m glad of that at least … I think the weather unit stuff appears “significant” for me to work on and see if that does help … and then I think it may just be keeping the number of objects “low” especially with doors and placeables.

Maybe aim for 1000 - 1200 objects maximum (outdoors) and maybe 400 - 500 objects maximum indoors (subject to doors and placeables used).

As I say, these figures can easily be surpassed within a SP environment, but they appear to impact heavily in a MP environment. I will keep playing around for a while (especially as I develop module 2) and see if I can find if any “bad object” appears to be damaging things.

Anyway, thanks again for helping out - let me know if you have any more thoughts… :+1:

EDIT: Oh and before I forget … I think the number of henchmen and/or summons (or even the number of faction members in general) may also have a bearing on these times. I may try experimenting on with and without summoned creatures and maybe force remove them on a transition for MP games if the load times are significantly improved or not …

:sleeping:

1 Like
#16

@kevL_s

Having slept, I had a thought … Perhaps the area “Phase” updates are any changes to objects since original bake and/or last loaded times. This is what it may appear to be to me. I tried putting that theory to test and added some switchable lights to two identical areas and jumped between the two. Here are the area stats.

image

For 20 object we can expect just under a second additional load time (working to the 0.04 per object as a very rough indication).

UPDATE: OK, I was going to post my results, which appeared to support the theory. However, during testing, I decided to also reduce the area entry fade from black so I could get a more accurate result of area entry. And interestingly, with the fade from black time significantly reduced, the load times appeared greatly improved! I have no idea if this is impacting it, but will continue to experiment with all this and come back to you. (#)

By the way, the results of my previous alterations are in with respect to the actual MP saved game, and here they are …

Original Entry time (Outdoor to Indoor): 78 Seconds.
Weather unit and summons fixed (Outdoor to Indoor): 45 Seconds! (*)

Original Entry time (Indoor to Outdoor): 45 Seconds.
Weather unit and summons fixed (Indoor to Outdoor): 30 Seconds!

(*) This is the figure I am trying to improve with respect to area load phase slowdowns. If the latest FadeFromBlack investigation does not bring any results, I will continue to investigate the “area changes” angle.

(#) OK, this is a weird result … Stopping the FadeFromBlack function improves my “test” areas lighting tests, but does nothing for our current existing saved game. However, in my test areas, I have prevented any placeable/light updates upon an area load, which means all object alterations are made “manually” (by the player) rather than by scripts. i.e. Lights are “switched” on or off according to a variable via scripting compared to a player interacting with a switch on game. This reminds me of this function …

image

So, now I think this function may be something I need to consider more closely for a MP environment. Again, I will get back to you … although, I am not sure if this function actually does anything with actual lighting does it? I’ll add it and see if it makes any difference.

#17

g’morning Lance,

re. RecomputeStaticLighting()
i doubt it does anything given that graphics was completely overhauled for Nwn2 – but maybe

1 Like
#18

@kevL_s

Yes, I abandoned looking into that one after I read it had a bug anyway.

And, after further testing, I don’t think there is much other “time” added due to light switching anyway, beyond extra objects (flames added vfx per torch kind of thing), which still does add time of course. Also, I realised that there were extra objects added between area loads with respect to creatures (villagers) too in certain circumstances. A second here and a second there soon add up.

I was trying to investigate why an outdoor area load varied between 23s (on first entry) and 30s (on a subsequent entry), and just had to conclude it was mainly influenced by number of objects again, rather than anything direct with scripting - apart from what scripting added or removed in the way of area objects.

So my next step (if I really wanted to pursue this) is to consider removing other potential “dynamic” objects between area transitions … like villagers, which I have also currently left in place if they were not needed to move.

Or, just reduce the number of objects in each area anyway. Actually, I will try that with my test areas - throw a lot of different object types and see which ones make more or less difference. If the difference is significant, then I guess I may be committed to changing as many placeables (that I left to figure the walkmesh) with environmental objects and a walkmesh cutter instead.

In some ways, I kind of hope it won’t make any difference, because I feel cheated that just because the game is MP that it should have such an impact. :innocent: Anyway, I will check and update again … Back soon …

EDIT: I forgot to add that quickly re-enabling a fade for a MP environment seems to help in some situations, especially where light objects are involved. However, I have tested so many scenarios that my head may have just imagined it. :exploding_head:

UPDATE: I am definitely beginning to think the presence of creature objects adds a lot to load times. Even though I have their HBs effectively “off” when PCs are absent, they eat into the load time. Adding only 12 villagers adds around 10 seconds by the looks of it! MISTAKEN: Oh! I just tested again and the time did NOT improve when removing the creatures, and so there must be something else … still searching …

#19

sounds like a good direction for figuring things out

The main concern is that an area failed to load after ~5 minutes. Something is seriously borked after ~2 minutes /imo

 
ps. Don’t forget that creatures can be disabled when PCs leave an area w/ SetScriptHidden(oCreature, TRUE, TRUE)

and reenabled w/ SetScriptHidden(oCreature, FALSE) when PC(s) re-enter the area.

 
Perhaps the problem is just too many objects (of type).

#20

@kevL_s

I am hoping that the 5 minute problem was due to a number of “smaller” issues that I have been dealing with along the way that may have just confused the area load rather than just slow it down. I am thinking that because the area loads are slower, the timing of some other checks were/are interfering - and there was definitely some database code that was old and needed updating.

For instance, I believe I may have traced that extra 5 seconds delay due to the weather units setting up again and adding objects before the area (in its mind) has finished loading - even though, my additional area load script takes place a few seconds later. < At least that is what is supposed to be happening, but I will take a closer look at that as well now. E.g. If I comment out the weather unit creation (which rebuilds the units on entering the area again), I regain 5 seconds on the load time - I really need a way of forcing the function to wait until the game believes it has finished loading the area.

Hiding creatures - Yes, I had started to do that with some creatures, but as the “villager” code was quite ingrained (and finally working as expected), I did not want to start messing with it - although it is still an option if it comes to it again. Although, I think they were not making that much difference in the end.

Too many objects … yes, I am still going to test the types in my test areas … but I am trying to “fix” the weather unit rebuilding time first, so it no longer interferes with the current loads.

Back in a bit …