The Arclight Project

[leo_x]

Overview

The arclight project is a tools-based journey through Neverwinter Nights from first principles.
Scrolldown for some screenshots. It’s also an attempt to create a technical vision of what tooling should be. The goal is learning, the joy of building in NWN, but the terminus is, clearly, “Neverwinter Nights 3”.

This is a large work-in-progress:

• Somethings aren’t complete, I’ve taken a breadth first approach so as to get a better sense of usage.
• Somethings are insanely basic, but that’s good because anyone reading here now could understand the lot of it.
• Somethings are still in the Diamond era.

I’m posting here in case there’s anyone out there wanting to rekindle the grand ambitions of the early days of NWN, there are tons of talented people not on the dev team.

This project isn’t intended to compete with any other project. Somewhere in one of the repositories
is a folder with historical screenshots dated 4/18/13. Three years prior, almost to the day,
of an important date in NWN history.

As stated in a previous post, if anyone is offended by what I’ve said, harsh opinions or whatever, in the past or due to my opinion of what happened at the start of NWN:EE, please take this as a public apology if you have any interest in this project.

The Foundation

The foundational library must be performant, cross-platform, and written in a language
that is easy to integrate with other game related libraries. It must also be written in
a direct, simple, straight-forward, low-abstraction style that any junior developer could
easily understand and contribute to since none of this is rocket surgery. For this the project
has selected C++. This is rollnw.

The Text Format

The text format should be a mirror of the object and not a mirror of a serialization format.
It should be trivial to use in any language, on any platform, in any scenario (i.e. a website, a linter,
or something that no one would have thought of), in a manner consitent with the C++ and Python libraries, and have the ability to create schemas. For this the project has chosen JSON, since a number of languages allow direct instantiation of JSON to objects and if they don’t most offer some method of trivial de/serialization. Example.

The Adhoc Tools

A diverse community of people working on different projects with different needs and goals,
requires the ability for any person to have the tools to make adhoc, one-off, tools. For this the
project has selected Python, since it’s the most popular programming language in the world and its
support for notebook based development (programmatically making a dialog). This is rollnw.py, distant predecessor pynwn. Available
on pypi: pip install rollnw.

Tools

The main tool and its widgets must be stable, performant, cross-platform. For this the project has
chosen Qt and Krita as a rough architectural model (this project is light years from that).

Other tools usable outside of the context of editing a module are available:

• arclight-tools - Arclight and other tools written in C++ and a collection of Qt Widgets.

The Renderer

Currently the built in renderer only supports the most basic models. No dynamic models, emitters, etc, etc…yet. It uses OpenGL 3.3 just as NWN:EE does. Ultimately, the goal is to replace this with a more advanced graphics API. Or if Project Fresh Look for NWN2 is realeased, that will be investigated too.

Features:

• Basic Meshses
• Skin Meshes
• Un-lerped Animations
• Very basic area viewing

The Widgets

The below examples are just the beginning.

Area View

Insanely basic area viewer. Terrible camera. Very little graphics know-how, but a start.

Creature View - Stats

For fans of axs’ modified toolset this layout will be very familiar.

Creature View - Feats

Add remove feats with sorting and a fuzzy search.

Dialog View

A tree view for NWN dialogs.

Container View

A view over a NWN container (i.e. erf, key, zip).

[night76]

Unfortunately there are a lot of interesting projects, both NWN and NWN2 out there that are no longer supported. Some are even at a good point and would just need people with a bit of will to be able to move forward and improve one of our favorite games to take it to even greater heights.
From what I’ve seen following them a bit, the main problem is people’s pride and fear of seeing their work taken away to the detriment of the community.
We should understand that projects of this magnitude cannot be managed and carried out by single people, but by a group of developers with different skills because, apart from geniuses I don’t know, it is truly impossible to have one man projects of this caliber.
I had already seen the project you propose, like others, but as said above it needs competent people to be able to continue… so I’m sorry but I’m afraid that your appeal will fall on deaf ears…

[clippy]

@leo_x

The goal is learning, the joy of building in NWN, but the terminus is, clearly, “Neverwinter Nights 3”.

Forgive me for only skimming the post and the repos, so if this is explicitly called out as a (non-)goal, just say so, but…

I’d like to suggest a more intermediate goal than NWN3: Once you have the tools, formats and workflow that works better than NWN’s, you can make a NWNX(-like?) adapter such that nwserver understands your backend, but then translates that into a netcode stream that existing clients can interact with.

This can be done incrementally, where you can piecemeal replace bits of the server logic, while keeping clients as a terminal only. It might be possible to entirely replace nwserver.exe eventually and have something completely custom that speaks the NWNEE network protocol (I’m not sure off the top of my head about encryption and MS listing, might still need to call into nwserver for that).

Then (and IMHO only then) you can look into building a client. The client could also speak the nwn netcode, but that’s not necessary, it could be two separate protocols, and you could have two different clients connecting to the same server.

All of this is obviously dozens of engineering years out, but maybe clearer goals and roadmap can help rally people around a cause.

[leo_x]

I take your point to heart and agree with your general assessment, but to quote Beckett “what’s the point of losing heart now? That’s what I say. We should have thought of it a million years ago in the nineties.”

[Maiyannah]

I would kill for a toolset that is more stable than Aurora. Or even … one that gives error messages that one has any idea what’s going on with them.

[leo_x]

I appreciate your feedback. The Neverwinter Nights 3 thing was a bit tongue-in-cheek, but we will dream big on this thread. Every project that starts today or in the recent past like Fresh Look should all be guided by some vague future possibilities. Since NWN:EE is still actively developing it doesn’t matter so much on this side of the community.

My concern about network compatibility and saved game compatibility, is that they would be the biggest turds in the punch bowl, to put it bluntly.

I agree 100% that all these projects need to cross-cut. I don’t think people even realize how much is already available to us now, even just conceptually. But, maybe, is missing the right companion piece. Take Nasher, imagine a toolset that can work on an arbitrary folder structure, transparently over gff or json, that gets packed at the end into an erf. Just a masher config and a module.ifo at root.

Or take in your own case with nwnccc a lot of people just didn’t get it, couldn’t get it really. Imagine if there was initiative here called “The Vault is the HAK”. Where the old projects are laid down, and we just have “an armory”, “a bestiary”, etc. where people subscribe and can download your tools files. People would get it.

I don’t think time is of any concern, would anyone believe NWN:EE is around 8 years old??

[leo_x]

I’m curious if anyone has any thoughts on palettes? Or if any discussions are/were had to improve them in NWN:EE? I’m assuming there is a general consensus that currently they stink.

I was thinking a vague filesystem model and an nwnexplorer model for module dependencies would be best, like so:

But… there are multiple consumers… builders and DMs.

nwn2 folks, if you care to, feel free to share your experience, I can’t load the nwn2 toolset in windows 11 on steam or gog anymore, so I couldn’t cross reference to see if Obsidian took a better approach.

[Semper]

@leo_x

I can’t load the nwn2 toolset in windows 11 on steam or gog anymore, so I couldn’t cross reference to see if Obsidian took a better approach.

You need to install legacy Directx9c for the toolset to work.

[Maiyannah]

If I changed anything about palletes, I would change it so I could change the categorization without having to pull teeth.

[leo_x]

Thank you for that!

[clippy]

@leo_x

I’m curious if anyone has any thoughts on palettes?

Forget all about them? Just use tags and fuzzy search (of tags, name, resref, etc). Ideally also a preview window, including the model image, when you hover over the entry in the list.

[Maiyannah]

Just a search, at all, would be such an improvement in any large module.

[leo_x]

I’ve added some new stuff, small highlights

Local Variable Editor:
I’ve opted to highlight duplicate entries with an error message and a warning message for any string variable that can trivially be converted to an integer or a float (a bug from the past). It makes me wonder what other ideas are out there for improving local variable editing, maybe local variable sets (like script sets)?

A jupyter notebook showing how to create icons from an item. I’ve left all the notebook output in git to better illustrate how it would look when run. Per-part coloring isn’t supported yet, but should be pretty simple. Screenshot:

If there’s anyone out there, passing by, that might be interested in this project, hopefully, this reiterates some of the goals of the project:

• Accessibility: plt decoding on cpu is super simple, understandable by anyone, there is no rocket surgery to any of this. Implementation should aid the understanding of the file format.
• Edification: jupyter notebooks are a huge win for communicating stuff to others.
• Foundational: Things that are outside the current scope of the project are still enabled: imagine a web app that shows servervault characters with their equipped inventory. Too many projects in NWN, even great ones, are never built on…
• Shareability: Clippy mentioned in another post recently about similar projects to this: anyone working on those is welcome to take anything they find here to further their own projects.

[leo_x]

I’ve been thinking about rules systems and the shape they take. I’m curious if there’s anyone else out there that has, in some hammock somewhere, had in mind how they might wish the game looked or how they might implement a rules system.

Looking at what I can find in NWN:EE, Oorth’s/Plenarius’ NWNX Feats/SkillRanks plugins are conceptually the most promising, since it allows both programmatic (nwscript) and configuration (2da) points of customization. I don’t personally care for ruleset.2da at all, nor for its initial implementation. Don’t get me wrong having it is better than not.

Does anyone know of anything else publicly available? The dot net folks seem to be guided more by having C# be a super-super charged nwscript? It looks as tho NWN2 doesn’t diverge much from NWN1 with regard to rules?

I started in this project with a generalization of Oorth’s/Plenarius’ plugin. Starting first with notion of modifier sources: i.e. ability, class, combat mode, environment, feat, race, combat situation (i.e. flanked), skill, etc. Then all those things which can modified, NWNX Feats has a pretty good list, it just needs some additions like critical hit related stuff.

The place I’ve come to is every modifier of the slightest complexity ends up as a function, because there is no way to express it in terms of configuration and nwscript is way too weak to express it programmatically. Maybe a `RunScriptChunkWithReturn" or some hack around could work?

In any case, I ended up with something like this:

nw::ModifierResult class_stat_gain(const nw::ObjectBase* obj, int32_t subtype)
{
    auto cre = obj->as_creature();
    if (!cre) { return 0; }
    if (subtype < 0 || subtype > 5) { return 0; }
    nw::Ability abil = nw::Ability::make(subtype);
    int result = 0;
    for (const auto& cl : cre->levels.entries) {
        if (cl.id == nw::Class::invalid()) { break; }
        result += nw::kernel::rules().classes.get_stat_gain(cl.id, abil, cl.level);
    }
    return result;
}

/// ...

void load_modifiers()
{
    auto& rules = nw::kernel::rules();
    rules.modifiers.add(mod::ability(
        class_stat_gain,
        "nwn-ee-class-stat-gain",
        nw::ModifierSource::class_));
}

I’m thinking now that this isn’t better than my old project Solstice which aimed to re-implement everything in Lua with a completely new API designed around the strengths of a significantly more powerful language and runtime (LuaJIT).

I think now Roblox’s Lua fork Luau now supersedes LuaJIT as the better technological direction. I’m going to try to port Solstice to NWNX:EE to explorer that idea. Some of unique features it had at the time (roughly half a decade before EE), and all kinds of other stuff:

• hidden helmets
• custom combat modifiers for most attributes (i.e. race, class, etc)
• sticky combat modes
• custom combat modes
• custom special attacks
• custom can-use-class abilities
• custom max feat uses
• custom damage types
• custom innate immunities
• effect immunity percentages
• etc, etc, etc.

Some of this is now in EE, of course. And a historical throwback: at the start of EE, someone even tried to pitch this project to Trent Oster and Mark Brockington but with a ridiculous asterisk. I’ll let him tell that story if he cares to.

If there’s anyone interested in that NWNX plugin or pondering rules systems, feel free to chime in. Surely, there are people out there that have pondered this in the context of NWN:EE and NWNX:EE???

Like imagine some off-cuff JSON model, ideally all in separate files globbed into the rule system, like:

{
	"label": "MyWeaponMasterFeat",
	"name": 0x01000001,
	"uuid": "not gonna pretend I remember what a uuid looks like", // This is gets saved in the GFF/archive, freeing the world from 2da line entries?
	"modifier_type": "crit_range",
	"modifer_function": "something convertible to nwscript and useable with RunScriptChunk"
},
{
	"label": "UseableFeat",
	"name": 0x01000002,
	"uuid": "not gonna pretend I remember what a uuid looks like", // This is gets saved in the GFF/archive, freeing the world from 2da line entries?
	"on_use": "something convertible to nwscript and useable with RunScriptChunk"
	"maximum_uses": "something convertible to nwscript and useable with RunScriptChunk"
}

[clippy]

Since you asked…

…this is absolutely not usable for you in anything resembling its current form, but might give you some ideas. Especially if combined with Lua.

As a preface, I’ll note that I never liked how nwnx_{feats,skillranks} were designed. In general, I don’t like any of the NWNX features that offer config knobs to tweak the behavior. It means you’re always constrained by the behaviors the author has programmed in, and when you start adding more features to accommodate every module’s desired behavior, it gets really messy really fast.[0]

A better alternative would be low level hooks and the ability to plug scripts (chunks) as handlers for stuff. e.g. instead of a bitmask for when a feat is usable (outdoors at night only), you’d hook into a nwscript handler for

CanUseFeat(object creature, int feat, object versus = OBJECT_INVALID, location loc = LOCATION_INVALID);

This works fine for most uses, but if the feat check is done every time someone shoots an arrow at you, it can be a real performance drain in some scenarios. LuaJIT might make it negligible though.

Anyway, my hot take is that NWNX (or whatever) should either provide low level building blocks, or reference implementations. Then, a PW would take this reference implementation (in C++), fork it to a PW-specific NWNX plugin and just change stuff there. Rather than trying to abstract away every possible hardcoded behavior into a config knob, just let everyone hardcode their own desired behavior.

With that in mind, here’s a trimmed down example of how that could work for feats:

DEFINE_FEAT(555, GiantSlayer,
    {
        Name = "Giant Slayer";
        Description = "You gain +2 AB and +2 AC against giants";
    }
    int GetABVersus(CNWSCreature* c, CNWSCreature* target) override {
        return target->m_pStats->m_nRace == RacialType::Giant ? +2 : 0;
    }
    int GetACVersus(CNWSCreature* c, CNWSCreature* attacker) override {
        return attacker->m_pStats->m_nRace == RacialType::Giant ? +2 : 0;
    }
);
DEFINE_FEAT(123, Dodge,
    {
        Name = "Dodge";
        Description = "You gain +1 dodge AC if not flatfooted.";
        Icon = "ife_dodge";
        Prerequisites.Dexterity = 13;
    },
    int DodgeAC(CNWSCreature* defender, CNWSCreature* attacker, uint32_t flags) override {
        return flags & FLATFOOTED ? 0 : +1;
    }
);

With some macro magic, you can generate the C++ logic, the 2DAs and the nwscript API from that. While it does involve the extra step of compiling and then reloading the plugin, the basic editing of the feats themselves is much easier than dealing with 2DA+TLK+nwscript.

The actual implementation would look something like

int32_t CNWSCreature::GetBaseAttackBonus() {
    int32_t bab = 0;
    xForeachClass([&](CNWClass* cls, int lvl){ bab += cls->GetAttackBonus(lvl);});
    xForeachFeat([&](Feat* f){ bab += f->BaseAttackBonus(this);});
    return bab;
}

I can give you a fully working skeleton for this setup, but maybe better if you write one yourself first so you don’t get polluted by my ideas.

---

[0]: Sometimes, that’s the only realistic option anyway, such as e.g. clientside progfx.2da, but on a server there’s always alternatives.

[leo_x]

You are always welcome to respond to any post I make. Or anyone for that matter that would like to spitball ideas.

I feel like a config interface can help non-technical peoples, but otherwise I agree that interface is incredibly limiting. I think we may be converging to a similar place. Do you feel like there could/would be a real downstream in the per-PW fork scenario? I’d be concerned people would make some minor change and just not know how to deal with the upstream (in case of bug fixes, etc).

Thinking about all this, and in the context of the game, I feel like you have to take the performance hit and make every “hook/override” point as discreet as possible (aside from generic systems).
Like structs of function pointers/script chunks/lua_Refs:

struct AttackFuncs {
    FunctionPtr<std::unique_ptr<nw::AttackData>(nw::Creature* attacker, nw::ObjectBase* target)> resolve_attack;
    FunctionPtr<int(const nw::Creature* obj, nw::AttackType type, const nw::ObjectBase* versus)> resolve_attack_bonus;
    FunctionPtr<int(const nw::Creature* obj, const nw::ObjectBase* versus, nw::AttackData* data)> resolve_attack_damage;
    FunctionPtr<nw::AttackResult(const nw::Creature* obj, nw::AttackType type, const nw::ObjectBase* vs, nw::AttackData* data)> resolve_attack_roll;
    FunctionPtr<nw::AttackType(const nw::Creature* obj)> resolve_attack_type;
    FunctionPtr<std::pair<int, bool>(const nw::ObjectBase* obj, const nw::ObjectBase* target, bool vs_ranged)> resolve_concealment;
    FunctionPtr<int(const nw::Creature* obj, nw::AttackType type, const nw::ObjectBase* vs)> resolve_critical_multiplier;
    FunctionPtr<int(const nw::Creature* obj, nw::AttackType type)> resolve_critical_threat;
    FunctionPtr<void(const nw::Creature* obj, const nw::ObjectBase* versus, nw::AttackData* data)> resolve_damage_modifiers;
    FunctionPtr<int(const nw::ObjectBase* obj, nw::Damage type, const nw::ObjectBase* versus)> resolve_damage_immunity;
    FunctionPtr<std::pair<int, nw::Effect*>(const nw::ObjectBase* obj, int power, const nw::ObjectBase* versus)> resolve_damage_reduction;
    FunctionPtr<std::pair<int, nw::Effect*>(const nw::ObjectBase* obj, nw::Damage type, const nw::ObjectBase* versus)> resolve_damage_resistance;
    FunctionPtr<std::pair<int, int>(const nw::Creature* obj)> resolve_dual_wield_penalty;
    FunctionPtr<int(const nw::Creature* attacker, nw::AttackType type)> resolve_iteration_penalty;
    FunctionPtr<std::pair<int, int>(const nw::Creature* obj)> resolve_number_of_attacks;
    FunctionPtr<nw::TargetState(const nw::Creature* attacker, const nw::ObjectBase* target)> resolve_target_state;
    FunctionPtr<nw::DamageFlag(const nw::Item* weapon)> resolve_weapon_damage_flags;
    FunctionPtr<int(const nw::Creature* obj, const nw::Item* weapon)> resolve_weapon_power;
};

I’m intrigued by your feats idea, I’m sure other people in the community would be interested in seeing your implementation for this too.

[leo_x]

I came across a very interesting talk by folks at Dice:
scopestackspublic.pdf (296.8 KB) and am fairly enamored with this idea, and tried to implement it: here. So I thought it was worth sharing here. I’ve gotten loading default resources (key files and what not), a basic module (DockerDemo), rules (not a hardcoded system like NWN), etc, etc (a lot more work to do of course). down to about 200ms on an old macbook pro with an ssd. Does anyone have experience with this sort of stuff, that they would like to share? I’m posting this, partly, to reiterate the core of this project is NOT just some file format library, but a library for instantiating modules.

[leo_x]

Hello all,

Small update if anyone is interested.

rollnw

Some performance improvements to the nwscript lexer. While the project is and always will be focused primarily on implementation simplicity, I think a gentle introduction to SIMD is still worth it (even if it doesn’t necessarily improve performance! I chose the xSIMD library). As it stands nwscript.nss is around 40ms to resolve completely on a Ryzen 6000 9, once the new allocator models are complete I think that will drop to 30ms. 10ms to lex, 10ms to parse, and 10ms to fully resolve the AST. It’s an atypical script but obviously an important one.

Even if you aren’t interested specifically in this project, knocking new ideas that could later be ported to nwn_script_comp would probably be an order of magnitude or two easier. No offense to the original compiler: it’s a product of its time and a singular mind.

Arclight.py

I’ve restructured the nwscript lsp project into a more general python tool project. All the tools will be installable with pip install arclight

nwscriptd

nwscript-language-server → nwscriptd

nwscript-lint.

A wrapper around the script parser and resolver (i.e typechecker, name resolution, etc)

usage: nwscript-lint [-h] [-I INCLUDE] [--no-install] [--no-user] scripts [scripts ...]

A linter for nwscript.

positional arguments:
  scripts               List of scripts to lint.

options:
  -h, --help            show this help message and exit
  -I INCLUDE, --include INCLUDE
                        Include path(s).
  --no-install          Disable loading game install files.
  --no-user             Disable user install files.

Sample output:

Obviously some of what are “errors” here is “just like my opinion, man”.

[leo_x]

Update for anyone interested.

Arclight.py

2dilate

Dusted off and tightened up my old 2da merger, 2dilate. I decided to work on this because I think 2da is possibly a useful format in a future tense, unlike say GFF which is decidedly legacy and not particularly good. The underlying rollnw 2da parser on a Ryzen 6000 9, with compiler optimizations (-O2), can reach over 500mb/s.

The merger can also convert to and from Excel xlsx format.

Merge precedence is determined by twodaname.2dx, twodaname_00.2dx, …, twodaname_XX.2dx when merging multiple 2dx files.

Original design goal: A file format that is simple, compact, and familiar. Something that could ideally be distributed with custom content to ease merging or shared by community members. And also trivially implementable in any programming language/environment.

More examples and details here

Hopefully, this does what any (good) tool should, even if one doesn’t use it: provides a frame of reference for thinking about how to best do whatever it is we want to accomplish, which in this case is to enable any person, regardless of technical ability to use and integrate all the amazing things that have been shared on the vault and subsequently tell their stories.

Basic example:

2DX V2.1
---
description: Add orb spells.
tlk:
  AltMessage: 1400
  Name: 1400
  SpellDesc: 1400
---
            Label               Name        IconResRef         School        Range        VS        MetaMagic        TargetType        ImpactScript        Bard        Cleric        Druid        Paladin        Ranger        Wiz_Sorc        Innate        ConjTime        ConjAnim        ConjHeadVisual        ConjHandVisual         ConjGrndVisual        ConjSoundVFX           ConjSoundMale           ConjSoundFemale         CastAnim        CastTime        CastHeadVisual        CastHandVisual        CastGrndVisual        CastSound        Proj        ProjModel             ProjType        ProjSpwnPoint        ProjSound        ProjOrientation        ImmunityType        ItemImmunity        SubRadSpell1        SubRadSpell2        SubRadSpell3        SubRadSpell4        SubRadSpell5        Category        Master        UserType        SpellDesc        UseConcentration        SpontaneouslyCast        AltMessage        HostileSetting        FeatID        Counter1        Counter2        HasProjectile
####        Acid_Orb            0           is_acidorb         V             S            vs        0x3d             0x32              sp_acidorb          ****        ****          ****         ****           ****          4               4             1500            hand            ****                  vco_mehanacid02        ****                  sco_mehanacid03        vs_chant_evoc_lm        vs_chant_evoc_lf        out             1000            ****                  ****                  ****                  ****             1           vpr_ectoacid01        homing          hand                 spr_los          path                   Acid                1                   ****                ****                ****                ****                ****                2               ****          1               1                1                       0                        ****              1                     ****          ****            ****            1
####        Cold_Orb            2           is_coldorb         V             S            vs        0x3d             0x32              sp_coldorb          ****        ****          ****         ****           ****          4               4             1500            hand            ****                  vco_mehancold02        ****                  sco_mehancold03        vs_chant_evoc_lm        vs_chant_evoc_lf        out             1000            ****                  ****                  ****                  ****             1           vpr_ectocold01        homing          hand                 spr_los          path                   Cold                1                   ****                ****                ****                ****                ****                2               ****          1               3                1                       0                        ****              1                     ****          ****            ****            1
####        Electric_Orb        4           is_elecorb         V             S            vs        0x3d             0x32              sp_elecorb          ****        ****          ****         ****           ****          4               4             1500            hand            ****                  vco_mehanelec02        ****                  sco_mehanelec03        vs_chant_evoc_lm        vs_chant_evoc_lf        out             1000            ****                  ****                  ****                  ****             1           vpr_ectoelec01        homing          hand                 spr_los          path                   Electricity         1                   ****                ****                ****                ****                ****                2               ****          1               5                1                       0                        ****              1                     ****          ****            ****            1
####        Fire_Orb            6           is_fireorb         V             S            vs        0x3d             0x32              sp_fireorb          ****        ****          ****         ****           ****          4               4             1500            hand            ****                  vco_mehanfire02        ****                  sco_mehanfire03        vs_chant_evoc_lm        vs_chant_evoc_lf        out             1000            ****                  ****                  ****                  ****             1           vpr_fireball          homing          hand                 spr_los          path                   Fire                1                   ****                ****                ****                ****                ****                2               ****          1               7                1                       0                        ****              1                     ****          ****            ****            1
####        Sonic_Orb           8           is_sonicorb        V             S            vs        0x3d             0x32              sp_sonicorb         ****        ****          ****         ****           ****          4               4             1500            hand            ****                  vco_mehansonc02        ****                  sco_mehansonc03        vs_chant_evoc_lm        vs_chant_evoc_lf        out             1000            ****                  ****                  ****                  ****             1           vpr_ectosonc01        homing          hand                 spr_los          path                   Sonic               1                   ****                ****                ****                ****                ****                2               ****          1               9                1                       0                        ****              1                     ****          ****            ****            1

File Format Structure:
Header: 2DX V2.1

Optional YAML Metadata: Extent is determined by --- ... ---. The only two reserved enteries are row and tlk, everything else is for the consumer/producer.

Columns: Only those columns that need merging are needed. Any new column is added as the last collumn, per the 2da spec. best practices. Imagine adding in new spell columns to an old spells.2da, only those new columns need to be put into the 2dx file.

Row numbers: Row numbers when specified do not have to be contiguous or ordered, that is unlike 2da they are very meaning: they determine where to merge. Row numbers can also be unspecified in two ways: ‘****’ works with YAML metadata row to produce row number = YAML meta row + 2dx row number and ‘####’ which simply appends the rows to the 2da being merged into, to produce row number = last 2da row + 2dx row number + 1

Rows: Rows values are merged iff the current value of the 2da being merged into matches base resources (the distro comes with all .36 2das), the only way to override this is by passing --force on the command line. For row values containing strrefs this can be controlled by the tlk section of the YAML metadata, in those cases row value will be determined by yaml meta tlk offset + row value + custom tlk special value Row values also have a secondary method of being unspecified: #### which instructs the merger to ignore that row and column for merging purposes. Imagine something like the below, which is not a particularly good example:

2DX V2.1
---
description: |
    Modify base class recommended starting stats.
---
           Str   Dex   Con   Wis   Int   Cha
0          17    10    16    8     13    8  
1          14    10    ####  8     14    16 
2          ####  ####  12    ####  14    12 
3          ####  8     14    17    13    8  
4          17    10    16    8     13    8  
5          10    17    12    14    13    8  
6          16    8     10    14    14    14 
7          12    ####  12    14    14    8  
8          10    18    ####  ####  ####  8  
9          13    8     12    8     13    18 
10         ####  ####  ####  8     18    8  

Stray thoughts / Topics of conversation:

• A consumer of custom content should only have to adjust two knobs as far as merging goes: where the rows go in a 2da and were the strrefs go in the custom tlk. We’ll come to TLKs and model renaming in time…
• Run-time merging is not valuable. Game systems like this should be accrete only, with serious consideration being given to removing or changing anything.
• The weakness of the 2da system, as a lot of people used to note, the fixity of row numbers is a serialization problem and not a problem with the format itself.

Many thanks to @Shadooow back in the day, many of these ideas here were his.

[leo_x]

For anyone interested (). I decided it’s time to circle back to the main project and implement equipment/inventory views

. It’s not very difficult to beat the nwtoolset widgets on performance and aesthetics, but I am curious why the toolset inventory panel seems to be so slow in NWN:EE?

There is also a notebook dealing with some basic inventory related things in Python.

There’s a lot still to be thought about and to implement, lfg.