I am trying to use a VB.NET program to retrieve the Mod_Name field from a Mod file.
I believe my initial issue is that I am unable to determine how to locate the start position of the IFO record within the file.
I have been using the NWN Omnibus to understand how these file are structured, but am too stupid to work out how to derive the start position of the IFI portion of the file.
Any help in understanding how I could achieve this will be very welcome and much appreciated.
Think I know what the problem is. You’re reading in old chars (which was 1 byte back in the day) like new chars (which are 2 bytes nowadays).
@pscythe
I may well be missing your something, but I am not sure that explains the problems. Accessing the Mod Description works perfectly. Also, the information as defined in the NWN Omnibus, specifies 1 byte rather than 2.
Have you been able to verify this?
Remember that NwN was built before the days of the various utf encodings. Internally NwN is still in ANSI/ASCII i.e. byte sized characters. I had a problem with editing 2da files recently, the fix for which was to tell Notepad++ to use ANSI encoding as the default as in one update or another they had switched to one the utf encodings.
Trent has said in the streams that they are looking into this but it is my guess that if they do change it, that it will be quite some time before it happens. It will involve quite a lot of work.
TR
@Tarot_Redhand That is exactly my point.
But somehow @Surazal manages to retrieve the ERF file description using BinaryReader.readChars which reads in 2-byte Unicode chars instead of the 1-byte ANSI char that NWN was using. Does VB’s Option Compare Text changes that to 1-byte chars?? Anyway I am wring a mod reader from scratch strictly using readBytes so there is no ambiguity.
I haven’t tried running your code but that was the first thing I spotted when studying it.
The other problem was that the code was trying to read the ERF resources list without going through the Keys list first to locate module.ifo. You had to search the keys list for resref module.ifo then that gives you a index into the resources list which would give you a offset into the actual IFO.
That’s good info - I will take another look at the Omnibus and see what I can work out. Many thanks.
According to my copy of a VB programmers reference we are dealing with streams which I never use. Reading about the ReadChars method it appears that it reads a single character and advances the reader’s position according to the streams encoding and the character. So it would seem that stream classes somehow know the encoding of the text being read in.
TR
The default encoding is actually UTF-8 which can be 1 to 6 bytes depending on the character itself. So this explains why it works for @Surazal as the module descriptions are in ASCII which is 1-byte chars.
anyway this is what I do to read 4-chars as NWN sees chars using the BinaryReader.
System.Text.Encoding.ASCII.GetString(reader.ReadBytes(4))
Okay, it is done.
F:\ModReader> ModReader.exe "To Heir is Human Custom.mod"
Module Description=
"To Heir is Human"
by BioWare
Set in the heart of Cormanthor, near Myth Drannor, a band of rangers, searching
for a kidnapped child, has been defeated by a drow army. The few remaining surv
ivors have given up hope of rescuing the heir to a barony. Who dares face the mi
ght of the drow to save an innocent child?
Recommended Levels: 5 - 15
Number of Players: 1 - 4
Mod_Name=To Heir is Human
The code is a bit messy but it was mostly a hack
Option Compare Text
Option Explicit On
'Option Strict On
Imports System.IO
Public Class ErfReader
Private Const BufferSize As Integer = 4096 * 2
Public reader As BinaryReader
Private fStream As FileStream
' For localised descriptions of the module
Private locStringCount As Int32 ' LanguageCount
Private locStringSize As Int32 ' LocalizedStringSize
Private locStringOffset As Int32
Private keysCount As Int32
Private keysOffset As Int32
Private resourcesOffset As Int32
' ERF keys have a fixed size
Private Const keySize = 16 + 4 + 2 + 2
Sub debug(str As String)
Console.WriteLine("Got: |" + str + "|")
End Sub
Function checkType(TypeStr As String)
Return (TypeStr = "MOD " Or TypeStr = "ERF " Or TypeStr = "SAV ")
End Function
Function checkVersion(VersionStr As String)
Return (VersionStr = "V1.0")
End Function
Public Function open(fname As String)
fStream = New FileStream(fname, FileMode.Open, FileAccess.Read, FileShare.Read, BufferSize, FileOptions.RandomAccess)
reader = New BinaryReader(fStream)
Dim type As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(4))
Dim version As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(4))
' Not sure if these are signed?
locStringCount = reader.ReadInt32
locStringSize = reader.ReadInt32
keysCount = reader.ReadInt32
locStringOffset = reader.ReadInt32
keysOffset = reader.ReadInt32
resourcesOffset = reader.ReadInt32
' What is this??
'Dim descOffset As Int32 = reader.ReadInt32
Return checkType(type) And checkVersion(version)
End Function
Public Sub close()
reader.Close()
fStream.Close()
End Sub
Public Function GetDescription()
reader.BaseStream.Position = locStringOffset
Dim attempts As Int32 = 0
Do While attempts < locStringCount
Dim languageId As Int32 = reader.ReadInt32
Dim size As Int32 = reader.ReadInt32
Dim description As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(size))
If (languageId = 0) Then ' English
Return description
End If
attempts = attempts + 1
Loop
Return "***No English Description found for Module.***"
End Function
' Returns the ResId from KeyList for the given resref and type
Public Function FindFile(name0 As String, type As Integer)
Dim name As String = name0.ToLower
reader.BaseStream.Position = keysOffset
Dim seen As Int32 = 0
Do While seen < keysCount
Dim resref As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(16))
Dim resid As Int32 = reader.ReadInt32
Dim resType As Int16 = reader.ReadInt16
reader.ReadBytes(2) ' discard unused
If (resref = name And type = resType) Then
'Console.WriteLine(name + " of Type=" + type.ToString + " Found at " + seen.ToString + " with resId=" + resid.ToString)
Return resid
End If
'Console.WriteLine(resref + resType.ToString)
seen = seen + 1
Loop
Return -1
End Function
Private Const resourceElemSize = 4 + 4
' Returns the FileOffset for the given ResId
Public Function GetResourcePosition(ResId As Int32)
Dim resource = New ResourcePosition
reader.BaseStream.Position = resourcesOffset + ResId * resourceElemSize
resource.offset = reader.ReadInt32
resource.size = reader.ReadInt32
Return resource
End Function
End Class
Public Class ResourcePosition
Public offset As Int32 ' Offset from file
Public size As Int32
End Class
Public Class IfoReader
Private reader As BinaryReader
Private position As ResourcePosition
Private structOffset As UInt32
Private structCount As UInt32
Private fieldOffset As UInt32
Private fieldCount As UInt32
Private labelOffset As UInt32
Private labelCount As UInt32
Private fieldDataOffset As UInt32
Private fieldDataCount As UInt32
Private fieldIndicesOffset As UInt32
Private fieldIndicesCount As UInt32
Private listIndicesOffst As UInt32
Private listIndicesCount As UInt32
Public Sub New(r As BinaryReader, resPos As ResourcePosition)
reader = r
position = resPos
End Sub
' Res is pointing to the module.IFO within the ERF
Public Function ReadHeader()
reader.BaseStream.Position = position.offset
Dim type As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(4))
Dim version As String = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(4))
structOffset = reader.ReadUInt32
structCount = reader.ReadUInt32
fieldOffset = reader.ReadUInt32 ' field array
fieldCount = reader.ReadUInt32
labelOffset = reader.ReadUInt32 ' label array
labelCount = reader.ReadUInt32
fieldDataOffset = reader.ReadUInt32
fieldDataCount = reader.ReadUInt32
fieldIndicesOffset = reader.ReadUInt32
fieldIndicesCount = reader.ReadUInt32
listIndicesOffst = reader.ReadUInt32
listIndicesCount = reader.ReadUInt32
Return checkIfoHeader(type, version)
End Function
Public Function GetModName() As Object
If Not ReadHeader() Then
Console.WriteLine("Not IFO file")
End If
'Goto the very first struct
reader.BaseStream.Position = position.offset + structOffset
Dim structType As Int32 = reader.ReadInt32 ' structType is actually signed
Dim structDataOffset As UInt32 = reader.ReadUInt32 ' index into field indices array
Dim structFieldCount As UInt32 = reader.ReadUInt32
If Not structType = &HFFFFFFFF Then
Console.WriteLine("**** Failed to find top struct !!!")
End If
'Console.WriteLine("struct offset=" + structOffset.ToString)
'Console.WriteLine("struct type=" + structType.ToString)
'Console.WriteLine("struct count=" + structCount.ToString)
'Console.WriteLine("struct field count=" + structFieldCount.ToString)
'Console.WriteLine("struct data offset=" + structDataOffset.ToString)
' Looking into the field indices array
reader.BaseStream.Position = position.offset + fieldIndicesOffset
' retrieve all the indices first
Dim seen As Int32 = 0
Dim indices As List(Of UInt32) = New List(Of UInt32)
Do While seen < structFieldCount
Dim index As UInt32 = reader.ReadUInt32
indices.Add(index)
'Console.WriteLine("field index=" + index.ToString)
seen = seen + 1
Loop
' now inspect the fields array
Dim currPosition As UInt32 = position.offset + fieldOffset
For Each index In indices
reader.BaseStream.Position = currPosition
Dim fieldType As Int32 = reader.ReadInt32 ' signed
Dim fieldLabelIndex As UInt32 = reader.ReadUInt32 ' index into label array
Dim fieldDataOffset As UInt32 = reader.ReadUInt32 ' byte offset into Field Data Block
currPosition = reader.BaseStream.Position
Dim label = GetLabel(fieldLabelIndex)
'Console.WriteLine("Label = " + label)
If (label = "Mod_Name") Then
Return GetFieldDataCExoLocString(fieldDataOffset)
End If
Next
Return "*** Mod_Name field not found ***"
End Function
Private Function GetFieldDataCExoLocString(offset As UInteger) As String
reader.BaseStream.Position = position.offset + fieldDataOffset + offset
Dim totalSize As UInt32 = reader.ReadUInt32 ' total size not including totalSize itself
Dim strref As UInt32 = reader.ReadUInt32 ' into dialog.tlk
Dim count As UInt32 = reader.ReadUInt32 ' Number of substrings here
'Console.WriteLine(count.ToString + " substrings")
Dim seen As UInt32 = 0
Do While seen < count
Dim stringId = reader.ReadUInt32 ' unsigned?
Dim stringLen = reader.ReadUInt32 ' unsigned ?
Dim contents = System.Text.Encoding.ASCII.GetString(reader.ReadBytes(stringLen))
'Console.WriteLine("stringId=" + stringId.ToString + " |" + contents + "|")
If (stringId = 0) Then
Return contents
End If
seen = seen + 1
Loop
Return "*** Mod_Name value not found ***"
End Function
Private Const labelSize = 16
Private Function GetLabel(offset As UInt32)
reader.BaseStream.Position = position.offset + labelOffset + offset * labelSize
Return System.Text.Encoding.ASCII.GetString(reader.ReadBytes(labelSize))
End Function
Function checkIfoHeader(type As String, version As String)
Return (type = "IFO " And version = "V3.2")
End Function
End Class
Public Module ModReader
Public Sub main()
Dim arguments As String() = Environment.GetCommandLineArgs()
Dim reader As ErfReader = New ErfReader
If (Not reader.open(arguments(1))) Then
Console.WriteLine("wrong right file")
Return
End If
Console.WriteLine("Module Description=")
Console.WriteLine(reader.GetDescription)
Console.WriteLine()
Dim resId = reader.FindFile("module", 2014) ' 2014 for IFOs
Dim resPos = reader.GetResourcePosition(resId)
Dim ifo = New IfoReader(reader.reader, resPos)
Console.WriteLine("Mod_Name=" + ifo.GetModName)
End Sub
End ModuleThe Bioware docs are fairly reliable as far as this task is concern. The only mistake was in the GFF docs where struct-type is actually a signed 32bit int as it is used rather than the unisgned one it says in places.
@pscythe
Thanks for this and all the effort you have made to help me - very much appreciated. Will take me some time to work through the code so I understand it. I had managed to compute the IFO location before you submitted your code, but was stuck on processing the IFO information - so getting your solution is very welcome.
Once I understand your code, I will integrate into NIT and then start changing how the .sav name is mapped to the NIT mod. Will also have to code some version migration code to tidy up what is no longer needed.
I will release the automated .sav to Mod name change before starting to think about a Basic user interface option.
Let me if you got any questions. The GFF is a whole lot of indirection. LOL
I have been using a modified version of @pscythe’s code, which seems to work for almost all cases. However, there are some exceptions to reliably getting the Mod_Name and Mod_Description fields.
In particular, The_Lord_of_Terror_2_0_5.mod and some of the descriptions for the NWM modules.
Does anyone have a routine to reliably get the Mod_Name and Mod_Description information for module files? If not, is anyone prepared to write one or help me see where I am going wrong?
If interested, you can download the Visual Basic source.
Many thanks for any help and suggestions.