Change in File URL on Vault

Has there been a change to the way in which file attachment URLs are stored on the vault?

The file name used to be a part of the URL, but it seems this has been changed to only include the FID.

If this is a new way of storing the file links, is there a way to reliably access the file name from the URL information?

Many thanks for any help - the change has caused most of NIT download rules to fail (because they use what used to be the filename portion of the URL).

I’ve contacted Niv on Discord and linked him to your post, so perhaps he will have an answer about this.

FP!

Hi,

I had to run a security update a couple of days ago. There was no mention of any non-security changes.

Could this be solved with config?

Do you have an example of what the old download URL looked like?

I’m assuming you are referring to this? (example)

https://neverwintervault.org/sites/all/modules/pubdlcnt/pubdlcnt.php?fid=33312

@niv
Many thanks for responding. Can’t be solved with NIT config - working on a complete rewrite of file download link processing. Would be great if it could be restored with a site config adjustment.

I’ll try to restore the old behaviour. Please give me an example of what you are trying to parse!

I don’t have an example because it is hardcoded, but it looked something like this…
https://neverwintervault.org/.../filename.extension&nid=30393

My recollection may not be quite right, but this should give you an idea.

The new link looks like this…
https://neverwintervault.org/sites/all/modules/pubdlcnt/pubdlcnt.php?fid=30393

I found the offending change: https://www.drupal.org/commitlog/commit/10486/e75f9c817b03ea997d1e80d8cb1daeaaf1831e9a

Cannot locate a issue report that explains the why of it though. Suppose it might have been a parsing fix.

What does NIT use the filename for? Would adding it as a hashtag (i.e. ?fid=123123#My_cool_file.mod) work as well? Is there some other way your requirement could be fulfilled?

NIT uses the filename for an automated download operation as well as to determine whether the file has already been downloaded.

In addition, the old format allowed me to query the file url to get the bytesize of the download.

Without the old url format I am into a fair amount of recoding (which I have started). Even if the url is modified as you suggested, the class that performs the download would have to change to accommodate the new format.

If it can’t be changed, I will assume that the metadata, which seems to include the file name as part of the url’s description text is valid. In addition, there is length specification under type in the html, which I assume is the correct byte size. Also, the new urls, start with /sites instead of https.

Let me know what your plans are. If your suggested url change included additional info, such as the filename and size, it would make parsing simpler, but if it is the same info I mentioned above, then I can accommodate it.

This is the problem with web page scrapers :frowning:

@niv
Just has another thought. Can I use the meta-data in the Project Page’s html to reconstruct the download url in the old format? If I can, will the old format URL still work?

Then all I need is to find out exactly what the old download format was.

Does anyone know what the old format file link url looked like?

I am keen to see if it still works, but can’t remember the exact format. It should be something like…

https://neverwintervault.org/<can’t remember this part>/filename.extension&nid=30393

Also, is the &nid correct or was it formatted differently?

Any help will be very much appreciated.

I took a closer look at NIT’s code and worked out what the link used to be…

https://neverwintervault.org/sites/all/modules/pubdlcnt/pubdlcnt.php?file=https://neverwintervault.org/sites/neverwintervault.org/files/project/29607/files/the_sinister_secret_of_saltmarsh_0.7z&nid=34342

by changing nid to fid, the old format still works.

@niv
Should I assume that things are going to stay as they are now? If so, I will start implementing the changes required to deal with the new link formats.

1 Like

@niv
Is there any way to determine whether a fid refers to rolovault file?

The link changes have really impacted the way NIT and its download rules work - feeling a little depressed regarding whether I will be able to restore NIT’s Project Download functionality :sob:

I just got a report that the download links file for Sanctum may have been broken by this same issue:

I agree that it would be good if this change could be reversed.

Here’s an example:

https://neverwintervault.org/sites/all/modules/pubdlcnt/pubdlcnt.php?file=https://neverwintervault.org/sites/neverwintervault.org/files/project/335/files/sanctumv50_ms_hak.7z&nid=335

As you noted, this is no longer working.

Andarian

@Andarian
If you change nid=335 to fid=30718 it should work. The same result is achieved with the new format, which only uses the file identifier…
https://neverwintervault.org/sites/all/modules/pubdlcnt/pubdlcnt.php?fid=30718

That means that I’ll have to re-build and re-release the download package files for my mods to use the new links. I can do that, but I’d rather not if the prior format is going to be restored – especially if the new format is going to cause such a problem for NIT.

I am travelling today but i will try to patch the old behaviour in this weekend. I think it was changed over to use the file id so no parser errors happen with weird filenames.

I decided to go ahead and re-build the Sanctum download file using the new link formats. It was a bit tedious, but straightforward. Project page links were fine, but I can confirm that all of the direct file links in it had been broken by the Drupal change. I think that anyone else linking directly to file resources on the Vault will have the same issue with it.

I also decided to do some long overdue maintenance cleanup on my project page. I had left a large number of obsolete links on it because I am not aware of any way to update them without losing their associated download counts. I took a screenshot of those counts as of this morning, which I’m going to post on the page, and then I’m going to delete all of them. If anyone knows of a way to update files in the future without losing their download counts, I would be very interested in knowing about it.

EDIT:

Will the patch preserve both link formats? I fixed it to use the new one because I have a player waiting on it. If I’ll need to change it back after the patch, just let me know. Thanks!

Let me know if you do plan to restore the old format permanently or not. I will need to undo a series of significant changes to restore the old behaviour.

Just to be clear - the old format is much easier to deal with as well as allowing me to update dialogues more rapidly. So my preference is the old format providing you will be able to maintain that state of play for some time.

Either way, your help and support is very much appreciated.

2 Likes