SS1 EE Reactor code randomization fix

[673f23ce6684cvoodoo47]

Apparently, the reactor code in SS1 EE is not randomized anymore;

https://www.gog.com/forum/system_shock_series/currently_known_problems_and_solutions/post108
https://steamcommunity.com/app/410710/discussions/0/496880203079253325/
https://steamcommunity.com/app/410710/discussions/0/412448158150855864/

To fix this, download the attached file, place it into your SS1 EE\res\data folder and overwrite (starting a new game is required). you can also patch a savegame with the reactor-rng tool available here.

[673f23ce66be0dertseha]

Oi, that's something interesting.
The archive does indeed have a static code in the keypad by default. It is overridden by the random number as soon as you enter the reactor level the first time.
The static code is "725 336", and the random number comes from two game variables that are initialized when you start a new game.

Since the people with the same code also read it from the screens it means the part that loads the code into the keypad is the same, and the two game variables actually hold the code. It's now unclear where this fixed code comes from:
* The engine was modified to set a fixed value
** It may also be due to an involuntary change - i.e. a bug that was introduced
* A set of triggers in the archive overwrite the random number to this fixed value.

My personal guess is that the engine was modified.
I'll fire up my tools to have a look at the archive as well as a savegame from this version. I should be able to find some time for that the upcoming days.

[673f23ce66ef5dertseha]

Update: I've had a look at the archive and two test savegames.
For my installation I can confirm that the code is the same as for the other people.
I did a search in the archive and a quick glancing look at the first level, and I couldn't find an action that would set the two variables. Finally, I did a binary compare between the EE archive.dat and the one from the original CD and found no differences.

This then concludes that the fixed code comes from the engine. Whether by accident or intentional is up to the maintainers.

Thinking about it, there is a chance to "fudge" a semi-random number. It is possible to set game variables, yet for the reactor code we'd need some random seed. I was thinking about taking the security value into account, yet this one has low entropy, especially since the reactor code should be fixed before the first CPU nodes are destroyed. Then I remembered that you can start timer with a randomness in them!
My thinking goes: Start 6 timer on level entry with various randomness. They increment each digit at each tick. Then, set a tile-entry-trigger at some tile hacker has to go through, like the passage before the mutant cyborg guarding the CPU room. This tile-entry-trigger stops the 6 timer.
This whole chain needs some extra logic to keep the numbers sane. While the variables are handled as binary values, the actual code is stored as BCD (Binary Coded Decimal - i.e. the code "123" is stored as 0x0123).
To make things even more interesting, level 1 has only 20 slots free for such marker objects to implement this chain.

Challenge accpted 

[edit]
Though, thinking some more - what is more likely:
* People stumble over EE, read about the problem before starting a new game, and download the patched archive to use
* People are at some arbitrary point in the game and would like to have their savegame retro-fitted (would require a small executable patcher, which is easier to do)
* People don't care. Judging by the comments, those new to the game hardly see a difference and are even given a chance to avoid backtracking if they don't like to - and veterans note down the digits right away anyway, since they're spoiled already.

[673f23ce67021voodoo47]

so basically, the engine loads the resources in a way that doesn't trigger the randomization. interesting indeed.

also, this is unintended - afaik, NDS are aware, but as they are busy with other things, they may not fix it as it's not critical.

[673f23ce67279JDoran]

I agree that this should be fixed, though if you Dertseha make it as some sort of patch for the .exe (or whatever files) then people can make their own mind up whether to use it to fix the bug, or if they'd prefer it to always be the same to make the game easier for them (I don't see why, but I believe anyone should be allowed to play a game how they like*, except for cheating in multiplayer, of course).

BTW, you've no doubt thought of this of course, but if you do make a patch then please ensure that it checks that the file(s) it's patching are the right version (such as the latest version of the .exe), as otherwise some people will get confused when it doesn't work, resulting in some confused posts here and at it's GOG forum.

Did NDS have the source code when they modded SS to become SS EE? It seems unlikely, but if they did, then it's very strange that they didn't notice this bug.



* One of the reasons why I dislike Trophies/Achievement Points in modern console games is that they've killed off in-game cheats, whereas in my favourite first person shooters, I love playing them with my favourite weapon from that game and infinite ammunition. Plus if you could cheat in (the otherwise brilliant) Bioshock 2, then you could use the Drill Specialist gene tonic much earlier, and so give a whole new challenge to the game (provided you don't upgrade the drill, as it can become *very* powerful).

[673f23ce6739bvoodoo47]

as far as I know, EE is completely based on Shlink, no SS1 source code stuff is involved. and bugs like this one slipping in is pretty normal - if they don't stop progress, they are hard to spot, unless you have a tester that is pretty much obsessed with the game.

[673f23ce67925dertseha]

I agree that this should be fixed, though if you Dertseha make it as some sort of patch for the .exe (or whatever files) then people can make their own mind up whether to use it to fix the bug, or if they'd prefer it to always be the same to make the game easier for them (I don't see why, but I believe anyone should be allowed to play a game how they like*, except for cheating in multiplayer, of course).

From a technical perspective, I settled for a patcher for save-game files. This would allow for retro-fitting a random code to any save out there. (Only if it is a resource file, and if it is a save game file)
And from a "purist" perspective I'm with you to say that content-wise this is not the same game as the original.

Yet from a practical perspective, I'm still at my realization from my above text: What for?
My reasoning - all in my opinion - goes: Regardless of the capabilities of the engine, as soon as you learn about the importance of the digits, you'll "always" write down the digits as soon as you clear the respective CPU nodes in any future game session. You've spoiled yourself this revelation and avoid the necessary backtracking. And the possibly very, very few people that are so immersed to still "role-play" this revelation could also go the extra mile and go back to each screen. Such people already made the mental effort not to "know" what these digits are for, so they could ignore the fact that they'd find always the same digits.
Learning that the engine has this bug, that the code is always static, implicitly also tell you the importance of the digits, making you a player that would write down these digits like noted above.
And for the most part, you'll have to either destroy the nodes anway to progress, or they are so conveniently on your path that it's irrelevant whether you turn aside to destroy them with a few shots.

Trying to describe this patch then without a spoiler would go like this: "Run this patch on your newly started game in order to restore a necessary back-tracking that you'll most likely avoid anyway when you know about the reason." or "...in order to make you remember stuff."

Then again... I've spent over two years writing a game editor that has hardly any uses, why not also trying my hand on a small patcher

[673f23ce67db2voodoo47]

pretty sure I'll get to it sooner or later. also, if the reactor fix is something that can be patched into the exe or other game files (making the code random for anyone who starts a new game), then there is no reason to not go ahead - GOG/steam have no problems deploying our patches.

What for?

why do men like us do anything - because we can.

[673f23ce67f54dertseha]

Ah, these little experiments help to better understand the engine.
I've started with a new tool that would patch existing savegame files to receive a new random code.
Things that I considered so far:
* Changing the random number in the game variables
* Changing the codes that the panel on reactor level expects. This panel is loaded with the random code on first level entry.

Trying it out for the first time by checking the first digit on level 1, I realized the following:
While I've already identified the action that lets a screen display its level-specific digit, so far I thought it would just tell it "show whatever your digit is". Yet in fact it loads the actual digit into the screen object. So, changing the code after the fact does not change what might already be displayed.
This means the patcher needs to walk all relevant levels and check/modify the screen state. Oh well.

[673f23ce68338JDoran]

Ah, these little experiments help to better understand the engine.
I've started with a new tool that would patch existing savegame files to receive a new random code.
Things that I considered so far:
* Changing the random number in the game variables
* Changing the codes that the panel on reactor level expects. This panel is loaded with the random code on first level entry.

Trying it out for the first time by checking the first digit on level 1, I realized the following:
While I've already identified the action that lets a screen display its level-specific digit, so far I thought it would just tell it "show whatever your digit is". Yet in fact it loads the actual digit into the screen object. So, changing the code after the fact does not change what might already be displayed.
This means the patcher needs to walk all relevant levels and check/modify the screen state. Oh well.

Maybe instead of a save-game patcher, then, you could just make a save-game maker, one that always makes the same identical save-game, a save-game of the start of the game (where you start at when you start the game properly), but the save-game creator let's you first specify the skill levels you want, then it creates a random code number for the game, and produces a game-save for the user, called "Start-1" (or a similar name).

I can't remember, are things like key configuration (as in the keyboard control configuration), and the screen resolution stored in the save-game files? If so, then the save-game maker could read these in from the System Shock data files, and incorporation them into the new game-save, for the user's convenience.

[673f23ce6852ddertseha]

Interesting ideas there, JDoran.
Though, that tool would essentially be a "game starter" (requiring to set options as you'd have at "new game"), yet the created savegame would miss all the initial stuff that the engine does the first time hacker "enters" the first level. The engine most likely would not start various "scripts" when loading such a created game.

Either way, I've already extended this patcher to modify any already displaying screen across Citadel. Though, I currently don't have time to smoke test this - which is what I want to do before doing a "beta" release for any of you to test. So, it will take a few days.

And apart from that, I've already spent some time thinking about how to set up the starting archive to create a random code. This would then be a replacement of archive.dat file.
Having a quota of only 20 objects is tricky. I've come up with a script that does the first three digits using 17 objects, with the second three then planned for the second level. Yet I haven't checked whether lvl2 has this much space as well. All in good time.

By the way: would it be copyright infringement if I were to post a modified archive.dat ? It is after all the whole of Citadel (and the "game"), just missing all the resources.

[673f23ce68644voodoo47]

the mods subforum is full of modified chunks of the original game(s), so it should be fine.

[673f23ce687c2dertseha]

Ok, I've released version 1.1.0 that includes now the command-line tool "reactor-rng", which randomizes the code on savegames. It can be run at any stage of the story.
I did not play the game in a linear fashion up to the relevant point to test it, I warped myself around levels, directly to the relevant spots to verify that the tool works as expected.

Next up I'll have a go at extending archive.dat to randomize a code from the start.

[673f23ce6898adertseha]

And here is the modified archive.dat file.

I tested the file with both the vanilla release, as well as the Enhanced Edition. I did not play through the entire story, I only tested the expected side-effects by looking at the save-games and then smoke-tested levels 1-6 by running through them on mission setting 0.

I am considering to record another video, describing the solution. Though, it may take some further time as I've got a cold right now.
Also: I did not manage to do it within the quota of 20 marker objects, I needed more and cheated a bit.  Still, I've also learned a few more things about the engine and how Citadel works - Which reminds me of the video series "Blueprints of Citadel" that I had in mind.

How to proceed now with this overall issue? Whenever (and wherever) a new post pops up about the code, redirect them to the tool & archive?

[673f23ce68a8cvoodoo47]

probably.

so let me get this right - using the modified archive.dat will randomize the code when starting a new game, and reactor-rng will do it on a game in progress (save), correct?

[673f23ce68b9adertseha]

Correct.
archive.dat has a fix built into Citadel. (To be technically correct, the code is not set at the start of the game, it is being generated while the player runs through levels 1 and 2). When this is used, the tool reactor-rng is not necessary.

When starting the game with the vanilla archive.dat, then you need to run reactor-rng on the save-games.

[673f23ce68c77voodoo47]

thanks - and moved to LG Classics.

[673f23ce68d44JDoran]

Thanks for this, Dertseha!

[673f23ce68e90TurboSnail]

Hi,

Thanks for your work!

When I use the "archive.dat" and start a New Game the Data Reader hardware will not stay in the Item sub-windows. It will quickly change back to the Weapon display. This occurs if I select the "E" hud item or select the "Data Reader" from the Hardware tab. All other "Hardware" seem to work ok and stays in the Item sub-windows. Any clues of how to fix this? Thanks!

File versions:
"archive.dat" [CRC-32: c310807a]
"sshock.exe" [CRC-32: 32d28255]
"sshock.dat" [CRC-32: d4cd1261]

[673f23ce6924cdertseha]

When I use the "archive.dat" and start a New Game the Data Reader hardware will not stay in the Item sub-windows.

Thank you for this finding! This is very annoying, yes.

I've made some tests and this issue is not related to the enhanced edition, it also happens with the original release. Whatever was selected previously in one of the side-MFDs, it is immediately reactivated. One also can't trick it by preapring the "Item" view (which the data reader uses) on either sides.

The bad news is that at the moment I don't know what specifically causes it. It is related to the reactor fix, yet I don't understand why it happens in order to fix that.
The good news is that this "only" happens during levels 1 and 2 - and only the first times around. As soon as you have reached level 3 or R the MFD will behave normally - even when going back to 1 or 2. (This because the "fix" is inactive at this point - the reactor code has been finalized)

In order to work with the data reader, for now I can only offer you to quickly click on the emails/log/data buttons before the side-MFD switches back. You can achieve that by moving the mouse cursor to the suspected button position, and then pressing the number 8 on the keyboard, which also activates the data reader. Then immediately click on the button before it disappears.
Or use the vanilla archive.dat, save the game right after starting, and run the console-utility on this savegame file.

I'm sorry, this is bad quality control on my part.
First I'll have to figure out why this happens, and then try to circumvent another limitation. Yet I fear that we'll run into the limits of both what can be packed into the archive, and the engine.

[673f23ce694aaJosiahJack]

I think there is a timer or distance check that returns the side MFD's to their previous states (I would test this to see if it is time based or distance but I'm away from my computer this weekend).  The Data Reader is a kind of "bonus" tabset.  It would technically make better sense to have it as separate tabs in the MFD sides and center.  As it is, opening the data reader (hud button, hardware inv, or pressing 8 ) causes the Data reader to hijack the MFD temporarily.  I'm wondering if your changes somehow affect the checks for when to return the hud to normal inadvertently.

Does the new archive.dat cause any problems when searching.  I'm curious if the distance check for how far away you are from a searchable (the cell it is in rather) also applies to when you activate the Data Reader and if searching has the same problem at all.

[673f23ce696dfdertseha]

Welp, analysis of a few code lines later and desperation kicks in.

It seems that any modification of the quest variables (in which also the reactor code is stored) deactivates the email reader in the side-MFD.
(For the code-curious: trigger.c::qdata_set() calls newmfd.c::mfd_notify_func() calls email.c::mfd_emailware_expose() with a 0 control value -> causing the email reader to close itself)
It is likely that this also deactivates other stuff shown in the "Item" tab, though that depends on how they react to a zero control value.

In other words, the current solution in archive.dat is not fix-able, it needs a complete revamp. (Good thing I haven't done the how-to video yet )

On the plus side though, I could refute a myth I had: I thought changing quest variables did not support the modulo operation - yet the codes does!
This means it should be feasible to solve this with very quick timers while hacker is still in the first room and without an email client, for all six digits.

* dertseha heads off to the editor

[673f23ce6985bdertseha]

* dertseha comes back from the editor

dang - I had again forgotten that the code is stored in binary-coded-decimal, and this makes things extra complicated.

I'll have to sleep over this one, perhaps I get another idea how to do all the stuff within the constraints.

[673f23ce69a86dertseha]

Phew. First I thought I could do it, then hit another roadblock, and finally had a great idea. And the solution now is elegant enough, as well as way simpler than the original one.

The new archive.dat is stored here: https://github.com/inkyblackness/mod-examples/tree/master/CitadelRNG
@voodoo47 , please update the attachment in your original post.

The code is now generated while hacker is in the first room.
The only downside of the new solution is when a player races to the far end, the code not might be randomized enough. The assumption is that the player spends a few seconds in this room before venturing forth.
Also, thanks to the incredible capabilities of "mr. gibbon", you still can fetch the email reader and encounter this "auto-turn-off" bug before leaving the room. Though, this case might be rare.
(i.e., yes, I now tested with the email reader as well )