SS2 - Trying to improve the multiplayer

[67418a4a60986figgis]

Hello.

For the past few days I've been trying to make the multiplayer component in SS2 somewhat more palatable.
Unfortunately the bulk of the netcode would be difficult to improve upon without having the source code, but it appears that there are still a few things that can be done, such as fixing 4 (or more) players support. I'm going to list my findings and mods here in case someone finds them useful.
All this applies to NewDark v2.48.
Apologies if it's in the wrong forum section.

Problem 1: The player cap.

Despite the player cap technically being 4, it seems impossible to get a game going with more than 3: whenever the 4th joins, someone else gets kicked out of the lobby. While it defaults to 4, nothing seems to actually be hard-wired to it being 4, so it can be easily changed.

The value can be found at offset 0x2E121 into ss2.exe and changed from 04 to a larger number, such as 0A. The same limit value is at offset 0x2F440 in Thief2MP.exe.

After changing it to 5 I've discovered that now 4 people can get into the lobby and the 5th player joining triggers the same bug described above. I suppose the "solution" is to just set it to N+1, where N is the desired player cap. The same seems to apply to Thief 2.

While I could now get more than 4 people into the lobby ( https://i.imgur.com/UXcuLOh.png ), players 5 and up wouldn't properly spawn in game. This seems to be because the game only has 4 network avatars defined in the object hierarchy ("Default Avatar 1-4"), so unless you add more or force everyone to use net_simple_avatars 1, the game will try to default to "Default Avatar 5" for player 5 and fail. This does not happen in Thief 2 because it defaults to "Garrett" for everyone. With net_simple_avatars I could properly get into a netgame with 5 instances of the game running locally ( https://i.imgur.com/M07DEDS.png ).

Problem 2: Actually getting into the game.

In my experience it is a rare occasion that everyone gets through the character creation sequence and the first "Synchronizing" screen without crashing, freezing or desyncing in the process. This might have to do with the game effectively "pausing" the netcode until medsci1.mis is reached: while you're in earth.mis and station.mis, (most) messages are not sent and (mostly) any messages received are stored for later and their processing is delayed until the load into medsci1. Or it might be that message send timeouts are expiring for messages that arrive while you're in a loading screen.

Either way, it seemed to me that one possible way to make this part more stable is decreasing the amount of loading screens and waiting time involved. To this end I made a small mod that gives you an option to pick your character creation branches using a simple dialog menu at the start of earth.mis ( https://i.imgur.com/m8VQ2RB.png ), which then loads you directly into medsci1.mis. I don't have any statistics to prove that it increases the connection success rate, but it certainly at least helps cut down the waiting time and also works in SP. It is attached to this post.

As for message timeouts, there are two: hard-coded 5 seconds for blocking messages, 60 seconds by default for non-blocking. Interestingly, Thief 2 v1.27 uses 0 (infinite) for both of them. It seemed to me like a good idea to at least increase them.
The former can be increased by changing the 5000 at offset 0x2C7B4 in ss2.exe, the latter by either changing the default value of 60000 at 0x2CAE3 or by adding e.g. net_player_timeout 600000 into your config file. I changed them to 60000 and 600000 respectively.

General gameplay improvements.

As has been mentioned in this thread, multiplayer lacks much difficulty.

One aspect of this is the respawning system. There's virtually no punishment for dying. To address this I made a script that modifies the respawn logic:

• you can only respawn automatically if the QBRM is active on the map, while you're dead the game checks for that every second (adjustable);
• reconstructing costs 15 nanites (adjustable): if you don't have them, you can't respawn even when it's active;
• if another player frobs an already-activated QBRM, all dead players will respawn at it;
• if the map has no QBRM, you respawn as normal (can be turned off).

Upon moving into a different level all dead players still respawn as normal. Not sure yet if that's possible to override via script.
Said script is part of the ss2_netfixes mod attached to this post.

Another good point made below is that there should be some sort of difficulty scaling depending on player count. I've yet to investigate how this could be implemented.

Problems unresolved.

After doing these things I was able to get a 4-player game with real players going over the Internet using ZeroTier and it was fairly stable, at least as far as SS2 multiplayer goes. However, there are still many problems left to fix:

• Additional players can't seem join on an existing save, even if they get a copy of someone else's. Perhaps this could be fixed by editing a new avatar and player object into one of the players' existing saves? Could this be done automatically by hooking into the connection procedure and transmitting the save file first?
• When loading an existing save or level transitioning, existing player states can swap between players or duplicate. They seem to be assigned by player number, so maybe this depends on the order of players in the lobby?
• Sometimes a remote player can get stuck on geometry and then warp to the proper position after a while. Could this be fixed completely, or at least could the time before warp be reduced?
• Minimizing the game seems to freeze parts of the networking system. For example, saving will not go through until everyone alt-tabbed out tabs back into the game. Maybe there's already a setting to disable that? Otherwise could probably be patched out by stubbing out some event handlers in the game's WndProc.
• General instability and crashes. This can be partially worked around by mods (e.g. disabling the Engineering cutscene seems to help).
• Chat is disabled in NewDark v2.48 for some reason. Could probably be worked around with a mod, but is a fairly low-priority issue.
• Would be nice to have an option to make the pause menu run on top of the game without pausing it. Not sure if that's possible to do.

As I've said before, my knowledge of SS2 modding and Dark Engine in general is still very limited, so if anyone else has ideas on how these could be approached, it would be nice to know. I will update this post if I figure out solutions to any problems listed. The saving issues are probably the most annoying in my experience and I'm going to look into that next.

Attached files:

• ss2_quickstart_v2.zip: the quick-start menu mod.
• ss2_netfixes_v1.zip: alternate respawn logic, additional net avatars and Engineering cutscene skip from SS2 Cutscene Skipper.
• dark_mp_patched_v1.zip: patched ss2.exe and thief2mp.exe. You can also patch the EXEs manually by editing them with a hex editor as described above.

Thanks to Nameless Voice for NVScript, from which I shamelessly stole the proper level transition procedure, and to RoSoDude for the Alternate Start mod, from which I shamelessly stole the idea of attaching the script to a random light object in the level.

[67418a4a60bd6ZylonBane]

SS2:EE is probably going to completely overhaul the multiplayer code, so any attempts to fix it in Old/NewDark would just be wasted effort.

[67418a4a60e70figgis]

SS2:EE is probably going to completely overhaul the multiplayer code, so any attempts to fix it in Old/NewDark would just be wasted effort.

I would certainly hope so, but I assume its release is still quite a ways off.

[67418a4a6107avoodoo47]

it's fairly safe to say SS2EE will be out later this year. so yeah, any attempts to improve the MP in Dark games is indeed a wasted effort, especially if we are talking about band aids like hexediting and script-circumventing issues.

also, any MP session with 3 or more players is not worth anything without making the game adjust spawns and AI strength accordingly first - with so many players, there is no challenge left, everything will just succumb to a barrage of wrench hits.

so to make 3+ player MP into something someone would actually want to play and enjoy, you would also have to pretty much overhaul the entire game as well.

do you have a decade and absolutely no life?


that said, we certainly don't want to discourage you from SS2 modding, but directing your efforts at something else would probably be a better choice.

[67418a4a616bafiggis]

it's fairly safe to say SS2EE will be out later this year. so yeah, any attempts to improve the MP in Dark games is indeed a wasted effort, especially if we are talking about band aids like hexediting and script-circumventing issues.

I don't personally consider it a waste of my time since I find stuff like this interesting to do. It's understandable that someone else would, though.
My original objective was to fit all my friends that wanted to play into one game session and I figured that someone else might have had the same thought and would find this interesting. Then it got me thinking about how much exactly could be fixed up without completely rewriting the whole thing.

so to make 3+ player MP into something someone would actually want to play and enjoy, you would also have to pretty much overhaul the entire game as well.

This is probably a valid point for a lot of people, however I don't think it would have to be that drastic, at least for me. I find it pretty fun even with default difficulty, not to mention there are at least a couple of mods out there that attempt to increase the difficulty of MP, so it could be possible to build on top of them.

[67418a4a61825voodoo47]

sure. but chances you get a 4 player session going in a manner that would allow game completion is astronomically low. just making sure you are aware.

also, most people really don't care about MP that much, and seems like that includes the Raven, so I wouldn't expect any MP improvements in the foreseeable future if we are talking NewDark here. SS2EE is your best bet, really.

[67418a4a61b1asarge945]

While I do appreciate the effort, I also disagree with the player increase. I feel like the game is already easy enough with 1 player. I would probably cap it at 2 max for multiplayer, and maybe double the ecology spawn rate.

[67418a4a61c56voodoo47]

I actually agree with that, a polished up, fully overhauled 2 player MP would be an interesting way of playing SS2.

that is what I would focus on, if I were NDS (assuming they want to deliver fully enhanced MP experience, not just fix the technical issues).

[67418a4a61d93ZylonBane]

Wanting to shove four players into a game of SS2 is just incomprehensible to me. Not only is SS2 not designed as a multiplayer experience, there are hundreds of games that ARE designed for multiplayer. So go play one of those instead.

[67418a4a62668RocketMan]

SS2:EE is probably going to completely overhaul the multiplayer code, so any attempts to fix it in Old/NewDark would just be wasted effort.

it's fairly safe to say SS2EE will be out later this year. so yeah, any attempts to improve the MP in Dark games is indeed a wasted effort,

What in the fuck???

we certainly don't want to discourage you from SS2 modding,

Well then how about opening with something other than telling the new member he's wasting his goddamn time?  That'll make him stay and do more cool stuff right?  I mean even something like "Cool bro... but you're wasting your time" would be infinitesimally better than just opening with discouragement.

I for one think it's pretty great to have a career selection widget that works in SP.  I also think it's great that he learned a couple of things that make the game unstable... something worth sharing with the community and being indexed in google for anyone who wants to know.

[67418a4a62787voodoo47]

it was an accurate assessment of the situation. he is completely free to keep poking around MP if he wishes to, of course.

[67418a4a628d2RocketMan]

Yes, it was accurate.  I don't think I've ever questioned the accuracy of such statements.  This is about social etiquette and how to avoid putting others into a defensive state emotionally.  Stuff like indirectly saying GTFO... because legitimate reason blah blah, isn't appropriate, even if you're 100% correct or just stating a fact.

[67418a4a62ad7voodoo47]

well, I really don't know what else is there to say.

just for the record, anyone is free to do whatever if they deem it fun, there doesn't really have to be a reason or a goal. but if it's something completely out of whack, someone is going to point it out - a good example would be that SS1 mod that was trying to change the original, unsynced audiolog text so there would be no masculine references to Shodan anywhere. which made absolutely no sense, and it ultimately went nowhere (but it forced me to actually do a proper version).

another example would be that guy who was basically redoing his own SHTUP from scratch a few thousands of years ago. I mean sure dude, but ZB kind of already got this, maybe you wanna try something else?

the intention here is not to discourage, but rather to make sure the state of resources at hand is understood properly, saving time, effort, and preventing frustration. remember that other guy who did his FM in olDark a few years ago just because he didn't bother to ask for the latest tools?


to wrap it up, an useful MP related project would be to examine the ecologies and AIs, and create a dml mod that would adjust the difficulty accordingly in a smart way. as SS2EE will (most likely) accept dml mods, this would be something that would be useful in the future as well.

[67418a4a638a7RocketMan]

I don't want to belabour the point but since you've given the same response a few times now (people are free to do what they want), I'll try a different approach.

People having free will has nothing to do with their inclination to exercise that free will.  And that inclination is emotionally based.  That's why robots and computers have no motivation.  So there is a causal relationship between how a person feels and what choices they make. 

Example:  New guy makes a replacement wrench.  Feedback from another user:  "I think this wrench is uglier than the original and the UV mapping is done poorly.  Based on this, you shouldn't bother spending any more time on it."

100% factually correct right?  No problem... right?  After all, the guy has FREE WILL.  He is free to keep working on what he's interested in.  In reality however, that post will very likely cause a number of emotional responses.  It may make the person feel:  anger, depression, hostility, self-doubt, resentment... you get the idea.

Now let's take a look at a different response: "Welcome to the forums!  Good to see active modders after all these years.  As for the wrench, I noticed something weird going on with the lighting.  Have you checked your UV mapping?  Also the texture is a bit incongruent with the original artistic design in my opinion.  However I'm looking forward to see where your next attempt goes!"

You get the same points across but without obliterating the person's ego and self-esteem.  You might argue that the truth hurts and you can't fix something that is self-evidently shit.  This is true but as emotional beings we have a duty of care to convey information without causing harm when and where possible.

It wouldn't be something worth harping on except that in a forum, the whole point (one of the chief ones anyway) is to create a stimulating and fun environment to enjoy whatever topic or theme the forum is about.  Sticking to the 4 corners of the forum rules is necessary but not sufficient.  You can commit offenses against others without ever breaking the rules.  That's why I contend that it's in the forum and its members' best interests to maintain a non-judgemental tone whenever possible.  Non-judgemental in the context I'm using it doesn't mean you can't judge others' work.  It means that your judgements are of the "stuff", not the person who gave it.  And to understand why saying "this works sucks" is an attack on the person and not the stuff, you have to understand how people attach their own identity to the "stuff" they make.  That's why, when you criticize a piece of work, you have to put a face on that work, so that you are mindful of how you would treat that stuff, if it were a person.  Because in fact, it kind of is.


EDIT:  I don't want to give the impression that I'm not reading your post properly so really quickly I'll touch on the points:

just for the record, anyone is free to do whatever if they deem it fun, there doesn't really have to be a reason or a goal. but if it's something completely out of whack, someone is going to point it out - a good example would be that SS1 mod that was trying to change the original, unsynced audiolog text so there would be no masculine references to Shodan anywhere. which made absolutely no sense, and it ultimately went nowhere (but it forced me to actually do a proper version).

Nothing wrong with pointing out things that don't make sense but it's really not that hard at all to just explain these things with neutral language.

another example would be that guy who was basically redoing his own SHTUP from scratch a few thousands of years ago. I mean sure dude, but ZB kind of already got this, maybe you wanna try something else?

Yeah, again, makes sense to me.  But to avoid discouraging someone who is up to their waist in a redundant task, why not get to know them a bit and then invite them to do something else?  Some forums have self-introductions when you register, where you can say what your skillsets are or what interests you but it never hurts to ask, if this info isn't volunteered.

the intention here is not to discourage, but rather to make sure the state of resources at hand is understood properly, saving time, effort, and preventing frustration. remember that other guy who did his FM in olDark a few years ago just because he didn't bother to ask for the latest tools?

It's nice that seasoned users like yourself are around to guide noobs in the right direction to save everyone a ton of grief.  The thing is, good intentions aside, our actions speak louder.  God knows how many times I've been taught this lesson.

an useful MP related project would be to examine the ecologies and AIs, and create a dml mod that would adjust the difficulty accordingly in a smart way. as SS2EE will (most likely) accept dml mods, this would be something that would be useful in the future as well.

Stating this verbatim from the beginning would have been a huge step in the right direction as it's not nearly as provocative.

[67418a4a639fdvoodoo47]

yes. and whatever question that person might have, no matter how weird or redundant the project may be, I will answer to my best knowledge if I can.

can't do much more than that. also, not putting any faces on, what you see here is what you are going to get. funny how an internet forum avatar represents the most real version of me that exists.

[67418a4a6459asarge945]

I actually agree with that, a polished up, fully overhauled 2 player MP would be an interesting way of playing SS2.

that is what I would focus on, if I were NDS (assuming they want to deliver fully enhanced MP experience, not just fix the technical issues).

Serious question:

How much of this is possible with Squirrel/DML?

Is there a way to disable the engine-level player respawning? That would be the big one, as I feel coop is at it's best when there's a way to allow the players to actually lose (they could still use QBRs etc).

I don't want to belabour the point but since you've given the same response a few times now (people are free to do what they want), I'll try a different approach.

People having free will has nothing to do with their inclination to exercise that free will.  And that inclination is emotionally based.  That's why robots and computers have no motivation.  So there is a causal relationship between how a person feels and what choices they make. 

Example:  New guy makes a replacement wrench.  Feedback from another user:  "I think this wrench is uglier than the original and the UV mapping is done poorly.  Based on this, you shouldn't bother spending any more time on it."

100% factually correct right?  No problem... right?  After all, the guy has FREE WILL.  He is free to keep working on what he's interested in.  In reality however, that post will very likely cause a number of emotional responses.  It may make the person feel:  anger, depression, hostility, self-doubt, resentment... you get the idea.

Now let's take a look at a different response: "Welcome to the forums!  Good to see active modders after all these years.  As for the wrench, I noticed something weird going on with the lighting.  Have you checked your UV mapping?  Also the texture is a bit incongruent with the original artistic design in my opinion.  However I'm looking forward to see where your next attempt goes!"

I think what ZB and Voodo are trying to do (albeit in a somewhat blunt way) is stop people from putting in a bunch of effort, then uploading it to the forums, then having nobody use it.

As discouraging as it is to be told that your work is pointless, it's 1000 times worse putting in (in some cases hundreds of hours of) effort only to have nobody care once you're done.

I can speak from experience on this. I have put literally weeks of fulltime work into this to get 4 downloads. That's doing far more damage to my willingness to mod going forward than any comments about quality ever have. I'm sure @RoSoDude feels similar stress and discouragement after he has put significantly more work into RSD and it's still relatively unknown. That's what sucks for modders, not some blunt criticism. If I was modding something that ultimately turned out to be pointless, I would want to know upfront.

On a COMPLETELY unrelated note, I don't really think EE is going to take over and become the de-facto standard way to play the game like everyone here seems to think. Unless they can get essentially perfect mod compatibility (a nearly impossible task), there will always be demand from people for the NewDark version. Especially when there are people who can't really play vanilla anymore. The assumption is that EE will support DML, maybe even squirrel, but to what extent? Will it be a perfect implementation? That's an extremely lofty goal, not one I think they will reach. Squirrel especially has a lot of weird quirks to it (both in the language itself and how it interacts with SS2) that I doubt will be replicated 1:1.

[67418a4a65023RocketMan]

As discouraging as it is to be told that your work is pointless, it's 1000 times worse putting in (in some cases hundreds of hours of) effort only to have nobody care once you're done.

I hear you.  My point was more about the style of communication rather than the content.

On a COMPLETELY unrelated note, I don't really think EE is going to take over and become the de-facto standard way to play the game like everyone here seems to think.

Agreed.  I guess time will tell but I'm not planning to switch.

[67418a4a65183voodoo47]

as far as I can tell, the only thing you can currently do is dml tweak the AIs to pose more threat. neat things like allowing the psi guy to cast spells onto a fellow player is probably not doable without source code level overhaul.

SS2EE will have to support squirrel, as this is essential for running SCP, dmls also have to be supported as a lot of mods use them, same goes for mtls. it is possible they will not be supported fully (meaning, a conversion of some kind may be necessary), but some support will have to be there.

[67418a4a65b06figgis]

That is sure a lot of posts. Remember: these are optional changes, I'm not forcing them into the next version of NewDark and making you play the game with 32 players at gunpoint. Also like I already said I don't consider it a waste of time.

Regardless, I think looking into scaling the difficulty with amount of players in general is definitely worth it. Even something as trivial as scaling the enemy health values and spawn rates, if such a thing is possible to do. Perhaps it could be possible with scripts? If SS2EE will have support for those, maybe the resulting mod could be used in that as well.

allowing the psi guy to cast spells onto a fellow player

Is there a way to disable the engine-level player respawning?

These are also very good ideas. I feel like it's probably doable with some gross script hacks, but I'm not sure of the details.

As for progress on the list of unsolved problems, I determined at least one possible cause of the crash when an extra player joins an existing save.
There's a block of data in the save containing the "player number <-> ObjID" relations for every player avatar, under the "AVATARS" tag. When the game loads the save in a netgame, it will unconditionally attempt to extract the avatar data out of that block for every joining player, even if the player index is outside of the bounds of the avatar array, which triggers an out of bounds read and makes it explode most of the time.

A possible solution is to patch the "player created" message handler to create a new avatar object for player numbers that are out of bounds, since it already has that logic in it for when you start a new game, but that goes beyond hex editing an int, and so far my attempts have been unsuccessful. Not to mention making more edits to the EXE like that would lead me into DLL hook territory.

Another possible solution is editing the save, which would involve changing the saved number of players, adding a new player object and avatar data block. As far as I can tell there aren't any existing tools that'd allow me to do that and I'm not sure how to make changes like this sync properly.

Update: after looking into the respawn thing a little bit, the default respawning can be disabled by preventing the "FinishDying" timer message from being handled by the PlayerScript. This can be done by blocking it in the OnTimer handler of a custom script, but for that to work said script has to execute before PlayerScript in the player object, i.e.

Code: (gamesys.dml) #

+ObjProp "The Player" "Scripts"
{
  "Script 0" "NetPlayerRespawn"
  "Script 1" "PlayerScript"
}
Feels like this is bound to break something.

[67418a4a65db3figgis]

After screwing around more with the respawning stuff, I made a small mod that adds alternate respawn logic:

• you can only respawn automatically if the QBRM is active on the map, while you're dead the game checks for that every second (adjustable);
• reconstructing costs 15 nanites (adjustable): if you don't have them, you can't respawn even when it's active;
• if another player frobs an already-activated QBRM, all dead players will respawn at it;
• if the map has no QBRM, you respawn as normal (can be turned off).

Upon moving into a different level all dead players still respawn as normal. Not sure yet if that's possible to override via script, nor if it's really needed.
There's also not really much you can do while being dead, unsurprisingly. Makes me wonder if some sort of spectator mode would be possible to implement.
I also don't know how to properly subtract nanites from the player that frobs the QBRM to revive everyone, so for now it's free.

I threw this together with the additional net avatars .dml and the Engineering cutscene skip from SS2 Cutscene Skipper into a more general "netgame improvements" mod, ss2_netfixes. It is now attached to the original post. It should hopefully be SCP-compatible.

I only tested it with 2 players and didn't test it with any other mods, but I reckon it won't work with any that require attaching additional scripts to the player object. The way it detects that the mission lacks a QBRM is also an untested hack (you just place an object named NetNoQBRM into the mission), since I couldn't figure if it's possible to iterate over all objects of an archetype, or even just check if one exists on the map.

[67418a4a65f18Pacmikey]

.

[67418a4a66539sarge945]

After screwing around more with the respawning stuff, I made a small mod that adds alternate respawn logic:

• you can only respawn automatically if the QBRM is active on the map, while you're dead the game checks for that every second (adjustable);
• reconstructing costs 15 nanites (adjustable): if you don't have them, you can't respawn even when it's active;
• if another player frobs an already-activated QBRM, all dead players will respawn at it;
• if the map has no QBRM, you respawn as normal (can be turned off).

Upon moving into a different level all dead players still respawn as normal. Not sure yet if that's possible to override via script, nor if it's really needed.
There's also not really much you can do while being dead, unsurprisingly. Makes me wonder if some sort of spectator mode would be possible to implement.
I also don't know how to properly subtract nanites from the player that frobs the QBRM to revive everyone, so for now it's free.

I threw this together with the additional net avatars .dml and the Engineering cutscene skip from SS2 Cutscene Skipper into a more general "netgame improvements" mod, ss2_netfixes. It is now attached to the original post. It should hopefully be SCP-compatible.

I only tested it with 2 players and didn't test it with any other mods, but I reckon it won't work with any that require attaching additional scripts to the player object. The way it detects that the mission lacks a QBRM is also an untested hack (you just place an object named NetNoQBRM into the mission), since I couldn't figure if it's possible to iterate over all objects of an archetype, or even just check if one exists on the map.

This sounds extremely based.

Scaling health via script is probably possible - as long as you can get the number of players via script (not sure if you can), since Enemy Health Randomiser already sets enemy health, so I know it can be done.

[67418a4a668ffZylonBane]

This sounds extremely based.

What.

[67418a4a66b8dsarge945]

What.

https://www.urbandictionary.com/define.php?term=based