[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

On 12/4/2019 at 9:56 PM, NegativeOne said:

 If you find a nice way to replace sqlite with mysql or postgresql, please share. I don't have much time to experiment with that at the moment.

'Nice' would be an exaggeration, but for now and simply testing these changes should do. If there are no other ideas on how to collaborate on this, you can just use the linked github repository which I used for experimenting with this and will continue to use so we can all utilize and improve the same lines of code without the hassle of file sharing (imho version control makes life easier a lot ;)).

Mysql-Database Usage for now is simply an additional 5-line file "db.cfg" in the executable directory with the following format:

DB_IP
DB_PORT
DB_NAME
DB_USER
DB_PASS

Because of the dependency change from SQLite to MySQL however, it will probably not work right now with Linux (or at least not with the mentioned interop library which would have to be replaced with another one for MySQL), but if it does not yet runs ootb with another lib I am pretty sure to have that fixed for Linux by tomorrow, too, because I will have tested (and adjusted if necassary) it with your mono solution until then.

[Edit]: Actually, that dependency causes no issues, MySQl.Data.MySqlClient seems to be available in mono, I could just run it with my existing installation to which I think I have not added any non default libs in the past.

Later, I would like to add a more generic solution for configuration and also clean up the code a bit, but for now, it seems okay.

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

Hey, that's very nice, in a few days from a no-lobby-situation to being able to host everything under Linux. I really have to check that out.
And yes, Mysql mostly works, I just have to fix one issue I could not get right yesterday with it and I will have time today evening or on the weekend for that.
Maybe we should move the code to a public git repo? Seems akward if everybody works on his local copy and has to up- and download files selectivly... ;)

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

 

That is useful information NegativeOne, thanks. Glad to see that Public Domain Entry there, I oversaw that yesterday.

 

On 12/2/2019 at 7:37 PM, NegativeOne said:

I also wanted to test it with the CM Patch for Sacred 2 Gold, so I made this to enable the buttons on the multiplayer screen: https://www.dropbox.com/s/0kawzl5zkn99pwh/Sacred 2-CM-OpenNet-UI.7z?dl=1

It's for CM Patch 1.60 and if you unpack it into the game directory, it will create this file:

Sacred 2 Gold\pak\Windows\OutGameMenu\Internet.xml

 

And that is even better, because I have zero knowledge regarding modding sacred2, so this will probably help a lot, I will check that out. If you want to test interaction between players alongside with these unknown packages, we could also meet up on a test server.

Maybe we can take a few days to fiddle around with all these new possibilities and after that seeing how much resources the systems need to run fine and start a little public list or something alike regarding the things we need or can supply as individuals for a centralized community hosted infrastructure? Seeing how small the community is, it would probably be a good idea to keep the players together, I.e. don't split them over too many lobbies or servers but provide them as stable as possible with Sacred ;). I mean, this could be a perfect self hosted Christmas present to all of us...

[Edit] FYI: Just in case someone did not know: The sacred 2 gameserver runs without additional configuration under linux within wine (just tested it on a debian stretch server). So you can host the lobby standalone on a windows machine and the gameservers on a separate one. This makes hosting a little bit cheaper as you need a server with a windows licence only for the lobby itself but not for its registered gameservers which will probably consume the most ressources (and at least for me that is also way less of a hassle to manage... ;)). For the lobby however it seems rather difficult to let it run in a non windows environment on the first glance, because of all this .NET stuff, but if it is only the lobby running on windows (even the database could be separated easily from windows by using another driver than sqlite and as a nice bonus would allow shared user accounts between lobby instances) it might be acceptable in terms of costs. I still want to experiment with that a little more.

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

It took me a while because of networking and the weird issue that I needed to compile the debug target myself instead of the precompiled (release?) binary you provided (probably some missing dependencies on my side or something like that), but I was able to get this running under Win 10 and Windows Server 2016 or something like that. First I did a local test, then on the server and set up a lobby there and registered multiple game servers to it (not from localhost, but from a different location meaning even splitting the load between lobby and actual gameservers works which is great) - and accessed them successfully over WAN, so yes this works!

Reconnecting to the lobby after tabbing out of the game seems to cause issues sometimes (connecting with no response), but those are solvable by providing wrong credentials once (which will trigger the "invalid parameter" response) and after that connecting will work again. But it is also possible, that this was already the case with the original lobby, as I remember, it was always a little bit wonky.

In my opinion your work is pretty much the greatest contribution to Sacred 2 since the CM Patch, I can't say "thank you!" enough.

Do you have any specific plans on public deployment if there are no serious bugs found after further testing? Or are we free to use your code to host services as we like? With a bit of coordination aside from private usage, we could create some kind of unofficial "hub" domain for game servers hosted by the community which would be epic. Also, if the CM-Patch guys are still active, they could probably bring back the "Open Net" Button which they removed from the Multiplayer menu which could allow usage not only for the base game, but also the more stable and better versions

@ameaeth: Sorry for my massive delay with the VPN Server, but at least I have a perfect excuse now to delay that even further: The VPN will not be necessary anymore because we will have something way better now thanks to cocomed which is worth every server owners full attention ;)

I have to go to work in four hours, so testing today ends here for me, but I probably have to start experimenting with dockerizing your application in the next days as I prefer my linux servers or rent  another windows machine if that is not possible.

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

1 hour ago, cocomed said:

Sacred 2 Lobby Emulator v1.0

Download on Mega

cocomed, this is far beyond awesome, thank you sooo much. In the last weeks I did nearly nothing, just took a quick glance at your logs (but nothing more) and did not even find the time to finish that lousy guide regarding VPN or answer your first post and in the meanwhile, you did all this by yourself? Have you already worked longer on this previously? Is this massive Payloads.cs exported from ghidra/IDA or something like that? Because all that looks like quite a bit of work.

Seems like I have to skip my plans for today, because I need to compile and test your application right now...

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

On 11/9/2019 at 12:27 PM, ameaeth said:

This sounds perfect and of course I'll wait for your notes patiently. I was sad that multiplayer wasn't really available anymore but if it works with a VPN maybe there is an option also to eventually have a Dark Matters community VPN for this purpose. 

Regarding the idea of a community VPN, imho this is a rather difficult topic. Or at least, there are quite a few reasons I have avoided providing web services like that publicly in the past and currently only hosting them for close friends I know and trust (but feel free to soothe my mind on this ;)):

Usually I would not only provide the peer-to-peer connection between clients, but also route all network traffic of the client over the VPN (including traffic of web browsing for example). On the one hand this simplifies the configuration effort on the client-side and on the other hand I remember a few cases client computers redirected part of the game network traffic wrong resulting in some games not working correctly over the VPN which I can circumvent by just forcing everything to resolve over my server. But of course, this means I have to install some kind of bandwidth quota for not having that one guy watching a youtube-stream at 8k-resolution or seeding a few torrents in the background ruin the fun of everyone else with latency spikes during gameplay.

Also, there is legal stuff: If I provide a VPN, I am responsible for the traffic it routes. If somebody decides to not just use it for gaming but route illegal traffic over it instead, in the worst case I have to deal with it. Of course, I could try to at least be able to proof it is not my fault in case something bad happens by logging usage extensively and of course stating I prohibit such usage of my service, but that would lead straight to the next issue, namely protection of privacy and over engineered laws like our German DSGVO, meaning, I would have to provide some kind of opt-in / terms of service to be fine in terms of law.

Furthermore, since everyone exposes his computer like in a local network to other participants of the same network, security issues arise. I remember someone once in a virtual network exposing his complete C:\ partition as a shared network drive without password protection. That was probably convenient for him in his home network to copy files from one pc to another, but is of course a complete disaster with unfamiliar internet users in the very same network. Briefly speaking, the VPN is just an additional risk for inexperienced internet users.

Also, as the operator of such a server, it will consume quite some of your time to host it. Some Users will probably complain to you over certain issues which are caused by themselves, but can't be fixed by themselves needing you to support them like for example:
- their own firewall blocking port 6800 for a hosted server
- their antivirus snake-oil program decides to silently block your openvpn application but not tell anyone about it,
- a broken* operating system which resets the network configuration all the time (*the user will claim his PC is alright :P)
- that one special guy insisting you have to make your guide working for his beloved windows vista
Such and other things are borderline fine for a few friends but a massive headache with dozens or hundreds of people... ;)

Concerns like that led to my wish/hope to reverse engineer the old lobby network protocol instead of building various duct-tape services around it. It would be a safe network (for both server administrator and client), limited to network traffic of only Sacred itself and minimal to none client side effort.

And btw. I think it might take me a few days more writing my stuff, because instead of just cleaning up my notes of my current setup (like I intended originally) it appears to me now like a much better approach to configure a fresh server from zero up and documenting what has to be done to do so (and actually checking it still works like intended a few years earlier), to be sure I don't miss out any things which might have been already configured on my currently running servers implicitly in the past but are also relevant for the networking of the VPN.

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

40 minutes ago, ameaeth said:

Hi xrystal,

do you by chance have a guide or similar that'd enable people to set up such VPN in order to play Sacred 2 coop? Or would any VPN guide from the web will do?

Best regards,

Martin

Hey ameaeth, sure, if you can wait round about till tommorrow evening, I can write up my own notes into something comprehensible and provide files which should work out of the box for especially this purpose on any cheap virtual server for max. 3-4€/month. And yes, generally speaking VPN guides from the net will do (probably easier if you know a bit about networking and server administration, because often you find configurations which are similar to your own goals and environment but not exactly equal needing a few adjustments), but you need to make sure you choose a configuration with peer-to-peer enabled and a TAP interface since Sacred needs to Broadcast network trafic which will not work on TUN based solutions.

If you want to try out how this works for the client after setup beforehands, you (or of course anyone else) can also pm me and I can send you a temporary access to a running vpn (for testing purposes only, not as a long term solution and also tomorrow since I am not here for the rest of this day ;)). And, furthermore, seeing that german flag on your profile picture, at least in personal messages don't hesitate to write me in german as I will probably be able to answer faster in that language

Also, totally unrelated at this opportunity @gogoblender, thanks for the warm welcome

[SACRED 2 ONLINE MULTIPLAYER: HOW-TO, DISCUSSION & DEVELOPMENT]

Yes, Sacred 2 Multiplayer is a thing again in almost all of it's glory and this is how to play in the OpenNet without using any additional software:

Joining

1. Launch Sacred 2 (Gold or any version patched to Ice&Blood 2.65, Steam also supported) and go to Multiplayer > OpenNet
   If there is no OpenNet Button, download this file linked here, copy it into your game directory with the same folder structure and restart Sacred 2 before proceeding.
2. Create a new account:
   You need to fill out all fields, but may enter whatever you like - however: please do not enter your actual cd-key or real email address, just use fake data instead as it does not matter and I dont want to know those anyway, I just can't prevent the game on your end asking for it and sending it to me ;)
3. Enter the lobby with your credentials and select an existing character or create a new one.
4. Join any server which matches your game version (see server title and description).
   Joing a non-matching server will crash your game without further notice, so go to a "vanilla" server with an unmodded game and to a "cm160hf" one with the CM Patch installed for example.
5. Loot Ancaria

      Currently, there are pre-configured servers available for following configurations which are hosted together with the lobby (I.e. should be always online): 

•     "Vanilla" default game without any mods (all difficulties, free & campagin)
•     "CM Patch 1.60hf" modded game (all difficulties, free & campagin)
•     "Sacred2 Enhanced Edition 3.1" (all difficulties, free & campagin)

      Still missing on the list is Dimitrius's Addendum, but it will be added eventually.

Hosting a Server

1. First, ensure the port your game shall run at is accessible. By default, you want to make sure port 7011 is free in your firewall at least for UDP traffic. If you host it at home, you will have to make sure your internet router does allow & redirect that traffic, too. How exactly to do this depends very much on the network equippment you use, so it is not possible to go into more detail here for me in general - just ask please if you have issues with a specific setup and with sufficient information provided we might be able to help you with that.
2. Host your server, there are multiple ways to do so:

• Straight-forward approach, ingame in the Lobby

  After hosting, you will join your new server automatically and it will keep running aslong as your own game does. Others will see it in the lobby now and will be able to join aslong as everything with the port is fine. Remember however, that most internet service providers change their customers IPv4 address every 24hours which may result in sheduled disconnects of your server. If you or your players dont like this, running the gameserver on a server in some datacenter with a static address might be preferable. Depending on the available operating system, dedicated solutions like the following two might serve that purpose for you.

• Dedicated on Windows:
  Open a commandline (Windows+R Shortcut, input "cmd" and hit enter), navigate to your sacred2 installation folder where the s2gs.exe is located. Start it in the commandline with parameters. Example:

s2gs.exe -connmode=opennet -diff=silver -name="Example Server" -description="It would be kind to put a list of mods here if required" -numplayers=5 -type=free -lobby=us.sacred2.net -lobby_port=6900 -externalip=YOUR_IPV4_ADDRESS_HERE -port=7011 -lobby_name=YOUR_ACCOUNT_USERNAME -lobby_pwd=YOUR_ACCOUNT_PASSWORD

For a full list of parameters, see this post which is still up-to-date and valid.

• Dedicated on Linux (for example a server you own with a static ip address)

The parameters are the same, but since we are on Linux we will need to use wine to launch that windows binary. For that purpose, I have added a example server which is deployed via docker-compose. Just edit the .env to your liking (at least set your servers ip address) and you should be good to go. See this post for more about that.

Hosting a Lobby

1. You may also setup a complete lobby on a server yourself with it's own game server registry which you or other players could use then to play together by altering the client side "network.lobby_ip" in the        "OptionsCustom.txt" to an address you host it on. This is technically a bit more involved and I suggest reading the whole discussion in this thread if you want to do that as it describes how this all currently works (my lobby server runs that way, too). A copy of the lobby files is hosted at github (yes, needs update, I know ;)) and it is based on this user's post.

I know many information in my post are a bit too general and would also be easier to understand if not only in textform - they will be fleshed out a bit more, soon enough, but for now, this loose collection will have to do. 
Oh and btw., even while only testing this in a very small scale I still already had the pleasure to meet a few nice people on the servers and for me thats basically what this is all about, bringing the players together, so in terms of purpose this little project already succeeds

Thanks to all of you who made it possible to get it this far (and maybe even further)!

Old original entry post as of 10/13/2019, for reference:

Spoiler

 

Hello,

first things first, I know this is an oddly specific request, however, since asking is cheap and neither time nor success is crucial on this (still it would be very convenient ;)), I will put it anyway: Does anybody has - maybe somewhere hidden in his always growing pile of data or on a old, dust covered hdd lying lonley in some dark corner - network traffic probe files which may randomly include sacred 2 lobby traffic and would be willing to share them (for wireshark, the filter to check for potential traffic would be "tcp.port == 6800" in files, if I remember right, from somewhere between october 2008 and april 2012) for analysis purposes?

I am currently looking into the tincat3.dll and it seems like an actually not impossible, but still time-consuming task to reconstruct the packages sent. I am willing to try this, but also I am more or less only able to work on my rare free weekends on this, so this would take a long time and long time projects in my experience tend to fail. Further more, since the first Sacred - accoding to this thread - uses the older version tincat2.dll which seems quite similar, it would be likewise useful traffic as well, probably not only for me. However, because of it's age I assume the odds are even worse?

Playing with friends on a VPN works nice & flawless, but is for sure not convenient enough for the majority of other players and I kinda miss the simplicity of the old lobby system - even if it never was that busy there, the people you met were always nice to play and talk with and you did not have to scare them away with technical client-side stuff like you are nowadays forced to...