Using Deep AI to Retexture Sacred 2

[Flix 5,116]

Using Deep AI to Retexture Sacred 2

Skip to the bottom if you don't care about the how or why and just want to see cool pictures.  

Introduction:
Traditionally, modders who wanted to improve texture quality in games had to resort to: 1) upscaling the old textures using basic resizing functions in image editors, or else 2) completely retexturing from scratch using new source images.

The former option is easy. The modder would resize the texture in Paint or Photoshop, maybe apply a sharpening filter, and then present their grainy product, generally to a chorus of boos about how the textures had no more detail than before. Oftentimes they looked even worse.

The second option can have varying results depending on the skill of the modder, but is generally an extremely demanding project, requiring many hours of toil to retexture a whole game.

And so for years, players could only but pray that the developers saw fit to release high-res or elite texture packs.

Then Artificial Intelligence changed everything.

Convolutional Neural Networks (CNN's) allow for the upscaling of images using artificial intelligence. The AI has been trained by feeding it millions of pre-downscaled images until it learns to faithfully reproduce the original high quality image. This upscaling process is typically followed by a noise removal function that can be calibrated as needed to preserve details while eliminating unnecessary artifacts.

The resulting image is a true high resolution version of the source texture, with none of the blur, graininess, jagged stair-stepping, or other visual noise that traditional resizing produces.

============================

Application:
Like most aspects of Sacred 2, this rough-cut gem of a game has textures of varying quality. By default the game has two texture options, "Minimal" and "Default."  A third option, "Elite" is added if the elite textures are present.

Ascaron's official Elite Texture pack did indeed provide high quality textures for about 80% of the game, and I recommend everyone download the optimized elite texture pack, right now, available HERE.

However, all the textures from the Ice and Blood expansion, as well as many creatures and items released in various content patches, were not covered by the Elite Textures. There were also a number of textures that were already so small to begin with, that even the "elite" variants were quite low in detail.

Let's take typical Sacred 2 weapon texture dimensions:
Minimal - 128x128
Default - 256x256
Elite - 512x512

Items added in Ice and Blood would be stuck at the Default setting even with "Elite" selected. There are also vanilla items that started out much smaller, such as 32x32, therefore reaching only 128x128 even at "elite" quality. The most unfortunate items suffer from BOTH a very small starting texture AND a lack of an elite texture. So some legendary weapons that otherwise have very cool and unique designs are almost painful to look because of their grainy 64x64 textures.

This is the actual full-size texture for the Signet of Thunder legendary. 

 

This is where CNN's save the day.

============================

Examples:
Textures in Sacred 2 generally respond well to AI upscaling. By selecting an appropriate target size (typically 512x512 for smaller items, up to 2048x2048 for very large creatures), and then applying varying degrees of noise removal, textures can be upscaled to a satisfying effect.

Click and scroll to compare.

Vanilla Max Quality on Left   <-------->   New AI Upscaled on Right

     

     

     

       <---Generated Normal Map / Original Normal Map--->  

     

     

     

     

     

     

     

     

     

It's not always perfect; some of the really bad cases will require correctional painting by hand, like the grid on Arnum's Prevalence.

     

==========================

These are just the highlights from the first round of upscaling. I have some more textures in mind for a second round, including some Ice & Blood creatures and mini-bosses. I'll post more results as I produce them. If you've seen any other noticeably low-resolution textures in the game, let me know and I should be able to upscale them!

[xrystal 59]

That's pretty impressive. I have seen a few interesting models trained by coworkers for instance to read rust and dirt covered product numbers stamped into industrial products from bad quality images of these or detecting and categorizing street signs in real time and determined those to be actually good examples of practical and useful applications of this whole "black magic" ai-topic in real life, but creating details almost out of nothing in the context of a wonderful computer game makes me think this might be an even better personal favorite example for non-academic, actually graspable quality of life improvements made possible by algorithms... ;)

[Dax 481]

I assume it works with S1 textures too. May I send some in?

[Flix 5,116]

4 hours ago, Dax said:

I assume it works with S1 textures too. May I send some in?

I can take a look.  You could just make a list of the file names, since I have the S1 pak files extracted.

[Dragon Brother 619]

Thats quite impressive. Is this a simple enough process to do Flix? As in just upload files somewhere and it gives you a link to download the upscaled images?

[Flix 5,116]

1 hour ago, Dragon Brother said:

Thats quite impressive. Is this a simple enough process to do Flix? As in just upload files somewhere and it gives you a link to download the upscaled images?

Yeah it is pretty simple for the modder, considering the results.  There are some state-of-the-art, high end programs like Topaz Gigapixel that can accomplish this, but I'm neither rich nor a pirate, so I just shelled out $6 to bigjpg.com

Steps:

• bigjpg can't handle DDS files, so first convert the source texture file to a lossless format (I use BMP).
  • Note: You will lose the alpha channel and therefore transparency information. This means if there's already a layer mask you should remove it before converting.
• Upload the BMP file to bigjpg.com and click "Start." You can upload multiple files at once.
• Choose the size of the rescaled output. I typically choose 4x or 8x depending on how small the original image was.  I've yet to need to 16x.
• Choose the amount of Noise Reduction. Typically you want it at "High" or "Highest" but sometimes for intricate textures you lose too much detail so may need "Medium" or even lower.  I've also spent some time merging multiple outputs with different noise levels by hand to get the best result.
• Click "OK" to process a single image, or "Enlarge All Images Below" to do a whole batch with the same settings.  Then you can download the resulting JPGs.
• Next you have to convert back to DDS format.  It may also be necessary to restore the alpha channel and any transparency from the original texture. I also spend time adjusting colors that may have been muted, lines that aren't quite straight, etc.
• Sometimes I will also use a GIMP plugin to generate a new normal map, since bigjpg does not handle the game's original normal maps well.

And that's it!  There's a very limited version of the site you can use for free, which was enough to convince it was worth the 6 bucks for 2 months.

[dimitrius154 612]

Very Interesting. I see some medium-level blurring introduced, is that adjustable?

[Flix 5,116]

35 minutes ago, dimitrius154 said:

Very Interesting. I see some medium-level blurring introduced, is that adjustable?

Generally, the lower the "Noise Reduction" setting, the less unwanted blur and detail loss.  However, this also means more of the unwanted grain and other noise remains and becomes embedded in the image.  If an item is meant to look battered, worn and dirty, this noise may be ok.  For slick and shiny items with smooth, straight lines, I have to turn up the noise reduction more.  It's a constant search for balance.  In some cases I've tried merging multiple outputs, some with high noise reduction, some with little to none.

Some perceived differences in the above images may also be due to me generating new normal maps for some items.  I did this because the original normal maps for some items are very poor quality and cause noticeable jaggedness, even with the new diffuse maps.  I was unable to get the original normal maps to upscale. After converting back to DDS they had lost some information and resulting in the item being extremely dark in-game.

Even so, the proper normal maps are better at representing height and depth than anything I can generate with a plugin.  The white shield with blue orbs and gold inlays above is the best example of this.  With the original normal map, the gold inlays have more height, but the overall look is almost as jagged as the original picture, even with the new diffuse map.

[Androdion 875]

I guess my question about this would be just how taxing the new textures are to the game engine. Is there any noticeable difference in the game engine's response with the same settings and upscaled textures?

[Dragon Brother 619]

22 hours ago, Flix said:

Yeah it is pretty simple for the modder, considering the results.  There are some state-of-the-art, high end programs like Topaz Gigapixel that can accomplish this, but I'm neither rich nor a pirate, so I just shelled out $6 to bigjpg.com

Steps:

<snip>

Very simple, I like it!

2 hours ago, Flix said:

Even so, the proper normal maps are better at representing height and depth than anything I can generate with a plugin.  The white shield with blue orbs and gold inlays above is the best example of this.  With the original normal map, the gold inlays have more height, but the overall look is almost as jagged as the original picture, even with the new diffuse map.

Ah that would explain it - there was only one thing I felt different (in addition to the improved texture quality), but I couldn't describe it but reading this explains it perfectly

[Flix 5,116]

1 hour ago, Androdion said:

I guess my question about this would be just how taxing the new textures are to the game engine. Is there any noticeable difference in the game engine's response with the same settings and upscaled textures?

None whatsoever.  The difference between loading the original 60kb texture and the new 600kb texture for a single weapon is negligible.  Most weapon textures are already that size or larger (1.33MB is pretty common for weapons with Elite enabled).

None of the new textures are going beyond a size that's already established in Elite Textures.  Basically those items that were stuck in "minimal" are now up to snuff with the other 90% of item textures.

[Androdion 875]

That's pretty awesome then! 

[gogoblender 3,068]

Love this Ben. It's so elegant and razor honed on this game... targetting that remaining 20 percent to make them perfect... I love how the closer you are getting to this game...the more you "see" and want to elevate.

What a way to bring Sacred into the now  

And... is there other possibilities with Deep AI?  I read about Topaz... fascinating

 

ggoo

[Flix 5,116]

On 11/22/2020 at 8:44 PM, gogoblender said:

And... is there other possibilities with Deep AI? 

Oh yes. I'm convinced the only limits to the possibilities are our own imaginations.  It's advanced faster than anyone would have guessed.

Here's a bite-size video about the basic potential:

 

 

[gogoblender 3,068]

Re embedding... you're right.. I wasnt able to get the link to show as text while maintaining the link, but I placed youtube video in... I'll ask Schot to take a look at the code to see about mending this

gogo

[desm 329]

I'm not into all that, it exceeds me but it's brilliant. Where you are taking the game is awesome! Keep up the great work, long live Sacred!

[Schot 407]

Amazing exploration Flix.  Right up my alley, haha.  One of my clients is heavily into the Topaz collection.  She calls on me to clarify things for her on occasion.  When first saw my client had bought Gigapixel AI I admittedly saw it as a gimmick to add onto her collection of Topaz products but as the software evolves it's becoming more and more useful.  And what great use you're making of it!  The reprocessed textures are much improved.  I'm most impressed by the first sword in your list.  Interesting that the flat gold lines going towards the tips of the blades became embossed in the up-scaled version.  It doesn't seem to me as though there was any embossing of those lines in the original.  Pretty cool stuff!

    

P.s.
Regarding the strange forum behaviour of auto embedding youtube links; try the follwoing:

1. paste youtube link
2. link becomes a video
3. scroll down to just under the now embedded video
4. look for the message "Your link has been automatically embedded.   Display as a link instead.
5. Click on "Display as a link instead" to get rid of the embedded video and leave a link in it's place.

Somewhat annoying.  I know!

[Flix 5,116]

15 hours ago, Schot said:

The reprocessed textures are much improved.  I'm most impressed by the first sword in your list.  Interesting that the flat gold lines going towards the tips of the blades became embossed in the up-scaled version.  It doesn't seem to me as though there was any embossing of those lines in the original.  Pretty cool stuff!

It's been over 2 weeks since I first tried it and I'm still consistently impressed every time I get the results of a new batch.

About the embossing: I believe that is the result of improved normal maps.  Like the diffuse map, the original normal map was quite small, jagged, and low detail.  I couldn't upscale it, so I generated a brand new normal map in GIMP.  This is one instance where the generated normal map improves on the height and depth of the original details.

[Timotheus 416]

This just looks awesome. No other word for it. GG!

[Daedalus 88]

What a coincidence!! Because, believe or not, I started forming myself to work in nothing less than A.I. / Machine learning! 

And it's good to see it used to improve games like Sacred 2

That being said, I suspect that it's not just CNN that are being used but something rather new called GAN's -> Generative Adversarial network. And I assume that maybe what you are using originally comes from something called a 'model zoo' which is basically open source pretrained models. 

GAN's are a pretty complicated and new field in A.I. And while I have yet to be able to really make good models with it, what you're trying to do give me ideas and motivation to look further into the subject. 

And if people are interested, I can give an overview on how they work. Not tonigh (it's late here), but I'll start working on something to explain, in layman terms, how they work.

It will basically a story that use the past to explain the present and future (Hint: Go to places where you can 'speak easy' and drink some good booze. also, make sure to watch some old movie about ... I dunno cops who cannot 'be touched' or something  ).

I'll start working on it and post a good explanation when ready if ppl are interested (and of course, if I come to the point where I can make a model myself, which will eventually be doable, I might be willing to give a hand just for the fun of it and the practice ) 

[Flix 5,116]

Well here's a longstanding wish of mine fulfilled - no more blurry and jagged Dragon Mage tattoo!  The DM has a large amount of very low-res equipment as well.  I'll probably be reviewing those textures for upscale candidates this week.

Before:                               After:

   

[gogoblender 3,068]

55 minutes ago, Flix said:

Well here's a longstanding wish of mine fulfilled - no more blurry and jagged Dragon Mage tattoo!  The DM has a large amount of very low-res equipment as well.  I'll probably be reviewing those textures for upscale candidates this week.

Before:                               After:

   

Its a pretty good difference, another nice elegant touch

... I can see, monsieur Flix, that this is simply the genesis of an enormous  beginning...

 

gogo

[lacr 60]

Sadly, there are still many upscaling artifacts, while first 2 shields have changed awesomely, the fourth shield is looking even worse than original : it has lost it's volume, looking flat and very artificial

On 11/21/2020 at 8:27 AM, Flix said:

     

About dragon mage, well, I dunno, the AA of a modern card is doing all the necessary work:

Edited November 27, 2020 by lacr

[lacr 60]

P.S. : in this game we are watching our characters from a great distance, so texture resolution is not that very important. I have recently started to play total war warhammer 2 and my first impression was "wow, what a graphics!", but after playing for some time, I have noticed that most of the time, I actually don't see all that coolness due to the distance of my point of the view and, practically, the game is looking like total war medieval 1(released in 2002 y.)

[Flix 5,116]

7 hours ago, lacr said:

the fourth shield is looking even worse than original : it has lost it's volume, looking flat and very artificial

That's not the fault of the upscaling though.  That's me fooling around with normal maps since I haven't been able to upscale the originals yet. Later today I'll post what it looks like with the upscaled diffuse map and the original normal so it's a better comparison.

Almost all of the screenshots above have new normal maps that I generated so it's not always a bad thing. Sometimes the plugin just can't mimic the volume of the originals.

  

7 hours ago, lacr said:

in this game we are watching our characters from a great distance, so texture resolution is not that very important.

The "Default" texture setting would be appropriate for you then, I think.  If you're zoomed that far out then you'd likely be seeing the lower resolution DDS mipmaps anyway.