Static model import/export for Blender: development

[674261108c663nemyax]

As some of you know, I'm working on a Dark Engine static model addon for Blender:
http://sourceforge.net/p/blenderbitsbobs/wiki/Dark%20Engine%20model%20importer-exporter/
The tool is still incomplete. This thread is for discussions of ways to improve it, update announcements, and bug reports.

Latest snag (resolved 15 May 2014)
One problem that I haven't been able to tackle for a long time is packing lighting normal vectors (the ones at the corners of polygons). In Dark Engine models, the three-component vector is crammed into four bytes (with a terrific loss of precision for anything other than zero, one, 0.5 and the like). For example, the 00 0c 00 00 sequence in a .bin would be 0.0, 0.0, -1.0 (straight down).
Here's what the leaked model header says about the compression format:

// Each element, x,y,z has 1 bit of sign, 1 bit of integer, and 8 bits
// of fraction, for 10 bytes total.  So the x, y, and z values of each
// lighting normal can be extracted in the following manner:
#define X_NORM(norm) ((short)((norm>>16)&0xFFC0))/16384.0
#define Y_NORM(norm) ((short)((norm>>6)&0xFFC0))/16384.0
#define Z_NORM(norm)  ((short)((norm<<4)&0xFFC0))/16384.0

This bit about light extraction doesn't extract enough to shed on how you actually pack three floats to get there. Previously, I tried using the leftmost bit in each component for the sign and the remaining nine bits for the absolute value (an integer on the 0 to 256 scale), like so:

Code: [Select]

def pack_light(xyz):
    result = 0
    shift = 22
    for f in xyz:
        sign = int(f < 0) * 512
        val = int(abs(f) * 256)
        result |= (sign + val) << shift
        shift -= 10
    return pack('<I', result)
This resulted in obvious shading artifacts (http://5.firepic.org/5/images/2014-02/21/h26n6fgntr52.png). Then I tried using two's complement values for negative components:

Code: [Select]

def pack_light2(xyz):
    result = 0
    shift = 22
    for f in xyz:
        fi = int(f * (511 + int(f < 0)))
        result |= (1024 * int(fi < 0) + fi) << shift
        shift -= 10
    return pack('<I', result)
And this is the current situation. The results are better, but the lighting is still weird on movable subobjects.
It turns out that the compression format is a strange mix of plain values (as written by the first snippet) and two's complement values (as written by the second one), often for the same vector. Worse still, the two's complement values are not necessarily used for negative components. Can someone here help me understand when to use which method (or suggest a better solution)?

[674261108c997Olfred]

Could you give me the bits for each corner for both of the bin's inside the zip?
Numbered like in the picture.

[674261108cb50nemyax]

In that model there are only two shared lighting normals, which are the same as the polygon normals in the absence of smoothing:

• (0.0, -1.0, 0.0) / bytes (LE): 00 00 30 00 / binary: 00000000001100000000000000000000 / dec: 3145728
• (0.0, 0.0, 1.0) / bytes (LE): 00 04 00 00 / binary: 00000000000000000000010000000000 / dec: 1024

[674261108cc7bOlfred]

I see

So for this example.
test3.bin should result in something like
0.0, 1.0, 0.0
0.0, 0.0, -1.0
?

I somehow expect for test4.bin to get 0.5

[674261108cddeZylonBane]

Seems like looking at the part of the leaked source that actually uses the unpacked data would provide the best information on how to pack it.

[674261108d060nemyax]

Seems like looking at the part of the leaked source that actually uses the unpacked data would provide the best information on how to pack it.

Oh I tried that, and it was a humbling experience =)
I should probably have another go.

[674261108d1fbnemyax]

Update 0.1.20140515
Resolved issues:

• The lighting normal packing method is closer to the original; this should mean fewer smoothing artifacts and differences from traditionally-processed models
• Lighting on child subobjects is no longer broken

(Yes, these turned out to be two separate bugs.)

Please re-export any enhancement pack models that you used this addon for.
Download: https://sourceforge.net/projects/blenderbitsbobs/files/

[674261108d2f7Olfred]

Sweet.
But this means it's still not like the original?

[674261108d41cnemyax]

Well, I mimicked what I observed in the bsp output: 0 is 0.0, 1..256 goes up to 1.0, and 1023..768 goes down to -1.0. Wherever I looked, that seemed consistent.

[674261108d53dOlfred]

I see. Really nice work, I've been looking forward to this for so long.

[674261108d6adOlfred]

I'm kinda sad, but it seems like I already found some bugs again.

1. When you look at the nori (the green square sticking out of the bowl) from the front it looks fine. But when you look at it from the back it doesn't turn the alpha channel invisible and you can see a fine grey line around it. It's kinda hard to see on this model but would be bothersome if you have something with a bigger invisible area on a texture.
I converted the model to a bin through bsp.exe and there it is rendered fine. (Included this one in the bugreport.zip under "bsp converted").

2. When I place an Empty/vhot it doesn't get the position where it was placed.
You can also see it in the picture. The blue circle is where ShockEd displays the vhot, but it should be in the middle of the bowl. And that's where it is on the blend file.

[674261108d7f1nemyax]

Olfred
Try the attachment for the vhot issue. If it's OK, I'll push an update.
I'm not really sure what I can do about the fading alpha.

[io_scene_dark_bin-0.1.20140516.zip expired]

[674261108d90dOlfred]

I tested it out and the vhots are working properly now.
Thanks for the quick update!

I tried to reproduce the same error again I get with the transparent issue.
Strange thing. I made a new model and just created some similiar scenario in many different ways and never get something like that. I can only reproduce it with the noodle bowl model.
I'll be trying around a bit more tomorrow.

[674261108dac4nemyax]

Update 0.1.20140516
I've uploaded the vhot correction update.
Download: https://sourceforge.net/projects/blenderbitsbobs/files/

Olfred
The difference between the bsp and dark_bin seaweeds is that the bsp one is split across the middle. When the addon gets support for BSP sorting, it should do the same kind of slicing, and the models should have fewer problems of this sort in the engine.

[674261108dbfcOlfred]

I tried messing around a bit with it again to get rid of this bug or do some workaround. Now something really strange happened. Maybe it will help you out with something.

Whether it will be rendered correctly is somehow dependent of the angle you look at it.
I tried adding sharp edges or seperating it into different objects, but nothing helped.

[674261108dd33ZylonBane]

Forcing an object to render alpha against itself is always a bit of a crapshoot, especially 8-bit alpha. When I was reducing the texture for this down to 256x256, I got rid of the alpha layer entirely, and it looked fine in-game. This object has no need for it.

That being said, have you tried adding Renderer/Transparency (alpha):1 to it?

[674261108e114Olfred]

This object has no need for it.

You certainly don't see the importance of nori!

That being said, have you tried adding Renderer/Transparency (alpha):1 to it?

I don't even get what you mean.

[674261108e208ZylonBane]

How are you viewing that model in-game? Are you using DromEd?

[674261108e38dvoodoo47]

yeah, careful with that - editor transparency is not the same as ingame transparency, learned that the hard way when trying to do something about those trams.

[674261108e49eZylonBane]

I meant, was he launching into game mode from DromEd, or just by starting the game. The screenshots are obviously from game mode due to the crosshair and bloom.

[674261108e585Olfred]

I started up ShockEd and then went into game mode.
Don't know how else to test things as I have very limited knowledge about editing levels or game related stuff.

[674261108e6dcOlfred]

ZylonBane mentioned something about rendering alpha against itself and that it can give you troubles.
But when I used bsp.exe it worked without a problem.
I have spend some time trying to figure out why it's like this and finally came to a solution.
Using the 3ds2e tool and then looking at the .e file you only have like two objects (one for each texture).
But I still had the model seperated into different objects which just share a texture.

So what I did now is to merge the noodles model and the nori model into one object, and TA-DA! it's working.

EDIT:
On second thought... this doesn't make sense. Even when using bsp.exe it's still two different objects and they don't even share the same texture....
Well... IT'S WORKING! WHO CARES!!

[674261108e7e5nemyax]

Update 0.1.20140527
Fixed a bug where import crashed due to incorrect material parsing.
https://sourceforge.net/projects/blenderbitsbobs/files/

[674261108e8d7Olfred]

Always nice to see some improvements
And didn't you mean 0.1.20140527?