The end of blue tunnelmouths...

[guillyman]

Hello everybody,
I'm new to this so please bear with me! I've been reading with interest the correspondence regarding the "blue tunnel mouth" / "house in a blue box" problem, where a colour which should be transparent is actually showing up and marring what would otherwise be a very good route.
Various solutions have been put forward, including resetting the monitor resolution, and suggesting incompatibility problems with the video card.
However, I believe we're all missing the point. All the processing of the information intended to go to the display monitor is taken care of in memory. All the foreground objects are presented first, then any objects further back are introduced and only those parts which are coincident with the pixels defined as 'transparent' by our program are forwarded to our screen, a technique known as "ray tracing." It is at this point that our graphic card takes over. To keep it simple, the card will more often than not, rather than recognise a certain colour, take its index number from the palette used in creating a particular bitmap. This is not possible if an object has been made with bitmaps in 24-bit mode colour, so some confusion can result if a route is shown on different computers. The solution is at source: please, you lovely route builders, save your bitmaps with a colour depth of 256. By this we will not lose any quality, these are small bitmaps and many of them in the field do not use more than a few dozen colours.
O.k. enough of the theory, now for the fix... Firstly identify the particular bitmap in your 'object' folder which is giving the problem and open it in your favourite painting program. I use Paint Shop Pro. (It sometimes pays to treat the lot - sometimes the iffy bitmap can be a swine to identify!) Use the eyedropper tool and right-click on what should be our background, or transparent colour. This may not always be blue - it may pay to check the b3d / csv file which uses the bitmap. Check the image information: the image must have sides measuring in powers of 2, i.e. 2, 4, 8, 16, 32, 64, 128, etc. Now drop down the 'colors' menu and select 'set palette transparency.' If the bitmap contains greater than 256 colours you will be prompted to 'reduce the image to a single paletted background layer.' Click 'yes' and ok 'optimised median cut,' then check 'set the transparency value to the background color' and ok this, then save your file and Bob's your uncle, no more blue tunnel mouth! And since the manipulation of the graphics is done by the program it should also run on any computer regardless of graphic card...
Have fun, everybody, looking forward to some feedback. And maybe one of you boffins out there will explain to me why the size of those bitmaps must be to powers of 2!
Regards, Alan. :razz:

 

[Dennis]

Can't disagree with any of that ^^^, normally I do all my texture editing in 8 bit RGB mode and have been known to acidentally forget to subsequently convert to 256 indexed colour mode, which generally gives no problems with my set-up.

It's very rare that more than 256 colours are needed in a texture and when it is, breaking it down into smaller parts offers a work around; the only time something with more colours is really needed are backgrounds; combining various shades of green and brown for the ground with blue, white and grey sky is difficult in 256 colours.

 

[Simon_G]

Textures should certainly be reduced to 256 colour, but I didn't think that BMP format supported transparency. Which presumably is why it has to be specified in the object file.

 

[guillyman]

Yes, I think that's about right, Simon, it seemed that people had been getting confused with screen resolution and palette colour depth, i.e. the number of colours used within a certain palette used to create a bitmap. And Dennis, comforting to receive support from somebody so eminent in the field! Love your tutorials, keep them coming!
Just a final appeal to all you routebuilders, without whom none of this would be possible, please create your textures with 256 colour depth, then all the files will be truly 'portable.' Thanks, Alan.

 

[reschanger]

I see things differently. The problem is that BVE does not seem to support bitmaps with alpha channels (like PNG), but only BMP. However, BMP does not support transparency, so seeing through a texture is achieved through SetDecalTransparentColor (or something) in the corresponding CSV file, which serves the purpose of setting one specific RGB color to transparent.

The fact that sometimes blue (or other colors) ringes appear at the edges of tunnel archs etc. results most often from the fact that texture makers do not consider that transparency will only work for that one specific color set up and not for any colors close to it. So when a graphics program creates a little offset to this value, e.g. when painting, blurring, resizing etc., there will be pixels that are not of the exact transparent color and yet are intended to be transparent.

Saving bitmaps in 256 colors can solve this problem, because a palette is created that suits the bitmap, and most often, texture colors are quite different from the transparent color, so it may be the case that the palette generator creates lets say only one specific tone of blue (the transparent color), even when there are some colors not exactly blue in the image. The inexact blues will then become exact blues if there are closer to the exact blue than to any other colors in the palette. However, if the texture has many pixels that are not exactly blue, then the palette generator will assign more different shades of blue in the palette. As only the one particular shade of blue will be transparent, the others (at the rim of the tunnel for instance) will not be transparent in BVE.

So, it's most often the result of the texture creator who was not careful enough. Saving in 24 bit will then produce even more artifacts, because colors close to the transparent color will not be rounded to it at all. But if all pixels that should be transparent are of one exact color, then using 24 bit color images does not pose a problem.

I find 256 color images a thing of the past. Except in the case of hand-drawn route or cab textures, there will be plenty of colors in a photograph, and even saving this texture in a suited palette will not produce a good quality when playing fullscreen with a monitor of high contrast ratio. Saving in 24 bit does produce good results, not the other way round (for guillyman to notice). I actually hate driving BVE routes and see color banding all over the place, just because the texture maker decided on making 8 bit images, not 24 bit ones.

And by the way:

All the foreground objects are presented first, then any objects further back are introduced and only those parts which are coincident with the pixels defined as 'transparent' by our program are forwarded to our screen, a technique known as "ray tracing."

I'm sorry, but raytracing is something very different. What you describe sounds if at all then something like what the z-buffer does. Objects are rendered in the order there are stored in, neither from back to front nor from front to back. The z-buffer stores the depth of the currently evaluated pixel and when another object is rendered to the same pixel, it is first checked if the new pixel's distance is smaller than the already rendered one, so the pixel closer to the camera will be painted over. Raytracing is used to determine which object is visibile for a particular pixel, which can involve quite time-consuming calculations for intersections, and it is not employed in current PC or video games, because no hardware support is available for it and would not achieve interactive framerates. The z-buffer is done in hardware.

 

[Dennis]

I won't dispute that using 256 colour images is a serious compromise regarding the general apearance of the view, but a requirement remains to retain usable framerates through minimising the number of faces drawn and the size of the textures used.

For an example of what can be achieved, the Watford Junction - Rugby screenies shown here look fantastic, although this level of quality of pushing the limits.

The 'lighting' possibilities offered by using .x format are also underused in BVE - this may be partly due to the time consuming nature of building curved surfaces; quite a task for 'amateurs' with little time available and few learning resources.

Considering BVE more generally, there are only a small number of active developers - the time needed to build route miles / scenarios has to be balanced against not only the quality of the objects built but also 'playability' i.e. acceptable framerates, quality sounds, robust and error free running. Finally, the routes would be nothing without top notch traction - essential for a cab view!

BTW - thanks to Reschanger and all the other more technically minded people for moving things forward; with further development, the BVE scene can stay healthy.

 

[guillyman]

Yes, thank you Reschanger and Dennis for your comments and your support and trouble to explain things better than I can. On the ray tracing issue I was merely trying to illustrate that any colour manipulation should be done by the program and any problems arising with using using different video cards merely highlight a deeper problem and are not the fault of the video card itself. I believe I am correct in saying that these use an index to the colour required to be transparent and these may be different from one computer to another. Over to you, Reschanger!
I must say that backgrounds and earth textures, with the whole array of colours, where it would be unusual to require a tranparency can quite happily stay 24-bit. But as you say, Dennis, a lot of these tunnel mouths, trees, cab fronts, etc, can quite happily be put as 256 colour depth, as these can be made quite small, and I can quite honestly say that I have never really noticed a marked reduction in visual appreciation. Sooner that than big blue patches, it works for me.
Thanks again guys for your troubles. I would love to hear back from anyone who has tried the fix above, to see if it does really work on all machines. Happy simming everybody! Alan.

 

[reschanger]

Maybe I misinterpreted the problem. In my post above I thought about blue ringes at the edge of where transparency should start in a texture and where the opaque part ends. But the problem described seems to be a more general to me now: Everything that should be transparent is not. Right?

If so, this is neither a problem of 8 bit images nor 24 bit images, but can result from another fact: When BVE is set to 16 bit color mode or the desktop properties specifiy 16 bit colors ("High color"), there can be ambiguity on how the graphics card handles 16 bit. This is because in 24 bit we have 8 bit per RGB channel. In 16 bit, one cannot assign the same amount of bits to each of the three channels without wasting a bit. So in 16 bit, we can have 5 bits per each channel, or 5 bits for red and blue and 6 for green. The problem is that each color in a bitmap is now remapped to the closest available color from the 65536 colors 16 bit mode provides. This is ambiguous, however. For 5 bits per channel, one may think of colors which are usually in the range from 0 to 255 now as 0,4,8,...,252 (gives 2^5=32 different shades). No final 255 included here. There are different ways how to map 8 bits per channel (256 colors) to 5 bits (32 colors) or 6 bits (64 colors), however. If one wants to use the full range from 0 to 255 it get's harder, because 255/64 is not an integer (try assigning 64 values from 0 to 255 evenly distributed). You might create 0,4,8,...,255 by not always incrementing in steps of 4 here, but sometimes 5, so you can end up at 255.

Why am I telling this? The thing is that I don't know how DirectX or the graphics driver remaps the full range from 0 to 255 per channel when in 16 bit mode. But I know there are different possibilities and some graphics drivers/cards may do it differently than others. The problem is, when a BVE texture uses pure blue {0,0,255} (RGB) for transparency, this value might get rounded to {0,0,248} (at least in my example), but when the transparency in the CSV file is set to {0,0,255}, the remapped blue might not get transparent because the values don't match. This is how I would explain it, but again, I don't know how BVE, DirectX, the graphics driver and the graphics card interact here.

I almost never had problems when running BVE in 32 bit mode (which is actually just 24 bits after all) AND having the desktop in "True Color" (24 bits), and I used many different graphics cards from both NVIDIA and ATI over the last years and even tried on three different computers. The only cases when objects did not show up entirely transparent, it was a problem with the texture, which had inexact blue ringes.

So I guess this didn't help either, but was fun to write. (Not so much actually, but whatever...)

 

[guillyman]

Crumbs! Did you follow all that? I think I'm with you, and can certainly appreciate all the problems you outline there. As for my suggested fix I tried to look at the problems as a programmer, albeit not a vastly experienced one. On the computers I have access to the only bitmaps which gave problems were ones saved as 24-bit images, although admittedly there were some which did behave. In all cases the change to an image of 256 depth (not screen resolution!) did the trick. I only actually found one bitmap, I think in one of the Brazillian routes, where the background intended to be transparent had actually 'spread' in various pixels to something other than pure blue, i.e. 0,0,255, presumably due to earlier conversion of the file from a different format. As this was a one-off I put this down to "just one of those things" and didn't attach too much significance to it.
I'm pleasantly surprised to receive the responses to my thread, but still wait with baited breath to see if anyone has actually had success with my "fix." How about you, Reschanger, 'nuff of the theory, now for the action!
With regards, Alan.

 

[reschanger]

I can't agree. For me there is no action to do. What your "fix" concerns, you say that 24 bit images should be saved as 8 bit images, but this only causes that (after a suitable palette is generated) inexact blues get rounded to exact blues, so the actual problem is worked around, namely that textures are sometimes not done right by the editor. And although converting to 8 bit images might be the easy solution even for those making the textures, working properly in 24 bit as an image designer will not pose a problem, but will actually give better quality images. Your "fix" is a work-around, not anything else. And for those using 16 bit (BVE) or High Color (Windows), I posted my 16 bit theory. Use True Color and 32 bit, and if you still experience the effect you describe, name the route and download location (and the object of your desire) and I will try to isolate the problem.

 

[Dennis]

I would absolutely love to see use of true colour images for textures but not at the expense of framerates; any sections of route I try and code which are packed full of detail results in diminishing framerates (something other authors also find - Birmingham New Stree on Cross-City being a classic example of this.

This can be partially overcome by reducing texture size, sticking to 256 colours and minimising the number of faces being drawn. This is a compromise I cannot see a solution to, except to wait for PC specs to improve.

The other feature which needs improving is the 600metre drawing distance; it is most unrealistic the way objects can appear as if by magic when driving along. The biggest drawback is not being able to build any 'big' scenic objects; hillls and tall buildings suddenly appearing out of nowhere is so unreal. Some kind of graduation of drawing distance based on face size would seem the best solution; something (I think) which is employed in other sims.

 

[reschanger]

The other feature which needs improving is the 600metre drawing distance; it is most unrealistic the way objects can appear as if by magic when driving along.

I'm working on that. In contrast to the display resolutions, this thing seems to be far more complicated. So far, I had no luck. Maybe in the near future though...

 

[guillyman]

Yes, framerates do improve slightly, that seems to be a hidden side effect. Love it when driving from Edinburgh, 'instant' appearance of Forth Bridge too, perhaps we should introduce some other objects to hide big buildings until they are 599 feet away!
I think you missed a point, Res, All the objects with transparencies that I've had troubles with, and I believe I speak for others, are all saved as 24-bit and all have a clearly defined 'transparent' area of pure blue, a consistent 0,0,255. Without exeption the problem was solved with the reduction to 256 colours in the palette, with hardly no loss in quality as most of them used less than 200 colours in the first place! It is not a case where the 'blue' is inconsistent. The only exception to this was one bitmap I could not cure until I changed its pixel size to that of powers of 2. This I could not explain, I think it's something to do with the way csv and b3d objects are written.
Regards, Alan.

 

[Simon_G]

perhaps we should introduce some other objects to hide big buildings until they are 599 feet away!

If you look at the way routes are laid out, this is done quite often, but it isn't always possible.