Friday 7 September 2012

Tech Feature: HDR Lighting


Introduction

Hello my name is Peter and I’m the new graphics and engine programmer. New is not really the correct word since I have been working at Frictional for a year now. During this time I have updated the engine and added a lot of new graphic features. This will be the first of my blog posts descripting the changes that have been made.


HDR


One of the biggest changes to the the new engine is the introduction of HDR (High Dynamic Range) Lighting. This is a technique to increase the detail of the lighting system. The benefit of using HDR is that bright things can be really bright, dark things can be really dark, and details can be seen in both.

In nature there is no limit to how bright something can be. The difference between a 60 W light bulb and sunshine hitting the earth is around 10 000 luminance (cd/m^2). This means that we need a way to store high intensity values while keeping the quality and precision of the dark areas. Thankfully there already exists a method for storing such values - by using floating point numbers.

We use a 16-bit fp RGBA buffer to store our lighting. This gives us enough of a dynamic range without taking up too much memory.


Tone mapping

A normal computer monitor can display 8-bit colors between the value of [0..1]. Because of this the monitor can not display a 16-bit HDR image directly. To be displayed the image will have to be converted to 8-bit while keeping as much of the details as possible.

Tone mapping is the process of converting an image of dynamic range to one with a clamped range between [0..1].

The simplest method for doing this is to use the Reinhard tone mapping algorithm.

vec3 color = x / (x + 1.0)

No matter how high x gets the final value will always stay between [0..1]. The problem with Reinhard is that it desaturates your dark colors and removes contrast. In the brighter parts of the image Reinhard produces great result with its soft highlights.

What you want to have is an algorithm that preserves the saturation of the color in the dark areas and that keeps as much of the contrast as possible.

I ended up choosing an algorithm created by John Hable for Uncharted 2.

vec3 Uncharted2Tonemap(vec3 x)
{
float A = 0.15;
float B = 0.50;
float C = 0.10;
float D = 0.20;
float E = 0.02;
float F = 0.30;

return ((x*(A*x+C*B)+D*E)/(x*(A*x+B)+D*F))-E/F;
}
Left: Long curve, Right: S-curve near dark values



This algorithm is based on a filmic tone mapping algorithm created by Kodak. It keeps the nice highlights from Reinhard while using a slightly S-shaped curve too keep the dark colors saturated.



Exposure

If you have been sitting in a dark room for some time and walk out into the sun, your eyes will not be ready for the bright light and you will have to squint. After a while your eyes will have adjusted to the light and it will not bother you anymore.

Exposure is a way to control the intensity of light that gets passed through the lens. In the eye this is controlled by the size of the pupil and in a camera it is done by selecting for how long the sensor should be active.

There are a few different ways to control exposure in a game. It can be controlled automatically by storing the average luminance over a few frames and calculating a exposure from that value.
We choose to go with a much simpler method that lets the artists control the exposure by dividing the level into areas that have different exposures. So in a dark area the exposure can be increased and in an outdoor area the exposure can be decreased.

vec3 color = Uncharted2Tonemap(scene_color * exposure)


White Point

A white point is used to increase the contrast of the image. This is the value that is selected to be the brightest any pixel can be. Pixels brighter than the white point will be clamped to 1.0.

vec3 color = Uncharted2Tonemap(scene_color * exposure) / Uncharted2Tonemap(white_point)

HDR Bloom

When a pixel is brighter than the white point it can be used to generate an additional post effect called HDR Bloom. Pixels that get too bright should start bleeding over to other nearby pixels, producing a halo around them. It is a subtle effect that adds realism to the image.

To solve this I use a few render passes. First a bright pass is applied to the image which removes all pixels that are below the white point and scales down all the pixels that are above it. The result is then blurred to create the halos. The blurred image is then added to the original image. The HDR Bloom effect must be performed before the tone mapping.


Final Thoughts

I would say that HDRL and filmic tone mapping is the most important part of any rendering pipeline. It greatly increases the quality of lighting and will make your game look much more realistic.

But HDR and tone mapping is all for nothing if your calculations are not done in linear color space. My next tech feature will focus on gamma correction and the linear color space.

References


38 comments:

  1. This is the kind of love and attention that makes the games from Frictonal so perfect. You guys are the pillar of survival horror today, please, keep up the excelent work!

    ReplyDelete
  2. This is Greek to me, but from photography I know that light is everting you need do tell the story, so great you improve on that!

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Thanks for this nice explanation!

    PS: Just after finishing this post I re-read the post title and thought: He didn't explained the "HDR lightning". So I'm still curios what awesome thing this "HDR lightning" might be ;-)

    ReplyDelete
    Replies
    1. Maybe I should have been a little clearer in the HDR part.
      HDR Lighting stands for High Dynamic Range Lighting. It basically means that you can have really bright scenes or really dark scenes where you still can see a lot of details.

      A computer monitor can not show HDR images directlyy. That is why you have to use tone mapping to convert the HDR image to a normal 8-bit image, like the png files in this post.

      Delete
    2. Urfoex probably refers to your blog post title saying "lightning" instead of "lighting". ;)

      Delete
    3. No, Peter, Your explanation was already good above.
      But thanks for clearing that again :-)

      It's just like Christopoh sais. To much "lightning"

      Delete
  6. Thanks for blogging again a bit about your progress with HPL3 and your upcoming game. These posts are very much appreciated!

    To be honest, I hardly see any difference between the three tone mapping samples. I'd say the grunt looks best with the Uncharted2 lighting, while the dark corner in the background looks better with the Reinhard formula since its color is not clamped to pitch-black like in the other two images.

    ReplyDelete
  7. HDR is a nice feature to look at but how does it perform on low-end hardware (nVidia 8400/Radeon 3200)? If it is performing badly, why should anyone of your customers care about HDR lighting when your game mechanic is suffering due to the poor performance?

    Tim Cain put it quite succinctly in a recent RPG Codex interview: I care more about a game being fun than being beautiful, because no matter how good you look, people will move on to the next pretty thing and forget about you. If you make a fun game, people will remember that.

    ReplyDelete
    Replies
    1. Tim Cain has a very different design philosophy than Frictional Games, probably because he's not making horror games. Amnesia isn't supposed to be fun, it's supposed to be scary, and when it comes to creating a disturbing atmosphere graphics do matter.

      Delete
    2. Friction games don't make RPGs, they don't use traditional game design either. There is no fun core in their games like RPGs. They make ARGs. When you want make a deep immerse experience that can touch people in a emotional way, realism is a vital aspect. Besides, If graphics don't matter to you and your PC can't take it just turn down the graphics settings. Its that simple.

      Delete
    3. You are talking about cards that are already 6 years old. When a game using HPL 3 is released they will be even older. Amnesia was created to be run on mid range graphics at most 4 years old, expect about the same for this.

      Delete
    4. "HDR is a nice feature to look at but how does it perform on low-end hardware (nVidia 8400/Radeon 3200)? "

      You must be joking right? Does cards are really slow and old. So i ask why should the world of PC gaming stagnate because people don't know they can buy a modern dx10.1/11 card at a reasonable price today? Also i'm curious, HL2 a seven year old game came with HDR does that run bad on your +-5 old card?

      Delete
  8. Great stuff. I am of the mindset that great lighting defines more than atmosphere. Great lighting can turn around a lot of things in game development. It's nice to see you guys playing with this... which makes me wonder what would this be used for. Keep us posted. Right now i am looking into better lighting myself, but for the 3ds. Afraid we can't really do HDR... but we are testing other areas.

    ReplyDelete
  9. Great post, keep these tech posts coming! =D

    Also I have a suggestion, have you already seen the new pixel shader based SMAA anti-aliasing algorithm?
    I think it would make a nice addition to the engine as well.

    http://www.iryoku.com/smaa/

    ReplyDelete
  10. Hey Thomas, thanks for updating HPL3 engine, but I think HDR is making the image more brighter. And I would prefer a dark horror game.
    HDR is a very good effect in general, but it could cause inapt brightness in horror games and this would kill the feeling of exploration.

    To the anti-aliasing:
    Which version of OpenGL is HPL3 using?
    There is a new anti-alisaing released called TXAA:

    http://www.geforce.com/landing-page/txaa

    http://timothylottes.blogspot.de/2012/03/unofficial-txaa-info.html

    I don't know it it's compatible with OpenGL, but to make a qualitative anti-aliasing would be great. SMAA is only like FXAA, not really effective, but better than no anti-aliasing. I think this is a tech-feature also could improved.

    ReplyDelete
    Replies
    1. I will look up different AA techniques when most of the engine is completed. We will most likely have support for a post-process effect like FXAA.

      We will probably support more expensive and better looking techniques, like SMAA, for people with more powerful computers.

      I dont know if we will be using TXAA, since it only works with the newest Nvidia cards for now.

      Delete
  11. Sorry, I don't mean Thomas, I mean Peter.
    Nice to see a new engine programmer, but what is Thomas doing now?

    ReplyDelete
    Replies
    1. I am up to other kinds of programming (AI for instance), design work, planning and all kinds of stuff I would rather not think about :)
      Before I often had to rush through tech stuff or just focus on the most crucial features. Now with Peter doing tech full-time he can put more time in it and he also has this uncanny ability of finding problems in my old code. For instance the animation loading is a A LOT better now.

      Delete
    2. Hey Thomas, thanks for the response!
      Seems that you @ Frictional Games had found a very nice talented employee. If Peter is able to find problems in your old code, I think you should be happy that he joined your team.

      I'm interested to know what game you're actually working and hope it will have your typical creepy horror atmosphere with more thoughtful puzzle oriented stuff like Penumbra.

      You guys are my favorite developer, really understanding the meaning of a game. I hope your next project will also be a melancholic and emotional horror game with even typical involvement and immersion.

      Looks like we can suspect two new masterpieces next year!

      best regards
      Martin

      Delete
  12. I'm in awe about the care and attention to detail you guys put into these games.

    I will admit, I was disappointed when the release date for AMFP was pushed back, but it's very reassuring seeing the reason why. You guys really do everything you can to make an immerse, rich experience. Something as subtle as the 'HDR Bloom' effect really adds wonders to create a realistic environment.

    Seeing these kind of posts (as well as the thorough reviews and analysis) really convinces me you guys are some of the best video game designers today. True artists in your field.

    Keep up the great work!

    ReplyDelete
  13. Will HPL3 support DirectX and 11?

    ReplyDelete
  14. Hey Peter,

    You mentioned you guys use 16-bit floating-point (divided over) RGBA. To allow colored lights I can imagine using the 4 bits per color channel; but what is the alpha channel used for?

    I'd like to see more of these technical posts for example I'd love to see how a scene is actually rendered step by step. Something like this article on Doom3's rendering would be awesome: http://fabiensanglard.net/doom3/renderer.php

    ReplyDelete
    Replies
    1. We actually use 16-bit per channel, so a total of 64-bit per pixel.
      The RGB channels are used for color. In the alpha channel we store the SSAO. The reason for this is so that we can choose if a light should be affected by the SSAO or not.
      This turned out to be faster than having an additional texture to sample in the shader.

      Delete
  15. The updated lighting looks very slick and bright. So where does that belong in a Horror game? The first Amnesia worked so well in that all the lighting was extremely murky. Without seeing this new lighting at very low levels, in different colors (alot of orange and blue), in conjunction with effects such as fog, noise or grain I can't see it's full effect on the future of the engine. For instance: how would this new lighting be used to create the prison level in Amnesia? Would it be as atmospheric and oppressive in it's lighting?

    ReplyDelete
    Replies
    1. HDR doesnt make the game any brighter. It is still possible to have as dark and murky levels as in Amnesia.

      All HDR does is that it increases the quality and precision of the lighting and makes it seem more "realistic".

      Delete
  16. If I redownload and replay Amnesia will I get the new HDR effects?

    ReplyDelete
    Replies
    1. Nope :) This is for an upcoming, non-amnesia, game.

      Delete
  17. @Peter:
    Just one question: from what I understand, you're storing your lighting-related in textures, so does that mean that the lighting calculations are implemented in graphics hardware? I'm asking because I'm interested in the implementation/optimization details. How is this kind of computation-intensive processing usually implemented in games - in the GPU, via a fragment and/or vertex program (shader), or maybe via FPU instructuions, SSE, or something else? (This is all somewhat abstract to me at this point...) What would you say is the best approach?

    ReplyDelete
    Replies
    1. We store all our lighting data in a texture on the GPU. So just like you guessed that means that we perform all lighting on the GPU.
      We use fragment shaders to calculate the lighting per pixel. Our engine uses a deferred shading technique, which means that we first render all the geometry and store it in a few textures.
      We then use the geometry data from those textures and render each light individually using special lighting shaders.

      To answer your last question you should always perform lighting and shading on the GPU. It is much faster than the CPU when it comes to computations that can be performed by multiple threads, which is really easy to do when it comes to rendering and shading 3D meshes.

      Delete
  18. your lighting-related in textures

    ==

    your lighting-related data in textures

    ReplyDelete
  19. Amazing work, nice touch with better result in service of fear.

    Best Wishs,

    ReplyDelete
  20. Awesome lighting you can giving to here and at this time many post lighting are available in market but you can giving really nice information about post lighting.

    lighting catalog promo code

    ReplyDelete
  21. Great post! Can't wait for post about gamma and linear color space, as I always had problem with this. Thanks!

    ReplyDelete
  22. so when will we see the next tech post?. How about making a long post describing all the changes that have been made, instead of waiting months for each post.

    ReplyDelete

Note: only a member of this blog may post a comment.