ATI + GLSL + setlocale + linux

[Julien Isorce]

Hi,

Working on gst-plugins-gl ( http://diracvideo.org/git?p=gst-plugins-gl.git;a=summary , and the branch : http://diracvideo.org/git?p=gst-plugins-gl.git;a=shortlog;h=remove_gstfreeglut )
, I am trying to resolve a bug about "ATI + fragment shader (from GLSL)  +  setlocale (from #include <locale.h>).

Well, shaders are not working if  setlocale (LC_ALL, "");  is called. And no problem if   setlocale (LC_ALL, "C"); is called.
I can see any relation bettween the 2 things.

And setlocale (LC_ALL, ""); is called in gst_init. (cvs/gstreamer/gst/gst.c). I have spent a lot of time to find that the bug become when setlocale is called with arg "".

Anyway, I have made a minimal test which only requires  freeglut (or glut) and have a graphic card which supports OpenGL and GLSL.
I joined this test to this mail:

./test 0   give the wrong result:  red color in the window. ( setlocale (LC_ALL, "");  is called)
./test 1   give the right result:  yellow color in the window. ( setlocale (LC_ALL, "C"); is called)

I have the bug on ubuntu intrepid (and Linux julien-desktop 2.6.27-9-generic #1 SMP Thu Nov 20 21:57:00 UTC 2008 i686 GNU/Linux)
I join also my glxinfo output and my xorg.conf (I have little bit fired my xorg.conf  but I had same bug with the default xorg.conf)
I have an ATI 9800 pro
fglrxinfo:
display: :0.0  screen: 0
OpenGL vendor string: ATI Technologies Inc.
OpenGL renderer string: RADEON 9800 PRO
OpenGL version string: 2.1.8087 Release

(It seems there is no problem with an NVIDIA graphic card)

setlocale doc: http://www.opengroup.org/onlinepubs/007908799/xsh/setlocale.html

my config:

echo $LANG
fr_FR.UTF-8

So what do you think I can do ? Bug come from fr_FR.UTF-8 environnement ?
Where should I report the problem ?

Please try the test if you have an ATI and GLSL and linux.

Sincerely

Julien



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Filippo]

On Sat, Nov 29, 2008 at 4:05 PM, Julien Isorce <[hidden email]> wrote:
> Hi,
Hi Julien,

> ./test 0   give the wrong result:  red color in the window. ( setlocale
> (LC_ALL, "");  is called)
> ./test 1   give the right result:  yellow color in the window. ( setlocale
> (LC_ALL, "C"); is called)

It seems that the ATI GLSL Compiler does some kind of localized string
parsing on the shader source.  When you call setlocale (LC_ALL, NULL)
it just sets the french locale, where the decimal separator is the ","
character. In "C" locale the decimal separator is the "." character.

So the decimal part of the floats in the shader source is truncated
because dot is not interpreted as decimal separator.
>From your testcase:
gl_FragColor = vec4(1.0, 0.9, 0.1, 1.0); is yellow on a C/english
locale but becomes something like vec4(1, 0, 0, 1) (red) or like
vec4(1,0, 0,9, 0,1, 1,0) (red with alpha=9) on a French locale.

I don't know what could cause such a behaviour but I guess it's an ATI
compiler bug. You should report it to their Developer Support, quoting
from  http://developer.amd.com/support/Pages/default.aspx :
"For all GPU-related developer inquiries, email [hidden email] with
the subject line "GPU-Developer-Support""

Best Regards,

Filippo Argiolas

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Julien Isorce]

Hi Filippo,

Yup and it seems it was present in NVIDIA some years ago, but fixed now.

Anyway the bug is strange because even I tried to call setlocale(LC_NUMERIC, "C"); before compiling the shader and linking, and then restore by calling setlocale(LC_NUMERIC, "");, it still does not work.
I thinking the problem happens at runtime, and not when parsing shader (because no error report in the ati shader compiler log).
Maybe the compiled shader is not understandable when it was parsed with locale "".

If I call setlocale(LC_NUMERIC, "C"); in gstopengl.c, then the sharder works :P (Do you remember  the pb about "purple" result ? now I can see the right colors with this ATI card)

Is it a problem to never restore to "" ?

I am going to report the pb to the ATI Developer Support.

Sincerely

Julien

2008/11/29 Filippo Argiolas <[hidden email]>

On Sat, Nov 29, 2008 at 4:05 PM, Julien Isorce <[hidden email]> wrote:
> Hi,
Hi Julien,

> ./test 0   give the wrong result:  red color in the window. ( setlocale
> (LC_ALL, "");  is called)
> ./test 1   give the right result:  yellow color in the window. ( setlocale
> (LC_ALL, "C"); is called)

It seems that the ATI GLSL Compiler does some kind of localized string
parsing on the shader source.  When you call setlocale (LC_ALL, NULL)
it just sets the french locale, where the decimal separator is the ","
character. In "C" locale the decimal separator is the "." character.

So the decimal part of the floats in the shader source is truncated
because dot is not interpreted as decimal separator.
>From your testcase:
gl_FragColor = vec4(1.0, 0.9, 0.1, 1.0); is yellow on a C/english
locale but becomes something like vec4(1, 0, 0, 1) (red) or like
vec4(1,0, 0,9, 0,1, 1,0) (red with alpha=9) on a French locale.

I don't know what could cause such a behaviour but I guess it's an ATI
compiler bug. You should report it to their Developer Support, quoting
from  http://developer.amd.com/support/Pages/default.aspx :
"For all GPU-related developer inquiries, email [hidden email] with
the subject line "GPU-Developer-Support""

Best Regards,

Filippo Argiolas

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Filippo]

On Sat, Nov 29, 2008 at 9:30 PM, Julien Isorce <[hidden email]> wrote:
> Hi Filippo,
>
> Yup and it seems it was present in NVIDIA some years ago, but fixed now.

Yes, I found some MacSlow forum posts about that issue.

> Anyway the bug is strange because even I tried to call setlocale(LC_NUMERIC,
> "C"); before compiling the shader and linking, and then restore by calling
> setlocale(LC_NUMERIC, "");, it still does not work.
> I thinking the problem happens at runtime, and not when parsing shader
> (because no error report in the ati shader compiler log).
> Maybe the compiled shader is not understandable when it was parsed with
> locale "".

Note that locale "" should be your current locale (french)
Unfortunately is not so easy to understand what's happening because
the spec just describes *what* the api should do, not *how* to do it.
So it could that the shader is just validated when you call
glCompileShader and actually compiled when you use it for the first
time.
It would be interesting to see the output of shader compilation. With
nvidia there are a couple of environment variables to see the result
(assembly) of glsl parsing. Is there anything similar with ATI?
Anyway if vec4 (1.0, 0.9, 0.1, 1.0) becomes vec4 (1,0,0,9,0,1,1,0) it
is still valid GLSL code so it would explain why infolog contains no
error.

> If I call setlocale(LC_NUMERIC, "C"); in gstopengl.c, then the sharder works
> :P (Do you remember  the pb about "purple" result ? now I can see the right
> colors with this ATI card)

I always suspected it was some kind of ATI bug :P, I'm glad it's fixed
now, it was getting me crazy (still I believe there is something wrong
with ActiveTexture calls too because I had some issue even with my
nvidian card).

Best Regards,

Filippo Argiolas

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Julien Isorce]

Note that locale "" should be your current locale (french)

 

Yup in first email:
echo $LANG
fr_FR.UTF-8
and same result with locale "".

So it could that the shader is just validated when you call
glCompileShader and actually compiled when you use it for the first
time.

I am attaching to this mail a new testcase. It shows that calling glUseProgramObjectARB (p); is not enough to have the shader effectivelly compiled. I have added an empty call to glBegin/glEnd(). (maybe an other call to gl would be better)

So I can first call set locale "C" then compile/link the shader then call an empty glBegin/glEnd() then restore locale to "".

Maybe we should do the same thing in gstglshader.c (as in this second test case attached to this mail) instead of doing the fix I have just commited some minutes ago.
 

It would be interesting to see the output of shader compilation. With
nvidia there are a couple of environment variables to see the result
(assembly) of glsl parsing. Is there anything similar with ATI?
Anyway if vec4 (1.0, 0.9, 0.1, 1.0) becomes vec4 (1,0,0,9,0,1,1,0) it
is still valid GLSL code so it would explain why infolog contains no
error.

I have to find them. What are they for NVIDIA ?

I always suspected it was some kind of ATI bug :P, I'm glad it's fixed
now, it was getting me crazy (still I believe there is something wrong
with ActiveTexture calls too because I had some issue even with my
nvidian card).

doublecube example works well now on the new branch. I can see something on linux too now. (since setlocale :P)

I will forward you the ati dev answer when it's done. (I have added some of your remarks in the report)

Cheers

Julien

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel