Improper treatment of percent escaped characters by gstsouphttpsrc

[Fuuzetsu]

Greetings,

I'm a Clementine (Qt based music player, a fork of Amarok) user which
uses GStreamer as it's engine.

Today a user file a bug report about his .xpfs file not being properly
played. In it, he has a URL pointed at some location on the Internet.
The issue occurs when the URL has percent-escaped characters. In this
case they happen to be ‘%5B’ and ‘%5D’ which stand for ‘[’ and ‘]’
respectively.


Now, there are some things I'd like to establish.

1. Clementine (or rather GStreamer) handles streams just fine.
2. The user interface and the whole back-end IS parsing the .xpfs fine,
extracting the string as expected.
3. The string being passed to a GStreamer pipeline is still the same
string, before the audio starts playing and after it stops.


Here's a small Clementine debug output. It contains a few more messages
than it usually does as I spent quite a few hours ensuring that
Clementine actually handles the URL properly and passes it to GStreamer
as expected.

Url at start of Load "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3"
Url after Engine::Base::Load "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3"
gst_url  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )
About to create a pipeline from gst_url
Creating pipeline in GstEngine using CreatePipeline(url, nanosec)
url is  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )
Calling InitFromUrl with url:  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )
Managed to make a pipeline
Before BufferingFinished()
After BufferingFinished()
Done with pipeline creation; return true;
In Play
Setting QFuture
Leaving Play
position 0 scrobble point 31 status 0
1 "gstsouphttpsrc.c(1117): gst_soup_http_src_parse_status (): /GstPipeline:pipeline/GstURIDecodeBin:uridecodebin-0/GstSoupHTTPSrc:source:
Not Found (404), URL: http://www.notarealurl.net/insub/%255Binsub05%255DJacques_Demierre-1-black_white_memories.mp3"
Gstreamer error: "Not Found"
QTimeLine::start: already running
Stopping; old url_ is  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )

As can be seen above, I make sure a couple of times that the URL I have
is correct. Once GStreamer tries to reach it however, it reports a 404
error with a mangled URL (number 25 seems to be inserted for some
reason; I haven't tried with other escape codes).

Is there something fundamental I'm missing? Is that plug-in not supposed
to be able to handle such URLs?
I tried looking for the source of the mentioned file to check if it's
doing anything funky, but the GStreamer repos were quite difficult to
navigate.

I thought that I should ask on a mailing list before filing a bug report
as there really might be something simple I fail to notice as I'm not
familiar with GStreamer. If someone is able to stream such a URL, might
have to go back and spend more time on the Clementine side of things,
although I'm pretty confident that it's not the culprit here.


Thank you.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Thu, 2012-11-22 at 21:20 +0000, Mateusz Kowalczyk wrote:
> Not Found (404), URL: http://www.notarealurl.net/insub/%255Binsub05%255DJacques_Demierre-1-black_white_memories.mp3"

> Stopping; old url_ is  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )
>
> As can be seen above, I make sure a couple of times that the URL I have
> is correct. Once GStreamer tries to reach it however, it reports a 404
> error with a mangled URL (number 25 seems to be inserted for some
> reason; I haven't tried with other escape codes).

It looks like the % in %5B is getting escaped again, to %255B .

Perhaps look at the GST_DEBUG log to see what's going on, and what URI
is actually passed to GStreamer ?

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Krzysztof Konopko]

On 22/11/12 22:37, Tim-Philipp Müller wrote:

> On Thu, 2012-11-22 at 21:20 +0000, Mateusz Kowalczyk wrote:
>> Not Found (404), URL: http://www.notarealurl.net/insub/%255Binsub05%255DJacques_Demierre-1-black_white_memories.mp3"
>
>> Stopping; old url_ is  QUrl( "http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3" )
>>
>> As can be seen above, I make sure a couple of times that the URL I have
>> is correct. Once GStreamer tries to reach it however, it reports a 404
>> error with a mangled URL (number 25 seems to be inserted for some
>> reason; I haven't tried with other escape codes).
>
> It looks like the % in %5B is getting escaped again, to %255B .
>
> Perhaps look at the GST_DEBUG log to see what's going on, and what URI
> is actually passed to GStreamer ?
>
> Cheers
>  -Tim
>

Here's an experiment to try on the command line:

gst-launch-0.10 \
  souphttpsrc \

location=http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3
  ! fakesink

or

gst-launch-1.0 \
  souphttpsrc \

location=http://www.notarealurl.net/insub/%5Binsub05%5DJacques_Demierre-1-black_white_memories.mp3
  ! fakesink

This should isolate the problem down to the GStreamer plugin you're
using in your pipeline. I tried both and I can't see any additional %.
You can also try it with a real URL and autoaudiosink instead of
fakesink to actually listen to the media.

Now according to your logs, you're using QUrl. According to the
documentation [1] (Qt 4.7):

QUrl::QUrl ( const QString & url )
Constructs a URL by parsing url. url is assumed to be in human readable
representation, with no percent encoding. QUrl will automatically
percent encode all characters that are not allowed in a URL.

It looks like QUrl escapes %. Try using QUrl::fromEncoded() instead.

I tried to look into the sourcode [2] but I'm not sure if this is the
right place. It looks like everywhere the URL is conveyed within QUrl
but maybe at some point QString is extracted and used to construct
another copy of QUrl?

[1] http://doc.qt.digia.com/qt/qurl.html#QUrl-2
[2]
http://code.google.com/p/clementine-player/source/browse/src/engines/gstengine.cpp?name=gstreamer-light

HTH,
Kris

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Fuuzetsu]

It would seem that the issue was caused by Clementine after all and was recently fixed by a maintainer.

You can find the diff here if you are interested:
http://code.google.com/p/clementine-player/source/detail?r=d5412d3800a0

Sorry and thank you for your time.