Android Surface only shows black

[jobu]

Hey guys,

I am trying to display a stream from GStreamer in a VR View on my Oculus
Quest using a sample from their Mobile SDK.

What I already did:
* I can run the tutorial-3 sample, receive a video and display it in the
Surface of a SurfaceView (either videotestsrc or a stream from my Macbook)
* I can run the VrCinema Sample from Oculus Mobile SDK and display a local
video on a surface through Android SDKs MediaPlayer
* I merged the tutorial-3 sample code into the VrCinema sample until I got
no more errors and everything seems to be fine and wired correctly.

Now I am creating a Surface from a SurfaceTexture I got from the 3D Scene,
initializing it via nativeSurfaceInit() and starting via nativePlay(). The
App starts, I see the VrCinema on my Quest and, like I already said,
everything seems to be fine watching the Debug Logs as they are pretty much
the same as in the working vanilla version of tutorial-3.

But the Surface stays black. :(

If I use the network stream I am actually getting a lot warnings telling
"gst_rtp_jpeg_depay_process:<rtpjpegdepay0> discarding data packets received
when we have no header”
but as they are also printed in the working version of tutorial-3 I don’t
think they matter that much at this point. But as this only happens when I
am actually sending data, I assume that the stream is received and I am
doing something wrong when trying to display it in the surface. There are no
warnings when using videotestsrc.

In another thread someone got it to work by setting the size of the
SurfaceTexture, as it was set to 1x1. That happened in my case too, so I set
it to 800x600 before initializing it’s surface with nativeSurfaceInit and
the NativeWindow applies that values. But the Surface still only shows
black.


Both NDK stuff and video stuff are new lands for me, so I appreciate every
help and hint I can get. I will add some debug logs below this message, If
you need more information don’t hesitate to ask for it. :)





Here are the Pipelines I use:
"videotestsrc ! videoconvert ! glimagesink”
"udpsrc address=x.x.x.x port=5000 !
application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec !
autovideosink"




And this is the Output in the logs:
2020-02-25 16:25:36.479 16981-16981/? I/GStreamer: GStreamer initialization
complete
2020-02-25 16:25:36.498 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.447059427 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:265:native_init
Created CustomData at 0x7f58356dc0
2020-02-25 16:25:36.498 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.447194584 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:267:native_init
Created GlobalRef for app object at 0x100826
2020-02-25 16:25:36.498 16981-16981/? V/VrCinema:
startMovieAfterPermissionGranted
movieFilePath=/storage/emulated/0/Movies/SampleVideo_1280x720_2mb.mp4
2020-02-25 16:25:36.498 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.447533802 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:179:app_function
Creating pipeline in CustomData at 0x7f58356dc0
2020-02-25 16:25:36.498 16981-16981/? I/VrCinema: VrCinema::GetScreenSurface
2020-02-25 16:25:36.507 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.456588542 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:352:gst_native_surface_init
Received surface 0x7fd3e82cc8 (native window 0x7f5d1c5810)
2020-02-25 16:25:36.507 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.456676354 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:353:gst_native_surface_init
Received surface size width: 800
2020-02-25 16:25:36.507 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.456701823 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:354:gst_native_surface_init
Received surface size height: 600
2020-02-25 16:25:36.507 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.456724636 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:57:attach_current_thread
Attaching thread 0x7f5d075700
2020-02-25 16:25:36.521 16981-16981/? D/VrCinema: onResume()
2020-02-25 16:25:36.531 16981-16981/? D/VrCinema: onPause()
2020-02-25 16:25:36.564 16981-17014/? W/GStreamer+glcontext:
0:00:00.513460052 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.564 16981-17014/? W/GStreamer+glcontext:
0:00:00.513661667 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514489167 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514605417 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514648959 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514690052 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514730521 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514771146 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514811250 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514850104 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514891042 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.565 16981-17014/? W/GStreamer+glcontext:
0:00:00.514931667 0x7f58327ed0
../gst-libs/gst/gl/egl/gstglcontext_egl.c:325:gst_gl_context_egl_dump_config:<glcontextegl0>
unknown EGL_COLOR_BUFFER_TYPE value 3300
2020-02-25 16:25:36.573 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.522352709 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:234:app_function
Entering main loop... (CustomData:0x7f58356dc0)
2020-02-25 16:25:36.573 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.522462969 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:236:app_function
after g_main_loop_new
2020-02-25 16:25:36.573 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.522489375 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:57:attach_current_thread
Attaching thread 0x7f583600c0
2020-02-25 16:25:36.573 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.523001406 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:155:check_initialization_complete
Initialization complete, notifying application. native_window:0x7f5d1c5810
main_loop:0x7f5a312470
2020-02-25 16:25:36.574 16981-17010/? I/GStreamer: Gst initialized.
Restoring state, playing:false
2020-02-25 16:25:36.574 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.523244636 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:308:gst_native_play
Setting state to PLAYING
2020-02-25 16:25:36.574 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.523869636 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:238:app_function
after check_initialization_complete
2020-02-25 16:25:36.575 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.524184896 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:94:set_ui_message
Setting message to: State changed to READY
2020-02-25 16:25:36.643 16981-16981/? W/VrCinema: FRAME AVAILABLE!
2020-02-25 16:25:36.643 16981-16981/? D/GStreamer+tutorial-3:
0:00:00.592800104 0x7f5d075700
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:308:gst_native_play
Setting state to PLAYING
2020-02-25 16:25:36.644 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.593231094 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:94:set_ui_message
Setting message to: State changed to PAUSED
2020-02-25 16:25:36.646 16981-17010/? D/GStreamer+tutorial-3:
0:00:00.595034740 0x7f583600c0
/Path/ovr_sdk_mobile_1.30.0/VrSamples/VrCinema/Projects/Android/jni/../../../Src/tutorial-3.c:94:set_ui_message
Setting message to: State changed to PLAYING




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jobu]

I just got it to work!

I was actually setting the buffer size of the surfacetexture and thought
that this would be enough. Now I found out that I had to set the size of the
texture in the native code for VrCinema too - and now it works :)

This solution is not related to GStreamer, but I wanted to add it here -
maybe someone who encounters the same problem finds this and can use it. The
important part is the use of nativeSetVideoSize. Also I need to call this
function after requesting WRITE_EXTERNAL_STORAGE permission.


public void prepareSurface() {
    synchronized (this) {
      // allocate a new external texture, and create a
      // surfaceTexture with it.
      movieTexture = nativePrepareNewVideo(appPtr);
      if (movieTexture == null) {
        Log.w(TAG, "prepareSurface - could not create movieTexture ");
        return;
      }
      movieTexture.setDefaultBufferSize(800, 600);
      movieSurface = new Surface(movieTexture);
      nativeSetVideoSize(appPtr, 800, 600);
      nativeSurfaceInit(movieSurface);
    }
  }



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[chamarts]

Hi,

I am trying to do the samething with flutter and using tutorial3 as base and
created a platformview but somehow the view stays blank after
nativeSurfaceInit. but I am confused where and how I set the video size from
your post

thx
sri



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[fr0sty]

hey chamarts!

I have a similar problem. Did you find a solution yet?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel