Android 10 audio forwarding

[fenste]

Are you considering adding audio forwarding for Android 10 when they added https://developer.android.com/guide/topics/media/playback-capture ?

[rom1v]

Absolutely: #14 (comment)

[AndroidDeveloperLB]

Will it be possible officially, then? Even for Windows?

[rom1v]

If it works, yes. That would be great.

The problem is finding time to work on it (especially to investigate how to receive audio packets using private APIs without Android context). 😞

[AndroidDeveloperLB]

Why doesn't Google offer an official API for this? They have an adb command for recording, but not including audio...
How do you use adb for the mirroring anyway?

[rom1v]

How do you use adb for the mirroring anyway?

I explained here and here.

[AndroidDeveloperLB]

Wait, you somehow run code on the device, without installing anything?
How is it even possible?
I don't get it...

[LBensman]

Wait, you somehow run code on the device, without installing anything?
How is it even possible?
I don't get it...

Installing an "app" via an app store is not the only way to get something running on an Android device. You can push files to device via adb push and you can execute those files using app_process command on the device itself (usually via adb shell command executed on the host).

[AndroidDeveloperLB]

I said "without installing anything".
You can push a file and run something in it, without installing anything?

[LBensman]

I said "without installing anything".
You can push a file and run something in it, without installing anything?

Sure, why not? There are limitations, I think, as to what a process can do that is launched that way, so you can still be limited in what APIs you can call from such a process, but at least for grabbing screen surface and video encoding, it has sufficient means to do so.

[AndroidDeveloperLB]

This is incredible. What are the limitations, then?
It can get all that's on the screen. That's a lot, no?
How could it get the screen content? I thought that maybe you will be restricted to pure Java stuff, but this is even beyond what Android's framework offers without user's intervention.

[LBensman]

I'm not exactly sure of the limitations or what would constitute being a lot or a little, but take a look at SurfaceControl wrapper to see what it calls to get the screen data, as well how it is wired as input to encoder.

[AndroidDeveloperLB]

Amazing. Seems you used a lot of reflection though.
How did you find about this? How did you know it's even possible?

[LBensman]

Oh, I'm not the author of it - you'd have to ask @rom1v on how he did the research and the level of experimentation that it took :)