Want to be able to load a Network Encoder as a runnable class about sagetv

[CraziFuzzy]

Part of why I like the network encoder mechanic, is it's inherent disconnect from the core sagetv service. I honestly feel it is best to have the encoder install in windows as a separate service.

The fix I would do, however, is to have sage check network encoder's more often for presence, so your issue is resolved smoothly regardless of the order of start-up. It's not like a NOOP message and response are very taxing.

[Narflex]

I'm confused why you would want to run a Network Encoder in another thread rather than in another process? If you're going to add support for another type of capture in process...just make it part of the existing capture system rather than using the network encoder interface.

[jvl711]

Narflex,

The goal of this change was not necessarily to make a Network Encoder as a thread to the sage process. My goal is more to allow sage to manage the starting and stopping of a network encoder instance (Thread/Process). I wrote a Network Encoder (PrimeNetEncoder), and it can be launched as a seperate process, or as a runnable class from Sage. In my opinion allowing Sage to manage starting and stopping the thread is less work for the end user, and makes the Network Encoder more portable between platforms. If I were to leave it as a standalone process than the end user would need to figure out how to manage start and stopping the process (ie system reboot). If I make it a service than I need to figure out how to package it in a way that it works in Windows/Linux/Mac.

I have tried to use this approach in Sage7 and it seems to work fairly well except for the fact that the thread is start to late in the startup process for the thread to respond to discovery requests, or for Sage to connect to the Network Encoder. You need to go to the Tuner Setup page for Sage to recognize the tuner is present.

Is there another approach that you would suggest?

Thanks,
Josh

[Narflex]

Josh,

Since it's using the network encoder API...it should be OK with starting up after the regular SageTV startup has occurred. The corresponding 'NetworkCaptureDevice' instance in SageTV will always be active (if it's been configured) and then it will handle the actual network encoder instance being up/down at various times.

Where I can see there would be a problem is if this is relying on Network Encoder Discovery. So in reality...that's only a problem the first time it runs where it needs to discover it and then add it to the encoders config.

I'd be OK with adding some other property like 'load_at_startup_encoder_runnables'...but these need to be sure they do NOT call the SageTV API at all (except for property stuff) because it won't be in a state to handle many of the requests at the point in time it would be run.

[CraziFuzzy]

Jeff, I think the issue is that sage only checks the encoder is there every so often, and it's sort of a race condition on startup. The first run of the scheduler is started before the encoder is recognised as functioning, so that first scheduler is done without it being present, resulting in potential conflicts. Later, at some point, sage retests the encoder (via NOOP), and when it gets a valid response, will then continue using it on subsequent schedule runs. I'm not sure what triggers the noop, and I'm not sure if a new encoder found functioning re-triggers scheduler, but these would be where this should be fixed, not really altering sage's startup routine.

[CraziFuzzy]

Actually, I think I might have found the issue. Sage has code in the NetworkCaptureDevice.isFunctioning() function to launch a monitoring thread to check a non-responsive network encoder once a second and once the encoder is found, fire off the scheduler again. Unfortunately - and I'm not sure why it is written this way - this monitoring process is only enabled if sage.EMBEDDED is true.

[Narflex]

Yeah, that was added for Fiber...and many of the Fiber changes I conditionalized on Sage.EMBEDDED to prevent changing behavior in the PC product. But if they're useful...they can be easily modified to be included. :)

[jvl711]

Narflex,

I think I might have another idea that would make everyone a little happier. What if I call mmc.redetectCaptureDevices() after loading the runnable classes. This looks like it was meant for discovering new capture devices at runtime. I will only call this method if there was a configured value in the runnable class setting. That way it should limit startup time.

I will attempt to code and test this change.

[CraziFuzzy]

redetectCaptureDevices won't actually trigger a new isFunctioning check on a network encoder. It simply causes the manager to return all it's devices, which it will, but that device would still be flagged as non functioning.

[CraziFuzzy]

Simply removing the boolean check I listed above should solve the problem pretty well. I actually made a branch with the change on my end, but haven't had a chance to actually test it out before pushing it to the main branch.

[jvl711]

Chris,

You are correct, but I think we need both changes. Yours will make sure to try to reconnect to the network encoder, since it missed it's original try at connecting since it has not started yet.

My code will send out a discovery packet and discover NetworkEncoders that have not been configured yet. If I do not call the redetect it will not send out the discovery packet again.

Josh

[CraziFuzzy]

I'm not sure it will though. It looks like that call, when you follow it through to the network encoder manager, doesn't actually initiate the network discovery, and instead just passes back the encoders that are configured in the sage.properties, regardless of whether they are running or not.

I also didn't think your encoder was using the discovery system, which is why properties need to be added to the sage.properties file manually. The Network Encoder discovery broadcasts is a completely different beast from the NOOP --> OK pinging the IsFunctioning() call does.

[jvl711]

Chris,

Maybe I am missing something, but from what I can see the method detectCaptureDevices() in the NetworkEncoderManager will send the discovery packet and look for new devices to configure. I am in the process of building a test server. Hopefully I will know soon if it does.

And you are correct. The current version of PrimeNetEncoder does not implement discovery, but the one that I am testing does :). I was able to work out some issues in my discovery code once sage was open sourced...

Josh

[CraziFuzzy]

The discovery response won't get you past your issue of the encoder being configured, but being flagged as not functioning. In fact, i don't believe it will get you anything, because the encoder will already be set up with mmc entries in the sage.properties file. Once it gets all it's responses from the various responding encoders on the network, it then gets the properties from the encoders. It then compares the response to what's in the .properties file already, and if it is there, breaks out of that check and moves on to the next one. Nothing in that process is going to turn an already recognized, but not functioning encoder into a recognized functioning encoder. The only thing that does that is the isFuncitoning() call, which is currently only called on startup, and when you enter the source config menu.

[CraziFuzzy]

Quite simply, the redetectCaptureDevices() call running an extra time at startup really won't do anything, as the only time you'd need to actually detect new capture devices is when you are first adding a new one, in which case, you'd be actively working on the system to do so. If anything, the only time I can envision redetectCaptureDevices() should be called is when you navigate to the source config screen - though I don't think it does at this time. It appears to actually never get called in any part of the current sagetv code or stv, and instead the detection only happens when the CaptureManager inits on startup.

[jwittkoski]

I think Chris is correct here. If I remember correctly from testing prime_encoder, once a network encoder is discovered and has property entries, the NOOP pings will pick it up eventually if it doesn't respond when the Sage process first starts. If a new/unknown encoder isn't running when Sage starts, Sage won't see it until the next restart and discovery. (This was really annoying when I was testing the discovery code but isn't critical once a user has things set up.)

[Narflex]

Don't get your hopes up about the redetectCaptureDevices method either...it looks like it'll do what you want...but there's a reason it's unused. I don't recall exactly what all the other hangups are around it; but there are some (otherwise we would have enabled this for the purpose of hot plugging non-network encoder capture devices like USB or HDHR). I know at least it won't trigger creation of the corresponding live MediaFile objects used for the devices (but triggering a library import scan should correct that).

I think there's a couple cleaner solutions to this:

1. Making use of mmc/custom_capture_device_manager (and changing the code around it so it's always called, right now it's skipped on all platforms...and also making that property extensible to allow specifying multiple ones) and then loading up stuff that way.
2. Having another type of runnable that starts earlier in the init process
3. When you runnable starts up, if you haven't had the devices specified in the properties file yet...add them and trigger a restart of SageTV.

[jvl711]

Jeff,

I really think that number 2 is the option that makes the most sense. It makes use of all of the code that is already in place. It should have very little impact on system startup time.

I will test the other approach since I already coded it and let you know if for some reason it seems to be working. If not I will revert the changes and go with option 2.

Josh

[Narflex]

Josh,

Even if the other approach 'seems' to be working...I wouldn't suggest doing it, as SageTV wasn't written with the intention of the set of capture devices changing during runtime (they can change availability...but existence is another matter). Simply calling redetectCaptureDevices could actually destabilize things in the system, so i'd be very hesitant to approve that change without spending a bunch of time looking over the code to see if it's OK (which I can do if your testing works out...also be sure you test with SageTVClient because that adds another dimension of complexity).

[jvl711]

Jeff,

I am a little surprised, but my testing has actually gone well. Here is what I tested.

1. Start sage with no encoders defined. Sage picks up the network encoder from the discovery request. I add the tuner. I then tune to a few channels. This seems to be working fine. The network tuner is available during the whole process, and is usable.
2. Start sage with encoder defined. The tuner is immediately available for use. I no longer have to go into video sources for sage to see the tuner.

Is there any places were you feel the system will break that I should attempt to test further?

I would like to make an overload to redetectCaptureDevices which takes in a CaptureDeviceManager. I would like to reduce the redetection to only networkencoders. I think this might make this change have less impact.

Let me know what you think. I am OK with scrapping this change if this is not the direction you want to go.

Thanks,
Josh

[Narflex]

Josh,

OK, this sounds like it should be fine. Definitely add the overloaded redetectCaptureDevices message...and THEN call that from within your own code after you startup your discovery server. This way the redetection is only done when people are using your plugin and problems would be isolated to that (and I'd also encourage you to then NOT call that method if you see that your encoders are already configured, then this'll only change behavior on the first run with your plugin, so if it does cause any issues they'd just go away after a restart). :)

[jvl711]

Jeff,

Perfect. I will update MMC to add the overload.

Just curious. Are those classes/methods accessible in the SageTV7 jar? Or are they wrapped so that we can not access them?

Josh

[Narflex]

They will be obfuscated in the V7 jar file.

[jvl711]

We were able to come up with a way to resolve this issue from inside of the NetworkEncoder client. I consider this issue closed.