enternoescape / opendct Goto Github PK

I am unfamiliar with the way the options are supposed to work, but on the HDHomeRunDiscoveredDeviceParent there is an option to force the channelmaps on the device to a certain type on discovery. This is not actually doing anything. If I change the initial value in HDHomeRunDiscoveredDeviceParent.java to a value ("us-bcast", for instance), then it does set it to that - but it no matter what I set the property to in opendct.properties, it never returns anything but "".

If the capture devices don't need to be completely reloaded after standby, don't require them to be completely re-detected.

Support for eu-cable.

Currently it raises an exception with my eu-cable tuner:
INFO | jvm 1 | 2016/05/22 00:33:21.353 | 00:33:21.301 [HDHomeRunDiscoveryReceive-37] ERROR DeviceLoaderImpl - Unable to load the capture device 'HDHomeRun HDHR3-4DC Tuner 14104392-0', id -1945501120 => opendct.tuning.discovery.CaptureDeviceLoadException: The program currently does not know how to use the channel map 'eu-cable'.

I have noticed a lot of these errors in my logs. I really have no idea where this IP is coming from, as it's nothing like any IP on my network (which is all 192.168.99.xxx, with the prime being 192.168.99.30). The prime is functioning fine otherwise, and being reported properly in the logs, so not sure why this particular call is getting it wrong.

11:07:45.257 [HDHomeRunDiscoveryReceive-43] INFO  ChannelManager - Updating the HDHomeRun channel lineup HDHomeRun DRI Tuner 13112BC2 (dct_hdhomerun).
11:07:45.257 [HDHomeRunDiscoveryReceive-43] INFO  HDHomeRunChannels - Connecting to Prime DCT using the URL 'http://198.105.254.24:80/lineup.xml'
11:07:46.292 [HDHomeRunDiscoveryReceive-43] DEBUG HDHomeRunChannels - populateChannels created an unexpected exception => java.net.ConnectException: Connection refused: connect
    at java.net.DualStackPlainSocketImpl.connect0(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at sun.net.NetworkClient.doConnect(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient.<init>(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.followRedirect0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.followRedirect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at opendct.channel.updater.http.HDHomeRunChannels.populateChannels(HDHomeRunChannels.java:173)
    at opendct.channel.ChannelManager.updateChannelLineup(ChannelManager.java:484)
    at opendct.capture.HDHRNativeCaptureDevice.<init>(HDHRNativeCaptureDevice.java:225)
    at opendct.tuning.hdhomerun.HDHomeRunDiscoveredDevice.loadCaptureDevice(HDHomeRunDiscoveredDevice.java:86)
    at opendct.tuning.discovery.discoverers.HDHomeRunDiscoverer.loadCaptureDevice(HDHomeRunDiscoverer.java:666)
    at opendct.tuning.discovery.DeviceLoaderImpl.advertiseDevice(DeviceLoaderImpl.java:49)
    at opendct.tuning.discovery.discoverers.HDHomeRunDiscoverer.addCaptureDevice(HDHomeRunDiscoverer.java:520)
    at opendct.tuning.hdhomerun.HDHomeRunDiscovery$ReceiveThread.run(HDHomeRunDiscovery.java:429)
    at java.lang.Thread.run(Unknown Source)

I have a DVBT_HDHOMERUN device, and when I try tune it I get the error message:

HDHomeRun HDHR3-DT Tuner 11112583-0] ERROR HDHRNativeCaptureDevice - legacyTuneChannel: The channel number {} is not a valid ATSC channel.

looking at the code in \src\main\java\opendct\capture\HDHRNativeCaptureDevice.java line 1086

    switch (encoderDeviceType) {
        case DVBC_HDHOMERUN:
        case DVBT_HDHOMERUN:
        case ATSC_HDHOMERUN:
            if (channelNumber <= 1 || channelNumber > lookupMap.length) {
                logger.error("legacyTuneChannel: The channel number {} is not a valid ATSC channel.");
                return false;
            }

            fChannel = lookupMap[channelNumber].FREQUENCY;
            break;

the logger.error has the parameter capture to show the channel number but it is not passed in.

I would expect

logger.error("legacyTuneChannel: The channel number {} is not a valid ATSC channel.", channelNumner);

the prior line in the log is:

21:32:29.395 [SageTVRequestHandler-71:HDHomeRun HDHR3-DT Tuner 11112583-0] INFO  ChannelLineup - '8746-34-1200' was remapped to '34.1200'.

so I am not sure if the 8746 is getting grabbed, or the 34

In case things do not stop correctly, it would be nice to have this file saved because there are many changes that happen other than setting changes in the JSON interface that would be nice to have preserved despite a bad stop.

there are couple of NULL items in the EU_BCAST lookupmap used for New Zealand, and this cause a uncaught exception, which means the device stops scanning, so no channels are found.

When doing a channel scan from sage for ATSC tuning, there is some issue in the data returned, resulting in a junked up lineup in sage. Looks like it is not detecting and returning actual program information. Example:

17:04:06.271 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG SageTVRequestHandler - SageTV sent: 'AUTOINFOSCAN HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner|10'
17:04:06.272 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] INFO  HDHRNativeCaptureDevice - Capture device is was already locked.
17:04:06.272 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG HDHomeRunControl - key: '/tuner0/channel' value: 'auto:213000000' lockKey: '1062706233' sendLength: 49 address: 192.168.99.31
17:04:06.523 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG HDHomeRunControl - key: '/tuner0/status' value: 'null' lockKey: '0' sendLength: 25 address: 192.168.99.31
17:04:06.525 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG HDHomeRunControl - key: '/tuner0/streaminfo' value: 'null' lockKey: '0' sendLength: 29 address: 192.168.99.31
17:04:06.776 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG HDHomeRunControl - key: '/tuner0/streaminfo' value: 'null' lockKey: '0' sendLength: 29 address: 192.168.99.31
17:04:06.777 [SageTVRequestHandler-123:HDHomeRun HDHR-US Tuner 10173AD8-0 Digital TV Tuner] DEBUG SageTVRequestHandler - Replied: '13-0;13-0;13-0;13-0'

I'll be looking into it, just wanted to see if you've done much (or any) testing of ATSC tuners.

If anyone has already done this before please let me know this isn't a high priority, but it would be nice have consistent logs.

So, first time restarting my docker in a while, and opendct didn't restart with the new 0.5.23-1 build. Checked the log, and I'm getting:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 4
at opendct.tuning.hdhomerun.HDHomeRunDiscovery.extractBroadcast(HDHomeRunDiscovery.java:580)
at opendct.tuning.hdhomerun.HDHomeRunDiscovery.getBroadcast(HDHomeRunDiscovery.java:557)
at opendct.tuning.discovery.discoverers.HDHomeRunDiscoverer.(HDHomeRunDiscoverer.java:78)
at opendct.Main.main(Main.java:222)

Just a quick hunch, but on a 4 byte array, shouldn't we start with i=3?

The circular buffer has been improved to support seeking at any time. If that doesn't address the problem, I will be adding RTCP support.

This feature will allow you to address the tuners as a pool instead of directly. This would make the tuner provided to SageTV a virtual tuner since it doesn't have a direct assignment. This allows for specifically HDHomeRun users to access a group of tuners in order of preference and availability. In order to keep things familiar, we will use the same merit system that SageTV uses to determine the order of selection. One criteria for skipping a tuner with a higher merit than another tuner would be if the higher merit tuner is currently locked by another computer/device. The locks would only be overridden if there is no other option and the override would be performed starting at the highest merit first.

The reason for pools instead of just grouping all of the available tuners into one big pool is because you could have unexpected issues while tuning. For example, let's say we have 2 tuners. One DCT and one ClearQAM. The DCT has all channels available and the ClearQAM only has a subset available. SageTV tunes a DCT channel that happens to also be a ClearQAM channel and OpenDCT selected the ClearQAM tuner since the DCT tuner is currently in use by another program. No problem. It tunes it and it works. SageTV then tunes a ClearQAM channel and OpenDCT selects the DCT since it's all we have left; also no problem. SageTV then tunes a DCT channel while the DCT is already in use on a ClearQAM channel. The ClearQAM tuner can't tune the channel and it fails. The only way to work around this would be to have OpenDCT dynamically swap the tuners around, but that may be an unrealistic approach since we would need to make sure everything lines up perfectly or people will not be happy with their chopped up recordings. Also this would require more time than many people might prefer just to tune into a channel if everything is all jumbled up.

Care needs to be taken to ensure that this doesn't impact tuning performance in any significant way. I think our goal should be to have a result in less than 100ms. If we can keep everything under that time frame even if unexpected things happen like a tuner disappears from the network, we will maintain the nice responsiveness that we already have.

This feature would also provide the ground work for us to do other cool things like consolidation between multiple SageTV servers. For example you could have multiple SageTV servers using the same tuners. If a request hits the pool for a channel that's already tuned, the stream would just be split between the two servers instead of using up another tuner. The NOOP response could be used to let SageTV know when tuners are being used by another server. In the event that we can consolidate a channel, the NOOP response would come back for one of the tuners. I could only see this ending badly if both servers start recordings on completely different channels all at the same time.

Currently only the Ubuntu package will stop the service if it is running when upgrading. The Windows and CentOS/Fedora packages really should have this feature. For reference uninstalling on Windows currently will not stop or uninstall the service, but uninstalling on Ubuntu and CentOS/Fedora will stop and uninstall the service.

In Windows it is normal for an installer to also create Firewall exceptions, so I will build the functionality into the installation. In Linux it is normal/preferred to configure these settings yourself.

CentOS 7 and Fedora 22 use firewalld out of the box. We can create an xml file and add it to the package located at /etc/firewalld/services/opendct.xml so that enabling the exceptions will be as easy as typing the code block below.

firewall-cmd --get-default-zone
firewall-cmd --permanent --zone=<default_zone> --add-service=opendct
firewall-cmd --reload

Ubuntu 14.04 uses ufw by default to make configuration of the firewall easier, but it's not enabled out of the box. Their argument is the default installation isn't open enough to warrant it being enabled. I will add a script to enable/disable the needed ports since appending a new service to /etc/services could be problematic; especially if the created entries are modified and the package is being removed. The commands needed to enable the firewall for OpenDCT are in the code block below.

ufw allow 7818/tcp
ufw allow 9000-9100/tcp
ufw allow 8271/udp
ufw allow 8100-8500/udp

This feature would scan the opendct.properties file and open all required ports before the capture devices have actually been detected. You must configure sagetv.device.global.required_devices_loaded_count= to equal the number of capture devices expected or this feature will not know to block a request and wait for the capture device to become available.

Need to implement us-hrc and us-irc channel maps. most cable operators in the US are typically on us-hrc (done to shift away from broadcast frequencies to minimize interference on the cables).

The needed methods to set a requested program and and related PIDs are already in place and being populated. The FFmpeg consumer just needs to be enhanced to support this feature with a default to do a best effort if the request can't be completely satisfied after a certain amount of data has been processed.

The reasoning behind this change is to have a measure in place to be completely sure that we get all available streams. The most common possibility from missing a PID is getting Spanish for audio. These situations are rare, but still possible. This change will also allow for potentially even shorter tuning times since we would know that we can stop as soon as we have everything requested instead of stopping after a minimum amount of data has been processed and we have detected a video and audio stream.

Since the port opendct uses is not required to be 9000, and since Logitech Media Server DOES use 9000, it'd be nice if when opendct fails to open 9000, it fails over to the 9001, and so on. LMS is a rather common/highly used docker container, so this conflict is sure to come up more than just in my case.

This issue was introduced when we started to zero in on a program suggested by the capture device. The problem is that without that program, FFmpeg really does a bad job at figuring out what it actually should be streaming and waits until the buffer is exhausted to make a determination even if all possible streams have been detected.

When I installed this on Debian buster there was an error on the install that I didn't catch but because the systemctl, and server worked I didn't worry too much. Now however opendct has messed up my apt updates which can't continue until I fix it.

This is the apt errro

dpkg: error processing package opendct (--configure):
 installed opendct package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 opendct
E: Sub-process /usr/bin/dpkg returned an error code (1)

Checking the install script I see that this file is only created when there is a folder /usr/share/upstart which I don't have. Since we don't need upstart perhaps the postinst doesn't need to perform this step

update-rc.d -f opendct defaults

This is to address the fact that FFmpeg doesn't work very well with Music Choice content. Also the DCT provides many more PIDs than are actually a part of the requested stream for Music Choice content. To be fair none of the other DCTs for SageTV work well with this content when they don't use raw output. For example, SageDCT starts streaming after about 30 seconds, but then the audio drops out randomly and comes back minutes later. OpenDCT has a similar issue. It starts streaming a little faster, but eventually the audio drops out too. I would be inclined to say this is an issue with the atypical way the streams are assembled so that more of them can be packed into a single frequency and FFmpeg isn't re-assembling the streams reliably because of it.

As an alternative, we could detect very low data transmission speed and automatically switch FFmpeg to do a transcoding of the stream instead of just remuxing so it could fill in the gaps that cause these glitches. The files will be larger, but they will also be far more reliable and possibly even better than raw. I want to hear people's thoughts on this.

Many devices do not correctly support interlaced content and some devices have no MPEG-2 support at all. I assembled a proof of concept that works very well and will be included as experimental in 0.4.

My wife will often try to record 5 shows at a single time period, however, after using OpenDCT on a pretty beefy machine along with 2 HDHR Primes dedicated to our SageTV box, one of the 5 recordings will always get screwy (artifacts, blocking, pixilation, dead spots) when trying to record anything more than 4 shows. I'm trying to see what the limitation is on this, or if there is a setting I can tweek in the profile for OpenDCT to improve the number of simultaneous recordings?

Need to ensure opendct.properties is not included in the install, so it is retained on reinstalls.

if defaults are wanted in file form, instead of just in the code (sagetv simply sets defaults in code the first time a property is accessed), then it needs to be a different file that is copied to opendct.properties on launch if the .properties file is missing, and ignored if the .properties file is already there.

Minor issue, but for human readability, channel lists in the lineup files for available and unavailable channels should probably be sorted. I think it should probably be handled in ChannelLineup.getAllChannels(). A collection.sort with a custom compatator to sort by the first token(-) (primary channel number) and if equal by the second token (subchannel number).

Things like the offline scan running in a seemingly random order lead me to desiring this. I doubt getAllChannels is called often enough for the sort delay to matter.

I'm wondering if a producer could be easily made that simply reads from a 'file', and sends the stream to the consumer. Reason I'm asking would be to leverage OpenDCT installed on a mini-pc (like a raspberry pi, for instance) connected to a hd-pvr, which would be presented as /dev/video0. I was working up a very simple network encoder in perl to do this, but I think, especially once you've got a web config going for opendct, it might be a better option (keep everything familiar for existing opendct users).