DVBLink 6.0.0 DVB-T/T2 Freeview HD Crystal Palace Channels Missing after tuning

System Requirements:

  • DVBLogic, DVBLink 6.0.0
  • Be in the London Region tuning against Crystal Palace

The Problem:

After tuning DVBLogic DVBLink against Crystal Palace, you are missing a number of channels from the channel list despite having a strong signal. the missing channels may include:

  • 5STAR + 1
  • BT Showcase HD
  • Forces TV
  • FreeSports
  • More4+1
  • PBS America
  • QVC Beauty HD
  • QVC HD
  • Rocks & Co 1
  • London Live
  • POP Max
  • Sony Movie Channel+1
  • The Vault
  • Tiny Pop
  • True Crime
  • True Movies

More Info

The transponder definitions provided for Crystal Palace within DVBLink are currently out of date. consequently, DVBLink only scans 7 transponders instead of the current 9 on the transmitter.

The Fix

You can add the missing transponder data by following the following steps:

  1. Open the DVBLink TVScource configuration web interface and log-in if applicable
  2. Ensure that the “Advanced mode” check box in the top right corner is selected
  3. Go to Sources > TV Sources
  4. Click the config (spanner) icon next to your tuner/tuner group for Freeview
  5. Select search channels
  6. Under ‘select your TV provider’ choose “DVB-T UK (Crystal Palace)” and click next
    Note: The specific transponder settings below will not work for other transponders, however if you can identify transponders for another transmitter, the process is identical
  7. On the left hand side of the current screen there is a long, vertical black bar. Click on it!
    Black bar
  8. At the bottom click the “Edit transponders” button
  9. Add the two missing transponders to the end of the list:
    Note: This is valid as of January 2018, in the future it is liable to change.8=538000,H,9600
    9=586000,H,9600The entire configuration should look like:
  10. click ‘save’
  11. Click the ‘scan’ button on the right hand side of the screen
  12. TVSource will now add the missing channels (16 in my case) to the system
  13. Press OK and configure the new channels as normal

DVBLink 6.0.0 Recordings Database Repair Utility

System Requirements:

  • DVBLogic, DVBLink 6.0.0
  • Windows Scripting Host

The Problem:

DVBLink is a good product which works across a large number of platforms, and, considering how complex it is, it does a good job of interfacing between DVB-T/DVB-S terrestrial TV transmissions and the on-demand world of modern media consumption.

I’ve always had a few niggles with the system (‘clear data’ in Kodi anyone?), but have always found it good, save for one area. It’s recording database mechanic is at best flimsy and at worst out-right fragile.

The number of database corruptions, anomalies and times I have had to restore the database form backup in essence amount to every 6 months. It can be caused by a service crash, the PC restarting without a clean shutdown (power cut), loss of visibility of the disk where the recordings folder target sits (e.g. USB drive, iSCSI recording storage on a SAN/NAS or any other ejectable media) or as a result of installing a DVBLink update (I had a v5 update that trashed my recordings database about 2 months after I first started using it).

To provide a simple summary of the issues with the SQLlite database implementation:

  1. If the database is lost, that’s it. DVBLink/Kodi etc cannot ‘see’ your recordings, they are lost to the software even though they are still physically on disk and if you know where to look and how to open them, they remain watchable.
  2. Some failure situations can lead the database to have more in it than the file system does. This leads to the Kodi ‘click of nothing’, where clicking an item in the recordings section does just that, nothing (no error, just nothing)…
  3. … DVBLink has an option in the server settings to perform a consistency check. This is supposed to fix the ‘I have a database record but no file issue’… yet in reality, in my experience, this far too complicated and (I would assume) tries to be cleverer than asking “is there a recordings file on disk for this recording entry in the database?”. If you have ever tried to manually reconstruct the database with this option on, you’ll know that DVBLink simply deletes the entry…
  4. … Some failure situations – and the DVBLink provided consistency checking option – can lead the database to not have entries for recordings that do exist in the file system. Referring back to #1, you’ll recall that this means that DVBLink/Kodi etc does not know that they exist and you cannot watch the recordings. Equally there is no way to delete the orphaned files as they are not reported. Consequently your DVBLink system will slowly leak disk space. This is made worse by the fact that the database appears to track deletions, but doesn’t appear to have a (working) mechanism to restore the records again.
  5. There is no way to automatically recover from these situations.
  6. Once in these situations, there is no way to recover lost meta data on recording files that are orphaned in the recordings directory.

When a family member with a large pre-existing recording catalogue went away on 22nd December, returned on the 29th December to find that

  1. The old catalogue was no longer showing in Kodi
  2. No new programme recorded before 27th December was present

There were plenty of tears before bedtime.

Due to a connectivity problem between the DVBLink service and the recorder disk on the 27th, DVBLink had restarted and decided that as it couldn’t find the recordings folder, it would delete the now missing content form the recordings database.

It started recording from the 27th onwards onto the tiny local SSD that the OS is installed onto as a fail back (fair dues here DVBLink, good move), but come the 29th there were only 6 things (new content from the 28th and 29th) in the database. There should have been around 70 recordings.

There was no database backup available later than 12th December (by luck, I had made one on the 12th). There was no automatic database backup later than October.

More Info

I now needed to merge the 12th December backup and the 27th December+ master database back together. This would leave a data gap from 13th-26th December of missing records, so having done this in a SQLLite GUI editor, I started the DVBLink back up and… it promptly deleted 85% of all the material introduced from the 12th December backup.

The mystery of the database reconciliation option strikes again. By checking that all of the filenames in the database, I confirmed that the files were on the disk, including the thumbnails, but it decided to delete the imported old records anyway. Just not all of them for an unknown reason.

I turned off the reconciliation option and repeated the merging process again, and, it worked. The files played back in Kodi. I just cannot enable the database reconciliation feature. Ever.

At this point I turned my attention to the missing 13th-26th period. The file system revealed that this amounted to some 14 recordings. At this point I noticed one slight problem. I was expecting to be recovering around 70 files into the database, yet there were over 180, large, playable .ts files in the recording folder consuming some 360 GB of disk space. These files covered a period starting in April 2015 through to 29th December 2017.

Looking non-exhaustively at very old database backup files, I was able to find several of them, proving that they had at one point existed and had not been deleted by the user, but by the database consistence checked/reconciler. This posed the following chain of problems:

  1. If I import the contents of all of the old database backups that I have, I will be importing a mass of junk data.
  2. If I turn on the automatic (and unreliable) consistency checker to clean that up, I will wind up deleting all of the work thus far and likely only be a couple of steps further forward.
  3. I still have not solved the problem of the completely missing 13th December 2017 – 26th December 2017 database records.

I thus decided to write some code and use programming to help solve the problem. Thus the DVBLink_RecordingsDb Maintenance API was created. The API allows you to view:

  • All files in the file system
  • All records in the database
  • All files in the file system that are not in the database
  • All records in the database for which there is no associated file in the file system

In turn it allows you to:

  • Remove entries in the database where the is no resultant file (using no other logic beyond is the correct .ts file presence e.g. the presence of a valid recording timer entry in the database to test validity is not used).
  • Add entries to the database where there is a file in the file system but no existing database entry. The tool will attempt – i.e. best effort – to populate a new database record with the minimum amount of information necessary to allow DVBLink/Kodi to play the file. This is largely based upon the file name and spoofing channel data.
  • Repair file play length information.
  • Scan for an eliminate any duplicate records found in the database

System Requirements

The computer running the tool must:

  • Windows / Windows Server
  • Have the SQLite ODBC Drivers installed (use x86 or x64 depending on your platform)
    Download: http://www.ch-werner.de/sqliteodbc/
  • Be able to access the dlrecorder.db database file as a file system mount (e.g. a direct drive mount, SMB, NFS etc) – under Windows this is in “C:\Users\Public\Documents\DVBLink\dlrecorder.db” by default.
  • Be able to access the recordings folder (the folder where DVBLink writes .ts recording files) as a system mount – under Windows this is in “C:\Users\Public\Documents\DVBLink\recorded” by default.

Usage

To run the code you must be running Windows and ensure that you install the correct SQLite ODBC driver first

  1. Make a manual backup of your dlrecorder.db file. This is found at “C:\Users\Public\Documents\DVBLink\dlrecorder.db”
    Note: If you do not make a backup of the database file and something goes wrong, you will at best lose meta data and at worst lost recordings. Don’t risk it, make a backup!
  2. Download and fully extract the zip file using the link below
    Note: Do not run the programme directly from the zip file
  3. Edit the config.vbs file in notepad or your preferred text editor
  4. Set the DB_PATH variable to the file system path of the dlrecorded.db file used by your DVBLink install. The default is provided
    CONST DB_PATH = “C:\Users\Public\Documents\DVBLink\dlrecorder.db”
  5. Set the RECORDINGS_PATH variable to the file system path of the recorded folder where DVBLink records .ts files. The default is provided
    CONST RECORDINGS_PATH = “T:\DVBLink\recorded”
  6. Save and close the config.vbs file
  7. Double click on the DVBLink_RecordingsUtility.wsf file to start the utility

Repair Utility Screenshot

Download

You can download the DVDLink Recordings Utility below. The code is © C:Amie. Please do not redistribute this file, please link them to this page.

No warranty is offered of implied for using the code, nor loss of data/recordings from your DVBLink database. Please ensure that you make a backup of the DVBLink database before using the tool.

Taking all necessary precautions, use this tool at your own risk.

The tool has been tested on DVBLink 6.0.0 under Windows 10 1709.

If you found this tool useful, please consider donating towards the running costs of this site.

Download: DVBLink_RecordingsTool-1.0.0.zip (8.37KB)

Issues / Ideas

If you have any issues or ideas on how the tool can be made to be more useful, please get in touch.

A thought for the DVBLink team

If anyone from the DVBLink team sees this, I really like what you do – it is a great product, it just has a weak link at the moment. I appreciate fully that you are using SQLlite for search and interrogation performance reasons. You could however fairly easily eradicate these problems by:

  1. Adding granular control to the database consistency checker so that it isn’t so brutal – have whatever mode it currently exists in as an level 2 option and just set a test for the presence of the file in the file system as the level 1 option.
  2. Write an XML/JSON file into the file system with the same name as the recording .ts file or the .jpg thumbnail file. Keep all of the meta data on the recording in here as well as the database. This allows you to implement file portability. With this in place, the consistency checker can be easily re-written to check whether the .ts file has a meta data file too. If it finds a .ts file that it doesn’t know about that does have a meta data XML/JSON file, simply import the file into the database. The database consistency checker algorithm can do this itself on service start/periodically.

If you do that you have several new features:

  1. Portability of recordings without transcoding.
  2. Recording export/import.
  3. An easy, end-user achievable, self-service recovery path from outdated backups with a process that is simple to write up in a knowledge base article.
  4. Significantly less risk / reliance on the integrity of the database. Plus reassurance for you that there is significantly less likelihood that users like me will feel compelled to poke around in the database in the first place.
  5. Support for removable recording targets – when a disk is missing the database is cleared, when it comes back the recordings appear in the database again.
  6. An easy way to report to the user that there are genuinely orphaned files because there is a .ts file with no meta data.
  7. Users can drop in their own .ts file and write their own meta data XML/JSON file for it in a text editor using your schema (also allowing import from competitor products).
  8. A more robust database and file consistency checker / scavenger.
  9. Preservation of disk space use for the user, something that as I have outlined here, seems to leak over time.

Change the Kodi TV Guide / Programme Listing Font Size under Kodi 17 / Krypton

System Requirements:

  • Kodi Krypton, 17.0, 17.1, 17.2, 17.3, 17.4

The Problem:

If you are a Kodi user, you will have no doubt noticed that with the change in version from 16 to 17, the XBMB Foundation changed the default skin from Confluence to Estuary. While Estuary is a nice looking skin, with Kodi 17, the programme has become more limited in the font customisation’s that are available through the skin/UI settings menu. Where as under previous versions, you had some pre-defined normal, large, extra large style choices, under version 17 all you can do is switch between default font and the system default font under the guise of it having higher legibility.

If your use case is such that you have a TV that is a long way away, or more importantly, if you need to support a user with accessibility needs who relied on Kodi’s ability to resize the text size in order to navigation the system, then you are somewhat out of luck at the current time.

This article outlines how to change the font size on the Kodi TV Guide/Programme Listings page by modifying the UI skin configuration files directly. It is also intended as a very high level illustration of how to modify other UI elements.

The Fix

The process of changing the font size is fairly easy, one you know what you are doing. The process of changing the font size is a three step process

  1. Find the correct configuration file
  2. Establish what your legal parameter values are
  3. Edit the configuration file in the correct place

Find the correct configuration file

The Estuary skin configuration and interface files are located at .\addons\skin.estuary\, relative to the install path. For example, on Windows, this is most likely:

C:Program Files (x86)\Kodi\addons\skin.estuary\

The key repository that you need to review is the xml folder (C:Program Files (x86)\Kodi\addons\skin.estuary\xml\).

In this folder, you will find the layout parameter files for Kodi. For the purposes of this guide, the file that we need to edit is MyPVRGuide.xml. You need to identify the correct file for the section that you want to edit in order to proceed.

If you are running Windows and User Account Control (UAC) is enabled, copy the file to your desktop and then make a backup copy of it. If you try and edit the fire directly under Program Files (x86) you will get an access denied error when you attempt to save the changes.

Establish what your legal parameter values are

Note: If you just want to edit the font and don’t care about the ‘how’, you can skip this section. Just realise that you cannot put any value that you want in the font size field when editing the skin.

For the font size, Kodi does not use an integer or point value for the font size (12, 36, 48 etc). Instead, it uses an XML enumerable type definition in its DTD representing allowed values. This means that you need to know what the allows values are and that if you want to use a non-standard value, you have to perform a far more complex series of edits to allow Kodi to support a new font size. As this change is something that you will likely need to make every time Kodi receives an official update, I strongly advise not attempting to create your own DTD value and consequently how to do it will remain beyond the scope of this article.

In order to ascertain what the allowed values are, I used a Command Prompt string search to look for all instances of “>font” under the XML folder.

The following command

findstr /s /l ^>font "c:\Program Files (x86)\Kodi\addons\skin.estuary\xml\*.xml"

Yielded the following block of definitions after de-duplication

<name>font10</name>
<name>font12</name>
<name>font13</name>
<name>font14</name>
<name>font25_narrow</name>
<name>font27</name>
<name>font27_narrow</name>
<name>font37</name>
<name>font45</name>
<name>font60</name>
<name>font_clock</name>
<name>font_flag</name>
<name>font20_title</name>
<name>font25_title</name>
<name>font30_title</name>
<name>font32_title</name>
<name>font36_title</name>
<name>font45_title</name>
<name>font52_title</name>
<name>font_MainMenu</name>

These fontXXXXXX values represent the allowed enumerate values that the skin ill accept. Anything else will be ignored and substituted with a default (font12).

Edit the configuration file in the correct place

Go back to the skin .xml file you copied to your desktop. Open the file in Notepad or your preferred text editor.

You now need to find the section that controls the UI element that you want to modify. Read through it looking for clues in the XML naming and if all else fails, it is a case of trial and error to find it (unless you want to go and read the Kodi skin documentation).

If you want to edit the TV Guide programme listing entry under Kodi 17.4, then look for the following section:

<itemlayout height="62" width="60">
<control type="image" id="2">
<width>58</width>
<height>58</height>
<texture border="3" fallback="windows/pvr/epg-genres/0.png">$INFO[ListItem.Property(GenreType),windows/pvr/epg-genres/,.png]</texture>
</control>
<control type="label" id="1">
<left>6</left>
<top>0</top>
<width>50</width>
<height>36</height>
<aligny>center</aligny>
<font>font12</font>
<label>$INFO[ListItem.Label]</label>
</control>
<control type="image">
<left>6</left>
<top>35</top>
<width>20</width>
<height>20</height>
<texture>$VAR[PVRTimerIcon]</texture>
</control>
</itemlayout>

The <font>font12</font> line controls the text size. In the use case that I had, I found that changing it to <font>font30_title</font> was acceptable. Remember: You can only use one of the lookup values shown in the section above and as this is XML, it is case sensitive!

Change the value, save the file and copy it back to the XML folder. Now (re)start Kodi to view your changes.

If you edit this value, you will notice that when you highlight the programme in the TV guide, the highlight goes back to a smaller font size, while non-highlighted programme’s will display in the new, larger font size. This is because the highlight is controlled by a different section. To change the highlight, go back to the .xml file in Notepad and edit the following section accordingly:

<focusedchannellayout height="62" width="350">
<control type="label">
<left>2</left>
<top>-2</top>
<width>75</width>
<height>60</height>
<font>font12</font>
<label>$INFO[ListItem.ChannelNumberLabel]</label>
<textcolor>button_focus</textcolor>
<align>center</align>
<aligny>center</aligny>
</control>
<control type="label" id="1">
<left>68</left>
<top>-2</top>
<width>350</width>
<height>60</height>
<font>font12</font>
<label>$INFO[ListItem.ChannelName]</label>
<textcolor>button_focus</textcolor>
<aligny>center</aligny>
<textoffsetx>10</textoffsetx>
</control>
</focusedchannellayout>

Note: in version 17.4 this is immediately ABOVE the section you just edited!

Change font12 to match your new value, save, put the file back in the XML folder and (re)start Kodi. The highlight font size should now match the rest of the Programme Guide.

Once you know where to go, the process it fairly easy. Do keep pin mind though that when you update Kodi to a new version, it will overwrite your changes and you will need to go back in and edit the font sizes once again. Hopefully the XBMC UI team will get around to restoring some degree of internal configurability for this soon, as not everyone in this world has 20:20 vision!

Hyper-V Discrete Device Assignment (DDA) with a TV Tuner (Hauppauge HVR-4400)

System Requirements:

  • Windows Server 2016
  • Hauppauge HVR-4400 PCIe Tuner

The Problem:

I am a DVBLink user. DVBLink does not play nicely with Windows Service and consequently it wants to run on a client OS. This means that I have lots of server hardware running server Operating Systems and one device with 4 TV Tuners in it running Windows 10.

With the release of Windows Server 2016 came the promise of VMWare like PCIe Pass-through, allowing physical devices on the PCI bus to be attached to VMs. The plan is to attach the PCIe TV Tuner and attempt to get DVBLink working in a VM so that the physical unit can be decommissioned (saving the power bill).

More Info

As part of the process, I was considering building a new server at the start of 2017 to perform a consolidation against. The Windows 10 DVBLink machine would be one consolidated devices onto more powerful modern hardware. I would also need new TV Tuners as only 2 of the 4 in the DVBLink TV Server is PCIe, the rest are PCI. Again, there are opportunities to consolidate that into fewer PCIe devices too.

The driver for the new server was Hyper-V PCIe Pass-through, or “Discrete Device Assignment” (DDA) as Microsoft are calling it. It is however quite difficult to find out whether BIOS firmware supports the proper implementations of I/O-MMU VT-d to permit it, making the purchase a risk. Equally, there is no guarantee that DDA will work with a TV Tuner.

Consequently, I decided to borrow a dual CPU Dell PowerEdge R630 to perform the experiment as there were several reports on-line that the R6xx and R7xx have the proper VT-d and SR-IOV feature set for this type of activity. Well done Dell (why don’t you advertise this?!).

After updating firmware, adding the TV Tuner and installing Windows Hyper-V Server 2016 on the machine, the first step was to – as an experiment – attempt to install the TV Tuner drivers on Windows Server 2016 (which errored). After that it was time to run the DDA Survey Script from Microsoft.

Download: DDA Survey Script (GitHub)

 

This was promising. The script found two devices that it stated were capable of being used with DDA

PERC H730 Mini
Express Endpoint -- more secure.
And its interrupts are message-based, assignment can work.
PCIROOT(0)#PCI(0100)#PCI(0000)

and

Hauppauge WinTV HVR-4400 (Model 121xxx, Hybrid DVB-T/S2, IR)
Express Endpoint -- more secure.
And it has no interrupts at all -- assignment can work.
PCIROOT(0)#PCI(0200)#PCI(0000)

The next step was to dismount the device from the Hypervisor and make it available to Hyper-V

# Find the HVR-4400
$pnpdevs = Get-PnpDevice -PresentOnly | Where-Object {$_.Class -eq "Media"} | Where-Object {$_.Service -eq "HCW85BDA"}# ... or if you know the hardware ID
$pnpdevs = Get-PnpDevice -PresentOnly | Where-Object {$_.InstanceId -eq "PCI\VEN_14F1&DEV_888
0&SUBSYS_C1080070&REV_04\4&39CDA168&0&0010"}foreach ($pnpdev in $pnpdevs) {
Disable-PnpDevice -InstanceId $pnpdev.InstanceId -Confirm:$false
Write-Host 'Device ' $pnpdev.InstanceId ' Disabled. NOTE: If this hangs, reboot and try again'
$instanceId = $pnpdev.InstanceId
$locationpath = ($pnpdev | get-pnpdeviceproperty DEVPKEY_Device_LocationPaths).data[0]
Write-Host 'Dismounting Device At: ' $locationpath ' (' $instanceId ')'
Dismount-VmHostAssignableDevice -locationpath $locationpath
Write-Host $locationpath
}

Initially, it hung PowerShell (and the system) so I had to hard reset the server. In this instance it was in fact necessary to reboot after issuing

Disable-PnpDevice

After trying again and rebooting the Dismount-VmHostAssignableDevice failed with

dismount-vmhostassignabledevice : The operation failed.
The manufacturer of this device has not supplied any directives for securing this device while exposing it to a
virtual machine. The device should only be exposed to trusted virtual machines.
This device is not supported when passed through to a virtual machine.
The operation failed.
The manufacturer of this device has not supplied any directives for securing this device while exposing it to a
virtual machine. The device should only be exposed to trusted virtual machines.
This device is not supported and has not been tested when passed through to a virtual machine. It may or may not
function. The system as a whole may become unstable. The device should only be exposed to trusted virtual machines.
At line:1 char:1

It would not proceed past this point. The trick was to change the line to

Dismount-VmHostAssignableDevice -locationpath $locationpath -Force

The next step was to ensure that the VM’s Automatic Stop Action was set to anything other than “Save”

Set-VM -Name “10-TEST” -AutomaticStopAction Shutdown

… and at this point it was simply a case of creating a VM and assigning the device

Add-VMAssignableDevice -LocationPath “$locationpath” -VMName “10-Test”

At which point the device immediately popped up in Device Manager under Windows 10 in the Generation 2 VM

DDA PCIe Passthrough in Device Manager

…. before the VM blue screened a few seconds later.

Blue Screen of Death

I tried to use several versions of the HVR-4400 driver that I could find and it made no difference. The VM would crash whenever it attempted to talk to the card. The Hypervisor itself did not seem to be impacted by the Blue Screen event and itself did not crash.

I also tried fully removing the device from the Hypervisor using DEVCON and clearing out the driver using pnputil. Superficially, this action made it worse as the VM wouldn’t boot at all now if it had a driver on-file for the TV Tuner. Before it would at least boot.

So this project was a failure and I will not be investing in new server hardware just yet. I’ll wait to see if Microsoft improve the feature set as allegedly this type of insanity (and yes, it is insane) is possible in VMWare. I do not want to change away from Hyper-V at the current time though, so I will have to stick with a client machine as a service.

This does not mean of course that this cannot work in Hyper-V. The HVR-4400 is a card from 2011/2012. So it is not exactly new hardware. PCIe TV Tuners designed to modern electrical standards and for use on PCIe 3.0 bus architectures may provide better interoperability out of the box. I just don’t have any other cards to test with and am in a bit of a chicken and egg situation over wanting to invest in new cards and servers unless I know they will work nicely together.

If you are interested in this too and would like me to have a go testing your hardware, please get in touch via HPC:Factor.