Tag: VBScript

VBScript Timer() function precision

C:Amie/ / Microsoft, Programming

This article explores the numeric precision of the ASP and VBScript Timer() function, outlining that it may be more accurate than it initially appears.

 

Timer()

Prior to the arrival of PowerShell. The VBScript Timer() function wass the closest thing that script creators had for undertaking high precision timings in the Windows Scripting Host (WSH) environment.

The Timer() function returns a VBSingle – aka a Single Prevision floating point number or a “Real” – value as a representation of the systems real time clock. As a Single precision value, the permitted range is “-3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values“. If you print out the value of Timer() in ASP/VBScript

WScript.Echo Timer()

You will get a value like

52191.62

This value is a representation of the number of seconds that have elapsed on the local executing system since the local system click last hit midnight. Consequently, evaluating Timer() against a DateDiff() evaluation of the number of seconds since Midnight will result in the same answer (save for the decimal point).

WScript.Echo Timer()
WScript.Echo DateDiff("s", #2019-07-01#, Now())

Which results in

52191.62
52191

 

The hidden detail

Masked in the 2 decimal places default printout. Timer() is significantly more detailed than just two decimal points. Depending on the precision of your systems Real Time Clock (RTC) the precision may be up to 7 decimal places. You can view your systems capability by subtracting the Integer from the timer value:

WScript.Echo Timer() - Int(Timer())

Which may result in a value such as

0.6171875

At 7 decimal places, the precision of Timer() is – floating point number inaccuracy side – considerably better than that of VBDateTime. Under VBDateTime, the second is the atomic value, offering no more precision.

0         - Second
0.6       - Decisecond  / 1 tenth
0.61      - Centisecond / 1 hundredth
0.617     - Millisecond / 1 thousandth
0.6171    -               1 ten thousandth
0.61718   -               1 hundred thousandth
0.617187  - Microsecond / 1 millionth
0.6171875 -               1 ten millionth

This demonstrates that there is flexibility in VBScript for more precise clock operations. But is the resolution high enough?

 

Fidelity

The resolution and fidelity of the Timer() function is what makes it valuable (or not) to a programmer. On a modern system, with a High Precision Event Timer (HPET), the update interrupt will only fire so many times to update the clock.

Running an imprecise test as follows:

for i = 0 to 999
  WScript.Echo Timer() - Int(Timer())
next

The output value only changed between 10 and 31 cycles (reflective of the CPU scheduler performing other tasks during execution. The counter incrementation was consistent, updating 48 times with an increment of between 0.0117187 and 0.0195313 seconds. Once every 20.8 cycles on average.

0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.0195313
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.015625
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187
0.0117187

In practice this means that the viable, comparable resolution of Timer() is not much better than once aver 0.2 seconds. Over a longer time period, Timer() can offer higher accuracy: provided you aren’t polling for an update more than every 0.2 of a second.

Replace line breaks in PRE tags using RegEx in ASP/VBScript

C:Amie/ / Programming

This article discusses how to replace unnecessary line breaks (<br>, <br />) in HTML pre-formatted ‘PRE‘ tags using RegEx and ASP/VBScript.

The Problem

When I added a [code] tag to the HPC:Factor Community Forums markup some years ago. There was an obvious, but low-priority problem that had niggled at me. The BBS tag renders a pre-formatted ‘PRE‘ tag into the post HTML and styles it using a console font and fixed-size characters for improved legibility. The way that MegaBBS works however is that new lines (vbcrlf, vblf, \r\n, \n, chr(13) & chr(10), chr(10)) are replaced with ‘<br />‘ as a batch replace at the beginning of the forum sanitisation and rendering process.

This means that when the browser renders the <pre></pre> tag, it renders both the vblf and the <br />, leading to double line breaks.

For example

Dim i
i = 0
while (i < 1000)
  Response.Write i
  i = (i + 1)
loop

becomes

<pre><br />
Dim i<br />
i = 0<br />
while (i < 1000)<br />
  Response.Write i<br />
  i = (i + 1)<br />
loop<br />
</pre><br />

leading the browser to render

Dim i

i = 0

while (i < 1000)

Response.Write i

i = (i + 1)

loop

This wastes screen space and reduces legibility.

RegEx Fix

The solution is very simple, use RegEx to re-parse the [code] block after it has globally replaced the line breaks.

I added the following code at the bottom of the MBBS Code loop in the MBBSDecode function  in include.asp

if (vBBSDecodeArray(0, index) = "\[code\]") then	' This looks for the PRE tag for the code and then removes the <br />'s from it to return it to pure pre-formatted
	mBBSRegEx.pattern = "\<pre[^>]*\>((.|\n)*)\<\/pre\>"
	for each sNewText in mBBSRegEx.execute(MBBSDecode)
		MBBSDecode = Replace(MBBSDecode, sNewText.Value, (Replace(sNewText.Value, "<br />", "")))
	next
end if

To evaluate what this means line-by-line

  1. When it is parsing the [code] tag from the list of all BBS markup statements
  2. Set a RegEx pattern to search for the opening PRE tag with zero or more attributes e.g. <pre attribute1="one" attribute2="two"> ending with </pre>. The “((.|\n)*)” ensures that the search looks for all characters, including over new lines, for as many characters and new lines as is necessary to encounter the closing </pre> tag.
  3. For every positive match i.e. for every <pre>*</pre> match
  4.  In the matches string, replace <br /> with "", then replace the match in the original source string (MBBSDecode) with the fixed string
  5. Move to the next match until there are no more matches

 

To genericise the example

Dim strHtml
Dim match
Dim matches
Dim regEx

strHtml = "<body><p>hello</p><pre class="">line one" & vbcrlf & "<br />line two" & vbcrlf & "<br />line three" & vbcrlf & "<br /></pre><p>hello</p></body>"

set regEx = New RegExp
    regEx.Pattern = "\<pre[^>]*\>((.|\n)*)\<\/pre\>"
    regEx.IgnoreCase = true
    regEx.Global = true
set matches = regEx.Execute(strHtml)

For Each match in matches
    strHtml =  = Replace(strHtml, match.Value, (Replace(match.Value, "<br />", "")))
Next

' strHtml will now effectively be:
' "<body><p>hello</p><pre class="">line one" & vbcrlf & "line two" & vbcrlf & "line three" & vbcrlf & "</pre><p>hello</p></body>"

Fairly simple, but as with most things RegEx, a headache for most of us – unless you are using it all the time.

DVBLink 6.0.0 Recordings Database Repair Utility

C:Amie/ / Multimedia

System Requirements:

  • DVBLogic, DVBLink 6.0.0
  • Windows Scripting Host

The Problem:

DVBLink is a good product which works across a large number of platforms, and, considering how complex it is, it does a good job of interfacing between DVB-T/DVB-S terrestrial TV transmissions and the on-demand world of modern media consumption.

I’ve always had a few niggles with the system (‘clear data’ in Kodi anyone?), but have always found it good, save for one area. It’s recording database mechanic is at best flimsy and at worst out-right fragile.

The number of database corruptions, anomalies and times I have had to restore the database form backup in essence amount to every 6 months. It can be caused by a service crash, the PC restarting without a clean shutdown (power cut), loss of visibility of the disk where the recordings folder target sits (e.g. USB drive, iSCSI recording storage on a SAN/NAS or any other ejectable media) or as a result of installing a DVBLink update (I had a v5 update that trashed my recordings database about 2 months after I first started using it).

To provide a simple summary of the issues with the SQLlite database implementation:

  1. If the database is lost, that’s it. DVBLink/Kodi etc cannot ‘see’ your recordings, they are lost to the software even though they are still physically on disk and if you know where to look and how to open them, they remain watchable.
  2. Some failure situations can lead the database to have more in it than the file system does. This leads to the Kodi ‘click of nothing’, where clicking an item in the recordings section does just that, nothing (no error, just nothing)…
  3. … DVBLink has an option in the server settings to perform a consistency check. This is supposed to fix the ‘I have a database record but no file issue’… yet in reality, in my experience, this far too complicated and (I would assume) tries to be cleverer than asking “is there a recordings file on disk for this recording entry in the database?”. If you have ever tried to manually reconstruct the database with this option on, you’ll know that DVBLink simply deletes the entry…
  4. … Some failure situations – and the DVBLink provided consistency checking option – can lead the database to not have entries for recordings that do exist in the file system. Referring back to #1, you’ll recall that this means that DVBLink/Kodi etc does not know that they exist and you cannot watch the recordings. Equally there is no way to delete the orphaned files as they are not reported. Consequently your DVBLink system will slowly leak disk space. This is made worse by the fact that the database appears to track deletions, but doesn’t appear to have a (working) mechanism to restore the records again.
  5. There is no way to automatically recover from these situations.
  6. Once in these situations, there is no way to recover lost meta data on recording files that are orphaned in the recordings directory.

When a family member with a large pre-existing recording catalogue went away on 22nd December, returned on the 29th December to find that

  1. The old catalogue was no longer showing in Kodi
  2. No new programme recorded before 27th December was present

There were plenty of tears before bedtime.

Due to a connectivity problem between the DVBLink service and the recorder disk on the 27th, DVBLink had restarted and decided that as it couldn’t find the recordings folder, it would delete the now missing content form the recordings database.

It started recording from the 27th onwards onto the tiny local SSD that the OS is installed onto as a fail back (fair dues here DVBLink, good move), but come the 29th there were only 6 things (new content from the 28th and 29th) in the database. There should have been around 70 recordings.

There was no database backup available later than 12th December (by luck, I had made one on the 12th). There was no automatic database backup later than October.

More Info

I now needed to merge the 12th December backup and the 27th December+ master database back together. This would leave a data gap from 13th-26th December of missing records, so having done this in a SQLLite GUI editor, I started the DVBLink back up and… it promptly deleted 85% of all the material introduced from the 12th December backup.

The mystery of the database reconciliation option strikes again. By checking that all of the filenames in the database, I confirmed that the files were on the disk, including the thumbnails, but it decided to delete the imported old records anyway. Just not all of them for an unknown reason.

I turned off the reconciliation option and repeated the merging process again, and, it worked. The files played back in Kodi. I just cannot enable the database reconciliation feature. Ever.

At this point I turned my attention to the missing 13th-26th period. The file system revealed that this amounted to some 14 recordings. At this point I noticed one slight problem. I was expecting to be recovering around 70 files into the database, yet there were over 180, large, playable .ts files in the recording folder consuming some 360 GB of disk space. These files covered a period starting in April 2015 through to 29th December 2017.

Looking non-exhaustively at very old database backup files, I was able to find several of them, proving that they had at one point existed and had not been deleted by the user, but by the database consistence checked/reconciler. This posed the following chain of problems:

  1. If I import the contents of all of the old database backups that I have, I will be importing a mass of junk data.
  2. If I turn on the automatic (and unreliable) consistency checker to clean that up, I will wind up deleting all of the work thus far and likely only be a couple of steps further forward.
  3. I still have not solved the problem of the completely missing 13th December 2017 – 26th December 2017 database records.

I thus decided to write some code and use programming to help solve the problem. Thus the DVBLink_RecordingsDb Maintenance API was created. The API allows you to view:

  • All files in the file system
  • All records in the database
  • All files in the file system that are not in the database
  • All records in the database for which there is no associated file in the file system

In turn it allows you to:

  • Remove entries in the database where the is no resultant file (using no other logic beyond is the correct .ts file presence e.g. the presence of a valid recording timer entry in the database to test validity is not used).
  • Add entries to the database where there is a file in the file system but no existing database entry. The tool will attempt – i.e. best effort – to populate a new database record with the minimum amount of information necessary to allow DVBLink/Kodi to play the file. This is largely based upon the file name and spoofing channel data.
  • Repair file play length information.
  • Scan for an eliminate any duplicate records found in the database

System Requirements

The computer running the tool must:

  • Windows / Windows Server
  • Have the SQLite ODBC Drivers installed (use x86 or x64 depending on your platform)
    Download: http://www.ch-werner.de/sqliteodbc/
  • Be able to access the dlrecorder.db database file as a file system mount (e.g. a direct drive mount, SMB, NFS etc) – under Windows this is in “C:\Users\Public\Documents\DVBLink\dlrecorder.db” by default.
  • Be able to access the recordings folder (the folder where DVBLink writes .ts recording files) as a system mount – under Windows this is in “C:\Users\Public\Documents\DVBLink\recorded” by default.

Usage

To run the code you must be running Windows and ensure that you install the correct SQLite ODBC driver first

  1. Make a manual backup of your dlrecorder.db file. This is found at “C:\Users\Public\Documents\DVBLink\dlrecorder.db”
    Note: If you do not make a backup of the database file and something goes wrong, you will at best lose meta data and at worst lost recordings. Don’t risk it, make a backup!
  2. Download and fully extract the zip file using the link below
    Note: Do not run the programme directly from the zip file
  3. Edit the config.vbs file in notepad or your preferred text editor
  4. Set the DB_PATH variable to the file system path of the dlrecorded.db file used by your DVBLink install. The default is provided
    CONST DB_PATH = “C:\Users\Public\Documents\DVBLink\dlrecorder.db”
  5. Set the RECORDINGS_PATH variable to the file system path of the recorded folder where DVBLink records .ts files. The default is provided
    CONST RECORDINGS_PATH = “T:\DVBLink\recorded”
  6. Save and close the config.vbs file
  7. Double click on the DVBLink_RecordingsUtility.wsf file to start the utility

Repair Utility Screenshot

Download

You can download the DVDLink Recordings Utility below. The code is © C:Amie. Please do not redistribute this file, please link them to this page.

No warranty is offered of implied for using the code, nor loss of data/recordings from your DVBLink database. Please ensure that you make a backup of the DVBLink database before using the tool.

Taking all necessary precautions, use this tool at your own risk.

The tool has been tested on DVBLink 6.0.0 under Windows 10 1709.

If you found this tool useful, please consider donating towards the running costs of this site.

Download: DVBLink_RecordingsTool-1.0.0.zip (8.37KB)

Issues / Ideas

If you have any issues or ideas on how the tool can be made to be more useful, please get in touch.

A thought for the DVBLink team

If anyone from the DVBLink team sees this, I really like what you do – it is a great product, it just has a weak link at the moment. I appreciate fully that you are using SQLlite for search and interrogation performance reasons. You could however fairly easily eradicate these problems by:

  1. Adding granular control to the database consistency checker so that it isn’t so brutal – have whatever mode it currently exists in as an level 2 option and just set a test for the presence of the file in the file system as the level 1 option.
  2. Write an XML/JSON file into the file system with the same name as the recording .ts file or the .jpg thumbnail file. Keep all of the meta data on the recording in here as well as the database. This allows you to implement file portability. With this in place, the consistency checker can be easily re-written to check whether the .ts file has a meta data file too. If it finds a .ts file that it doesn’t know about that does have a meta data XML/JSON file, simply import the file into the database. The database consistency checker algorithm can do this itself on service start/periodically.

If you do that you have several new features:

  1. Portability of recordings without transcoding.
  2. Recording export/import.
  3. An easy, end-user achievable, self-service recovery path from outdated backups with a process that is simple to write up in a knowledge base article.
  4. Significantly less risk / reliance on the integrity of the database. Plus reassurance for you that there is significantly less likelihood that users like me will feel compelled to poke around in the database in the first place.
  5. Support for removable recording targets – when a disk is missing the database is cleared, when it comes back the recordings appear in the database again.
  6. An easy way to report to the user that there are genuinely orphaned files because there is a .ts file with no meta data.
  7. Users can drop in their own .ts file and write their own meta data XML/JSON file for it in a text editor using your schema (also allowing import from competitor products).
  8. A more robust database and file consistency checker / scavenger.
  9. Preservation of disk space use for the user, something that as I have outlined here, seems to leak over time.

Script to split single file multiple contact VCF file into individual VCF files for each included contact (Import Android Contacts Export file into Microsoft Outlook)

C:Amie/ / Microsoft, Software

System Requirements:

  • Windows Scripting Host 5.6 or higher
  • Windows 95, NT 4, 98, 98SE, ME, 2000, XP, Vista, 7, 8, 8.1, 10
  • Windows Server NT 4, 2000, 2003, 2003 R2, 2008, 2008 R2, 2012, 2012 R2, 2016

The Problem:

I did an export from the People / Contacts on Android which produced a single .VCF vContact file for all of the contacts. On attempting to import it into Outlook, Outlook ignored all but the first entry in the file.

Additionally, you may not want to import all of the contacts in the monolithic VCF.

The Fix

The VB Script file below will take the .VCF file and split it into individual .VCF files, one for each contact.

The script will create a sub-directory in the same folder that the script is executed from and all of the new .vcf files will be placed in here using the name of the user.

Download

  • No warranty is offered or implied for the use of this script. No liability is inferred for damages caused as a result of its use. Use it at your own risk
  • You may use the script for free for personal use
  • You may not redistribute this script. If you want to link to it, please link to this page.

Download: SplitVCard version 1.0.1 (2.57 KB)

Usage Instructions

You can either drag and drop the monolithic VCF file onto the SplitVCard.vbs file in Windows Explorer or use the command line syntax as follows to specify the path of the source file.

cscript.exe SplitVCard.vbs “C:\Bla\Bla\Bla\Contacts.vcf”

Release history

1.0.0 – 21/11/2016. Initial release

1.0.1 – 22/06/2017. Added some additional illegal character protections when writing individual files back to the file system for ?, / and \ characters

Limitations

As this is a VB Script file, the naming conventions are currently only setup for ASCII. If you wanted to use Unicode or another encoding, you will need to edit the script.