Unable to update NuGet or Packages in Powershell due to “WARNING: Unable to download the list of available providers. Check your internet connection.”

When attempting to install or update PowerShell Modules, NuGet or NuGet packages in PowerShell 5. You receive one or more of the following errors

WARNING: Unable to resolve package source 'https://www.powershellgallery.com/api/v2/'.

The underlying connection was closed: An unexpected error occurred on a receive.

WARNING: Unable to download the list of available providers. Check your internet connection.

Equally, you may receive the same error when attempting to run a WGET or an Invoke-WebRequest command e.g.

wget https://www.google.com/

You are unable to install/update the software component or make an outbound internet connection.

This issue may be especially prevalent on IIS installations serving HTTPS websites.

The Fix

Conventional troubleshooting is fairly well documented on-line

  1. Ensure that you are actually able to open a https webpage in a web browser
  2. Ensure that your DNS is working correctly.
  3. Check to see whether wget can connect to a non-https site e.g.
    wget http://www.google.com/
  4. Check to see whether or not you need to use a Proxy server. If so, you must configure PowerShell to use your Proxy Server before you proceed. This may require you to to configure PowerShell with your Proxy Server credentials.
    $webclient=New-Object System.Net.WebClient
    $webclient.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

A less obvious issue to explore related to the default operating system security configuration for using SSL.

More Info

By default, Windows Server and Windows client will allow SSL3, TLS 1.0, TLS 1.1 and TLS 1.2. The .net Framework is also configured to allow these protocols, and, by default, any outbound request for a SSL site will attempt to use SSL3/TLS 1.0 as its default protocol.

In secure environments, where system administrators have enabled recommended best practice on Windows systems to disable the use of SSL1, 2,3 and TLS 1.0. PowerShell is not currently clever enough to internally compare its configuration to that of the operating system. consequently, when attempting to make an outbound https request in such an environment. PowerShell will attempt to use one of the older protocols which has been disabled by the operating system’s networking stack. Instead of re-attempting the request using a higher protocol. PowerShell will fail the request with one of the error messages listed at the beginning of the article.

As NuGet and Update-Module both attempt to make connections to Microsoft servers using HTTPS, they too will fail.

Encountering this issue on a SSL enabled IIS install will be more common, as it is more likely that system administrators will have applied best practice and disabled legacy encryption protocols on these servers. their public facing, high visibility should demand such a response.

To fix the issue there are two options:

  1. Reconfigure and reboot the system to re-enable client use of TLS 1.0 (and possibly SSL3) via
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\<protocol>\Client

    DisabledByDefault = 0
    Enabled = ffffffff (hex)

  2. Alternatively, you must set-up each PowerShell environment so that the script itself knows not to use the legacy protocol versions. This is achieved via the following code which restricted PowerShell to only using TLS 1.1 and TLS 1.2.
    [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls11,Tls12'

Simple Fourm YouTube API Video Embed Sample Script

System Requirements:

  • HTML

The Problem:

www.hpcfactor.com’s forum software dates from 2004. At some point in the interim I rigged it to support a basic

BBS markup tag which uses a <embed> tag and the legacy a Flash object to inject a video from YouTube onto the page:

<object width="640" height="390">
 <param name="movie" value="{param}"></param>
 <param name="allowFullScreen" value="true"></param>
 <param name="allowScriptAccess" value="always"></param>
 <embed src="{param}" type="application/x-shockwave-flash" allowfullscreen="true" allowScriptAccess="always" width="640" height="390"></embed>
</object>

It is however currently 2016 and HTML 5 is the norm now, so at the behest of a user I wanted to update it.

More Info

Using the formal YouTube APU Reference for iframe Embeds sample. I needed to make something quick and easy that could be injected into the (extremely limited) BBS Markup system on the 12 year old hpcfactor.com forum. The only default substitution available without re-writing the forum core is {param} which takes the <URL> value from the BBS

markup.

View: YouTube Player API Reference for iframe embeds

 

The code sample shown below was the quickly generated solution. The steps it goes through are:

  1. Drop a DIV in-line on the page for the YouTube API to replace with the iFrame
  2. Drop the JavaScript in-line as well (it has some unnecessary duplication but won’t execute)
  3. The JavaScript checks to see if the YouTube API is already on the stack, if it is not it loads the YouTube API, if it is, it skips to the call to the JavaScript to embed the iFrame
  4. If the youTube API is not loaded, it loads the youTube API, sets up an array structure to act as a callback and pointer reference source to track all video files requested to be embedded on the page
  5. It sets up the onYouTubePlayerAPIReady listener to listen for the ‘ready’ callback from the youTube API, at which point JavaScript will be told to parse the sources array and create the iFrame’s
  6. A registration function is provided. This receives video registrations even before the YouTube API is ready. As soon as the YouTube API callback to the onYouTubePlayerAPIReady listener, the video registrations are processed. Until it does, they queue.
  7. A function to test to see if the video has already been embedded is provided and is flagged once it has been embedded

<div id="videoPlayer_{param}"></div>
<script type="text/javascript">
// Load the IFrame Player API code asynchronously.
if (document.getElementById('youTubeExternal') === null) {
var tag = document.createElement('script');
tag.id = 'youTubeExternal';
tag.type = 'text/javascript';
tag.src = "https://www.youtube.com/player_api";
var headTag = document.getElementsByTagName('head')[0];
headTag.appendChild(tag);
var arrVideoStack = new Array();
arrVideoStack[0] = new Array(); // 0 = player reference, 1 = Display Elm ID, 2 = Video ID, 3 = IsLoaded
arrVideoStack[1] = new Array();
arrVideoStack[2] = new Array();
arrVideoStack[3] = new Array();
var iVideoCount = 0;
var bolYtReady = false;// Replace the 'ytplayer' element with an <iframe> and
// YouTube player after the API code downloads.
function onYouTubePlayerAPIReady() {
bolYtReady = true;
loadVideos();
}function registerNewVideo(strTargetElmId, strVideoId) {
strVideoId = strVideoId.replace('https://www.youtube.com/watch?v=', '');
strVideoId = strVideoId.replace('https://www.youtube.com/v/', '');
strVideoId = strVideoId.split('&')[0]; // In case there are any more parameters
arrVideoStack[0][iVideoCount] = null;
arrVideoStack[1][iVideoCount] = strTargetElmId;
arrVideoStack[2][iVideoCount] = strVideoId;
arrVideoStack[3][iVideoCount] = false; iVideoCount++;
if (bolYtReady) {
loadVideos();
}
}function loadVideos() {
if (bolYtReady) {
if (iVideoCount > 0) {
for (i = 0; i < iVideoCount; i++) {
// If it hasn't already loaded, load it
if (!arrVideoStack[3][i]) {
arrVideoStack[0][i] = new YT.Player(arrVideoStack[1][i], {
height: '640',
width: '390',
videoId: arrVideoStack[2][i]
});
arrVideoStack[3][i] = true;
}
}
}
}
}
}

registerNewVideo('videoPlayer_{param}', '{param}');
</script>

If you would like to test it without the BBS markup parts in-situ you can copy and paste the following version into a .html file and double click on it

<div id="videoPlayer_Test1"></div>

<div id="videoPlayer_Test2"></div>

<div id="videoPlayer_Test3"></div><script type="text/javascript">

// Load the IFrame Player API code asynchronously.

if (document.getElementById('youTubeExternal') === null) {

var tag = document.createElement('script');

tag.id = 'youTubeExternal';

tag.type = 'text/javascript';

tag.src = "https://www.youtube.com/player_api";

var headTag = document.getElementsByTagName('head')[0];

headTag.appendChild(tag);

var arrVideoStack = new Array();

arrVideoStack[0] = new Array(); // 0 = player reference, 1 = Display Elm ID, 2 = Video ID, 3 = IsLoaded

arrVideoStack[1] = new Array();

arrVideoStack[2] = new Array();

arrVideoStack[3] = new Array();

var iVideoCount = 0;

var bolYtReady = false;// Replace the 'ytplayer' element with an <iframe> and

// YouTube player after the API code downloads.

function onYouTubePlayerAPIReady() {

bolYtReady = true;

loadVideos();

}function registerNewVideo(strTargetElmId, strVideoId) {

strVideoId = strVideoId.replace('https://www.youtube.com/watch?v=', '');

strVideoId = strVideoId.replace('https://www.youtube.com/v/', '');

strVideoId = strVideoId.split('&')[0]; // In case there are any more parameters

arrVideoStack[0][iVideoCount] = null;

arrVideoStack[1][iVideoCount] = strTargetElmId;

arrVideoStack[2][iVideoCount] = strVideoId;

arrVideoStack[3][iVideoCount] = false; iVideoCount++;

if (bolYtReady) {

loadVideos();

}

}

function loadVideos() {
if (bolYtReady) {
if (iVideoCount > 0) {
for (i = 0; i < iVideoCount; i++) {
// If it hasn't already loaded, load it
if (!arrVideoStack[3][i]) {
arrVideoStack[0][i] = new YT.Player(arrVideoStack[1][i], {
height: '640',
width: '390',
videoId: arrVideoStack[2][i]
});
arrVideoStack[3][i] = true;
}
}
}
}
}
}

registerNewVideo('videoPlayer_Test1', 'dQw4w9WgXcQ');
registerNewVideo('videoPlayer_Test2', 'm2ATf01v4hw');
registerNewVideo('videoPlayer_Test3', 'YVhxcpItk_M');
</script>

0x80090020 when attempting to load a .PFX Private Key Certificate into a CAPICOM_MEMORY_STORE using Store.Load() or Certificate.Load() using CAPICOM 2.1.0.2

System Requirements:

  • Windows Server 2008, R2
  • Windows Vista
  • Windows 7
  • Windows 8, 8.1
  • 7.0, 7.5, 8.0
  • ASP 3.0 (Classic)
  • CAPICOM 2.1.0.2

The Problem:

Ah encryption, that most noble of things. One thing that is sure to drive every developer close to the brink on the odd occasion. The one time where clear, concise API documentation should be considered mandatory – and the one place where good API documentation it seems is an obligation itself not to provide. Be it Microsoft, Java, BouncyCastle, PHP it would seem they are all blighted with the same issue.

Attempting to use legacy API on an unsupported platform should seem like an exercise in masochism, however, you know how much I like to avoid using .net whenever I can.

If you attempt to do this

Dim cert
set cert = Server.CreateObject("CAPICOM.Certificate")
call cert.load("c:\myPrivateKey.pfx", "test", CAPICOM_KEY_STORAGE_EXPORTABLE)

or this

const CAPICOM_MEMORY_STORE = 0
const CAPICOM_LOCAL_MACHINE_STORE = 1
const CAPICOM_STORE_OPEN_READ_WRITE = 1
const CAPICOM_KEY_STORAGE_EXPORTABLE = 1Dim store
set store = Server.CreateObject("CAPICOM.Store")
call store.Open(CAPICOM_MEMORY_STORE, "MemoryStore1", CAPICOM_STORE_OPEN_READ_WRITE)
call store.load("c:\myPrivateKey.pfx", "test", CAPICOM_KEY_STORAGE_EXPORTABLE)

you will get back

error '80090020'
/file.asp, line ###

If you send in a .cer file instead of a .pfx, it works without error but doesn’t allow you to access the Private Key.

More Info

Taking the two code samples in order

Dim cert
set cert = Server.CreateObject("CAPICOM.Certificate")
call cert.load("c:\myPrivateKey.pfx", "test", CAPICOM_KEY_STORAGE_EXPORTABLE)

Should you be getting a 0x80070056 error, your password is wrong. If the file doesn’t have a password, only send parameter 1 (which is about to cause you a problem). To resolve the 0x80090020 error while using a CAPICOM_MEMORY_STORE, you need to stop CAPICOM from attempting to insert the certificate as a resource for a user. If the IIS worker process that you are using doesn’t connect to a user account and has no permissions, the default parameter CAPICOM_CURRENT_USER_KEY or 0 will throw 0x80090020.

To change the scope, ensure that you use the fourth parameter and set the value to CAPICOM_LOCAL_MACHINE_KEY.

const CAPICOM_CURRENT_USER_KEY = 0
const CAPICOM_LOCAL_MACHINE_KEY = 1Dim cert
set cert = Server.CreateObject("CAPICOM.Certificate")
call cert.load("c:\myPrivateKey.pfx", "test", CAPICOM_KEY_STORAGE_EXPORTABLE, CAPICOM_LOCAL_MACHINE_KEY)

To resolve the second issue, modify the original code to make use of the now fixed certificate.load() call and import it vie the long route.

const CAPICOM_MEMORY_STORE = 0
const CAPICOM_LOCAL_MACHINE_STORE = 1
const CAPICOM_STORE_OPEN_READ_WRITE = 1
const CAPICOM_KEY_STORAGE_EXPORTABLE = 1Dim cert
Dim store
set store = Server.CreateObject("CAPICOM.Store")
call store.Open(CAPICOM_MEMORY_STORE, "MemoryStore1", CAPICOM_STORE_OPEN_READ_WRITE)set cert = Server.CreateObject("CAPICOM.Certificate")
call cert.load("c:\myPrivateKey.pfx", "test", CAPICOM_KEY_STORAGE_EXPORTABLE, CAPICOM_LOCAL_MACHINE_KEY)

call store.add(cert)

If you receive 0x80070005, you are either getting an Access Denied error to the MEMORY_STORE or you are attempting to import a certificate into the instantiated store which already exists. Similarly, if you receive 0x80070056, your password is wrong.

‘Bug’ in ASP 3.0 Application.Contents iterator causes undesired deletion patterns when Application.Contents.Remove() is called from within a for each / for loop

System Requirements:

  • Windows NT 4.0 Server SP4+
  • Windows Server 2000
  • Windows Server 2003
  • Windows Server 2008, R2
  • Windows Server 2012, R2
  • Windows Server 2016
  • Windows Server 2019
  • Windows 2000 Professional
  • Windows XP
  • Windows Vista
  • Windows 7
  • Windows 8, 8.1
  • windows 10
  • IIS 4.0, 5.0, 5.1, 6.0, 7.0, 7.5, 8.0, 10.0
  • ASP 3.0 (Classic)

The Problem:

I remember, long ago in approximately 2001 – in my less competent days – fighting to make something work and ultimately concluded that it was a hapless endeavour and ultimately went about it in a different way. The task was to clear down all but a small number of elements from the ASP 3.0 Application.Contents object.

What I concluded then, is something that I’ve only just re-remembered now after finally making a determined effort to hunt down a bug in a module on HPC:Factor – which is being used elsewhere – and in which a recent change brought the issue back to light.

There is an iterator issue come bug (depending on your point of view) in the ASP 3.0 Application object.

More Info

We’ll lead by example with this one. After spending an hour or so reacquainting myself with the problem while fixing (read making more robust) the HPC:Factoor class module, a fairly simple process can be used to demonstrate it. Whether or not you see this as a natural feature, or a sincere bug is something that I’ll leave to you. There are always ways around this sort of thing, so I guess that what counts is whether you think it should be fixed in the iterator or by the end user.

Take the following code

Option Explicit
Dim strKey
Application.Contents.RemoveAll()
Application.Contents("one") = "a"
Application.Contents("two") = "b"
Application.Contents("three") = "c"
Application.Contents("four") = "d"
for each strKey in Application.Contents
  Response.Write strKey & " == " & Application.Contents(strKey) & "<br />"
next

It’s obviously going to print out the following

one == a two == b three == c four == d

So what if we now do this:

for each strKey in Application.Contents
  Application.Contents.Remove(strKey)
next
for each strKey in Application.Contents
  Response.Write strKey & " == " & Application.Contents(strKey) & "<br />"
next

Clearly it should print

For dramatic effect, that’s “absolutely nothing being printed”. The application object should be completely empty.

Wrong! It prints:

two == b
four == d

What’s going on is quite simple. The for each iterator being called from the Application.Contents collection is indexed, in other words when items are added or removed they are given a numeric, integer based index in order to aid lookup.

This index becomes stateful as it initially exists at call time for the “for each” provider and its content is copied out to the iterator, By Value (ByVal). It should really be passed out By Reference (ByRef) i.e. via a Pointer.

What this means (using comments to explain the process) is that the following logic occurs:

Option Explicit
Dim strKey
Application.Contents.RemoveAll()            ' Delete all indexes, release pointers to all data
Application.Contents("one") = "a"           ' Create Index 1, Key:"One", Value:"a"
Application.Contents("two") = "b"           ' Create Index 2, Key:"two", Value:"b"
Application.Contents("three") = "c"         ' Create Index 3, Key:"three", Value:"c"
Application.Contents("four") = "d"          ' Create Index 4, Key:"four", Value:"d"
' Application.Contents.Count = 4

for each strKey in Application.Contents     ' Create an iterator of the index [1 - 4]
  ' Iterator Index i = 1
  Application.Contents.Remove(strKey)     ' Remove item at index 1
  ' Index 1 removed, compact index
  ' Index 1, Key:"two", Value:"b"
  ' Index 2, Key:"three", Value:"c"
  ' Index 3, Key:"four", Value:"d"
  ' Application.Contents.Count = 3
  ' Move to NEXT
  ' Iterator Index = 2 (i = (i + 1))
  Application.Contents.Remove(strKey)     ' Remove item at index 2
  ' Index 2 removed, compact index
  ' Index 1, Key:"two", Value:"b"
  ' Index 2, Key:"four", Value:"d"
  ' Application.Contents.Count = 2
  ' Move to NEXT
  ' Iterator Index = 3 (i = (i + 1))
  ' 3 is greater than 2 (the index is > count), exit

The problem is that the Index is being compacted on a successful call to .Remove(). The count of the number of items in Application.Contents is being updated to reflect the correct number of items, but the iterator isn’t being told i = (i – 1) after the successful completion of the Remove() method.

The same thing happens if you use “for” rather than “for each”:

Option Explicit
Dim i
Application.Contents.RemoveAll()
Application.Contents("one") = "a"
Application.Contents("two") = "b"
Application.Contents("three") = "c"
Application.Contents("four") = "d"
for i = 1 to Application.Contents.Count
  Application.Contents.Remove(i)
next

This also results in data still remaining inside the Application Object due to the same error, except here we are directly calling the iteration number ourselves via i so we also get 2 and 4 left in the collection as with calling .Remove() from the “for each”.

If the Application.Contents.Remove method supported a success/failure return type – for example a boolean true for item removed and boolean false for no such item in collection, then the fix would be simple:

for i = 1 to Application.Contents.Count
  if (Application.Contents.Remove(i)) then
    i = (i - i)
  end if
next

Sadly the method doesn’t support a return type.

The Fix

This bug means that there are only two ways to deal with it The first way would be to iterate across the collection, store the Keys in an array and then in a second pass remove all of the items that you want to delete by using an external array.

It does the job and allows you to continue to use keys, but why use two loops when you can use one? In the knowledge that the following is true:

  1. The index is compacting
  2. The iterator is not being reduced by 1 after a successful call to .Remove()

The second and simplest approach to solve the problem is to force the for loop to decrement it for you. In other words, reverse iterate instead of forward iterate through the collection.

Dim i

Application.Contents.RemoveAll()
Application.Contents("one") = 1
Application.Contents("two") = 2
Application.Contents("three") = 3
Application.Contents("four") = 4
for i = Application.Contents.Count to 1 step -1
  Application.Contents.Remove(i)
next

for i = 1 to Application.Contents.Count
  Response.Write Application.Contents.Key(i) & " == " & Application.Contents.Item(i) & "<br />"
next

By going backwards, the index is decremented and so is the external iterator, meaning that they keep in sync with each other.

To adapt this further, if you only want to remove certain items from the collection and want to delete based upon the key, use the following.

Dim i
Application.Contents.RemoveAll()
Application.Contents("one") = 1
Application.Contents("two") = 2
Application.Contents("three") = 3
Application.Contents("four") = 4
for i = Application.Contents.Count to 1 step -1
  if ((Application.Contents.Key(i) = "one") OR (Application.Contents.Key(i) = "three")) then
    Application.Contents.Remove(i)
  end if
next

for i = 1 to Application.Contents.Count
  Response.Write Application.Contents.Key(i) & " == " & Application.Contents.Item(i) & "<br />"
next

Using our example above, the output will correctly be:

two == b
four == d