Exploration of which request headers you need, should and cannot set when using MsXml2.XmlHttp, Microsoft.XmlHttp and MsXml2.ServerXmlHttp

System Requirements:

  • Windows NT 4.0 Server SP4+
  • Windows Server 2000
  • Windows Server 2003
  • Windows Server 2008, R2
  • Windows Server 2012, R2
  • Windows 2000 Professional
  • Windows XP
  • Windows Vista
  • Windows 7
  • Windows 8, 8.1
  • IIS 4.0, 5.0, 5.1, 6.0, 7.0, 7.5, 8.0
  • ASP 3.0 (Classic)
  • VBScript

The Problem:

I was browsing around the web earlier looking for inspiration on making PowerShell send asynchronous XmlHttp requests when I came across a block of VBScript examples. In them, I observed that the coder was religiously attempting to set the Content-Length request header, something that when using MsXml I don’t usually do – it’s a different story with PHP.

Equally, I observed that the programmer was using Len() to define the length of the content body for text, but ASP usually operates using an internal UTF-16 encoding to represent strings, thus does it follow that Len() will give the correct value?

So I thought that I would explore the issue to see for my own amusement, what was actually going on.

More Info

Let us take a standard code block to use for our tests and use the services of the good people at httpbin.org for a public message echo system.

Option Explicit
Dim x
Dim strRequest : strRequest = “a=1&b=2&c=3” ‘ <- 11 charactersset x = CreateObject(“MsXml2.XmlHttp”)
call x.open(“POST”, “http://httpbin.org/post”, false)
call x.setRequestHeader(“Content-Length”, 11)
call x.setRequestHeader(“Accept-Language”, “EN-GB”)
call x.setRequestHeader(“Connection”, “keep-alive”)

x.send(strRequest)

The above gives us a boiler plate for what we believe the correct header should looklike written in VBScript. When executed, HttpBin responds with:

{
“args”: {},
“data”: “a=1&b=2&c=3”,
“files”: {},
“form”: {},
“headers”: {
“Accept”: “*/*”,
“Accept-Encoding”: “gzip, deflate”,
“Accept-Language”: “EN-GB”,
“Cache-Control”: “no-cache”,
“Connection”: “close”,
“Content-Length”: “11”,
“Host”: “httpbin.org”,
“User-Agent”: “<omitted>”,
“X-Request-Id”: “<omitted>”
},
“json”: null,
“origin”: “<ommitted>”,
“url”: “http://httpbin.org/post”
}

HttpBin provides us with an echo of the header and WireShark provides us with a wire capture of the transport packet being sent between the XmlHttpResponse provider and the HttpBin server. With these two tools we can analyse the header states.

Content-Length

If you take out the line call x.setRequestHeader(“Content-Length”, 11) and do not specify a content-length, both HttpBin and Wireshark’s trace of the connection report Content-Length: 11\r\n, which is correct. So logically, this has been inserted by the XmlHttp parser during the request to .send().

So what happens if we attempt to override it. If we inject call x.setRequestHeader(“Content-Length”, 15), which is 4 characters too long, again both WireShark and HttpBin report

Content-Length: 11\r\n

The same occurs if we under-report the content length, i.e. 10 characters. Therefore it is safe to conclude that MsXml2.XmlHttp’s send() operator handles this for you and any processing of the Len() of the message body before sending is just wasting CPU time, memory and lines of code as it is ultimately an action that will be duplicated.

To further test the theory to see if it is required in down-level versions of MsXml, I repeated the experiment using HttpBin and Wireshark with the following COM objects:

Microsoft.XmlHttp
MsXml2.XmlHttp            <-which is logically v3.0
MsXml2.XmlHttp.3.0
MsXml2.XmlHttp.6.0
MsXml2.ServerXmlHttp      <-which is logically v3.0
MsXml2.ServerXmlHttp.3.0
MsXml2.ServerXmlHttp.6.0

None of them could be coerced to include an incorrectly sized Content-Length and all of them appended the correct 11 character content length on their own when no attempt was made to manually provide it.

We can therefore categorically conclude that with MSXML 3.0 and higher, you do not need to waste time calculating the content length.

A an aside, when dealing with character data, the use of Len() appears to be correct for UTF-8 encoding. LenB() should be used when passing in binary data i.e. ADODB.Stream data, FileSystemObject data etc.

Content-Type

If you do not manually specify a Content-Type header, all MSXML providers appear to transmit POST encoded data as

Content-Type: text/xml; charset=”utf-8″\r\n

As we were sending POST data (i.e. a HTML form), this is incorrect as the content-type should be “application/x-www-form-urlencoded; Charset=UTF-8”. Thus is is appropriate and necessary to include the Content-Type header manually as shown below.

call x.setRequestHeader(“Content-Type”, “application/x-www-form-urlencoded; Charset=UTF-8”)

Accept-Language

Accept language was decidedly different depending on which provider you are using. You can safely override Accept-language by manually specifying it using setRequestHeader(“Accept-Language”, “EN-GB”).

Microsoft.XmlHttp         <- NOT SET
MsXml2.XmlHttp.3.0        <- NOT SET
MsXml2.XmlHttp.6.0        <- Derived it from OS i.e. Accept-Language: en-gb\r\n
MsXml2.ServerXmlHttp.3.0  <- Derived it from OS i.e. Accept-Language: en-gb\r\n
MsXml2.ServerXmlHttp.6.0  <- Derived it from OS i.e. Accept-Language: en-gb\r\n

Thus, if you are using the legacy Microsoft.XmlHttp or the client MsXml2.XmlHttp.3.0, it is necessary to manually enter the Accept-Language if of course it is needed.

Connection

All examples set the connection value to Connection: Keep-Alive\r\n when communicating with the HTTP/1.1 protocol. If you need to force the remote server to close the TCP connection immediately after the response has been received, you must manually set Connection: Close\r\n using setRequestHeader(“Connection”, “Close”).

Cache-Control

Cache-control is only set by the client providers and can be added manually to the server providers.

Microsoft.XmlHttp         <- Cache-Control: no-cache\r\n
MsXml2.XmlHttp.3.0        <- Cache-Control: no-cache\r\n
MsXml2.XmlHttp.6.0        <- Cache-Control: no-cache\r\n
MsXml2.ServerXmlHttp.3.0  <- NOT SET
MsXml2.ServerXmlHttp.6.0  <- NOT SET

Accept-Encoding

Accept-encoding is only set by the client providers and can be added manually to the server providers.

Microsoft.XmlHttp         <- Accept-Encoding: gzip, deflate\r\n
MsXml2.XmlHttp.3.0        <- Accept-Encoding: gzip, deflate\r\n
MsXml2.XmlHttp.6.0        <- Accept-Encoding: gzip, deflate\r\n
MsXml2.ServerXmlHttp.3.0  <- NOT SET
MsXml2.ServerXmlHttp.6.0  <- NOT SET

Accept

Accept is set to Accept: */*\r\n by all providers and can be overridden if required.