VMWare Enhanced Authentication Plugin Fix

Plugin Installation

Web browser SSO to vCenter used to be provided via a browser API called NPAPI, but this API has now been deprecated by all browsers. As such VMWare was required to adopt an alternative approach to providing an SSO capability. This is now provided by installing the VMWare Enhanced Authentication Plugin. If this plugin is not installed the vCenter logon page prompts to download and install it:

The plugin package (currently VMware-EnhancedAuthenticationPlugin-6.5.0.exe) actually comprises two components:

  • Client Integration Plugin
  • Enhanced Authentication Plugin

The installation process installs a Windows service called “VMware Cip Message Proxy Service” and adds the following entries to the local hosts file:

127.0.0.1 vmware-plugin
::1 vmware-plugin

Once installed, the “VMware Cip Message Proxy Service” service presents a WebSocket interface on the local system. This interface brokers the authentication to the vCenter server using the user’s browser Windows session credentials. This is instigated by background calls to the WebSocket service from the vCenter logon page. The following endpoints are called from that page:

  • wss://vmware-plugin:8094 (alias for localhost address)
  • vmware-plugin://csd

If all is working then the “Use Windows session authentication” checkbox will be enabled:

Unfortunately in my case all was not well and this option was disabled:

Problem and Resolution

Now there can be a multitude of reasons for this (certificate issues being the most common), but this is what I observed and how I fixed the issue.

The first thing I noticed was that, when I attempted to load the logon page, the “VMware Cip Message Proxy Service” service would stop. Further to that, an error was logged in the Application event log stating that there was a fault with the “vmware-cip-msg-proxy.exe” application. A corresponding event was logged in the System event log – “The VMware Cip Message Proxy Service service terminated unexpectedly”.

Further googling, procmon tracing and log file analysis suggested that the
“VMware Cip Message Proxy Service” appeared to being denied access to the “%localappdata%\VMware\CIP” folder in my profile path (I believe it is trying to create a session lock or log file, but don’t know for sure).

Now the default plugin installation configures the “VMware Cip Message Proxy Service” service to run as the Local Service account:

However this account does not have permission to write to my (or any other) profile folder path. So I reconfigured the service to run as Local System instead:

Having restarted the service and refreshed the logon page, the “Use Windows session authentication” checkbox was finally enabled and I was able to log in with my current Windows session.

I don’t know why this change is necessary, how it’s supposed to be configured, or whether there is a better workaround, but it’s a solution that I’m happy to work with and thought it might be useful to share my findings.

5 thoughts on “VMWare Enhanced Authentication Plugin Fix

  1. Pete Long

    This is a great post! – Wish it was one of mine. Typically the ones you did the most work on, and are most proud of are the most obscure ones! ┬áNice on Nick.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *