VDT and lsdoctor: self-service diagnostic tools

Two self-service diagnostic tools for a vCenter appliance, both plain Broadcom KB attachments. Between the two, they’re where most real answers come from once something is actually broken – reach for VDT first for a broad health sweep, then lsdoctor if VDT’s Lookup Service/AD check (or any other SSO symptom) points deeper into the Lookup Service / vmdir layer. Neither is tied to any one migration or upgrade step – both apply whenever a vCenter/PSC exists, regardless of track (VCF, VVF, or standalone vSphere).

VCF Diagnostic Tool for vSphere (VDT)

A general vCenter appliance health-check script – the closest self-service equivalent to an SDDC Manager fleet precheck when there’s no SDDC Manager to run one from (e.g. the standalone VVF / manual GUI upgrade path). Per Broadcom KB 344917, “Using the VCF Diagnostic Tool for vSphere (VDT)”:

1. Download the VDT version matching the source vCenter build from the KB attachments, copy it to the appliance (WinSCP or equivalent) into /root, then extract it:

   cd /root/
   unzip vdt-<version_number>.zip
   cd vdt-<version_number>

2. Run it:

   python vdt.py

   Prompts for the administrator@<sso-domain> password – “Many checks will still run even if credentials are not supplied,” but supply it for full coverage.
3. Review the PASS/FAIL/WARN results. Checks that matter most for an upgrade specifically: DNS, NTP, disk space, certificates, Lookup Service / AD integration, vCenter services, and VCHA – all things a prerequisites checklist assumes are healthy; VDT is how to actually confirm that instead of assuming it.

A FAIL here is cheaper to fix now than mid-migration – run it per vCenter before the window opens, not after something’s already gone wrong. If the FAIL is in the Lookup Service / AD integration check specifically, that’s the signal to move on to lsdoctor below rather than treating it as a standalone DNS/AD problem.

lsdoctor: Lookup Service / SSO / vmdir troubleshooting

Lookup Service Doctor (lsdoctor) is Broadcom’s general-purpose tool for data stored in the PSC database, plus data local to a vCenter – SSL trust mismatches between services, broken/stale service registrations, and inconsistent solution users. Reach for it whenever something in the SSO/Lookup Service/vmdir layer looks wrong and doesn’t cleanly match a more specific, already-diagnosed symptom – see Field notes: Identity Broker / VCF SSO for symptoms this tool has already resolved (Identity Broker encryption key desync, NTP-adjacent SSO token failures, ELM drift after breaking Enhanced Linked Mode).

For a worked example of the read-only check run at a specific point in a migration’s flow, see IWA to AD-over-LDAPS migration → Validate SSO/Lookup Service health with lsdoctor. This doc is the general reference; that one shows it in context.

Supported versions and where to run it

• vCenter Server 6.7 (Windows-based or VCSA) and later – covers 7.x, 8.x, and 9.x. Per Broadcom KB 320837 (source below), support for the very latest vCenter build can lag slightly behind release, so confirm compatibility with the target build before relying on it right after a new release.
• Needs shell/SSH access to the target node.
• Works across embedded PSC, external PSC, and mixed Enhanced Linked Mode deployments spanning multiple SSO sites – most operations can run from any node within the same SSO site, not necessarily the Primary.

Download and setup

1. Download the tool attachment from Broadcom KB 320837, “Using the ‘lsdoctor’ Tool”.
2. Copy it to the target vCenter (WinSCP or equivalent), SSH in, and unzip it.
3. Run it from inside the extracted lsdoctor-main directory – it must be run from there, not a copy of individual files elsewhere.

Always run the read-only check first

python lsdoctor.py -l

(-l / --lscheck) only reports – it makes no changes, so it’s safe to run without a fresh snapshot beyond whatever backup discipline the surrounding procedure already requires. Resolve anything it flags before assuming a separate, unrelated cause for whatever symptom brought you here – an unresolved Lookup Service inconsistency tends to surface later as a confusing, hard-to-place permission or authentication failure rather than a clean error at the point it was actually introduced.

Field-observed symptom: “Node In Multiple Sites”

The read-only check can report the same vCenter node registered under more than one SSO site, e.g.:

SSO CHECKS
    VC Lookup Service Check
        • SSO Site: default-first-site
            • [FAIL]    vcenter.example.com (VC Server or CGW)
                [FAIL]    Node In Multiple Sites
                            Please run python lsdoctor.py -r option 2 on this node
                            Affected Nodes: {'default-first-site': 'vcenter.example.com', 'domain': 'vcenter.example.com'}
                            Documentation: https://knowledge.broadcom.com/external/article?legacyId=80469
        • SSO Site: site-b
            • [FAIL]    vcenter.example.com (UNKNOWN)
                [FAIL]    Node In Multiple Sites
                            Please run python lsdoctor.py -r option 2 on this node
                            Affected Nodes: {'site-b': 'vcenter.example.com', 'default-first-site': 'vcenter.example.com'}

The tool’s own output names the fix directly: run python lsdoctor.py -r and choose option 2 from the rebuild menu for the affected node – per KB legacyId=80469 that the tool’s output itself links to. Take the same-instant, whole-SSO-domain snapshot from the warning below before running any -r option, same as any other repair mode.

Field-observed symptom: orphaned service registrations

A [WARNING]-level “3rd party/Orphaned service registrations” line for a service the tool can’t associate by hostID/nodeID/serviceID (third-party integrations like a storage vendor’s vSphere plugin are a common source) is informational, not necessarily something to fix – confirm the service is genuinely orphaned (the integration was removed) before touching it, rather than treating every warning as an action item.

Decommissioned Site Recovery Manager / VMware Live Site Recovery is a common, confirmable source of this same warning: an old SRM/VLSR appliance that was decommissioned improperly, or that became unreachable before it could unregister itself cleanly (e.g. deleted or powered off before running its own uninstall/unregister flow). Per Broadcom KB 337576, “Cleaning up decommissioned SRM registrations”, symptoms besides the lsdoctor warning include SSL errors unregistering via the appliance’s own VAMI, a "Failed to connect to Site Recovery Manager Server at https://<IP>:9086/vcdr/vmomi/sdk" connection-refused error in the vSphere Client, and stale entries under vCenter → Administration → Client Plug-Ins. Applies to SRM 8.x, VLSR 9.0.2.4, and vCenter 7.x/8.x. Relevant if disaster recovery is in scope for the upgrade – see Disaster Recovery for the SRM/VLSR convergence workstream this can surface during.

Root cause, quoted verbatim: “Stale registrations and solution users remain orphaned in the vCenter VMDIR database when an appliance is decommissioned improperly or is no longer reachable.” Three-step cleanup, in order:

1. Unregister the extension via the vCenter MOB. Navigate to https://<vCenter_FQDN>/mob → Content → ExtensionManager → More (to see the full list) → find the extension prefixed com.vmware.vcDr, open it, and check the Server field’s IP address to confirm it’s the stale instance before touching anything. Back in ExtensionManager, click UnregisterExtension, paste that extension’s ID, Invoke Method.
2. Remove the service registration via CLI. SSH to the vCenter, then:

   cd /usr/lib/vmware-lookupsvc/tools/
   ./lstool.py list --url http://localhost:7090/lookupservice/sdk --no-check-cert --ep-type com.vmware.dr.vcDr

   Identify the Service ID matching the stale IP, then:

   ./lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id <Service_ID> --user 'administrator@vsphere.local' --password '<password>' --no-check-cert

3. Remove the solution user via VMDIR/LDAP, if step 2 didn’t fully clear it. Connect an LDAP browser (e.g. JXplorer) to the vCenter – Host <vCenter_IP>, Port 389, Base DN dc=vsphere,dc=local – navigate to vsphere → Configuration → Sites → [Site_Name] → LookupService → ServiceRegistrations, and delete the identified Service IDs.

Verify the IP/Service ID at every step before deleting anything – this removes registrations directly from the SSO domain’s database, with no undo beyond whatever snapshot/backup discipline the surrounding procedure already requires.

Field-observed symptom: Machine ID mismatch (VMAFD vs. Likewise registry)

The read-only check’s VC Machine ID Check category can report two related failures:

VC Machine ID Check
    [PASS]    Machine ID Check
    [FAIL]    Compare Machine ID (VMAFD vs. registry)
                Machine ID doesn't match between the VMAFD service and the likewise registry!  Follow the KB to correct.
                Documentation: https://knowledge.broadcom.com/external/article/312479
    [FAIL]    Compare Machine ID (vpxd.cfg vs. registry)
                Machine ID doesn't match between vpxd.cfg and the likewise registry!  Follow the KB to correct.
                Documentation: https://knowledge.broadcom.com/external/article/312479
    [PASS]    vpxd.cfg SSO Domain Check

This is not just an lsdoctor curiosity – per Broadcom KB 312479, the same underlying desync can block a 7.0.x → 8.0.x vCenter upgrade outright, failing Stage 2 pre-checks with Exception occurred in postInstallHook, and it’s exactly what VDT’s own Machine ID Check validates (see VDT above) – so this can surface from either tool. Root cause, quoted verbatim: “The MachineGuid value stored within the local Likewise registry (vmdir service) is missing or desynchronized from the authoritative Machine ID managed by VMAFD and referenced in vpxd.cfg.”

Take an offline (powered-off) snapshot first – of every replication partner if the vCenter is part of Enhanced Linked Mode, not just the one node being fixed. Then, per KB 312479:

1. Get the authoritative Machine ID from VMAFD:

   /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost

2. Check whether MachineGuid exists in the registry:

   /opt/likewise/bin/lwregshell ls "[HKEY_THIS_MACHINE\Services\vmdir]"

3. If missing, add it with a temporary placeholder value:

   /opt/likewise/bin/lwregshell add_value '[HKEY_THIS_MACHINE\Services\vmdir]' MachineGuid REG_SZ 1

4. Set it to the real value from step 1:

   /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\vmdir]' "MachineGuid" <Correct_UUID>

5. Restart services:

   service-control --stop --all && service-control --start --all

6. Re-run the read-only lsdoctor check or VDT to confirm both FAILs clear before proceeding.

Repair modes – all more invasive than the read-only check

None of these reboot the appliance VM, but every one of them requires a service-control restart afterward – a vCenter management-plane outage for however long that restart takes (running VMs on ESXi are unaffected; this is vCenter/PSC service downtime, not workload downtime), and the restart scope differs by flag – some are single-node, some are domain-wide:

The restart itself is the standard appliance-wide cycle:

service-control --stop --all && service-control --start --all

Plan for this like any other vCenter/PSC service restart – schedule it in a maintenance window, expect the vSphere Client and API to be unreachable for the restart’s duration, and don’t run a repair mode against a production SSO domain expecting zero interruption. -t and -p are the ones most likely to surprise – they demand a site-wide/LB-wide restart, not just the node you ran the command on.

Before using any repair mode, snapshot the whole SSO domain at the same instant. Quoted verbatim from KB 320837: “Before using lsdoctor to make any changes, ensure you have taken proper snapshots of your SSO domain. This means that you must shut down all VCs or PSCs that are in the SSO domain at the same time, then snapshot them, and power them on again. If you need to revert to one of these snapshots, shut all the nodes down, and revert all nodes to the snapshot. Failure to perform these steps will lead to replication problems across the PSC databases.”

That’s a same-instant, whole-domain snapshot – not one node at a time, and not a live/running-state snapshot. Reverting works the same way: shut every node down first, revert every node, then power back on.

Output includes JSON report files (exact paths are printed by the tool at run time); the -r/--rebuild interactive menu’s four sub-options should be chosen based on what the read-only check surfaced, not guessed at.

Sources

• Using the VCF Diagnostic Tool for vSphere (VDT) (KB 344917)
• Using the “lsdoctor” Tool (KB 320837)
• Cleaning up decommissioned SRM registrations (KB 337576)
• Machine ID mismatch between VMAFD and the Likewise registry (KB 312479)