Home > Live@EDU > Why PCNS Stopped Working

Why PCNS Stopped Working

I recently visited a customer site who has student email hosted by Microsoft’s Live@EDU program.

While on-site, they reported the ILM/OLSync/PCNS (Password Sync) had stopped working. Users were able to change their LiveID password with Microsoft, but the one-way password-reset sync from Active Directory no longer worked.

For those unfamiliar with any of those above phrases, this article is not for you. But you can read about them here:

A quick overview of PCNS requirements and configuration:

  1. PCNS schema extension is in place
  2. PCNS is installed on all domain controllers in the user’s domain
  3. Inclusion Group(s) Defined
  4. Password Synchronization is configured within the ILM software

For my customer, this was all configured. And PCNS worked when I left them a few months back. What changed?

I stopped by the ILM server to have a look at things. PCNS is not so great about reporting errors by default, so I wanted to enable more logging. To do this you need to create a registry key called FeaturePwdSyncLogLevel.

FeaturePwdSyncLogLevel isn’t documented much on TechNet as far as I’m aware, but it is explained on this public forum here:

http://social.technet.microsoft.com/forums/en-US/identitylifecyclemanager/thread/4686507a-2acc-4a4d-9e64-4f6f15d5e165

PCNS

For PCNS, four logging levels are controlled by adding the EventLogLevel (REG_DWORD) entry to the following registry subkey:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PCNSSVC\Parameters

· 0 = Minimal logging

· 1 = Normal logging (default)

· 2 = High logging

· 3 = Verbose logging

MIIS 2003

For MIIS 2003, four logging levels are controlled by adding the FeaturePwdSyncLogLevel (REG_DWORD) entry to the following registry subkey:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\miiserver\Logging

· 0 = Minimal logging

· 1 = Normal logging (default)

· 2 = High logging

· 3 = Verbose logging

So after configuring my ILM 2007 FP1 server (despite the above quote saying “MIIS 2003”) with this value (and restarting) I once again attempted a password reset.

Now I am able to see the following error in the application log:

image

Log Name: Application
Source: MIIServer
Event ID: 6905
Task Category: (8)
Level: Information
Keywords: Classic
Computer: gcilm.domain.com
Description:
A password notification was received but could not be processed because the corresponding object is a disconnector.
Additional information:
Reference ID: {6BC69D88-9392-4FEE-B050-940537F4063F}
Source Object GUID: {75562CF8-BF46-4CE2-94FA-89B3EABA60D8}
Source DN: CN=10187,OU=Y2013,OU=Students,OU=User Accounts & Groups,DC=domain,DC=com
Source MA Name: OnPremise

What we see in this error is that ILM cannot sync the password to the Live@EDU mailbox because ILM thinks the account is a “disconnector”.

Essentially this means ILM doesn’t have a matching object for this user.  This does not mean the mailboxes in Live@EDU are gone or damaged – just that ILM doesn’t have a relationship between them and an on-premise AD account anymore.

For more about disconnectors, see the article: Don’t call me disconnector !!!

So how do we fix this?

Objects become disconnected in OLSync when the rules defined by the OLSync code are violated.  So let’s look at the rules.  Our best public source for these rules is in this article, titled: How Outlook Live Directory Sync Works.  Within you’ll find a section called OLSync filtering logic:

When OLSync runs, ILM filters out objects in the following order. After an object is filtered out, ILM won’t evaluate it again, nor will the object be copied to the ILM metaverse for synchronization.

1. Recipient objects that don’t have required attributes ILM reads the following recipient objects. If any of the required attributes are empty (null), the recipient object is filtered out.

Recipient object type Required attributes
Mailbox-enabled user mail, legacyExchangeDN, proxyAddresses
Mail-enabled user mail, targetAddress
User (AD DS or Active Directory only; no Microsoft Exchange installed) mail
Mail-enabled contact mail, targetAddress
Distribution group, dynamic distribution group, or security group mail, proxyAddresses, mailNickName

2. Recipient objects where the adminCount attribute is set to 1 The adminCount attribute is used to identify users in protected administrator groups, such as the Domain Admins and Administrators. If the adminCount attribute is set to 1 on any recipient object, it is filtered out.

3. Mailbox-enabled user objects that are specified as mailbox plans, discovery mailboxes, or arbitration mailboxes The msExchRecipientTypeDetails attribute is used to identify mailboxes that are specified as mailbox plans, discovery mailboxes, or arbitration mailboxes. These mailbox-enabled users are filtered out.

4. The mail attribute on an AD DS or Active Directory-only user that doesn’t match the provisioning domain In an on-premises environment where Microsoft Exchange hasn’t been installed, OLSync filters out all user objects where the mail attribute doesn’t contain an SMTP address that matches the provisioning domain.

5. The attribute used to generate the Windows Live ID doesn’t match any of the accepted domains The final pass filters out recipient objects that are configured for auto-provisioning but don’t have an accepted domain match in the attribute that is used to generate the Windows Live ID.
The attribute used to generate the Windows Live ID must contain a domain name that matches one of the accepted domains that you have configured in Outlook Live. As described in step 4, by default, OLSync looks to the user principal name (UPN) for a match unless you have set the MVWindowsLiveIdAttributeName parameter to use a different attribute. In this case, OLSync matches the SMTP address that is stored in the attribute that you have specified in the MVWindowsLiveIdAttributeName parameter. In any case, if OLSync can’t find a match to an accepted domain, the recipient object is filtered out.

After ruling out many of the above conditions, I thought more on item 2.  I checked and realized the adminCount attribute on ALL of my users now is populated with a “1”!

This attribute is automatically populated by the PDC emulator for all users in “restricted” groups.  Sadly, if you remove users from these groups, the same process doesn’t bother to clear that attribute!  You have to do it manually.

More on this topic here: AdminSDHolder – or where did my permissions go?

When clearing this attribute make sure you set it to null. ADSIedit has a “clear” button to nullify an attribute:

image

Of course I have a few remaining questions:

  1. Will this “1” return?
  2. How did this occur?
  3. How do I script this fix?
    The answer to #1 is yes – if you don’t fix the root cause.  Every hour the protected groups are evaluated and 1’s re-stamped if appropriate.  So we need to fix the cause before we fix the symptom.
    How did this occur?
    After looking at the group memberships of each user I verified users were NOT in a protected group; or at least until I checked nested group memberships – Bingo!
    Someone had added the Domain Users group to Print Operators; a protected group!  Why was this done – who knows.  But I removed this membership.
    Now time for cleanup.

Using the free Quest PowerShell tools, I wrote the following script to identify the scope of the problem:

Get-QADUser –SizeLimit 0 -IncludeAllProperties -SearchRoot “domain.com/User Accounts & Groups/Students” | where {$_.admincount -eq “1”}

And followed up with this one to clear the attribute:

Get-QADUser –SizeLimit 0 -IncludeAllProperties -SearchRoot “domain.com/User Accounts & Groups/Students” | where {$_.admincount -eq “1”} | Set-QADUser -ObjectAttributes @{adminCount= $null}

Note, in the scripts I am only looking in a specific OU.  You’ll need to adjust this for your environment.  In fact, you can remove the SearchRoot switch all together if you want, as the adminCount will be re-populated for users in protected groups in the next hour anyway.

After running this script you can re-run the “startsync.ps1 –firstrun” to repair the relationships between the objects.

That’s it – bye for now!

Categories: Live@EDU Tags: , ,
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 62 other followers

%d bloggers like this: