Backup and Restore Instructions for the DirSync Database

April 17, 2014 Leave a comment

Today, Microsoft released a 9 page guide on backing up and restoring the Microsoft Azure Active Directory Sync tool. You can get it here.

Some things to keep in mind:

  • This guide applies to DirSync when used with the full version of SQL only.  This means it does not apply to most installations.
  • You don’t need to backup or restore DirSync.  If you simply install a new instance and configure it appropriately, the objects will re-sync.  Doing a backup/restore can save time however, if you have a very large number of users (I wouldn’t bother with less than 100k).
  • Ironically, this guide doesn’t actually tell you how to backup or restore the database.  You need some SQL-aware backup product to do that.  Instead, this guide helps you make use of a restored database in a DirSync environment (working with miisclient.exe, handling keys, etc).

DirSync 1.0.6593.0012

February 3, 2014 2 comments

Late Monday, Microsoft released another update to the DirSync software, this time with a build number of 6593.0012. You can download it in from the usual link.

DirSync 1.0.6593.0012

As with previous DirSync updates, there has been no official announcement of the release, however the “use at your own risk” Wiki does mention one of the new features:

Version 6593.0012
Date Released 2/3/2014
Notable Changes

New features:

  • Additional Attributes are synchronized on User and Contact objects

Attributes documented here

The new attributes referenced in the link are userCertificate and userSMIMECertificate. Interestingly pwdLastSet was also added, however there is no mention of that one in the article. These additions serve an unknown purpose for now, however one might speculate that they are in support of new capabilities soon to be available in the service?!

Before you upgrade, you may wish to get a “before and after” review of the attribute inclusion list. The best way to review this is in the “Configure Attribute Flow” area of each management agent. At the end of this post, I have also shared an experimental PowerShell method of getting this information.

It is noteworthy that the author of this update, a Microsoft Program Manager for DirSync, is linking to yet another community wiki page instead of the seemingly defunct Knowledge Base article KB-2256198. Sadly, it would appear that the crumbling integrity of the TechNet/Support documentation may be latest casualty in a growing list of IT Pro-related cuts Microsoft has made along their quest to the cloud…

<#
Description:
This script counts and dumps the attribute inclusion lists from each MA.
It does not evaluate attribute flow or applicable object types.

February 3 2014
Mike Crowley

http://mikecrowley.us

#>

#Import Modules
Import-Module SQLps -WarningAction SilentlyContinue

#Get SQL Info
$SQLServer = (gp 'HKLM:SYSTEM\CurrentControlSet\services\FIMSynchronizationService\Parameters').Server
if ($SQLServer.Length -eq '0') {$SQLServer = $env:computername}
$SQLInstance = (gp 'HKLM:SYSTEM\CurrentControlSet\services\FIMSynchronizationService\Parameters').SQLInstance
$MSOLInstance = ($SQLServer + "\" + $SQLInstance)

#Get Management Agent Attribute Info
[xml]$OnPremAttributes = (Invoke-Sqlcmd -MaxCharLength 10000 -ServerInstance $MSOLInstance -Query "SELECT attribute_inclusion_xml FROM [FIMSynchronizationService].[dbo].[mms_management_agent] WHERE [ma_name] = 'Active Directory Connector'").attribute_inclusion_xml
[xml]$CloudAttributes = (Invoke-Sqlcmd -MaxCharLength 10000 -ServerInstance $MSOLInstance -Query "SELECT attribute_inclusion_xml FROM [FIMSynchronizationService].[dbo].[mms_management_agent] WHERE [ma_name] = 'Windows Azure Active Directory Connector'").attribute_inclusion_xml
$ADAttributes = $OnPremAttributes.'attribute-inclusion'.attribute
$AzureAttributes = $CloudAttributes.'attribute-inclusion'.attribute

#Output to Screen
Write-Host $ADAttributes.count "Attributes synced from AD to the Metaverse" -F Cyan
Write-Host $AzureAttributes.count "Attributes synced from the Metaverse to Azure" -F Cyan
Write-Host "See" $env:TEMP\DirSyncAttributeList.txt "for detail" -F Cyan

#Output to File
"******AD Attributes******" | Out-File $env:TEMP\DirSyncAttributeList.txt
$ADAttributes | Out-File $env:TEMP\DirSyncAttributeList.txt -Append
" "| Out-File $env:TEMP\DirSyncAttributeList.txt -Append
"******Azure Attributes******" | Out-File $env:TEMP\DirSyncAttributeList.txt -Append
$AzureAttributes | Out-File $env:TEMP\DirSyncAttributeList.txt -Append

##END

DirSync 1.0.6567.0018 Has Been Released

November 22, 2013 9 comments

As some of us noticed, last week, Microsoft quietly removed the latest version of DirSync without so much as a tweet explaining why. Word on the street is that there were issues in the “Export” stage in the synchronization process (see KB 2906832). Today it would appear those issues have been resolved, as v1.0.6567.0018 just hit the web. You can download it here, though I’d advise caution, given Microsoft’s approach to communicating (lack-thereof) bugs.

As stated in the updated Wiki, the following improvements exist in this version:

New features:

  • DirSync can be installed on a Domain Controller (must log-off/log-on AFTER installation and BEFORE configuration wizard)
    • Documentation on how to deploy can be found here

Contains fixes for:

  • Sync Engine memory leak issue
  • Sync Engine export issue (FIM 2010 R2 hotfix 4.1.3493.0)
  • “Staging-Error” during large Confirming Imports from Windows Azure Active Directory
  • password sync behavior when sync’ing from Read-Only Domain Controllers (RODC)
  • DirSync setup behavior for domains with ‘@’ symbol in NetBois names
  • Fix for Hybrid Deployment Configuration-time error:
    • EventID=0
    • Description like “Enable-MsOnlineRichCoexistence failed. Error: Log entry string is too long.  A string written to the event log cannot exceed 32766 characters.”

Exchange Proxy Address Report Update

November 15, 2013 1 comment

I’ve made an update to the popular “Exchange Proxy Address (EmailAddresses) Report” script.  If you’re into in that sort of thing, check it out:

http://mikecrowley.wordpress.com/2012/04/16/exchange-proxy-address-alias-report

Sample output to screen

Sample output shown in Excel

Upgrading DirSync to the Latest Version

November 5, 2013 9 comments

EDIT (Nov. 22 2013): DirSync 1.0.6567.0018 Has Been Released

EDIT (Nov. 11 2013): DirSync 1.0.6553.2 has been removed from Microsoft’s download site and version history comment removed from the Wiki.  Not sure why.

Early this morning, Microsoft released an updated version of Windows Azure Active Directory Sync tool (DirSync to you and me). Version 1.0.6553.2 (or later) can be downloaded from the usual link. It comes with 4 known improvements:

  1. Fix to address Sync Engine memory leak
  2. Fix to address “staging-error” during full import from Azure Active Directory
  3. Fix to handle Read-Only Domain Controllers in Password Sync
  4. DirSync can be installed on a Domain Controller. Documentation on how to deploy can be found here.

I am most excited about #4, as this enables me to build more interesting labs from my laptop, now that I don’t need a dedicated “DirSync Server”. You should note however, this is recommended only for “development” environments. After some further testing, I’d consider recommending this configuration for shops with multiple domain controllers and 50 or fewer users.

If you’re already running DirSync, and want to upgrade, you’re likely in one of two camps:

  1. You want to move DirSync from a dedicated server to a DC.
  2. You don’t want to move the DirSync server to a DC (or elsewhere), you just want the latest version.

If you’re in the first scenario, I’m going to assume you’re working in a lab or very small environment. This means you don’t need to worry about a lengthy synchronization process, and can easily take advantage of the built-in soft-match capability of the product. Your upgrade process is easy:

  1. Throw away your existing DirSync server.
  2. Install Dirsync on a DC.
  3. Run the Directory Sync Configuration Wizard

As soon as you finish the 3rd step, the initial synchronization will rebuild the database (and re-sync passwords), returning to where you left off!

NOTE: If you’re a big shop, you should consider that a full sync takes roughly 1 hour per 5,000 objects synced, according to a recent webcast by Lucas Costa. Soft-matches would likely go faster, but you’ve been warned…

Now, if you’re just looking to upgrade your version of DirSync to the latest version, you need to first ensure you are running versoin 6385.0012 or later. In-place upgrades aren’t supported on earlier versions. If this is you, refer to the soft-match advice I gave above. This is your upgrade path.

For those that are running 6385.0012 or later, upgrading is as simple as a few clicks of the mouse. For the nervous, here are some screenshots:

NOTE: The installer detects an existing installation.
This is the default path, but it should reflect your installation directory.
Hmm, that’s not good! Fortunately a reboot cleared this up for me, but if you’re not so lucky, you can examine the following logs:
  • coexistenceSetup
  • dirsyncSetup
  • miissetup
  • MSOIDCRLSetup

…which are located in the earlier discussed installation directory.

msiexec returned 1618
Much better!
For an upgrade, you’ll want to run this right away, since not doing so leaves you without a functioning DirSync server.
Global Office 365 Administrator credentials go here. This is stored on your DirSync server, so make sure PasswordNeverExpires attribute is set to $true on the Office 365 account (or your on-premises account, if you’re using a federated user)
On-Premises Enterprise Admin credentials go here:
Checking this box allows some attributes to be written back to your Active Directory, which is necessary for a Hybrid Exchange Server scenario.
Enable Password Sync… or Don’t.
NOTE: Upgrades and new installs require a Full Sync.
This post wouldn’t be complete without a plug for my free DirSync Report script! DirSync Report

DirSync and Disabled Users: The BlockCredential Attribute [Part 2]

October 23, 2013 2 comments

In this two-part article, I have laid out a scenario in which DirSync sets the Azure “BlockCredential” attribute of disabled Active Directory users. In Part 1, I explained how the Windows Azure Active Directory Sync tool (DirSync) causes this to happen. Part 2 (below) discusses how to change this behavior.


Last time, we saw that magic a rules extension prevents a user from logging into Office 365 if their on-premises Active Directory account was disabled. Below, I’ll show you how to override this attribute flow, but first a note on Microsoft Support:

NOTE: Changing the behavior of DirSync means that you may wander into “unsupported” terrain, but in my experience, unless an unsupported change is likely the cause for a given problem, Microsoft’s support staff have been understanding and have yet to terminate a support case without cause. Having said this, you should not expect Microsoft to incorporate your changes into their upgrade path, so be sure to document, backup, and plan upgrades accordingly.

As you’ll recall, the existing attribute flow is:

userAccountControl à Rules Extension à accountEnabled à Metaverse
Metaverse à accountEnabled à BlockCredential

We will adjust it to the following:

userAccountControl à Rules Extension à accountEnabled à Metaverse à <Nowhere>

In essence, we are allowing the rules extension to update the Metaverse, but not allowing the Azure MA to flow to the BlockedCredential attribute.  This ensures changes in the on-premises Active Directory (such as disabling accounts) will not prevent login to Office 365 (be sure this is actually what you want before you proceed).  Fortunately it also does not necessarily prevent an administrator from setting BlockedCredential manually on Office 365 users.

With our game plan, let’s begin by firing up the trusty miisclient.exe; usually located here:

C:\Program Files\Windows Azure Active Directory Sync\SYNCBUS\Synchronization Service\UIShell\miisclient.exe 

1) Click Management Agents.
2) Open the “Windows Azure Active Directory Connector” MA.
3) Click “Configure Attribute Flow” and expand “Object Type: User”.
4) Select the accountEnabled attribute.
5) Click “Delete”.6) Click “OK” until you are back on the main screen.
 

We’re almost done!  Two tasks remain:

  1. Test our change by:
    • Creating a new AD user, ensure they sync to Office 365 and that they can log in
    • Disable the user’s AD account, run another sync and ensure they can still log in.
  2. Determine how to update users that were disabled before our change.  If you simply want to re/enable all currently disabled accounts, the below PowerShell sample might work well:
Connect-MsolService
$BlockedUsers = Get-MsolUser -EnabledFilter DisabledOnly -All
$i= 1
$BlockedUsers | ForEach-Object {
 Write-Host ($_.UserPrincipalName + " (" + $i + " of " + $BlockedUsers.count + ")" )
 Set-MsolUser -UserPrincipalName $_.UserPrincipalName -BlockCredential $false
 $i = $i + 1
 }

Thanks to William Yang for his advice on this post.

Categories: Office365 Tags: , ,

DirSync and Disabled Users: The BlockCredential Attribute [Part 1]

October 23, 2013 7 comments

In this two-part article, I will describe a scenario in which DirSync sets the Azure “BlockCredential” attribute of disabled Active Directory users. In Part 1 (below) I explain how the Windows Azure Active Directory Sync tool (DirSync) causes this to happen. Part 2 discusses how to change this behavior.

As I’ve been discussing, DirSync can be more complicated than it appears. Even if you are familiar with the miisclient.exe console, some of FIM’s logic is hidden in “Rules Extension” DLL files such as “MSONLINE.RulesExt.dll“. These files can be reverse-engineered to some degree, however it can be very difficult.

That’s why it’s good to know you can avoid them all together if necessary! For example, imagine that I don’t want DirSync to prevent my disabled users from logging into Office 365. Perhaps you need to limit access to on-premises resources for a group of people, while still allowing everyone access to Office 365.

If this restricted group is only a handful of users, and you don’t need password synchronization, you might be best off creating them manually within the Office 365 portal. However if automation and password sync are important, this scenario presents a few credentialing challenges:

  • Because ADFS authenticates against local domain controllers, the accounts Must be enabled.
  • DirSync will sync passwords for disabled users, but as mentioned above, it also disables them in Office 365 (by setting their BlockCredential attribute).

The first bullet point is simply how ADFS works, therefore ADFS is out. This 2nd option, however, can actually be explored. WHY does DirSync do this? As far as I can tell, Microsoft hasn’t documented this part of the attribute flow, so let’s take a look ourselves.

Launch miisclient.exe and select the Management Agents tab. Double-click the “Active Directory Connector” MA and select “Configure Attribute Flow”, then expand to this section:

What we can see here is that FIM is reading the Active Directory attribute “userAccountControl” (where the disabled state is recorded) and updating the “Metaverse” attribute “accountEnabled” based on logic within the rules extension. For the sake of argument, why don’t we call this rules extension “magic”, because I have no idea what’s inside it – but let’s keep going.

Now let’s look at the “Windows Azure Active Directory Connector” MA in the same spot:

Well, that’s pretty simple. It’s taking the accountEnabled attribute OUT of the Metaverse and sending it to Azure. The type “Direct” means no magic. After some testing, I have determined that this attribute directly toggles the BlockCredential attribute I mentioned earlier.

userAccountControl à Magic à accountEnabled à Metaverse

Metaverse à accountEnabled  à BlockCredential

(AD / Azure)

Clear as mud, right? J

Here’s an example to be sure:

1) A user has just been disabled.
2) Later, DirSync runs, updating the “userAccountControl” value in the AD MA.3) The magic within the rule extension reads this and decides the “accountEnabled” Metaverse attribute needs to be updated to “false” which is then exported to Azure.
4) More magic within Azure, decides the user’s BlockCredential attribute needs to be updated. You can view this in the Office 365 Admin Portal or within PowerShell.
5) The user can no longer log into Office 365.

Note: This behavior is described in KB 2742372 

It looks like your account has been blocked

As you can see, this won’t work in our scenario. Fortunately, FIM is very flexible and we can change this behavior!

Continue on to Part 2 if you’d like to see how.

Categories: Office365 Tags: , ,
Follow

Get every new post delivered to your Inbox.

Join 51 other followers