Migrating Workflow Items

​When performing SharePoint migrations, you will inevitably come across lists with associated workflows needing migration.  These can be challenging to migrate – especially if you just looking to migrate the data, and not the associated workflow(s).  Metalogix has a nice write up of how to use their tool to copy workflows (http://www.metalogix.com/help/Migration-Manager-for-SharePoint/SharePoint-Edition/002_HowTo/003_MigrationActions/017_CopyWorkflows.htm), but this article doesn’t help if all you want is the data from a list.  The Migration Manager will not allow you to copy items from a list which are associated with a workflow, unless the workflow and associated content types are migrated as well.I recently ran into this situation during a migration.  We had a list of tasks that were being created by a workflow in our legacy system.  The target system was setup as a clean environment, and while it had the workflow in place, they weren’t setup identically as the source.  The Migration Manager software refused to migrate the tasks unless we migrated the workflow as well.  Our simplest solution was to break the workflow association on the source server, then use the Migration Manager to migrate the list items to the target server.Here’s how we did it: 

• We created a .NET console application (our “MigrationHelper”) to read the source list, grab each field we cared about (including all metadata) and write the data to a temporary list on the same server.  This temporary list could be created/re-created over and over with no ill effects.  It contained all of the data that we needed, but not the associated workflow.

• We used the Migration Manager to migrate the temporary items to the destination list.  Since the list definitions were different, we simply mapped fields from the source to the destination using the built-in mapping functionality of the Migration Manager tool.

• We created an incremental job to migrate changed data in the temporary source list to the destination list.  We had a large number of items in the list and wanted our “go-live” night to require the least amount of data transfer possible.  (I’ll write a future blog entry on how and why you should do this)

The end result was a process that required a small amount of code on our part (reading the source list, deleting/creating the temporary list, copying data to the temporary list) and left the bulk of the migration to the Migration Manager.  Focusing on the limitations of the tool allowed us to work quickly and efficiently.  Yes, we had to write some custom code, but we didn’t have to reinvent the wheel and write code that Metalogix had already written (see my previous blog entry on using the right tool for the job:http://www.timlinenterprises.com/consultingknowhow/Lists/Posts/Post.aspx?ID=5).Have you ever run into a similar problem?  What did you do to solve it?

The Right Tool For The Job

When planning a data migration effort in SharePoint, you will undoubtedly be confronted by the classic build vs. buy decision.  Ultimately, your decision will be based on the changes to the data as it is moved from system to system.  For simple migrations where you are just switching from one provider to another (running the exact same software on both the old and new servers), a simple DB backup of the SharePoint Configuration and Content DBs should suffice.  However, when some type of data transformation is necessary, the decision becomes less clear.

The last project I worked on seemed straightforward enough - a migration from SharePoint 2007 at hosting provider A to SharePoint 2007 at hosting provider B.  However, a few requirements surfaced that made this trickier than a simple DB backup and restore.

The main complexity surfaced when the schema at the new facility was different from the schema at the source facility.  Why?  Because the custom application was modified and many of the fields that used to be used were no longer necessary.  My first inclination was to copy the entire  Content DB, and simply pare down the data to the desired schema.  However, seeing how the new environment was being setup as a completely "clean" environment, this was determined to be an unsatisfactory solution.

The next simplest mechanism for moving the desired data was to export the required lists to Excel spreadsheets, remove the unnecessary columns, and import them at the destination site.  Easy, right?  Not so fast - unfortunately, SharePoint will set the modification and creation information based on the import date, not the dates in the source data.  Since we were dealing with very sensitive data with strict auditing requirements, this was deemed unsatisfactory.

This left us with fewer options.  We could have just written code to copy directly from the source to the destination - but that's never as easy as it first appears.  Many unforeseen problems often arise that can take up days of programming time.  Ultimately, we made the decision to purchase an off the shelf tool to assist in the data migration - "The Metalogix Migration Manager."

Using this tool greatly simplified our data migration - however, it was not without its share of limitations.  Like any tool, it requires enough understanding of the platform in order to work well.  For example, we still needed to understand how the Content Types were mapped; how Workflows were being used and what data was associated with those workflows; how users were assigned to tasks and how to deal with mapping non-existent users in the new environment.

In the end, we ended up writing a fair amount of code to make the migration run smoothly.  One of my colleagues felt that we should have just written the entire data migration by hand.  I disagree.  I feel the investment in the tool was well worth the money and saved us from dealing with the bulk of the data transfer. Instead, it allowed us to focus on some of the more tricky/more interesting aspects of the move.

Using our custom code and the off the shelf tool in conjunction ultimately led to a successful migration.  Could we have done it all on our own and not used the tool?  Sure, but at a significantly higher cost.  Plus, the client ended up with a license for a tool that could be used for future migrations.

Bottom line - if you are doing any type of data migration in SharePoint, make sure you have someone that understands the platform and consider getting that person the tools they need to be successful.  Trying to save a bit on the tool cost will almost certainly cost you more in the long run.

What do you think?  Have you had a trying experience with a data migration?

Migrating Your Systems - Are You Prepared?

Lately, I've been leading a number of data migration efforts centered around SharePoint 2007/2010 for a few companies with very sensitive data. These companies decided not to upgrade to new versions of software, but decided to change hosting providers. In so doing, they believed that their system migraiton would be a simple process. Not so fast - it's always more complicated than it appears on the surface.


Here's a list of questions to ask yourself when preparing for a system migration: 

• Are we bringing over all of our data, or just a subset?
• What kind of downtime can we accept? 
• Disaster recovery? 
• Which systems get moved first?  Next? 
• Who has access to the old systems? Are those people still around? Will they be part of the migration effort? Do they want to help us? 
• Who will have access to the new systems?
• How are we planning on moving the data? 
• Do we understand our systems and know which systems access which data? 
• Do we know what exact versions of software our current systems are running on? 
• Will our new hosting provider be able to match the versions or do we need to upgrade? 
• How do we ensure the integrity of the data? 
• How much data do we have - how many databases, How many Gigs? How many records? 
• How is the data physically going to be moved? 
• How can we protect the data?  Do we need to audit the entire process?
• Have we inventoried everything at our current provider? 
• Who will be managing the new systems? 
• How much time do we have for the move? 
• What kind of testing will be done to ensure the new systems are working properly? 
• Do we need to hire consultants to help with the move (hint: yes!)

If I could offer just one bit of advice to anyone planning a migration effort, it would be - do not underestimate the amount of effort required.   Hiring an expert who has experience leading migration efforts is well worth the investment.  The cost of hiring a consultant is more than offset by the cost of not having an expert on hand and ending up with major unforeseen problems.  


What do you think? 


Have you recently been part of a system migration? Was it successful?

Dual Booting an Asus Eee PC 1000h

I got my new Asus Eee PC 1000h and have to say - I love it!
It comes pre-installed with Windows XP on a 40 GB partition and has a free 34.5 GB partition just begging for OS X. Here's how to do it:


WARNING: I'm not responsible for any damage, loss of data, etc. that this could do to your new machine. Use these instructions AT YOUR OWN RISK. They worked for me, but I can't guarantee they will work for you. OK - you've been warned. Still want to continue? Good...

You should read & understand:
http://www.maceee.com/how-to-mac-os-x-1054-for-the-eee-pc-901-1000h-27.html
before continuing here. Make sure you have everything you need.

1. Boot into XP

2. Verify that your Eee PC is setup like mine - 3 partitions:
40 GB
34.5 GB
32 MB
The 34.5 GB partition should be the D: drive and should be completely empty. That's the partition we're going to use for OS X.

3. Next, we want to delete the 2nd partition and re-create it as an "af" partition and make it "active." Here's the deal - once you change the active partition to the 2nd partition, your machine will no longer boot.

Ready? Here's what you do

Make sure you're on the desktop of your Windows installation.
Press WIN+R to open the Run prompt, then type DISKPART and press Enter.
Now we're going to use DISKPART to create a HFS+ compatible Mac partition that we can boot off.

Follow these next set of commands:

DISKPART> select disk 0

DISKPART> select partition 2

DISKPART> delete partition

DISKPART> create partition primary id=af

DISKPART> select partition 2

DISKPART> active

DISKPART> Exit

4. Follow the excellent instructions here:
http://www.maceee.com/how-to-mac-os-x-1054-for-the-eee-pc-901-1000h-27.html
(I DID NOT DO STEP 6)
Be sure to "Erase" this newly created partition and format for Mac OS X.

5. So, you followed the instructions and can now boot into OS X, but NOT into XP. That's OK - we're going to fix that now. You just need to modify one file:
/Library/Preferences/SystemConfiguration/com.apple.Boot.plist. Here's my complete file (the bold faced lines are the lines I added):

<plist version="1.0">
<dict>
<key>Kernel</key>
<string>mach_kernel</string>
<key>Kernel Flags</key>
<string></string>

<key>Boot Graphics</key>
<string>Yes</string>
<key>Quiet Boot</key>
<string>No</string>

<key>Timeout</key>
<string>5</string>
<key>device-properties</key>
<string></string>
</dict>
</plist>



You can edit this file directly from a terminal window using:
sudo nano /Library/Preferences/SystemConfiguration/com.apple.Boot.plist

that should keep the file permissions and ownership intact and just allow you to add the 4 additional lines.

That's it!
The system will boot OS X by default, but if you press F8 when prompted, Darwin will show you a menu and you can select either Windows XP or OS X.

I'm writing these instructions using my eee pc now (OS X of course) - however, i'm doing this from memory, so, if I missed anything or there are any errors PLEASE let me know and I will fix it. Hopefully, everything will go smoothly for you, but just remember, it's only a computer and you can always fix it ;)

Andrew's New "Do"

Well, we gave in to Andrew's incessant begging... he's all set for his trip to Florida:

In the chair











The buzzer is out












Uh oh...












Yep... a Mohawk! Oy...

Almost there...

It's hard to believe it's been almost a year since I got into the accident. I had my 2nd to last appointment with the dentist - molds were made for the final crowns! Here's what it looks like underneath:



Notice the implant on the right side of the picture (left side of my mouth). Maybe I should just keep this look?

Catch-22

File this one under the most ridiculous insurance policy to date. I got a call from my insurance broker yesterday to discuss my dental claims. Turns out, the dentist used "dental codes" to file the claims with Blue Cross. Silly people - don't they know that you need to use "medical codes" in order for the medical insurance to pay?

The broker asked me to have the dentist resubmit all the claims with the medical codes, and then he would be able to work on getting Blue Cross to pay the claim. Easy enough, right? Here's the catch: my dentist doesn't have the medical codes. Sounds simple enough, right? I asked the broker for the medical codes. Here's about how the conversation went:

me: My dentist doesn't have medical codes.

broker: Can they ask around and get them?

me: Um, can't you just give them to the dentist?

broker: Oh, I don't have the codes and Blue Cross won't give them to me.

me: Huh? Why's that?

broker: If I had the codes, then I would give them out and dentists would file under medical policies. Then Blue Cross would have to pay.


Ah, yes. Another one of the insurance companies methods for avoiding payouts. "Yeah, it's covered if you file the claim properly, but we're not going to tell you or anyone how to do that, sohaveanicedaybye!"