04-20-2010 09:38 PM - edited 04-20-2010 09:39 PM
This probably sounds like a strange request, though, I have an implementation where we are converting an ACT! for Lotus Notes system to ACT! 2010 Premium ST (SP1 HF1). We've successfully exported out all of the data from Lotus Notes (along with the attachments), and are using the Import IT utility to bring the data into the new ACT! 2010 implementation. Everything maps beautifully, and once in the system, works fantastic, though, the problem is, getting it in quickly enough to satisfy the customer (and my sleep schedule
There were over 100,000 contacts, and it averages 5,000 imported every 2 hour(s), and I have to do the whole process twice (once to create the contacts and linked company, and then again to merge into the company the contact info from the contact, else I just have company records with names only).
This is a virtualized server provided to us by the customer that we are using as the ACT! server, that currently has the following properties in Control Panel > System.
Windows Server 2008 Standard, SP2, 64bit
Processor: Intel Xeon E5450 @ 3Ghz (4 processors)
The makers of the Import IT software have mentioned it is every RAM intensive, hence why we have so much memory thrown at this server. Though, with that said, in the Task Manager during one of our 5,000 chunks for testing, the ActSage.exe process (which Import IT runs through) never goes above 300MB (and usually hovers in the 200MB range) with 20% average CPU utilization, and SQL Server 2008 Standard process stays consistently around 4GB of RAM in Task Manager and averages 10% CPU utilization.
I've configured ActDiag to set the minimum SQL Server memory to 10GB hoping to force it, though, it still sticks around 4GB.
My initial thought was, if I could force these values to go higher (memory and CPU utilization process for ActSage.exe), I could process these imports much quicker. Now, it may just be that internally, the processes have simply hit the limit as to how quickly they can run, but I thought I would pose the question to see if anyone had any suggestions.
04-20-2010 10:27 PM
It's limited by SQL Express, using SQL Standard can use more.
You might also speak to the guys doing the itImport about using itService... by doing the import through the service, it's outside ACT! and is let limited by the ACT! UI memory usage
04-21-2010 03:22 AM
What's strange is that it is SQL Standard.
I worked with Import IT support and they also suggested using IT Service, and doing so did offer me the level of automation I was wanting (so I'm not starting the more approachable batches manually) and it increased the import speed roughly 300% I would say in my limited testing.
Now, if I could only figure out why I can't do a "Replace Field" for the "Record Manager" on "Company" records (I can on "Contacts", though, I see this was only added in Hot Fix 3), but I'll make another post for that
Thanks for taking the time to offer up your thoughts on my issues.
04-21-2010 04:00 AM
Yup, itImport with itService is the fastest way I've seen to do large data imports....
I have a tool that can do updates to the Record Manager field in Companies... it can also update the Limited Access settings (users/teams) for the Company as well as doing the same for all the Company Divisions AND all the contacts links to the Company/Divisions
But be a bit much if you just want to do Edit | Replace on the Companies, but if looking at a the full features, might be worth it