100 GB Database Converted Using 8-Core Computing System

20150313fr-david-r-barker-uiowaI needed to convert the format of a very large file, and no commercial services in the area were able to do it.

I tried on my own reasonably high-end computer, but had no luck – even if my system didn’t crash, which it did, the job might have taken weeks to complete.

Greg Johnson of the DWLLC and LMC used a state of the art system and was able to get the job done in just a few days.

Many thanks to Greg and the pleasant and efficient staff at the LMC and DWLLC!

David R. Barker — Former economist for the Federal Reserve and Professor of Economics and the University of Iowa

* * *

Technical Notes

20100104tu-gregory-johnson-by-makur-jain-IMG_1164-198x198-square
Greg Johnson

In these stories and articles posted on the Studio Users website, we like to share briefly from a technical perspective about how user support needs are met. This helps people understand our services better, and can also serve as a ‘how-to’ guide for others with similar support needs.

On March 4, I received a phone call from David Barker who was seeking assistance with converting a very large Microsoft Exchange Server data file.

The file was an Extensible Storage Engine (ESE) Database File (EDB) format file that was 100 gigabytes in size.

The goal was to convert the file into a Personal Storage Table (PST) format readable by Microsoft Outlook.

Part of the challenge was in creating smaller more manageable 10GB PST files that would be easier to open and manage in Outlook. There’s been a 20GB limit on PST file size for many years, and recently that limit was raised to 50GB by default, but larger file sizes can be problematic.

With over 634,000 records to convert and process, and given the nature of the data, it could take weeks for the data conversion to complete.

I’d recently configured and ordered an 8-core high-end solid-state power computing system with oversized power supply and about half a dozen cooling fans to keep the system temperature down when computational demands were high. The computer has a backup power system to avoid outages during long-run processes. I realized that this system would be ideal for processing this huge data set.

At David’s suggestion, we ended up using the Systools Exchange Recovery software. The older version of the program (version 4.5.0.0 of the software) had already been purchased and licensed. So, we tried that first. For some reason it didn’t seem to work. The conversion process would fail, and installed prerequisites were not being recognized. Diagnosing this, and communicating with technical support took a few days. We eventually switched to version 5.6 of the software. That installed and ran flawlessly.

The processing time was about 100,000 records for every 8 hours. So, it took about 50 hours to complete. In all, Given the previous conversion failures, I wanted to keep checking in on the system remotely. In all, from beginning to end this project spanned over about 6 days, including many evenings and a weekend. I was really pleased that I could help David with this project, and thankful for his patient and appreciative attitude.

Service Projects

When David Barker initially contacted me, he had no idea that I worked at the University of Iowa. He’d found my contact information as a local technology specialist in searching the Internet and called me as a last resort after exhausting all support options available to him. In talking with David, I learned of his affiliation with the University of Iowa, so I told him I’d like to volunteer to help him as a service project. I’ve been exploring ways to initiate more service projects and expand my volunteers efforts.

While this was a personal volunteer project that I ultimately completed off-campus and after hours, I thought it was important to share here on the LMC Studio website. As a University of Iowa alumni, and now as a full-time employee, I get to benefit from an enriching workplace that helps me grown in my career. I get to connect with other professionals and build collaborative relationships. It’s this environment of innovation and empowerment that equips me to take on projects like this one. There are many ways that members of the University of Iowa regularly volunteer and give back to the community. So, this is just one of many similar stories. Other inspiring examples are found in law, business, and medicine.

Giving

Those who are inspired by this kind of service project, and would like to support similar work, can consider donating to support the World Languages, Literatures, and Cultures Gift Fund.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s