Sample Solutions From Our Clients’ Perspectives: Massive Data Migration Project From Microsoft Access to Microsoft SQL Server for Honda North American Finance
♦ If you have worked with Microsoft Excel for a good amount of time it is likely that you have heard the trendy catchphrase “Big Data“. 1) Big Data, 2) Dashboards, 3) Power BI are the top three MS Excel catch phrases in the world of Microsoft programming, consulting and training. “Visualization Experts” is a close fourth.
♦ Big Data is as data scientists and data visualization experts like to describe it as a significant amount of data. How much that is no one knows or agrees, but it is really just a large amount of data, 30 mb or 300 TB, your perspective is what makes the data small, big or huge data.
♦ In reality databases are built to house excessively large amounts of detailed data so the phrase big data is not really all that meaningful or important, marketing really. But if you want to use the phrase, use the phrase. If big data is a a whole lot of data, then huge data is like big data on steroids.
♦ For this data migration project there was so much data that needed to be scrubbed and migrated that it took almost a year to complete. Now that is HUGE data!
♦ The client had allowed their data to get away from them, and it become unruly. What the client discovered after a lengthy and in-depth analysis of their North American operations was this, they were exposed to large and possibly critical data integrity, security issues as well as privacy concerns. In a nutshell, what they had were hundreds and hundreds of one off databases that housed all of their historical transaction data, going back decades, tens of millions of rows of data, millions and millions of records.
♦ This mixed up mess of databases was in fact the client’s data warehouse, across hundreds of CD/DVD files. This was a rude awakening and the alarms went off.
♦ A costly new ERP system was the solution. A costly multiyear project to migrate their systems from on-off Access databases to an expensive ERP data warehouse
- Over the years, dozens of one-off Microsoft Access databases were built to house client transaction data
- Each of the Access databases was unique; no simple standards of properly normalized data was visible
- The Access databases were saved to the desktop on the local hard drive, and then stored on CD/DVD.
- There was little to no documentation on how to use and to update the databases
- A single individual knew how to run each independent database, say a Senior Financial Analyst, or an accountant
- If a person was not at work, the work would not get done, period
- If a CD/DVD was lost, so was the underlying data
A little background, we have had so many clients over the years who were in very similar situations. And they all got to the same place by doing the same thing; they bypassed IT and built their own databases using Microsoft Access. All one-off databases, usually poorly built, built by the accountant or financial analyst responsible for outputting the data. Documentation, data integrity, security, all at risk due to lack of controls and standards. It was like the wild west of data rangling
♦ If you would like to discuss our data scientists and visualization experts to build one for you, give us a call at 877-392-3539.