Data Migration: Enabled 200,000 records in single upload
It is inevitable to migrate your existing data when you plan to implement a new customer relationship management (CRM) or enterprise resource planning (ERP) solution for your business. NetSuite data migration is a tedious task which includes CSV creation and uploading the data. It is very important to perform the migration properly, otherwise, the organization’s data may have permanently damaged.
One of our NetSuite clients approached us with a requirement to migrate data from their legacy system to NetSuite. They wanted us to migrate the data without the help of any third-party applications. The data migration is sometimes complicated as the data in old systems arranged in a complex format. Before migrating it is necessary to understand the data types which need to be imported. For e.g. the data you migrate can be accounting transactions related to data or customer records related data.
There are several options to migrate the data such as manually typing and entering the data into NetSuite platform, using third-party tools, implementing an extra module at an additional cost or writing code to move data. Using native scripts to upload the data would be a perfect solution for migrating the data from the client’s legacy system to the NetSuite platform considering a large amount of data and its security.
Our Challenges of Data Migration
Data migration becomes very complex when there is a large number of customs records to migrate. Our client wanted to migrate thousands of records from their legacy system to NetSuite. Manually to create CSV files and to upload the data is out of the question as there are thousands of records to migrate and also NetSuite allows a maximum of 25,000 records to upload at a time. If there are more than 25,000 records we have to do multiple uploads, which is entirely time to consume.
The next choice is using third party web services. But the client has specifically mentioned that we must ensure their data is secure. So the only option left was to write code to migrate the data. But how to develop a method to upload a large number of records simultaneously using script was our main challenge.
It is extremely important to realize where the data is coming from and what is the master data in the legacy system. We studied the entire data structures in the legacy system and our team developed a deep knowledge of the data sources in the legacy system. Next, the main task was to choose the most suitable import option for data migration.
Keeping in mind about the client requirements we developed a method utilizing native scripts for uploading the data. Using this method we can upload a large number of records simultaneously.
Jobin&Jismi developed a highly effective data uploading method using native scripts within a very short time. The main highlights of our implemented solution are
- Can use excel directly, no need to convert it to csv
- Can upload up to 200,000 records at a time
- Since it is using Suitescripts we can modify it to include any customization, like the default value for particular fields, any calculations etc
- Once the upload is completed you will get an email notifying the process status
- No third party servers are used so it is safe