Abstract image of color cut outs partially overlapping

I recently imported over 100,000 orders for a client who was migrating into WooCommerce. This is a complex process as disparate systems are never easy to get in sync, even when doing a one time export/import operation.

There’s many ways to approach the task. Here’s some findings that may help you with your next big data import project:

  1. Write a PHP script to convert the exported CSV (etc.) data into an importable CSV for WooCommerce.
    • Really helps you understand the source and destination data sets and where the differences are.
    • Makes a clean CSV import file with only the fields that you need to import.
    • Pay special attention to the serialized field “Line Items” well as you will need to map these particulars (item name, quantity, price, etc.).
  2. Only generate customer user accounts where required. User accounts slow down the process as the importer needs to check for preexistence each time.
  3. Use the official importer plugin to import from CSV. With this tool you can import Customer and Coupon records before importing the Order records so Order records link to those other records.
  4. Execute a dry run first so you can see any problems with your format before writing to the database. The dry run will not necessarily detect duplicates or conflicts though, but is a good idea for a first pass.
  5. Be prepared for some back and forth on common formatting issues:
    • Preventing duplicates that the CSV export may contain.
    • Date format may need conversion. For example watch out for dd/mm/yyyy format.
    • Double quotes within fields such as product titles and addresses.
    • Missing email addresses will not import.
    • Email addresses containing the + sign the importer will not like.
    • Map country codes. For example “United States” should import into Woo as “US”.
    • Zero left padding zip codes for certain USA states.
    • Split discounts over product line items versus adding as its own line item. Test case is for the order to be able to be recalculated if edited, or exported to remote services such as sales tax integrations.
    • Map order statuses. For example: awaiting shipping = processing, shipped = completed
  6. Run the import on a local host environment then export / import the database rows to a staging environment, then to production, instead or running the import process on real servers.
    • This will save your hosting account from excessive PHP processing, prevent emails going out or other workflows tied to order creation running, and maximizes processing speed via dedicated resources.
    • Avoid index conflicts by setting the database table auto indexes well above what exists on production for all affected tables: posts, postmeta, order_items, and order_itemmeta rows.