How to Skip duplicate records (URLs)

Closed

Import WP Addon - WooCommerce Updated 9 months ago 9 Replies

Dung asked 10 months ago on February 11, 2024 at 6:07 pm

1. With scraped data, I have a lot of duplicate records in CSV file.

As I use Product URLs from scraped data as Unique Identifier (ImportWP):

11 unique urls

2 duplicate urls

After running import (the second time), it displays 13 Update. How to set ImportWP to skip the duplicate URLs instead of updating (so it can speed up the import process) ?

I attached the importer.

2. I use product data as a way to show stock & out of stock.

Example:

Day 1: 20 products imported

Day 2: 15 products update (how to show the missing product as “out of stock” ?)

Day 3: 20 products update (how to change “out of stock” to “in stock” when the missing products become available again ?)

3. When I set Taxonomy (product category) as “Samsung > Samsung air conditioner”, I see that products page displays the parent category & child category as well. 

How to import the product in the child category “Samsung air conditioner” only ?

J

James replied Support Agent

10 months ago on February 11, 2024 at 11:20 pm

Hi Dung,

Please find attached the latest development version of the plugin, this has a few extra features which i have just been added to allow the setting of stock status instead of deleting missing products, and importing only the last term in a taxonomy hierarchy.

In all the attached code snippets they have been restricted the importer_id of 13, change this to the id of your importer, found from the url when visiting the edit screen e.g. /wp-admin/tools.php?page=importwp&edit=13&step=2.

1) To stop processing of duplicate records, this can be accomplished using the following custom code:

/**
 * Stop duplicate records being updated in the same import.
 */
add_action('iwp/importer/mapper/before', function ($data) {

    $importer_id = iwp()->importer->getId();

    // comment the following lines remove importer id restriction
    if ($importer_id !== 13) {
        return;
    }

    if ($data->getId() === false) {
        return;
    }

    $session = iwp()->importer->getStatusId();

    /**
     * @var \WPDB $wpdb
     */
    global $wpdb;

    if (is_multisite()) {
        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d AND site_id=%d", $importer_id, $data->getId(), $wpdb->siteid);
    } else {
        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d", $importer_id, $data->getId());
    }

    $col = $wpdb->get_var($query);
    if ($col === $session) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Duplicate Record skipped.");
    }
});

2) If you enable the delete permission, and add the following custom code to create a custom delete action, instead of products being deleted there stock status will be changed to out of stock (however during testing your importer is importing external products, which do not support a stock status).

/**
 * Enable the custom delete action
 */
add_filter('iwp/importer/enable_custom_delete_action', function ($enable, $importer_id) {

    // comment the following lines remove importer id restriction
    if ($importer_id === 13) {
        return true;
    }

    return $enable;
}, 10, 2);

/**
 * On delete set stock status to outofstock
 */
add_action('iwp/importer/custom_delete_action', function ($importer_id, $object_id) {

    // comment the following lines remove importer id restriction
    if ($importer_id !== 13) {
        return;
    }

    $product = wc_get_product($object_id);
    if (!$product) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: Product #" . intval($object_id) . " not found.");
        return;
    }

    try {
        $product->set_stock_status('outofstock');
        $product->save();
    } catch (Exception $e) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: " . $e->getMessage());
    }
}, 10, 2);

3) A new field has been added to the taxonomy term settings called "hierarchy relationship", where by default it will import all terms, but if you set this to "connect last term", it will only attach the child term to the post/product.

DN

Dung replied

10 months ago on February 12, 2024 at 4:32 am

I tried to change your code without success.

Can you change your code above to apply to multi importer_ids ? (Such as 13,14,15)

(because I have over 30 CSV files, so I don't want to create a lot of Snippet Codes.)

J

James replied Support Agent

10 months ago on February 12, 2024 at 9:03 am

function iwphd_is_active_importer($importer_id){
    return in_array($importer_id, [13,14,15]) ? true : false;
}

/**
 * Stop duplicate records being updated in the same import.
 */
add_action('iwp/importer/mapper/before', function ($data) {

    $importer_id = iwp()->importer->getId();

    // comment the following lines remove importer id restriction
    if (!iwphd_is_active_importer($importer_id)) {
        return;
    }

    if ($data->getId() === false) {
        return;
    }

    $session = iwp()->importer->getStatusId();

    /**
     * @var \WPDB $wpdb
     */
    global $wpdb;

    if (is_multisite()) {
        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d AND site_id=%d", $importer_id, $data->getId(), $wpdb->siteid);
    } else {
        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d", $importer_id, $data->getId());
    }

    $col = $wpdb->get_var($query);
    if ($col === $session) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Duplicate Record skipped.");
    }
});

/**
 * Enable the custom delete action
 */
add_filter('iwp/importer/enable_custom_delete_action', function ($enable, $importer_id) {

    // comment the following lines remove importer id restriction
    if (!iwphd_is_active_importer($importer_id)) {
        return;
    }
    
    return true;
}, 10, 2);

/**
 * On delete set stock status to outofstock
 */
add_action('iwp/importer/custom_delete_action', function ($importer_id, $object_id) {

    // comment the following lines remove importer id restriction
    if (!iwphd_is_active_importer($importer_id)) {
        return;
    }

    $product = wc_get_product($object_id);
    if (!$product) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: Product #" . intval($object_id) . " not found.");
        return;
    }

    try {
        $product->set_stock_status('outofstock');
        $product->save();
    } catch (Exception $e) {
        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: " . $e->getMessage());
    }
}, 10, 2);
DN

Dung replied

10 months ago on February 12, 2024 at 1:47 pm

Thanks for your quick support.

Function "Stop duplicate records from updating" working well.

----------

I forgot that Woo external product doesn't display the stock status (instock/outofstock).

Can you change function "On delete set stock status to outofstock" to "On delete remove price" when any records no longer exist in CSV files ?

- so I can easy to hide products (without price) if necessary.

 

J

James replied Support Agent

10 months ago on February 12, 2024 at 2:02 pm

To change the price instead of the stock status, replace thh following line:

$product->set_stock_status('outofstock');

with

$product->set_price(0);

 

DN

Dung replied

10 months ago on February 12, 2024 at 4:24 pm

I change code as your instruction, and testing many times, however, the "deleted" products still intact - displaying price instead of removing.Can you check & test the code again ?

function iwphd_is_active_importer($importer_id){

    return in_array($importer_id, [15669,12614]) ? true : false;

}

 

/**

 * Stop duplicate records being updated in the same import.

 */

add_action('iwp/importer/mapper/before', function ($data) {

 

    $importer_id = iwp()->importer->getId();

 

    // comment the following lines remove importer id restriction

    if (!iwphd_is_active_importer($importer_id)) {

        return;

    }

 

    if ($data->getId() === false) {

        return;

    }

 

    $session = iwp()->importer->getStatusId();

 

    /**

     * @var \WPDB $wpdb

     */

    global $wpdb;

 

    if (is_multisite()) {

        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d AND site_id=%d", $importer_id, $data->getId(), $wpdb->siteid);

    } else {

        $query = $wpdb->prepare("SELECT `session` FROM {$wpdb->prefix}iwp_sessions WHERE importer_id=%d AND item_id=%d", $importer_id, $data->getId());

    }

 

    $col = $wpdb->get_var($query);

    if ($col === $session) {

        throw new \ImportWP\Common\Importer\Exception\MapperException("Duplicate Record skipped.");

    }

});

 

/**

 * Enable the custom delete action

 */

add_filter('iwp/importer/enable_custom_delete_action', function ($enable, $importer_id) {

 

    // comment the following lines remove importer id restriction

    if (!iwphd_is_active_importer($importer_id)) {

        return;

    }

    

    return true;

}, 10, 2);

 

/**

 * On delete set stock status to outofstock

 */

add_action('iwp/importer/custom_delete_action', function ($importer_id, $object_id) {

 

    // comment the following lines remove importer id restriction

    if (!iwphd_is_active_importer($importer_id)) {

        return;

    }

 

    $product = wc_get_product($object_id);

    if (!$product) {

        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: Product #" . intval($object_id) . " not found.");

        return;

    }

 

    try {

        $product->set_price(0);

        $product->save();

    } catch (Exception $e) {

        throw new \ImportWP\Common\Importer\Exception\MapperException("Custom Delete Action Error: " . $e->getMessage());

    }

}, 10, 2);

J

James replied Support Agent

10 months ago on February 12, 2024 at 4:33 pm

Sorry the previus change was untested, to solve this you need change 

$product->set_price(0);

to

$product->set_regular_price(0);
DN

Dung replied

10 months ago on February 12, 2024 at 5:20 pm

Thanks for your quick support.

The code is working well now.

J

James replied Support Agent

9 months ago on February 13, 2024 at 9:57 am

Thanks, Glad its working,

if you have other questions just let me know.

To help our growth and others to find this plugin, please help by leaving a review (https://wordpress.org/support/plugin/jc-importer/reviews/).