I suggest you ...

Allow merging when duplicates found on contact import

506 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    anonymousanonymous shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →
    CasperCasper shared a merged idea: Merge imported contacts with existing contacts  ·   · 
    Antony HumphreysAntony Humphreys shared a merged idea: Allow users to Import updated CSV that updates existing records and adds new  ·   · 
    GregGreg shared a merged idea: Use Export/Import to update or revise database (overwrite existing info with new info)  ·   · 
    GlennGlenn shared a merged idea: When I import a contact and it matches an existing contact, it should merge the two.  ·   · 
    JennyJenny shared a merged idea: I would like to have the ability to update existing contacts when I import.  ·   · 
    Joshua WaldmanJoshua Waldman shared a merged idea: include a merge feature for contact imports  ·   · 
    Mark McCurryMark McCurry shared a merged idea: Update Contacts  ·   · 
    KimKim shared a merged idea: Import new items to existing contacts  ·   · 
    Duncan StockdillAdminDuncan Stockdill (Admin, Capsule) responded  · 

    we have looked into merging and concluded it’s not quite as straight forward as it might appear. Various implementations we’ve looked at aren’t without their problems.

    I’d like some feedback from supporters of this feature on the following ideas and concerns:

    Our initial thought was to match based on email address. However, if there are already multiple contacts in the database, what to do… do we ignore, or do we update each? If the name is different on the record being imported do we overwrite? Would it be better to match on name + email? Many of you are looking to import 100+ records at a time so presenting a choice or showing differences per record isn’t really viable.

    If the updated data in the import contains a ‘work phone’ column, should we update existing ‘work phone’ records or should we add a secondary ‘work phone’. What if the ‘work phone’ column is blank for a particular customer, but populated on their contact record in Capsule? Should we remove the existing number or ignore? Same thing applies to other contact details, tags and also custom fields, but for custom fields we don’t have the option to add a secondary value, we can only overwrite.

    I’m also concerned about the destructive nature of an import that overwrites. e.g. a user has a file that accidentally has data in the wrong columns such as an email address in the phone number field. Possibly we should be building an undo function alongside the import. However this would significantly increase the development effort. My current thinking is the user would need to resolve by re-running the import merge on the corrected file. Anyone see any issue with this?

    62 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • Robin CarswellRobin Carswell commented  ·   ·  Flag as inappropriate

        Two user stories
        1. Select default behaviour for a list
        As a list uploader, I would like to select a default selected behaviour for the list I upload (ignore / add), so that I don't have to click through the default behaviour currently implemented (a really long click marathon)

        2. Allow merge as default behaviour
        As a list uploader, I would like the option of merging "suspected duplicates", so that I don't have to manually merge each one later (another click marathon). This would expand the list above to allow auto-merge as a default option

      • GrahamGraham commented  ·   ·  Flag as inappropriate

        Hi Duncan, I'm with Ian Curry on this, it's a serious weakness. Whatever you call it (merge, bulk update, whatever) the impact is that once your contact / org data is loaded you cannot execute any changes (apart form add a tag, or delete) to the data except one at a time. For items with data tags it makes even less sense. The whole point of a data tag is that it can be used programmatically somehow. The solution might be to write a bulk update tool that uses the API, we're looking at this at the moment. If that works we'll licence it to other capsule users.

      • Ian CurryIan Curry commented  ·   ·  Flag as inappropriate

        New users to Capsule and seriously disappointed with the inability to import data to update people/organisations. A very simple implementation with warnings would be far better than doing nothing for 5 years!!

        So the scenario is I have multiple files that contain a column that is a key field e.g. email address. I have another column that I then want to load into a custom field or data tag where the key field matches.

        We are still in trial period and this is seriously impacting on our ability to add value to the data and ultimately run our business.

      • Mark E FellingMark E Felling commented  ·   ·  Flag as inappropriate

        The complexity of your questions comes down to different usage scenarios. SIMPLIFY. Import Profiles that contain predefined default of import logic for ease-of-use but also full flexibility (MIGRATION, UPDATE, or ADVANCED). These two different primary usage scenarios is what is causing all of the confusion and differences in user responses. Advanced allows your user to select an import profile for defaults then a pulldown box for each data field with selection to: (Overwrite, Ignore, Add Secondary). This then accommodates all usage scenarios including custom fields and custom data tags.

        Ability to undo an import mistake is important but is easy to solve with your current UI. Add "Imported" Data Tag to all imported records with a [Date - Time] data tag value. In this way on UI front end, all imported records will be easily identified, and using your search tool one can select all records imported at a specific date and time to "Undo Import" en masse if there was an error. During the import, make a copy in a separate "import backup's table" of all records that are modified (overwritten or values added) during the import with the same "Imported" Data Tag with date and time. To undo you simply first delete all records with the matching data tag and then restore the records from the backup table with the matching data tag. Or even better give the user the ability to do each of these two steps so they can choose to restore only overwritten records and leave all new records imported.

        MIGRATION SCENARIO:
        Name + E-mail match. Before import prompt user if e-mail matches but Name does not, (i.e. Bob Smith vs. Bob G. Smith vs. Smith, Bob) whether to create new contact or after import show a list of conflicts and allow user to select which one is correct. User can decide based on size of import. If name matches exactly but e-mail does not, same question for user: create additional contact or add e-mail to existing contact.

        All other fields Add Secondary by default (when possible) and overwrite custom fields and data tags by default. Or user can select Advanced and change any field to overwrite existing and not add secondary if they know the data being imported is more current or to ignore values being imported if they know the data in capsule is more current.

        UPDATE SCENARIO:
        -If Name Matchs but e-mail does not, then (User Specified before import: update existing e-mail or add secondary).
        -If e-mail matches but name does not then update name.
        -If neither name nor e-mail match create new Contact record
        -For all other records, overwrite existing by default. Or user can specify in Advanced to Add Secondary (when possible) or ignore for custom fields and data tags.

        ---------------------
        OK, I think I have solved your problem for all possible scenarios. Future import logic can be handled by adding a new import scenario without modifying the rest of the code. Now you guys get it implemented so we can all use it! Otherwise only option is to use Nutshell CRM or Podbox/tool to pull together/clean data first before importing (at least for migration scenario)

      • Anonymous commented  ·   ·  Flag as inappropriate

        To me the answer ie easy. Give the responsibility to the user (us) with three choices
        1) - For new emails: Add contact + opportunity
        2) - For existing emails: Update contact fields + Add opportunity
        3) - For existing emails: Update EMPTY contact fields + Add opportunity

      • Anonymous commented  ·   ·  Flag as inappropriate

        If the name is different on the record being imported do we overwrite?
        NO definitely match name + email, add secondary value if they are different from the one in existing record
        If the updated data in the import contains a ‘work phone’ column, should we update existing ‘work phone’ records or should we add a secondary ‘work phone’
        ADD a 2nd
        What if the ‘work phone’ column is blank for a particular customer, but populated on their contact record in Capsule? Should we remove the existing number or ignore?
        IGNORE

      • Anonymous commented  ·   ·  Flag as inappropriate

        I would also like to be able to do this - and cannot understand why it is not possible to automate a refresh of existing records, given that each contact, organisation and opportunity has a Unique Reference Number (URN) / ID.

        Is there a reason that the IDs are not visible within each record on screen?

        Is the ability to be able to import batch updates to existing contacts in development please?

        Thank you

      • RevensonRevenson commented  ·   ·  Flag as inappropriate

        We need some forward progress on this. Something is better than the current options.

        Maybe there could be a score based match. The more pieces of data that match (name, email, phone, address, etc), the higher the score. Then we get to chose. If a user selectable score for match exists, corresponding to multiple pieces of data matching, then merge.

        Please make some improvement!

      • Alec KinnearAlec Kinnear commented  ·   ·  Flag as inappropriate

        Hi Duncan,

        We use the Freshbooks integration very heavily and have run into problems with merging contacts (you have to merge in the Freshbooks one, not merge to). It makes merging very fussy. Could you set up the merging to grab the Freshbooks match and hang onto it except in case of conflict please? Ideally then there would be a question about which Freshbooks account to match.

        Thanks!

      • Web-ideasWeb-ideas commented  ·   ·  Flag as inappropriate

        I've spent a while checking Capsule to make sure it meets my client's requirements and it looks great. However I came upon this stumbling block today. They need to regularly upload updates to contacts and I'd assumed that Import would do the job but alas no! I'm a developer so I'm going to have to implement my own Update facility (for People) via the API. Would this be of use to anyone else?

      • Pippa Russell Pippa Russell commented  ·   ·  Flag as inappropriate

        Definately you need to create the ability to merge. Updating files singuarly in Capsule is too time consuming. Currently we are exporting lists and updating details in the CSV and want to reimport them. If a field is blank then it should retain the current inserted text. If there is text on any imports to merge it should overwrite the current data.

        Alternatively come up with a faster easier way for us to update many files rather than singluarly.

      • KaiKai commented  ·   ·  Flag as inappropriate

        One use case is the export of some contacts, edit them in excel, and then import the changes. In this case it would be sufficient to include some sort of internal ID/Number which can be used to match the imported contacts to existing ones.

      • gsbgsb commented  ·   ·  Flag as inappropriate

        Even if it has a potential for large scale distruction I would prefer a simple implementation to none. You could offer an option for handling duplicate contacts, individually and for all duplicates:
        - ignore
        - import and create duplicate
        - import and overwrite existing
        - import and merge by adding new info
        (You can think of shorter names)

        A second option
        Mac users have an easy way to merge contacts: export contacts as a vcard, import them to Addressbook. Addressbook does a very decent job of merging the contacts. I do not know whether Windows Outlook does this too, but I assume it does.
        Then you could reimport the changed contacts and update all info. This way you would leave the painful merging buiness to Apple/Microsoft.

        Another idea:
        Another program we use asks you to create a contact list with its name, in this case capsule to your Addressbook. With a function "import from addressbook" it reads all contacts in this grop and updates the info. Very convenient!

      • Zahra MamoojeeZahra Mamoojee commented  ·   ·  Flag as inappropriate

        Would it not be easier for the system to automatically upload but automatically create a list of the possible duplicates? This way it would avoid the need to change each record to "Import" and note it outside of the system and merge after import. The user can then check them once loaded. Perhaps a merge all function would suit some users with the option to uncheck and merge individually.

      • Mark WilliamsMark Williams commented  ·   ·  Flag as inappropriate

        Just to update this, since there hasn't been a comment in a few months. Any progress at all here yet? As an interim measure, how about introducing an "Import All" button. I can sort of cope with the duplicates, what I definitely can not cope wit is you forcing me to scroll through a very small box painstakingly looking for re warnings and needing to set them to "Import" - because I'd rather load duplicates than miss data to be honest. This would be particularly useful for Contacts,where your system checks on names - a very strange thing to do.

      • Ian WeatherhoggIan Weatherhogg commented  ·   ·  Flag as inappropriate

        It would make life MUCH easier if, at the preview step of an import, you stated how many duplicates had been found and gave us a single button that would flip them all from "Ignore" to "Import".
        ....when we're importing mailing lists with 1000's of entries, there are hundreds of apparent duplicates which are just coincidences, and it's extremely painful to have to scroll down and find each one.

      • Perch Ten DesignPerch Ten Design commented  ·   ·  Flag as inappropriate

        Here's a thought:

        * Run the import dry. My guess is that email is the most sensible id to use.
        * Make a record of all the conflicts you find, with each conflict also recording the type of conflict (e.g. name clash, multiple email addresses etc.).
        * Present the user a list of of the conflict types and give them the option in the UI to select ones to batch solve (e.g. overwrite all names, lossless add email addresses). The options available for each merge type can be displayed in a select box.
        * Also provide a UI to allow the user to drill down into each conflict type and make per-record choices to override the batch selection if they wish. Again, select box against each record.
        * Store the user preferences as a map of options keyed against a suitable unique conflict identifier
        * Run the actual import. Where a conflict is found check the option map for a specific user choice, if not found fall back to the batch solution.

        With each conflict type you find in the code as you're programming you will be able to identify the possible solutions. These are the options in the select boxes. They could include things like merge, ignore, overwrite, keep both records etc. Or with a custom field, if it is not possible to have multiple entries the options are simply 'overwrite, ignore'.

        I don't think this is actually as hard as it sounds in principle. I think the trick is to accept that you need to let the user decide from the word go and build your solution around efficiently gathering the data you need from the user about their choices. Simply asking the user for each record as and when you find it during the import is going to hurt. A lot. Better to build that map.

        Alternatively, just delegate it all to a third party like google contacts and call it integration :D

        Hope that helps.

      • FräuleinFräulein commented  ·   ·  Flag as inappropriate

        Please also consider duplicates on a basis of phone numbers. We are using the CRM for telemarketing and would like to have a dublicate review as we are entering a new contact based on phone number... Just to throw one more in.

      • MarkMark commented  ·   ·  Flag as inappropriate

        Hi, I have just started using Capsule and uploaded 2600 potential clients (schools in my country) from a spreadsheet. I will need to update information about them on an annual basis from a csv file to Capsule or even upload new info into a newly created field assigned to each client. You really need to have the ability to do this. I have read the comments here about unique emails and phone nrs etc. For my client organisations each has a unique organisation name or better still a unique identifier number field. When uploading why not have a allow the upload to assign a "key" field to assign the information to. This is a normal database function and surely cannot be that difficult. Then a mass assign or merge could take place. I have already mass uploaded and deleted three times to get my initial client list right. I cannot afford to muck around here.
        Thanks

      ← Previous 1 3 4

      Feedback and Knowledge Base