CSV/SFTP Sync Setup

How do I set up my CSV Sync?

Ryan Bickham avatar
Written by Ryan Bickham
Updated over a week ago

Our most flexible method of syncing your organization’s people data into Sift is through a CSV (comma-separated values) file. You may also choose to use our Secure File Transfer Protocol (SFTP) integration to safely send files to Sift on an automated basis.

Generating your CSV file (watch video)

From your organization’s system of record, such as your HRIS system, you’ll need to export a CSV that contains your people data. If you do not have an existing CSV to upload, you can also create one from scratch. If you need a place to start, click here to download an example CSV.

The only attributes we require for each person (for Primary Sources) are their first name, last name, and email address, but any additional properties will make your profiles that much richer! You will need to export or create a CSV file with the first row as headers and each subsequent row representing a single person.

A few important notes:

  • Each person (row) must have a unique email address and a unique Primary Key value. If there are any rows that contain duplicate values, or empty values, they will be ignored when you run your data sync.

  • Additionally, each person (row) must have a value for first name and last name.

  • If you would like to set up your Org Chart, you need a column in your CSV that includes a reference to the person's manager.

  • For Primary Sources, the full file must be uploaded on each sync. If Sift detects any rows that are missing, it will delete any associated Sift Profiles on subsequent data syncs.

  • For Complementary Sources, you may upload partial data, as they only append to your existing Profiles, and do not provision them.

Uploading a CSV file in the Admin Dashboard

There are a couple of different places you can upload your CSV on the Admin Dashboard:

  1. During the initial Directory/source setup wizard when the source is first created.

  2. After you've created your CSV source, you can update your people data by uploading a new CSV on the Source Dashboard by clicking the "Upload CSV File" button when a sync is not already running.

We'll walk through the first scenario noted above, but the process for the second is very similar.

Step 1: Upload your CSV file

Use the file upload area to import your CSV file. Only files with a ".csv" extension are supported.

The mappings page will automatically provide suggestions based on the headers in your CSV. To learn more about setting up mappings, click the link above.

Step 3: Wait for the sync to complete

After uploading your file, Sift will process the file and update your people data. This should only take a minute or two, but could take longer if you have a very large file. The Sync Status indicator at the top of the page will let you know once your sync is complete.

Debugging Failed Syncs

For further documentation on possible CSV sync errors, check out our troubleshooting article.

Check Required Data Mappings

If Sift finds people in your data source who do not have all of our required mappings (primary key, first name, last name, and email), they will be ignored. If no people are found that have all of these attributes, then the sync will fail. Ensure that all people you want to be synced into Sift have a value for all of these mappings.

Contact Us

If these steps do not fix the issue, you can contact us using the Chat button in the Admin Dashboard and we can help you investigate!

Additional Setup (Optional)

The below sections provide additional options for importing photo data into Sift, or automating the CSV sync process with SFTP/PGP. These are not required for setup.

Creating a photo CSV

After initially uploading your data CSV, you can optionally provide photos in a separate file. Our photo CSV files require a more specific format. You will need to include the following:

Primary Key column: You need a header named the same as your Primary Key Header in your data file. For example, if the Primary Key in your data file is called employeeId , the header in your photo file should be named the same.

Photo data: You need a header named exactly photoBinary. Values under this column should include a photo encoded in either base64 or hex. If encoded in hex, the values must start with the identifying hex prefix 0x.

Like a Complementary Source, each photo file will append to existing data. Therefore, it is only necessary to include new or updated images. Discluding a row from this file will not remove the photo from the associated Profile.

Uploading a photo CSV file manually

Step 1: Upload your CSV file

Click the Import Photo CSV button as seen below and go through the steps to upload your file.

Step 2: Wait for the sync to complete

After uploading your file, Sift will process the file and update your people data. The Sync Status indicator at the top of the page will provide an update, and once the sync has been completed, it will give you the time when your directory was updated.

Integrating with SFTP

Instead of uploading CSV files on our Admin Dashboard, you may instead choose to drop them into an SFTP directory in an automated fashion. To set up the SFTP integration, you'll need to generate an SSH key pair (RSA) and provide your public key to Sift. We will then provide you with connection details. Note that as Security Standards continue to evolve, Sift follows industry standards around ssh-key pairs, and no longer supports RSA/SHA1 by default, nor DSA key pair types. We recommend using one of the following key generation types(or greater) when generating your key pair:

ssh-keygen -t rsa -b 4096

ssh-keygen -t rsa -b 2048

ssh-keygen -t ecdsa -b 521

ssh-keygen -t ed25519

Automated HR system integrations

Many HR systems provide the ability to export reports on an automated basis via SFTP. After setting up the SFTP file drop on Sift as outlined below, you may be able to provide the information to your HR vendor and set up automated reports without much extra work. For detailed instructions on setup, see your HR system's documentation.

Sift Guides for Specific HR Systems

Step 1: Provide your public key to Sift

Paste your SSH public key in the text area below.

Step 2: Connect to your SFTP directory

Use the provided information to connect to your SFTP file drop. 

Step 3: Drop a file

Before dropping your file, be sure to set up your mappings, with the source column corresponding to the headers in your CSV file.

File naming

We recommend naming your files with some sort of timestamp identifier, such as:

Photo file names should include the prefix Photos_Full_

Data file names may be any other name. Ex: employees_{timestamp}.csv

Dropping a CSV file in the uploadProd folder will automatically trigger a data sync to begin within seconds. You should be able to follow the progress on your Admin Dashboard as usual. 

The uploadTest should be used for testing on your end or to debug any issues with the Sift team if needed. Dropping files here will not trigger a sync.

Encrypting with PGP (Optional)

If you would prefer an additional layer of security, you may encrypt your files with PGP before dropping them through SFTP. You will need to encrypt your file with our public key. To obtain this key and get help setting this up, please contact us.

Did this answer your question?