SPPull - simple client to pull and download files from SharePoint

Node.js module for downloading files from SharePoint document libraries.

New in version 2.7.0

• [Breaking change]: Removed legacy module.exports for straightforward usage in modern JS and TypeScript

import SPPull from 'sppull';
const sppull = SPPull.download; // <-- this is an entry point

• TSLint replaced with ESLint
• Integration tests are migrated to new version of Mocha and fixed

New in version 2.2.0

Files streaming download:

• Download files of any supported size
• Effective memmory consumption while fetching large files

New in version 2.1.0

Performance in SPO and HTTPS environments is improved. Download on multiple objects is x2 faster now!

New in version 2.0.6

Smart re-download mechanism. Existing files with no changes are ignored from the download.

Supported SharePoint versions

• SharePoint Online
• SharePoint 2013
• SharePoint 2016
• SharePoint 2019

How to use

Install

npm install sppull --save-dev

Demo

Usage

const { SPPull } = require('sppull');

const context = {/*...*/};
const options = {/*...*/};

SPPull.download(context, options)
  .then(successHandler)
  .catch(errorHandler);

Arguments

Context

• siteUrl - SharePoint site (SPWeb) url [string, required]
• creds
  • username - user name for SP authentication [string, optional in case of some auth methods]
  • password - password [string, optional in case of some auth methods] ...

Additional authentication options:

Since communication module (sp-request), which is used in sppull, had received additional SharePoint authentication methods, they are also supported in sppull.

• SharePoint On-Premise (Add-In permissions)
  • clientId
  • issuerId
  • realm
  • rsaPrivateKeyPath
  • shaThumbprint
• SharePoint On-Premise (NTLM handshake - more commonly used scenario):
  • username - username without domain
  • password
  • domain / workstation
• SharePoint Online (Add-In permissions):
  • clientId
  • clientSecret
• SharePoint Online (SAML based with credentials - more commonly used scenario):
  • username - user name for SP authentication [string, required]
  • password - password [string, required]
• ADFS user credantials:
  • username
  • password
  • relyingParty
  • adfsUrl

For more information please check node-sp-auth credential options and wiki pages.

Options

• spRootFolder - root folder in SharePoint to pull from [string, required]
• dlRootFolder - local root folder where files and folders will be saved to [string, optional, default: ./Downloads]
• spBaseFolder - base folder path which is omitted then saving files locally [string, optional, default: equals to spRootFolder]
• recursive - to pull all files and folders recursively [boolean, optional, default: true]
• ignoreEmptyFolders - to ignore local creation of SharePoint empty folders [boolean, optional, default: true]
• foderStructureOnly - to ignore files, recreate only folders' structure [boolean, optional, default: false]
• strictObjects - array of files and folders relative paths within the spRootFolder to proceed explicitly, [array of strings, optional]
• camlCondition - SharePoint CAML conditions to use [string, optional]
• spDocLibUrl - SharePoint document library URL [string, mandatory with camlCondition]
• metaFields - array of internal field names to request along with the files [array of strings, optional]
• createEmptyFolders - to create empty folders along with documents download task [boolean, optional, default: true]
• omitFolderPath - folder path pattern which is omitted from final downloaded files path [string, optional]
• muteConsole - to mute console messages during transport queries to SharePoint API [boolean, optional, default: false]
• shouldDownloadFile - a callback to decide if a file should be downloaded based on the metadata [optional, (props) => boolean, default: undefined]

Overloads / cases

• All files with folder structure from spRootFolder
• Files from spRootFolder folder, first hierarchy level only
• Folders structure from spRootFolder without files
• Files based on array of paths provided strictly [works with array of files only right now]
• Files based on CAML query conditions
• Pull for documents metadata to use it in callback's custom logic

Use case scenarios can be found on the Scenarios page. This page suggests combinations of options which are optimal for certain use cases.

successHandler

Callback gets called upon successful files download.

errorHandler

Callback gets executed in case of exception inside sppull. Accepts error object as first argument for callback.

Samples

Refer to the Scenarios page for suggested options combinations available with sppull.

Basic usage

TypeSctipt:

import { AuthConfig as SPAuthConfigurator } from 'node-sp-auth-config';
import SPPull, { ISPPullOptions, ISPPullContext } from 'sppull';

new SPAuthConfigurator().getContext().then(({ siteUrl, authOptions }) => {

  const context: ISPPullContext = {
    siteUrl: context.siteUrl,
    ...context.authOptions
  } as any;

  const options: ISPPullOptions = {
    spRootFolder: 'Shared%20Documents',
    dlRootFolder: './Downloads/Documents'
  };

  SPPull.download(context, options);

}).catch(console.log);

or ES6:

const { SPPull } = require('sppull');

const context = {
  siteUrl: "http://contoso.sharepoint.com/subsite",
  creds: {
    username: "[email protected]",
    password: "_Password_",
    online: true
  }
};

const options = {
  spRootFolder: "Shared%20Documents/Contracts",
  dlRootFolder: "./Downloads/Contracts"
};

/*
 * All files will be downloaded from http://contoso.sharepoint.com/subsite/Shared%20Documents/Contracts folder
 * to __dirname + /Downloads/Contracts folder.
 * Folders structure will remain original as it is in SharePoint's target folder.
*/
SPPull.download(context, options)
  .then((downloadResults) => {
    console.log("Files are downloaded");
    console.log("For more, please check the results", JSON.stringify(downloadResults));
  })
  .catch((err) => {
    console.log("Core error has happened", err);
  });

Passwords storage

To eliminate any local password storing if preferable to use any two-way hashing technique, like cpass.

Inspiration and references

This project was inspired by spsave by Sergei Sergeev and gulp-spsync by Wictor Wilén projects.

SPPull depends heavily on sp-request module and use it to send REST queries to SharePoint.