Upload large files to SharePoint online document library using PowerShell csom

In this post, we will discuss how to upload large files to SharePoint online document library using PowerShell CSOM. I have followed the PowerShell script from Asad Refai.

Upload large files to SharePoint online document library using PowerShell

Here we have a SharePoint online document library and will upload documents from the local drive. If the file size is less than 9 MB then it will upload directly to the document library. But if it is more than 9MB then it will upload in a chunk.

Below is the full PowerShell script to upload large files to SharePoint online document library. You can write the below PowerShell script by using visual studio code or PowerShell ISE.

$SiteURL = "https://onlysharepoint2013.sharepoint.com/sites/Bhawana/"
$libraryName="MyTestDocLib"
$SPOnlineUserName="Bijay@*********.onmicrosoft.com"
$SPOnlinePassword="***********"
$fileDirectory="C:\Users\Bijaya.Sahoo\Desktop\Bijay\Files"
Try{
Add-Type -Path 'C:\Users\Bijaya.Sahoo\Desktop\Microsoft.SharePoint.Client.dll'
Add-Type -Path 'C:\Users\Bijaya.Sahoo\Desktop\Microsoft.SharePoint.Client.Runtime.dll'
}
catch {
Write-host "No dlls found"
}
Function UploadFileInSlice ($ctx, $libraryName, $fileName, $fileChunkSizeInMB) {
$fileChunkSizeInMB = 9
# Each sliced upload requires a unique ID.
$UploadId = [GUID]::NewGuid()
# Get the name of the file.
$UniqueFileName = [System.IO.Path]::GetFileName($fileName)
# Get the folder to upload into.
$Docs = $ctx.Web.Lists.GetByTitle($libraryName)
$ctx.Load($Docs)
$ctx.Load($Docs.RootFolder)
$ctx.ExecuteQuery()
# Get the information about the folder that will hold the file.
$ServerRelativeUrlOfRootFolder = $Docs.RootFolder.ServerRelativeUrl
# File object.
[Microsoft.SharePoint.Client.File] $upload
# Calculate block size in bytes.
$BlockSize = $fileChunkSizeInMB * 1024 * 1024
# Get the size of the file.
$FileSize = (Get-Item $fileName).length
if ($FileSize -le $BlockSize)
{
# Use regular approach.
$FileStream = New-Object IO.FileStream($fileName,[System.IO.FileMode]::Open, [System.IO.FileAccess]::Read)
$FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$FileCreationInfo.Overwrite = $true
$FileCreationInfo.ContentStream = $FileStream
$FileCreationInfo.URL = $UniqueFileName
$Upload = $Docs.RootFolder.Files.Add($FileCreationInfo)
$ctx.Load($Upload)
$ctx.ExecuteQuery()
return $Upload
}
else
{
# Use large file upload approach.
$BytesUploaded = $null
$Fs = $null
Try {
$Fs = [System.IO.File]::Open($fileName, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read)
$br = New-Object System.IO.BinaryReader($Fs)
$buffer = New-Object System.Byte[]($BlockSize)
$lastBuffer = $null
$fileoffset = 0
$totalBytesRead = 0
$bytesRead
$first = $true
$last = $false
# Read data from file system in blocks.
while(($bytesRead = $br.Read($buffer, 0, $buffer.Length)) -gt 0) {
$totalBytesRead = $totalBytesRead + $bytesRead
# You've reached the end of the file.
if($totalBytesRead -eq $FileSize) {
$last = $true
# Copy to a new buffer that has the correct size.
$lastBuffer = New-Object System.Byte[]($bytesRead)
[array]::Copy($buffer, 0, $lastBuffer, 0, $bytesRead)
}
If($first)
{
$ContentStream = New-Object System.IO.MemoryStream
# Add an empty file.
$fileInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$fileInfo.ContentStream = $ContentStream
$fileInfo.Url = $UniqueFileName
$fileInfo.Overwrite = $true
$Upload = $Docs.RootFolder.Files.Add($fileInfo)
$ctx.Load($Upload)
# Start upload by uploading the first slice.
$s = [System.IO.MemoryStream]::new($buffer)
# Call the start upload method on the first slice.
$BytesUploaded = $Upload.StartUpload($UploadId, $s)
$ctx.ExecuteQuery()
# fileoffset is the pointer where the next slice will be added.
$fileoffset = $BytesUploaded.Value
# You can only start the upload once.
$first = $false
}
Else
{
# Get a reference to your file.
$Upload = $ctx.Web.GetFileByServerRelativeUrl($Docs.RootFolder.ServerRelativeUrl + [System.IO.Path]::AltDirectorySeparatorChar + $UniqueFileName);
If($last) {
# Is this the last slice of data?
$s = [System.IO.MemoryStream]::new($lastBuffer)
# End sliced upload by calling FinishUpload.
$Upload = $Upload.FinishUpload($UploadId, $fileoffset, $s)
$ctx.ExecuteQuery()
Write-Host "File upload complete"
# Return the file object for the uploaded file.
return $Upload
}
else {
$s = [System.IO.MemoryStream]::new($buffer)
# Continue sliced upload.
$BytesUploaded = $Upload.ContinueUpload($UploadId, $fileoffset, $s)
$ctx.ExecuteQuery()
# Update fileoffset for the next slice.
$fileoffset = $BytesUploaded.Value
}
}
} #// while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
}
Catch {
Write-Host "Error occurred"
}
Finally {
if ($Fs -ne $null)
{
$Fs.Dispose()
}
}
}
return $null
}
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$securePassword=ConvertTo-SecureString $SPOnlinePassword -AsPlainText -Force
$Context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($SPOnlineUserName, $securePassword)
foreach($fileName in Get-ChildItem $fileDirectory)
{
$UpFile = UploadFileInSlice -ctx $Context -libraryName $libraryName -fileName $fileName.FullName
$Context.Dispose();
}

Once you run the PowerShell script, you can see the files uploaded to SharePoint online document library.

upload large files to SharePoint online document library using PowerShell
sharepoint online upload large files to doscument library using PowerShell

Method invocation failed because [Microsoft.SharePoint.Client.File] does not contain a method named StartUpload

While uploading large files to SharePoint online document library using PowerShell in SharePoint online, I got the below error which says: Method invocation failed because [Microsoft.SharePoint.Client.File] does not contain a method named ‘StartUpload’

Method invocation failed because [Microsoft.SharePoint.Client.File] does not contain a method named StartUpload
Method invocation failed because [Microsoft.SharePoint.Client.File] does not contain a method named StartUpload

Method invocation failed because [Microsoft.SharePoint.Client.File] does not contain a method named StartUpload

Below is the code we can write using Visual Studio Code or Windows PowerShell ISE.

$ContentStream = New-Object System.IO.MemoryStream

# Add an empty file.
$fileInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$fileInfo.ContentStream = $ContentStream
$fileInfo.Url = $UniqueFileName
$fileInfo.Overwrite = $true
$Upload = $Docs.RootFolder.Files.Add($fileInfo)
$ctx.Load($Upload)

# Start upload by uploading the first slice.

$s = [System.IO.MemoryStream]::new($buffer)

# Call the start upload method on the first slice.
$BytesUploaded = $Upload.StartUpload($UploadId, $s)
$ctx.ExecuteQuery()

The error was coming in the below line:

$BytesUploaded = $Upload.StartUpload($UploadId, $s)

Here I was using the 15 versions of Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll. Here you need to use the 16 versions of both the dlls. Once I replaced with updated 16 versions of client dlls, it worked fine.

Read some PowerShell tutorials:

I hope this PowerShell script will be helpful to upload large files to SharePoint online document library.

>