Skip to content

Commit

Permalink
Merge pull request #17 from jordanhandy/upload-all
Browse files Browse the repository at this point in the history
Add commands with ability to mass upload assets
  • Loading branch information
jordanhandy authored Jun 12, 2024
2 parents 2e9c773 + 4878c13 commit 2f1b8f7
Show file tree
Hide file tree
Showing 16 changed files with 551 additions and 54 deletions.
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ Released under [MIT](/LICENSE) by [@jordanhandy](https://github.com/jordanhandy)
This plugin allows you to automatically upload images, video, audio and raw files pasted to Obsidian directly into your Cloudinary account (instead of stored locally). Note: There is functionality for media manipulation in this plugin using Cloudinary's custom parameters

## How it Works
### Single File Upload
![Action GIF](https://res.cloudinary.com/dakfccuv5/image/upload/v1636859613/Nov-13-2021_22-11-40_bpei0d.gif)
### Multi-file Upload
![Multi File](https://res.cloudinary.com/dakfccuv5/video/upload/v1718021709/mass-note-upload_qnx5ar.mp4)
## Configuration
1. Disable Obsidian Safe Mode
2. Install the Plugin
Expand Down
24 changes: 24 additions & 0 deletions docs/cloudinary-duplication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
label: Cloudinary Duplication
layout: default
order: 600
author: Jordan Handy
icon: gear
---
## Duplication in Cloudinary
Without proper configuration, it is highly likely that Cloudinary will not upload items as you expect. I recommend reading [Cloudinary documentation on Upload Presets](https://support.cloudinary.com/hc/en-us/articles/360016481620-What-are-Upload-presets-and-how-to-use-them).

This doc will explain some of the most important pieces of information.

### Use filename or externally-defined public ID
In your Upload Preset Settings under the Storage and Access menu, this option allows you to preserve filenames in Cloudinary. Normally, when you upload an item to Cloudinary, it is renamed with a unique public ID. If you would like to preserve filename, use this setting to allow for you to keep original filenames. The Obsidian plugin is configured such that filenames will be used if this setting is enabled.
![Externally defined public ID](https://res.cloudinary.com/dakfccuv5/image/upload/v1718108147/457f3fb2-f4c6-4235-8708-981ed138485d.png)

## Unique Filename
If **disabled**, the Unique Filename option will guard against duplication. With this enabled, Cloudinary will append random characters to end string of every upload. This means that even if you preserve file names, no two uploads of the same file will match and every upload will be unique. This could lead to mass duplication.

Keep this **disabled** so that if a file with the same filename already exists duplication is less-likely to occur
![Unique Filename](https://res.cloudinary.com/dakfccuv5/image/upload/v1718108164/46971520-e89b-4ae3-ad31-83a7790419d3.png)


Continue to [Plugin Commands](plugin-commands.md)
10 changes: 8 additions & 2 deletions docs/configuring-cloudinary.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ order: 800
author: Jordan Handy
icon: gear
---
## Cloudinary Confiuration Steps
## Cloudinary Configuration Steps
1. Log in to Cloudinary and find your Cloud Name here
![Cloudinary Dashboard](assets/cloudinary-dash.png)
2. Enable Unsigned Uploads
Expand All @@ -21,4 +21,10 @@ When the preset is created, it will have a "name" associated with it. Use this

!!! Note
If you have a folder name already configured on Cloudinary under the settings for your specific upload preset (can be configured on Cloudinary itself), this folder setting will be ignored.
!!!
!!!

## Cloudinary and Potential Duplicate Uploads
I recommend double-checking all of your Cloudinary Upload Preset settings before you begin to use the plugin.
[!ref Cloudinary Duplication](cloudinary-duplication.md)

Continue to [Plugin Commands](plugin-commands.md)
7 changes: 4 additions & 3 deletions docs/configuring-the-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,16 @@ To configure the Plugin, you'll need the following information:
- Cloudinary Cloud Name
- Cloudinary Upload Preset
- Cloudinary Folder Name
- Configure the behaviour that triggers a Cloudinary upload
- Copy/paste from clipboard
- Drag and Drop
- OPTIONAL: [Default Transformation Options](https://cloudinary.com/documentation/transformation_reference)
- OPTIONAL: Configure which asset types you want to be uploaded to Cloudinary
- Raw (non-media type files)
- Audio
- Video
- Images
- OPTIONAL: Configure the behaviour that triggers a Cloudinary upload
- Copy/paste from clipboard
- Drag and Drop
- OPTIONAL: Configure specific folders that different media types should be placed into

By default, only images are set to be uploaded to Cloudinary via a copy/paste from the clipboard, but this can be altered
![Cloudinary plugin configuration](https://res.cloudinary.com/dakfccuv5/image/upload/v1716510330/obsidian/wwhysme8vdrz6syd5skp.png)
Expand Down
14 changes: 14 additions & 0 deletions docs/plugin-commands/backup-vault-assets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
label: Backup Vault Assets to Cloudinary
layout: default
order: 700
author: Jordan Handy
icon: cloud
---
## Backup All Vault Assets to Cloudinary

!!!warning
This is an experimental feature. Your mileage may vary as to the success of this command because of the issues already outlined in [uploading single note content](upload-single-note-cloudinary.md) and [upload all note contents](upload-all-notes-cloudinary.md). This command attempts to read all "non-markdown" files in your vault and upload a copy of them to Cloudinary. The "Backup Folder" option is where these files are stored. The "Preserve File Paths" option attempts to maintain the same foldering structure you have in your vault, prepended with your specified backup folder as the root.
!!!

[See note on Cloudinary Duplication](../cloudinary-duplication.md)
15 changes: 15 additions & 0 deletions docs/plugin-commands/plugin-commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
label: Plugin Commands
layout: default
order: 1000
author: Jordan Handy
icon: gear
---
## Plugin Commands

The Cloudinary Uploader plugin has palette commands that can be run. Explore them below:
- [Upload single note assets to Cloudinary](upload-single-note-cloudinary.md)
- [Upload all note assets to Cloudinary](upload-all-notes-cloudinary.md)
- [Local asset file backup](backup-vault-assets.md)

Continue to [Upload single note assets to Cloudinary](upload-single-note-cloudinary.md)
4 changes: 4 additions & 0 deletions docs/plugin-commands/plugin-commands.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
label: Plugin Commands
order: 700
icon: gear
expanded: true
15 changes: 15 additions & 0 deletions docs/plugin-commands/upload-all-notes-cloudinary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
label: Upload all note assets to Cloudinary
layout: default
order: 800
author: Jordan Handy
icon: cloud
---
## Upload All Note Assets to Cloudinary

Within the command palette, run the "Upload all note files to Cloudinary" option. This will take all local assets located in the vault and attempt to upload them to Cloudinary.

See below for information:
[!ref Upload single note files](upload-single-note-cloudinary.md)

Continue to [Backup Vault Assets](backup-vault-assets.md)
27 changes: 27 additions & 0 deletions docs/plugin-commands/upload-single-note-cloudinary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
label: Upload single note assets to Cloudinary
layout: default
order: 900
author: Jordan Handy
icon: cloud
---
## Upload Single Note Assets to Cloudinary

Within the command palette, run the "Upload files in current note to Cloudinary" option. This will take all local assets located in the current note and attempt to upload them to Cloudinary.

!!!warning
It is **strongly** recommended to take a backup of your notes before you perform this action.
As part of the action, your local media assets are **not deleted** when the upload to Cloudinary happens, so you can still reference them if you wanted to. However, because of the nature of the upload, and the variability of syntax in how some users may choose to reference media, this may alter the formatting of your notes. Use with caution.
!!!

The success / failure of this plugin largely depends on the following factors:
1. Your Cloudinary plan -- Certain Cloudinary plans have quotas applied to their accounts. Consult Cloudinary documentation to understand what your limits are
2. The file types being uploaded -- Cloudinary only accepts certain file types to be uploaded to their servers. If you try to upload an unsupported file type, the upload for that specific file will fail. Consult [this documentation on media types](https://support.cloudinary.com/hc/en-us/articles/202520642-What-type-of-image-video-and-audio-formats-do-you-support) and [this documentation on raw types](https://support.cloudinary.com/hc/en-us/articles/202520572-Using-Cloudinary-for-files-other-than-images-and-videos) for more information.

## Warning
When you first use the option for mass note upload, you will be presented with a warning message explaining the potential dangers of your action. By default, this warning message will be displayed **every time** you invoke the action. If you would like to disable this message, toggle the "Hide command palette mass upload warning" option in plugin settings.
## Demo
View a demo of the command below:
![Demo video - mass upload](https://res.cloudinary.com/dakfccuv5/video/upload/v1718021709/mass-note-upload_qnx5ar.mp4)

Continue to [Uploading all note assets to Cloudinary](upload-all-notes-cloudinary.md)
2 changes: 1 addition & 1 deletion manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
"author": "Jordan Handy",
"isDesktopOnly": true,
"minAppVersion": "0.11.0",
"version": "0.3.1"
"version": "1.0.0"
}
7 changes: 4 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@
"eslint": "^7.30.0",
"obsidian": "obsidianmd/obsidian-api",
"obsidian-plugin-cli": "^0.4.5",
"typescript": "^4.1.5"
"typescript": "^4.1.5",
"retypeapp": "^1.11.0"
},
"dependencies": {
"axios": "^0.21.1",
"cloudinary": "^2.2.0",
"compressorjs": "^1.0.7",
"object-path": "^0.11.5",
"retypeapp": "^1.11.0"
"object-path": "^0.11.5"
}
}
189 changes: 189 additions & 0 deletions src/commands/utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
/*--------- Utilities for Commands and async uploads -----------*/

import { Notice, FileSystemAdapter, TFile } from 'obsidian';
import path from 'path';
import objectPath from 'object-path'
import { NoteWarningModal } from '../note-warning-modal';
import { v2 as cloudinary } from 'cloudinary';
import { audioFormats, videoFormats, imageFormats } from '../formats';
import CloudinaryUploader from '../main';

// Generates a modal when command is invoked
// References 'plugin' as we need access to settings on Cloudinary Uploader
export function uploadNoteModal(file?: TFile, type: string, plugin: CloudinaryUploader): void {
new NoteWarningModal(plugin.app, type, (result): void => {
if (result == 'true') {
if (file) {
uploadCurrentNoteFiles(file, plugin); // If a file was passed and modal agreed
return;
} else {
// If no file passed, but assets were to be uploaded
if (type == 'asset') {
fetchMessages(plugin);
//uploadVault(plugin); // Upload vault function
return;
} else if (type == 'note') { //! If no file passed, but 'notes' to be uploaded, this means all notes are requested.
const files = plugin.app.vault.getMarkdownFiles()
for (let file of files) {
uploadCurrentNoteFiles(file, plugin);
}

}
}
} else {
return;
}

}).open();
}

export async function uploadVault(plugin: CloudinaryUploader): Promise<string[][]> {
let successMessages = [];
let failureMessages = [];
//* Get all files in vault that are not
//* MD files, so they may be uploaded
const files = plugin.app.vault.getFiles()
new Notice("Upload of vault files started. Depending on your vault size, this could take a long while. Watch for error or success notices",0);
for (let file of files) {
if (file.extension != 'md') {
let filePath;
const adapter = plugin.app.vault.adapter;
if (adapter instanceof FileSystemAdapter) {
filePath = adapter.getFullPath(file.path);
}
await cloudinary.uploader.unsigned_upload(filePath, plugin.settings.uploadPreset, {
folder: plugin.settings.preserveBackupFilePath ? path.join(plugin.settings.backupFolder, path.dirname(file.path)) : plugin.settings.backupFolder,
resource_type: 'auto'
}).then(res=>{
// add to success messages array
successMessages.push('success')
},err=>{
// add to failure messages array
failureMessages.push(err.message);
});
}
}
// send messages
return [successMessages,failureMessages];
}

//* This function fetches messages from the mass upload job
//* as this could take a while if the vault is larger
export async function fetchMessages(plugin:CloudinaryUploader) : Promise<void>{
// After the upload action completes then
// retrieve data and display messages based on results.
uploadVault(plugin).then((data)=>{
if(data[0].length > 0 && data[1].length > 0){
new Notice("Cloudinary vault asset backup: There was some success in uploading vault files, as well as some errors. Open Developer Tools for error information in Console",0);
for(let msg of data[1]){
console.warn(msg);
}
}else if(data[0].length >0){
new Notice("Cloudinary vault asset backup: This was successfully completed. No errors to report",0);
}else if(data[1].length > 0){
new Notice("Cloudinary vault asset backup: This operation failed. Please try again",0);
}
});
}
export function uploadCurrentNoteFiles(file: TFile, plugin: CloudinaryUploader): void {
//! Read a cached version of the file, then:
/*
* Find a RegEx match for [[]] file refs
* Get file name and find full path of file
* Pass full path to Cloudinary for upload
* Based on answer returned, determine subfolder, then:
* Replace current strings with Cloudinary URLs
*/
let data = plugin.app.vault.cachedRead(file).then(() => {
const found = data.match(/\!\[\[(?!https?:\/\/).*?\]\]/g);
if (found && found.length > 0) for (let find of found) {
let fileString = find.substring(3, find.length - 2);
let filePath;
const adapter = plugin.app.vault.adapter;
if (adapter instanceof FileSystemAdapter) {
filePath = adapter.getFullPath(fileString)
cloudinary.uploader.unsigned_upload(filePath, plugin.settings.uploadPreset, {
folder: setSubfolder(undefined, filePath, plugin),
resource_type: 'auto'
}).then(res => {
console.log(res);
let url = objectPath.get(res, 'secure_url');
let resType = objectPath.get(res, 'resource_type');
url = generateTransformParams(url, plugin);
let replaceMarkdownText = generateResourceUrl(resType, url);
data = data.replace(find, replaceMarkdownText);
plugin.app.vault.process(file, () => {
return data;
})
new Notice("Upload of note file was completed"); // Success
}, err => {
// Failure
new Notice("There was something wrong with your upload. Please try again. " + file.name + '. ' + err.message, 0);
})
}
}
});
}
// Called to generate the output of the transformation parameters
// that are set on uploads
export function generateTransformParams(url: string, plugin: CloudinaryUploader): string {
if (plugin.settings.transformParams) {
const splitURL = url.split("/upload/", 2);
url = splitURL[0] += "/upload/" + plugin.settings.transformParams + "/" + splitURL[1];
}
if (plugin.settings.f_auto) {
const splitURL = url.split("/upload/", 2);
url = splitURL[0] += "/upload/f_auto/" + splitURL[1];
// leave standard of no transformations added
}
return url;
}
// Once upload completes, generate the resulting getMarkdown
// that will be written to file
export function generateResourceUrl(type: string, url: string): string {
if (type == 'audio' || isType(url, audioFormats)) {
return `<audio src="${url}" controls></audio>\n`;
} else if (type == 'video' || isType(url, videoFormats)) {
return `<video src="${url}" controls></video>\n`;
} else {
return `![](${url})`;
}
}
// Required as Cloudinary doesn't have an 'audio' resource type.
// As we only know the file type after it's been uploaded (we don't know MIME type),
// we check if audio was uploaded based on the most-commonly used audio formats
export function isType(url: string, formats: string[]): boolean {
let foundTypeMatch = false;
for (let format of formats) {
if (url.endsWith(format)) {
foundTypeMatch = true;
}
}
return foundTypeMatch;
}
// Determine the subfolder to place the uploaded
// file in, based on file type uploaded
export function setSubfolder(file: File, resourceUrl: string, plugin: CloudinaryUploader): string {
if (file) {
if (file.type && file.type.startsWith("image")) {
return `${plugin.settings.folder}/${plugin.settings.imageSubfolder}`;
} else if (file.type.startsWith("audio")) {
return `${plugin.settings.folder}/${plugin.settings.audioSubfolder}`;
} else if (file.type.startsWith("video")) {
return `${plugin.settings.folder}/${plugin.settings.videoSubfolder}`;
} else {
return `${plugin.settings.folder}/${plugin.settings.rawSubfolder}`;
}
} else if (resourceUrl) {
if (isType(resourceUrl, imageFormats)) {
return `${plugin.settings.folder}/${plugin.settings.imageSubfolder}`;
} else if (isType(resourceUrl, audioFormats)) {
return `${plugin.settings.folder}/${plugin.settings.audioSubfolder}`;
} else if (isType(resourceUrl, videoFormats)) {
return `${plugin.settings.folder}/${plugin.settings.videoSubfolder}`;
} else {
return `${plugin.settings.folder}/${plugin.settings.rawSubfolder}`;
}
}

}
Loading

0 comments on commit 2f1b8f7

Please sign in to comment.