bump to 0.3.7

README.md

@@ -14,7 +14,7 @@ This is yet another unofficial sync plugin for Obsidian. If you like it or find
 
 ## !!!Caution!!!
 
-As of Jan 2022, the plugin is considered in BETA stage. **DO NOT USE IT for any serious vaults.** **ALWAYS, ALWAYS, backup your vault before using this plugin.**
+**ALWAYS, ALWAYS, backup your vault before using this plugin.**
 
 ## Features
 
@@ -23,7 +23,7 @@ As of Jan 2022, the plugin is considered in BETA stage. **DO NOT USE IT for any
   - Dropbox
   - OneDrive for personal
   - Webdav
-  - A list of connectable / not-connectable / in-the-plan / never-possible services are noted down [here](./docs/services_connectable_or_not.md).
+  - [Here](./docs/services_connectable_or_not.md) shows more connectable (or not-connectable) services in details.
 - **Obsidian Mobile supported.** Vaults can be synced across mobile and desktop devices with the cloud service as the "broker".
 - **[End-to-end encryption](./docs/encryption.md) supported.** Files would be encrypted using openssl format before being sent to the cloud **if** user specify a password.
 - **Scheduled auto sync supported.** You can also manually trigger the sync using sidebar ribbon, or using the command from the command palette (or even bind the hot key combination to the command then press the hot key combination).
@@ -35,8 +35,7 @@ As of Jan 2022, the plugin is considered in BETA stage. **DO NOT USE IT for any
 
 - **To support deltions sync, extra metadata will also be uploaded.** See [Minimal Intrusive](./docs/minimal_intrusive_design.md).
 - **No Conflict resolution. No content-diff-and-patch algorithm.** All files and folders are compared using their local and remote "last modified time" and those with later "last modified time" wins.
-- **Cloud services cost you money.** Always be aware of the costs and pricing.
+- **Cloud services cost you money.** Always be aware of the costs and pricing. Specifically, all the operations, including but not limited to downloading, uploading, listing all files, calling any api, storage sizes, may or may not cost you money.
-- **All files or folder starting with `.` (dot) or `_` (underscore) are treated as hidden files, and would NOT be synced.** It's useful if you have some files just staying locally. But this strategy also means that themes / other plugins / settings of this plugin would neither be synced.
 - **Some limitations from the browser environment.** More technical details are [in the doc](./docs/browser_env.md).
 
 ## Questions, Suggestions, Or Bugs
@@ -63,7 +62,7 @@ Additionally, the plugin author may occasionally visit Obsidian official forum a
 - Prepare your S3 (-compatible) service information: [endpoint, region](https://docs.aws.amazon.com/general/latest/gr/s3.html), [access key id, secret access key](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/getting-your-credentials.html), bucket name. The bucket should be empty and solely for syncing a vault.
 - About CORS:
   - If you are using Obsidian >= 0.13.25, you can skip this CORS part.
-  - If you are using Obsidian < 0.13.25, you need to configure (enable) [CORS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enabling-cors-examples.html) for requests from `app://obsidian.md` and `capacitor://localhost` and `http://localhost`. Full example is [here](./docs/s3_cors_configure.md). It's unfortunately required, because the plugin sends requests from a browser-like envirement. And those addresses are tested and found on desktop and ios and android.
+  - If you are using Obsidian < 0.13.25, you need to configure (enable) [CORS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enabling-cors-examples.html) for requests from `app://obsidian.md` and `capacitor://localhost` and `http://localhost`, and add at least `ETag` into exposed headers. Full example is [here](./docs/s3_cors_configure.md). It's unfortunately required, because the plugin sends requests from a browser-like envirement. And those addresses are tested and found on desktop and ios and android.
 - Download and enable this plugin.
 - Enter your information to the settings of this plugin.
 - If you want to enable end-to-end encryption, also set a password in settings. If you do not specify a password, the files and folders are synced in plain, original content to the cloud.
@@ -72,15 +71,13 @@ Additionally, the plugin author may occasionally visit Obsidian official forum a
 
 ### Dropbox
 
-- **This plugin's function for Dropbox is not as mature as functions for S3.**
 - **This plugin is NOT an official Dropbox product.** The plugin just uses Dropbox's public API.
 - After the authorization, the plugin can read your name and email (which cannot be unselected on Dropbox api), and read and write files in your Dropbox's `/Apps/remotely-save` folder.
-- If you decide to authorize this plugin to connect to Dropbox, please go to plugin's settings, and choose Dropbox then follow the instructions.
+- If you decide to authorize this plugin to connect to Dropbox, please go to plugin's settings, and choose Dropbox then follow the instructions. [More with screenshot is here](./docs/dropbox_review_material/README.md).
 - Password-based end-to-end encryption is also supported. But please be aware that **the vault name itself is not encrypted**.
 
 ### OneDrive for personal
 
-- **This plugin's function for OneDrive is not as mature as functions for S3.**
 - **This plugin is NOT an official Microsoft / OneDrive product.** The plugin just uses Microsoft's [OneDrive's public API](https://docs.microsoft.com/en-us/onedrive/developer/rest-api).
 - This plugin only works for "OneDrive for personal", and not works for "OneDrive for Business" (yet). See [#11](https://github.com/fyears/remotely-save/issues/11) to further details.
 - After the authorization, the plugin can read your name and email, and read and write files in your OneDrive's `/Apps/remotely-save` folder.
@@ -89,7 +86,6 @@ Additionally, the plugin author may occasionally visit Obsidian official forum a
 
 ### webdav
 
-- **webdav support is considered experimental.**
 - About CORS:
   - If you are using Obsidian >= 0.13.25, you can skip this CORS part.
   - If you are using Obsidian < 0.13.25:
@@ -105,6 +101,12 @@ Additionally, the plugin author may occasionally visit Obsidian official forum a
 - In auto sync mode, if any error occurs, the plugin would **fail silently**.
 - Auto sync only works when Obsidian is being opened. It's **technically impossible** to auto sync while Obsidian is in background, because the plugin just works in the browser environment provided by Obsidian.
 
+## How To Deal With Hidden Files Or Folders
+
+**By default, all files or folder starting with `.` (dot) or `_` (underscore) are treated as hidden files, and would NOT be synced.** It's useful if you have some files just staying locally. But this strategy also means that themes / other plugins / settings of this plugin would neither be synced.
+
+In the latest version, you can change the settings to allow syncing `_` files or folders, as well as `.obsidian` special config folder (but not any other `.` files or folders).
+
 ## How To Debug
 
 See [here](./docs/how_to_debug/README.md) for more details.

docs/dropbox_review_material/README.md

@@ -0,0 +1,47 @@
+## Material Source
+
+To apply for the production use of Dropbox API, some descriptions are needed to be submitted to Dropbox. Coincidently, this can also be served as a "readme" to use this plugin with Dropbox.
+
+## Some Backgrounds
+
+1. Obsidian allows users to take notes using markdown files.
+2. The "app", remotely-save. is an unofficial Obsidian plugin, helping users download and upload ("sync") their notes from and to Dropbox.
+3. Technically, remotely-save is developed using web technologies.
+
+## Api Usages
+
+1. remotely-save uses "App folder", to avoid unnecessary accessing users' other files.
+2. remotely-save uses "account_info.read", to get the displayed username, so that users know which of their accounts has been logged in after OAuth steps.
+3. remotely-save uses "files.content.read", so that it can read the "last modified time", and the content of files on Dropbox. Under some conditions, the plugin would download the files. For example, the plugin would compare the last modified timestamps of the file in the local device and that on Dropbox, and if the timestamp on Dropbox is larger, the plugin would download the "newer" file from Dropbox to local.
+4. remotely-save uses "files.content.write", so that it can upload or overwrite the content of files on Dropbox. Under some conditions, the plugin would do that. For example, the plugin would compare the last modified timestamps of the file in the local device and that on Dropbox, and if the timestamp in the local device is larger, the plugin would upload the "newer" file from local to Dropbox, and overwrite that file on Dropbox.
+
+## Steps
+
+Here are the steps to see the functionality of remotely-save.
+
+Most steps have screenshots.
+
+1. Download the note-taking app Obsidian (Windows or Mac versions are both ok) from its official website: https://obsidian.md/ . It's free to download and use. Then install it.
+2. Open Obsidian, click the "Create" button under "Create new vault".
+3. Pick a vault name, "example-vault", and choose a location, then click "Create".
+   ![step03](./attachments/step03.png)
+4. Close any update new if prompted.
+5. Create a new note by clicking a button on the left. And write something on the note.
+   ![step05](./attachments/step05.png)
+6. Click "setting" (a gear icon) on the button left of the sidebar.
+   ![step06](./attachments/step06.png)
+7. In the settings panel, go to the "Community plugins" page, turn off the safe mode, and confirm to turn off the safe mode. Then click the "Browse" button for community plugins.
+   ![step07](./attachments/step07.png)
+8. Search "Remotely Save" and install on the result.
+   ![step08](./attachments/step08.png)
+9. After successful installing the plugin, go back to the "Community plugins" page, and enable the plugin.
+   ![step09](./attachments/step09.png)
+10. Go to newly added "Remotely Save" settings, select "Dropbox" in "Choose service", and click the "Auth" button.
+    ![step10](./attachments/step10.png)
+11. The standard auth flow address is shown, users should click the address, and finish the auth steps on the website. Finally, the Dropbox website should automatically redirect users back to the Obsidian app.
+    ![step11](./attachments/step11.png)
+12. The "Auth" button disappears. A new "Revoke Auth" appears.
+    ![step12](./attachments/step12.png)
+13. Go back to the main interface of Obsidian, a new "switch icon" should appear on the left sidebar. Click this, then the plugin would trigger the sync progress. It would compare meta info of local files and remote files (on Dropbox), and decide to download some files and/or upload some files.
+    ![step13](./attachments/step13.png)
+14. Create, edit, remove some notes, and repeat step 13, the files on Dropbox should also change to reflect the changes locally.

docs/dropbox_review_material/attachments/step03.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:655f6f487cc24af0ab86b8592622a7701c30f3369e7847a1fbb783d0962829c3
+size 52910

docs/dropbox_review_material/attachments/step05.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe971af94f1f7b042b7313268e4d2d8432d926ce2392e50ca2a4abc21c8a4aab
+size 57033

docs/dropbox_review_material/attachments/step06.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:025303053f2b2cedd2bdd61cd71e1f1c0fb12e45bfbc0e81f2c0d70747133505
+size 63458

docs/dropbox_review_material/attachments/step07.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:061707977c8046a6f924881f86aac962e2f05618e201080768246270de792040
+size 100162

docs/dropbox_review_material/attachments/step08.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8add61e6ef34b2470e5a5bea3e10955a86eed5ed1fc4dd398f30835485a6c455
+size 158422

docs/dropbox_review_material/attachments/step09.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:021ae1ce29d3932f088f0954787c475be826915b794320bf212d3ba7d58656da
+size 110529

docs/dropbox_review_material/attachments/step10.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b225d21de5778ddc032893e0f52150ab4260b0dfbe0e9141da4aa8b03131a9da
+size 147821

docs/dropbox_review_material/attachments/step11.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fda4f6e8cf68833f239bdc85c46802fe2131d149011074f747b1829822a38939
+size 59041

docs/dropbox_review_material/attachments/step12.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:576801d5ebd8cf7fcd3c9c0c6df325654338854f7d762a5680e9aab3c9e6508a
+size 142155

docs/dropbox_review_material/attachments/step13.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c93f681f0f7ce74cee0f2ed193f574ef915554f1a9ef96ba54afdede302f13a
+size 112057

manifest.json

@@ -1,7 +1,7 @@
 {
   "id": "remotely-save",
   "name": "Remotely Save",
-  "version": "0.3.6",
+  "version": "0.3.7",
   "minAppVersion": "0.13.21",
   "description": "Yet another unofficial plugin allowing users to synchronize notes between local device and the cloud service.",
   "author": "fyears",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "remotely-save",
-  "version": "0.3.6",
+  "version": "0.3.7",
   "description": "This is yet another sync plugin for Obsidian app.",
   "scripts": {
     "dev2": "node esbuild.config.mjs",
@@ -51,13 +51,13 @@
     "webpack-cli": "^4.9.1"
   },
   "dependencies": {
-    "@aws-sdk/client-s3": "^3.37.0",
+    "@aws-sdk/client-s3": "^3.54.0",
-    "@aws-sdk/fetch-http-handler": "^3.53.0",
+    "@aws-sdk/fetch-http-handler": "^3.54.0",
-    "@aws-sdk/lib-storage": "^3.53.1",
+    "@aws-sdk/lib-storage": "^3.54.0",
-    "@aws-sdk/protocol-http": "^3.53.0",
+    "@aws-sdk/protocol-http": "^3.54.0",
-    "@aws-sdk/querystring-builder": "^3.53.0",
+    "@aws-sdk/querystring-builder": "^3.54.0",
-    "@aws-sdk/signature-v4-crt": "^3.37.0",
+    "@aws-sdk/signature-v4-crt": "^3.54.0",
-    "@aws-sdk/types": "^3.53.0",
+    "@aws-sdk/types": "^3.54.0",
     "@azure/msal-node": "^1.6.0",
     "@fyears/tsqueue": "^1.0.1",
     "@microsoft/microsoft-graph-client": "^3.0.1",

src/baseTypes.ts

@@ -12,6 +12,7 @@ export interface S3Config {
   s3SecretAccessKey: string;
   s3BucketName: string;
   bypassCorsLocally?: boolean;
+  partsConcurrency?: number;
 }
 
 export interface DropboxConfig {

src/main.ts

@@ -585,6 +585,9 @@ export default class RemotelySavePlugin extends Plugin {
     if (this.settings.webdav.depth === undefined) {
       this.settings.webdav.depth = "auto_unknown";
     }
+    if (this.settings.s3.partsConcurrency === undefined) {
+      this.settings.s3.partsConcurrency = 20;
+    }
   }
 
   async saveSettings() {

src/misc.ts

@@ -268,3 +268,34 @@ export const getSplitRanges = (bytesTotal: number, bytesEachPart: number) => {
   }
   return res;
 };
+
+/**
+ * https://stackoverflow.com/questions/332422
+ * @param obj anything
+ * @returns string of the name of the object
+ */
+export const getTypeName = (obj: any) => {
+  return Object.prototype.toString.call(obj).slice(8, -1);
+};
+
+/**
+ * Startting from 1
+ * @param x
+ * @returns
+ */
+export const atWhichLevel = (x: string) => {
+  if (
+    x === undefined ||
+    x === "" ||
+    x === "." ||
+    x === ".." ||
+    x.startsWith("/")
+  ) {
+    throw Error(`do not know which level for ${x}`);
+  }
+  let y = x;
+  if (x.endsWith("/")) {
+    y = x.slice(0, -1);
+  }
+  return y.split("/").length;
+};

src/remoteForS3.ts

@@ -154,6 +154,7 @@ export const DEFAULT_S3_CONFIG = {
   s3SecretAccessKey: "",
   s3BucketName: "",
   bypassCorsLocally: true,
+  partsConcurrency: 20,
 };
 
 export type S3ObjectType = _Object;
@@ -291,7 +292,7 @@ export const uploadToRemote = async (
     const body = new Uint8Array(remoteContent);
     const upload = new Upload({
       client: s3Client,
-      queueSize: 20, // concurrency
+      queueSize: s3Config.partsConcurrency, // concurrency
       partSize: bytesIn5MB, // minimal 5MB by default
       leavePartsOnError: false,
       params: {

src/settings.ts

@@ -754,6 +754,30 @@ export class RemotelySaveSettingTab extends PluginSettingTab {
         });
     }
 
+    const partsConcurrencyDiv = s3Div.createEl("div");
+    new Setting(partsConcurrencyDiv)
+      .setName("Parts Concurrency")
+      .setDesc(
+        "Large files are split into small parts to upload in S3. How many parts do you want to upload in parallel at most?"
+      )
+      .addDropdown((dropdown) => {
+        dropdown.addOption("1", "1");
+        dropdown.addOption("2", "2");
+        dropdown.addOption("3", "3");
+        dropdown.addOption("5", "5");
+        dropdown.addOption("10", "10");
+        dropdown.addOption("15", "15");
+        dropdown.addOption("20", "20 (default)");
+
+        dropdown
+          .setValue(`${this.plugin.settings.s3.partsConcurrency}`)
+          .onChange(async (val) => {
+            const realVal = parseInt(val);
+            this.plugin.settings.s3.partsConcurrency = realVal;
+            await this.plugin.saveSettings();
+          });
+      });
+
     new Setting(s3Div)
       .setName("check connectivity")
       .setDesc("check connectivity")
@@ -1185,8 +1209,8 @@ export class RemotelySaveSettingTab extends PluginSettingTab {
       .addDropdown(async (dropdown) => {
         dropdown.addOption("s3", "S3 or compatible");
         dropdown.addOption("dropbox", "Dropbox");
-        dropdown.addOption("webdav", "Webdav (beta)");
+        dropdown.addOption("webdav", "Webdav");
-        dropdown.addOption("onedrive", "OneDrive for personal (alpha)");
+        dropdown.addOption("onedrive", "OneDrive for personal");
         dropdown
           .setValue(this.plugin.settings.serviceType)
           .onChange(async (val: SUPPORTED_SERVICES_TYPE) => {

src/sync.ts

@@ -32,6 +32,7 @@ import {
   mkdirpInVault,
   getFolderLevels,
   getParentFolder,
+  atWhichLevel,
 } from "./misc";
 import { RemoteClient } from "./remote";
 import {
@@ -43,10 +44,9 @@ import {
   DEFAULT_FILE_NAME_FOR_METADATAONREMOTE2,
   isEqualMetadataOnRemote,
 } from "./metadataOnRemote";
+import { isInsideObsFolder, ObsConfigDirFileType } from "./obsFolderLister";
 
 import * as origLog from "loglevel";
-import { padEnd } from "lodash";
-import { isInsideObsFolder, ObsConfigDirFileType } from "./obsFolderLister";
 const log = origLog.getLogger("rs-default");
 
 export type SyncStatusType =
@@ -933,6 +933,69 @@ const dispatchOperationToActual = async (
   }
 };
 
+const splitThreeSteps = (syncPlan: SyncPlanType, sortedKeys: string[]) => {
+  const mixedStates = syncPlan.mixedStates;
+  const totalCount = sortedKeys.length || 0;
+
+  const folderCreationOps: FileOrFolderMixedState[][] = [];
+  const deletionOps: FileOrFolderMixedState[][] = [];
+  const uploadDownloads: FileOrFolderMixedState[][] = [];
+  let realTotalCount = 0;
+
+  for (let i = 0; i < sortedKeys.length; ++i) {
+    const key = sortedKeys[i];
+    const val: FileOrFolderMixedState = Object.assign({}, mixedStates[key]); // copy to avoid issue
+
+    if (val.decision === "skipFolder" || val.decision === "skipUploading") {
+      // pass
+    } else if (val.decision === "createFolder") {
+      const level = atWhichLevel(key);
+      if (folderCreationOps[level - 1] === undefined) {
+        folderCreationOps[level - 1] = [val];
+      } else {
+        folderCreationOps[level - 1].push(val);
+      }
+      realTotalCount += 1;
+    } else if (
+      val.decision === "uploadLocalDelHistToRemoteFolder" ||
+      val.decision === "keepRemoteDelHistFolder" ||
+      val.decision === "uploadLocalDelHistToRemote" ||
+      val.decision === "keepRemoteDelHist"
+    ) {
+      const level = atWhichLevel(key);
+      if (deletionOps[level - 1] === undefined) {
+        deletionOps[level - 1] = [val];
+      } else {
+        deletionOps[level - 1].push(val);
+      }
+      realTotalCount += 1;
+    } else if (
+      val.decision === "uploadLocalToRemote" ||
+      val.decision === "downloadRemoteToLocal"
+    ) {
+      if (uploadDownloads.length === 0) {
+        uploadDownloads[0] = [val];
+      } else {
+        uploadDownloads[0].push(val); // only one level needed here
+      }
+      realTotalCount += 1;
+    } else {
+      throw Error(`unknown decision ${val.decision} for ${key}`);
+    }
+  }
+
+  // the deletionOps should be run from shadowest to deepest
+  // so we need to reverse it!
+  deletionOps.reverse(); // inplace reverse
+
+  return {
+    folderCreationOps: folderCreationOps,
+    deletionOps: deletionOps,
+    uploadDownloads: uploadDownloads,
+    realTotalCount: realTotalCount,
+  };
+};
+
 export const doActualSync = async (
   client: RemoteClient,
   db: InternalDBs,
@@ -987,95 +1050,47 @@ export const doActualSync = async (
       );
       log.debug(`finished ${key}`);
     }
-  } else {
-    let realCounter = 0;
 
-    log.debug(
+    return; // shortcut return, avoid too many nests below
-      `1. create all folders from shadowest to deepest, also check undefined decision`
+  }
-    );
-    for (let i = sortedKeys.length - 1; i >= 0; --i) {
-      const key = sortedKeys[i];
-      const val = mixedStates[key];
 
-      if (
+  const { folderCreationOps, deletionOps, uploadDownloads, realTotalCount } =
-        val.decision === undefined ||
+    splitThreeSteps(syncPlan, sortedKeys);
-        val.decision === "skipFolder" ||
+  const nested = [folderCreationOps, deletionOps, uploadDownloads];
-        val.decision === "createFolder"
+  const logTexts = [
-      ) {
+    `1. create all folders from shadowest to deepest, also check undefined decision`,
-        log.debug(`start syncing "${key}" with plan ${JSON.stringify(val)}`);
+    `2. delete files and folders from deepest to shadowest`,
+    `3. upload or download files in parallel, with the desired concurrency=${concurrency}`,
+  ];
 
-        if (callbackSyncProcess !== undefined) {
+  let realCounter = 0;
-          await callbackSyncProcess(realCounter, totalCount, key, val.decision);
-        }
-        realCounter += 1;
 
-        await dispatchOperationToActual(
+  for (let i = 0; i < nested.length; ++i) {
-          key,
+    log.debug(logTexts[i]);
-          vaultRandomID,
+
-          val,
+    const operations: FileOrFolderMixedState[][] = nested[i];
-          client,
+
-          db,
+    for (let j = 0; j < operations.length; ++j) {
-          vault,
+      const singleLevelOps: FileOrFolderMixedState[] | undefined =
-          localDeleteFunc,
+        operations[j];
-          password
+
-        );
+      if (singleLevelOps === undefined || singleLevelOps === null) {
-        log.debug(`finished ${key}`);
+        continue;
       }
-    }
 
-    log.debug(`2. delete files and folders from deepest to shadowest`);
+      const queue = new PQueue({ concurrency: concurrency, autoStart: true });
-    for (let i = 0; i < sortedKeys.length; ++i) {
-      const key = sortedKeys[i];
-      const val = mixedStates[key];
-      if (
-        val.decision === "uploadLocalDelHistToRemoteFolder" ||
-        val.decision === "keepRemoteDelHistFolder" ||
-        val.decision === "uploadLocalDelHistToRemote" ||
-        val.decision === "keepRemoteDelHist"
-      ) {
-        log.debug(`start syncing "${key}" with plan ${JSON.stringify(val)}`);
 
-        if (callbackSyncProcess !== undefined) {
+      for (let k = 0; k < singleLevelOps.length; ++k) {
-          await callbackSyncProcess(realCounter, totalCount, key, val.decision);
+        const val: FileOrFolderMixedState = singleLevelOps[k];
-        }
+        const key = val.key;
-        realCounter += 1;
 
-        await dispatchOperationToActual(
-          key,
-          vaultRandomID,
-          val,
-          client,
-          db,
-          vault,
-          localDeleteFunc,
-          password
-        );
-        log.debug(`finished ${key}`);
-      }
-    }
-
-    log.debug(
-      `3. upload or download files in parallel, with the desired concurrency=${concurrency}`
-    );
-    const queue = new PQueue({ concurrency: concurrency, autoStart: true });
-
-    // const commands: any[] = [];
-
-    for (let i = 0; i < sortedKeys.length; ++i) {
-      const key = sortedKeys[i];
-      const val = mixedStates[key];
-      if (
-        val.decision === "skipUploading" ||
-        val.decision === "uploadLocalToRemote" ||
-        val.decision === "downloadRemoteToLocal"
-      ) {
         const fn = async () => {
           log.debug(`start syncing "${key}" with plan ${JSON.stringify(val)}`);
 
           if (callbackSyncProcess !== undefined) {
             await callbackSyncProcess(
               realCounter,
-              totalCount,
+              realTotalCount,
               key,
               val.decision
             );
@@ -1098,8 +1113,8 @@ export const doActualSync = async (
         };
         queue.add(fn);
       }
-    }
 
-    await queue.onIdle();
+      await queue.onIdle();
+    }
   }
 };

tests/misc.test.ts

@@ -244,3 +244,25 @@ describe("Misc: get split ranges", () => {
     expect(k).to.deep.equal(k2);
   });
 });
+
+describe("Misc: at which level", () => {
+  it("should throw error on some parameters", () => {
+    expect(() => misc.atWhichLevel(undefined)).to.throw();
+    expect(() => misc.atWhichLevel("")).to.throw();
+    expect(() => misc.atWhichLevel("..")).to.throw();
+    expect(() => misc.atWhichLevel(".")).to.throw();
+    expect(() => misc.atWhichLevel("/")).to.throw();
+    expect(() => misc.atWhichLevel("/xxyy")).to.throw();
+  });
+
+  it("should treat folders correctly", () => {
+    expect(misc.atWhichLevel("x/")).to.be.equal(1);
+    expect(misc.atWhichLevel("x/y/")).to.be.equal(2);
+  });
+
+  it("should treat files correctly", () => {
+    expect(misc.atWhichLevel("x.md")).to.be.equal(1);
+    expect(misc.atWhichLevel("x/y.md")).to.be.equal(2);
+    expect(misc.atWhichLevel("x/y/z.md")).to.be.equal(3);
+  });
+});

versions.json

@@ -1,4 +1,4 @@
 {
   "0.3.2": "0.12.15",
-  "0.3.6": "0.13.21"
+  "0.3.7": "0.13.21"
 }