feat(server): Add support for client-side hashing (#2072)

* Modify controller DTOs * Can check duplicates on server side * Remove deviceassetid and deviceid * Remove device ids from file uploader * Add db migration for removed device ids * Don't sanitize checksum * Convert asset checksum to string * Make checksum not optional for asset * Use enums when rejecting duplicates * Cleanup * Return of the device id, but optional * Don't use deviceId for upload folder * Use checksum in thumb path * Only use asset id in thumb path * Openapi generation * Put deviceAssetId back in asset response dto * Add missing checksum in test fixture * Add another missing checksum in test fixture * Cleanup asset repository * Add back previous /exists endpoint * Require checksum to not be null * Correctly set deviceId in db * Remove index * Fix compilation errors * Make device id nullabel in asset response dto * Reduce PR scope * Revert asset service * Reorder imports * Reorder imports * Reduce PR scope * Reduce PR scope * Reduce PR scope * Reduce PR scope * Reduce PR scope * Update openapi * Reduce PR scope * refactor: asset bulk upload check * chore: regenreate open-api * chore: fix tests * chore: tests * update migrations and regenerate api * Feat: use checksum in web file uploader * Change to wasm-crypto * Use crypto api for checksumming in web uploader * Minor cleanup of file upload * feat(web): pause and resume jobs * Make device asset id not nullable again * Cleanup * Device id not nullable in response dto * Update API specs * Bump api specs * Remove old TODO comment * Remove NOT NULL constraint on checksum index * Fix requested pubspec changes * Remove unneeded import * Update server/apps/immich/src/api-v1/asset/asset.service.ts Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com> * Update server/apps/immich/src/api-v1/asset/asset-repository.ts Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com> * Remove unneeded check * Update server/apps/immich/src/api-v1/asset/asset-repository.ts Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com> * Remove hashing in the web uploader * Cleanup file uploader * Remove varchar from asset entity fields * Return 200 from bulk upload check * Put device asset id back into asset repository * Merge migrations * Revert pubspec lock * Update openapi specs * Merge upstream changes * Fix failing asset service tests * Fix formatting issue * Cleanup migrations * Remove newline from pubspec * Revert newline * Checkout main version * Revert again * Only return AssetCheck --------- Co-authored-by: Jason Rasmussen <jrasm91@gmail.com> Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
2025-12-08 20:29:05 +00:00 · 2023-05-24 23:08:21 +02:00
parent 49b74e9091
commit 1b54c4f8e7
33 changed files with 1396 additions and 58 deletions
--- a/server/immich-openapi-specs.json
+++ b/server/immich-openapi-specs.json
@@ -3251,6 +3251,49 @@
        ]
      }
    },
+    "/asset/bulk-upload-check": {
+      "post": {
+        "operationId": "bulkUploadCheck",
+        "description": "Checks if assets exist by checksums",
+        "parameters": [],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "$ref": "#/components/schemas/AssetBulkUploadCheckDto"
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/AssetBulkUploadCheckResponseDto"
+                }
+              }
+            }
+          }
+        },
+        "tags": [
+          "Asset"
+        ],
+        "security": [
+          {
+            "bearer": []
+          },
+          {
+            "cookie": []
+          },
+          {
+            "api_key": []
+          }
+        ]
+      }
+    },
    "/asset/shared-link": {
      "post": {
        "operationId": "createAssetsSharedLink",
@@ -6046,6 +6089,78 @@
          "existingIds"
        ]
      },
+      "AssetBulkUploadCheckItem": {
+        "type": "object",
+        "properties": {
+          "id": {
+            "type": "string"
+          },
+          "checksum": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "id",
+          "checksum"
+        ]
+      },
+      "AssetBulkUploadCheckDto": {
+        "type": "object",
+        "properties": {
+          "assets": {
+            "type": "array",
+            "items": {
+              "$ref": "#/components/schemas/AssetBulkUploadCheckItem"
+            }
+          }
+        },
+        "required": [
+          "assets"
+        ]
+      },
+      "AssetBulkUploadCheckResult": {
+        "type": "object",
+        "properties": {
+          "id": {
+            "type": "string"
+          },
+          "action": {
+            "type": "string",
+            "enum": [
+              "accept",
+              "reject"
+            ]
+          },
+          "reason": {
+            "type": "string",
+            "enum": [
+              "duplicate",
+              "unsupported-format"
+            ]
+          },
+          "assetId": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "id",
+          "action"
+        ]
+      },
+      "AssetBulkUploadCheckResponseDto": {
+        "type": "object",
+        "properties": {
+          "results": {
+            "type": "array",
+            "items": {
+              "$ref": "#/components/schemas/AssetBulkUploadCheckResult"
+            }
+          }
+        },
+        "required": [
+          "results"
+        ]
+      },
      "CreateAssetsShareLinkDto": {
        "type": "object",
        "properties": {