feat(server): Add support for client-side hashing (#2072)

* Modify controller DTOs

* Can check duplicates on server side

* Remove deviceassetid and deviceid

* Remove device ids from file uploader

* Add db migration for removed device ids

* Don't sanitize checksum

* Convert asset checksum to string

* Make checksum not optional for asset

* Use enums when rejecting duplicates

* Cleanup

* Return of the device id, but optional

* Don't use deviceId for upload folder

* Use checksum in thumb path

* Only use asset id in thumb path

* Openapi generation

* Put deviceAssetId back in asset response dto

* Add missing checksum in test fixture

* Add another missing checksum in test fixture

* Cleanup asset repository

* Add back previous /exists endpoint

* Require checksum to not be null

* Correctly set deviceId in db

* Remove index

* Fix compilation errors

* Make device id nullabel in asset response dto

* Reduce PR scope

* Revert asset service

* Reorder imports

* Reorder imports

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Update openapi

* Reduce PR scope

* refactor: asset bulk upload check

* chore: regenreate open-api

* chore: fix tests

* chore: tests

* update migrations and regenerate api

* Feat: use checksum in web file uploader

* Change to wasm-crypto

* Use crypto api for checksumming in web uploader

* Minor cleanup of file upload

* feat(web): pause and resume jobs

* Make device asset id not nullable again

* Cleanup

* Device id not nullable in response dto

* Update API specs

* Bump api specs

* Remove old TODO comment

* Remove NOT NULL constraint on checksum index

* Fix requested pubspec changes

* Remove unneeded import

* Update server/apps/immich/src/api-v1/asset/asset.service.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove unneeded check

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove hashing in the web uploader

* Cleanup file uploader

* Remove varchar from asset entity fields

* Return 200 from bulk upload check

* Put device asset id back into asset repository

* Merge migrations

* Revert pubspec lock

* Update openapi specs

* Merge upstream changes

* Fix failing asset service tests

* Fix formatting issue

* Cleanup migrations

* Remove newline from pubspec

* Revert newline

* Checkout main version

* Revert again

* Only return AssetCheck

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
This commit is contained in:
Jonathan Jogenfors
2023-05-24 23:08:21 +02:00
committed by GitHub
parent 49b74e9091
commit 1b54c4f8e7
33 changed files with 1396 additions and 58 deletions

View File

@@ -10,6 +10,7 @@ All URIs are relative to */api*
Method | HTTP request | Description
------------- | ------------- | -------------
[**addAssetsToSharedLink**](AssetApi.md#addassetstosharedlink) | **PATCH** /asset/shared-link/add |
[**bulkUploadCheck**](AssetApi.md#bulkuploadcheck) | **POST** /asset/bulk-upload-check |
[**checkDuplicateAsset**](AssetApi.md#checkduplicateasset) | **POST** /asset/check |
[**checkExistingAssets**](AssetApi.md#checkexistingassets) | **POST** /asset/exist |
[**createAssetsSharedLink**](AssetApi.md#createassetssharedlink) | **POST** /asset/shared-link |
@@ -93,6 +94,63 @@ Name | Type | Description | Notes
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
# **bulkUploadCheck**
> AssetBulkUploadCheckResponseDto bulkUploadCheck(assetBulkUploadCheckDto)
Checks if assets exist by checksums
### Example
```dart
import 'package:openapi/api.dart';
// TODO Configure API key authorization: cookie
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKeyPrefix = 'Bearer';
// TODO Configure API key authorization: api_key
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKeyPrefix = 'Bearer';
// TODO Configure HTTP Bearer authorization: bearer
// Case 1. Use String Token
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken('YOUR_ACCESS_TOKEN');
// Case 2. Use Function which generate token.
// String yourTokenGeneratorFunction() { ... }
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken(yourTokenGeneratorFunction);
final api_instance = AssetApi();
final assetBulkUploadCheckDto = AssetBulkUploadCheckDto(); // AssetBulkUploadCheckDto |
try {
final result = api_instance.bulkUploadCheck(assetBulkUploadCheckDto);
print(result);
} catch (e) {
print('Exception when calling AssetApi->bulkUploadCheck: $e\n');
}
```
### Parameters
Name | Type | Description | Notes
------------- | ------------- | ------------- | -------------
**assetBulkUploadCheckDto** | [**AssetBulkUploadCheckDto**](AssetBulkUploadCheckDto.md)| |
### Return type
[**AssetBulkUploadCheckResponseDto**](AssetBulkUploadCheckResponseDto.md)
### Authorization
[cookie](../README.md#cookie), [api_key](../README.md#api_key), [bearer](../README.md#bearer)
### HTTP request headers
- **Content-Type**: application/json
- **Accept**: application/json
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
# **checkDuplicateAsset**
> CheckDuplicateAssetResponseDto checkDuplicateAsset(checkDuplicateAssetDto, key)