KevinMidboe
a35d62a0bd
Extended description & added image and ndjson assets
2023-08-26 13:38:40 +02:00
KevinMidboe
097a83eb56
Merge pull request #4 from KevinMidboe/refactor/classified-and-simplified
...
Refactor: Classified and simplified
2023-08-26 10:53:52 +02:00
KevinMidboe
574eebed11
Boolean match string value of True
2023-08-26 10:53:22 +02:00
KevinMidboe
0cad37f67f
Filepath over ssh should be wrapped in qoutes.
...
This is relevant for files with spaces or special characters that we
want interpreted as a single parameter.
2023-08-26 10:52:13 +02:00
KevinMidboe
42f909a251
Escape spaces in file path for du command
2023-08-25 20:30:09 +02:00
KevinMidboe
db03d8debf
du command incorrect, host instead of path
2023-08-25 20:28:22 +02:00
KevinMidboe
b75115515e
Files list should be logged as string
2023-08-25 19:49:24 +02:00
KevinMidboe
cedefbc42b
Re-wrote everything to be defined in Classes, better logging & exec
...
Abstracted remote and local filesystems into a System class.
Transporter does the file transfering, simplified this code. Removes a
lot of the error handling that was there in replacement of state that
will be more resilient, predictable and less error-prone.
2023-08-25 19:39:07 +02:00
KevinMidboe
9d77dd210b
Avg speed file should return integer or None
2023-08-25 19:31:18 +02:00
KevinMidboe
ed2a1d9a53
Make elastic logging optional with config flag
2023-08-25 19:30:50 +02:00
KevinMidboe
948eeb1d31
Updated default configuration
2023-08-25 19:12:35 +02:00
KevinMidboe
e697a54332
.avgspeed.txt file gitignored
2023-08-25 19:10:32 +02:00
KevinMidboe
9f87219398
Merge pull request #3 from KevinMidboe/fix/folderless-files
...
Fix: Error when transferring without a parent folder
2023-08-22 23:53:00 +02:00
KevinMidboe
7ac6583796
Ensure always writing a number to file
2023-08-22 20:25:25 +02:00
KevinMidboe
12f6d1bf3f
Error when moving files without parent folders
...
This will check if file ends with a know video extension, if so
use os.makedirs to create a folder at destination and update rsync
destination with this folder.
2023-08-22 20:21:33 +02:00
KevinMidboe
7dcbd60de4
All log messages before moving a file should be debug
2022-12-10 11:59:33 +01:00
KevinMidboe
0f5f050448
Compute, log and save to file estimated & actual transfer time
2022-11-28 00:49:13 +01:00
KevinMidboe
6ff33dcef1
Use delugeClient-kevin package instead of exec in os thread
2022-11-28 00:48:24 +01:00
KevinMidboe
8af5301d34
More metadata logged to es, also es apikey setting
2022-11-28 00:44:30 +01:00
KevinMidboe
23d7b7513f
Calculate, log & save estimated & actual transfer speed
2022-11-28 00:43:04 +01:00
KevinMidboe
2fc9948836
File for storing previous avg transfer speed
2022-11-28 00:40:36 +01:00
KevinMidboe
92441a41f3
Updated readme
2022-11-27 19:18:24 +01:00
KevinMidboe
a11567dcf6
Moved log message for when we move an item to below the Already exists skipping message. If the file already exists we dont print Moving message when its acctually not going to be moved.
2019-05-01 21:17:55 +02:00
KevinMidboe
1846d15cac
Changed elastic param from file to path to be more consistent with elastic fields.
2019-05-01 21:16:45 +02:00
KevinMidboe
02f4b1b371
Posts to unique index every day with suffix of todays date
2019-05-01 21:15:25 +02:00
KevinMidboe
0816b77352
Fixed issue where not supplying es argument to logger threw an error.
2019-05-01 12:30:01 +02:00
KevinMidboe
75faa5c949
Fixed syntax error. Missing wrapping dict
2019-05-01 12:16:09 +02:00
KevinMidboe
0d394b8d1d
All logs that have filenames or paths now also supply these as args to the logger. They then receive their own field mapping in the document sent to elastic.
2019-05-01 12:13:59 +02:00
KevinMidboe
829a60f570
Created a middleware for our logger that can receive a es parameter. If supplied the parameter values is stripped from log string and its key and value are added to the document sent to elastic. This way filenames and lists can get their own field mapping and more easily searchable.
2019-05-01 12:12:28 +02:00
KevinMidboe
8030cf5b36
Safer parameter strings for rsync command. Wrap filename in qoutes.
2019-05-01 10:50:28 +02:00
KevinMidboe
ce2a319447
Conver session id to string when attaching to json object.
2019-05-01 00:30:25 +02:00
KevinMidboe
c3a23ef344
Fixed syntax error at if test. Incorrect closing paranthese position
2019-05-01 00:28:11 +02:00
KevinMidboe
5559030199
Create a session id each time ES handler is initialized
2019-05-01 00:26:31 +02:00
KevinMidboe
3c976b398f
We collect files to move. Some cases another transfer can start and finish before this instance gets to the file. To not transfere duplicates we now check if the file exists with getFiles before we start a transfere. If its at the remote path we simply continue to the next file.
2019-04-30 23:52:49 +02:00
KevinMidboe
506a029a88
Config gets a new elastic field for logging to dynamic elastic host.
2019-04-30 19:24:36 +02:00
KevinMidboe
206606ca1e
Moved function for fetching config into separate utils file.
2019-04-30 19:23:04 +02:00
KevinMidboe
df5246b304
Elastic logging handler now dependency free (urllib in favor of requests)
2019-04-30 00:28:43 +02:00
KevinMidboe
eefe4f7aa0
Created a custom elastic logging handler.
2019-04-30 00:23:30 +02:00
KevinMidboe
0a88e3b376
Changed severity levels on some logs
2019-04-30 00:22:21 +02:00
KevinMidboe
8c67226823
Log-log-log! If files are found or not we output two different logs. Also info log every time we dont find anything.
2019-04-29 22:39:20 +02:00
KevinMidboe
524cc82020
Replaced all prints with logger messages. Also added more info logging of what files are found
2019-04-29 22:26:23 +02:00
KevinMidboe
9c95395827
Logg deluge stdout to log file on remove and fixed issue where the name to delete from deluge was the absolute path not just the filename.
2019-04-29 22:13:40 +02:00
KevinMidboe
4e5f5e5172
Renamed case-sensitive file
2019-04-29 22:03:35 +02:00
KevinMidboe
dc777f0812
Improved log levels for subprocess calls.
2019-04-29 22:01:19 +02:00
KevinMidboe
af136e7ce2
Now uses logging to file.
2019-04-29 21:53:29 +02:00
KevinMidboe
48266f7d1a
Added project config file to gitignore
2019-04-29 21:52:44 +02:00
KevinMidboe
40846e490f
Logger now also creates log file if it does not exist yet
2019-04-29 21:52:07 +02:00
KevinMidboe
80013af814
If config does not exist we exit and tell user to follow readme
2019-04-29 21:33:20 +02:00
KevinMidboe
dce5a070ca
Added default config file with empty variables.
2019-04-29 21:09:43 +02:00
KevinMidboe
15eeb3681f
Version 0.1 of the transatlantic torrent express.
2019-04-29 21:08:25 +02:00
KevinMidboe