Update version

Added Verbose mode

README.md

@@ -6,13 +6,12 @@ This program downloads imgur, gfycat and direct image and video links of saved p
 ## What it can do
 - Can get posts from: frontpage, subreddits, multireddits, redditor's submissions, upvoted and saved posts; search results or just plain reddit links
 - Sorts posts by hot, top, new and so on
-- Downloads imgur albums, gfycat links, [self posts](#how-do-i-open-self-post-files) and any link to a direct image
+- Downloads **REDDIT** images and videos, **IMGUR** images and albums, **GFYCAT** links, **EROME** images and albums, **SELF POSTS** and any link to a **DIRECT IMAGE**
 - Skips the existing ones
-- Puts post titles to file's name
+- Puts post title and OP's name in file's name
 - Puts every post to its subreddit's folder
 - Saves a reusable copy of posts' details that are found so that they can be re-downloaded again
 - Logs failed ones in a file to so that you can try to download them later
-- Can run with double-clicking on Windows
 
 ## [Download the latest release](https://github.com/aliparlakci/bulk-downloader-for-reddit/releases/latest)
 
@@ -24,7 +23,8 @@ This program downloads imgur, gfycat and direct image and video links of saved p
   
 - **MacOS** users have to **[compile it from source code](docs/COMPILE_FROM_SOURCE.md)**.
 
-Script also accepts **command-line arguments**, get further information from **[`--help`](docs/COMMAND_LINE_ARGUMENTS.md)**
+### Additional options
+Script also accepts additional options via command-line arguments, get further information from **[`--help`](docs/COMMAND_LINE_ARGUMENTS.md)**
 
 ## Setting up the script
 Because this is not a commercial app, you need to create an imgur developer app in order API to work.
@@ -37,11 +37,24 @@ Because this is not a commercial app, you need to create an imgur developer app
 * Correct CHAPTCHA
 * Click **submit** button  
   
-It should redirect to a page which shows your **imgur_client_id** and **imgur_client_secret**
+It should redirect you to a page which shows your **imgur_client_id** and **imgur_client_secret**
   
-\* Select **OAuth 2 authorization without a callback URL** first then select **Anonymous usage without user authorization** if it says *Authorization callback URL: required*
+\* Select **OAuth 2 authorization without a callback URL** first then select **Anonymous usage without user authorization** if it says *Authorization callback URL: required*. If this does not work, it is safe to proceed with **OAuth 2 authorization without a callback URL**.
 
 ## FAQ
+### What do the dots resemble when getting posts?
+- Each dot means that 100 posts are scanned. 
+  
+### Getting posts is taking too long.
+- You can press Ctrl+C to interrupt it and start downloading.
+  
+### How downloaded files' names are formatted?
+- Self posts and images that are not belong to an album are formatted as **`[SUBMITTER NAME]_[POST TITLE]_[REDDIT ID]`**.
+  You can use *reddit id* to go to post's reddit page by going to link **reddit.com/[REDDIT ID]**
+  
+- An image in an imgur album is formatted as **`[ITEM NUMBER]_[IMAGE TITLE]_[IMGUR ID]`**
+  Similarly, you can use *imgur id* to go to image's imgur page by going to link **imgur.com/[IMGUR ID]**.
+
 ### How do I open self post files?
 - Self posts are held at reddit as styled with markdown. So, the script downloads them as they are in order not to lose their stylings.
   However, there is a [great Chrome extension](https://chrome.google.com/webstore/detail/markdown-viewer/ckkdlimhmcjmikdlpkmbgfkaikojcbjk) for viewing Markdown files with its styling. Install it and open the files with [Chrome](https://www.google.com/intl/tr/chrome/).  
@@ -53,20 +66,34 @@ It should redirect to a page which shows your **imgur_client_id** and **imgur_cl
   them, there.
 
 ## Changes on *master*
-### [23/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/7314e17125aa78fd4e6b28e26fda7ec7db7e0147)
+### [26/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/1623722138bad80ae39ffcd5fb38baf80680deac)
-- Split download() function
+- Improved verbose mode
-- Added erome support
+- Minimalized the console output
-- Remove exclude feature
+- Added quit option for auto quitting the program after process finished
-- Bug fix
 
-### [22/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/a67da461d2fcd70672effcb20c8179e3224091bb)
+### [25/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/1623722138bad80ae39ffcd5fb38baf80680deac)
+- Added verbose mode
+- Stylized the console output
+
+### [24/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/7a68ff3efac9939f9574c2cef6184b92edb135f4)
+- Added OP's name to file names (backwards compatible)
+- Deleted # char from file names (backwards compatible)
+- Improved exception handling
+
+### [23/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/7314e17125aa78fd4e6b28e26fda7ec7db7e0147)
+- Splited download() function
+- Added erome support
+- Removed exclude feature
+- Bug fixes
+
+### [22/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/6e7463005051026ad64006a8580b0b5dc9536b8c)
 - Put log files in a folder named "LOG_FILES"
 - Fixed the bug that makes multireddit mode unusable
 
 ### [21/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/4a8c2377f9fb4d60ed7eeb8d50aaf9a26492462a)
 - Added exclude mode
 
-### [20/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/commit/7548a010198fb693841ca03654d2c9bdf5742139)
+### [20/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/7548a010198fb693841ca03654d2c9bdf5742139)
 - "0" input for no limit
 - Fixed the bug that recognizes none image direct links as image links
 
@@ -76,7 +103,7 @@ It should redirect to a page which shows your **imgur_client_id** and **imgur_cl
 - Fixed the bug that prevents downloading some gfycat URLs
 
 ### [13/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/9f831e1b784a770c82252e909462871401a05c11)
-- Change config.json file's path to home directory
+- Changed config.json file's path to home directory
 
 ### [12/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/50a77f6ba54c24f5647d5ea4e177400b71ff04a7)
 - Added binaries for Windows and Linux

docs/COMMAND_LINE_ARGUMENTS.md

@@ -5,20 +5,24 @@ See **[compiling from source](COMPILE_FROM_SOURCE.md)** page first unless you ar
 ***Use*** `.\bulk-downloader-for-reddit.exe` ***or*** `./bulk-downloader-for-reddit` ***if you are using the executable***.
 ```console
 $ python script.py --help
-usage: script.py [-h] [--directory DIRECTORY] [--link link] [--saved]
+usage: script.py [-h] [--directory DIRECTORY] [--NoDownload] [--verbose]
-                 [--submitted] [--upvoted] [--log LOG FILE]
+                 [--quit] [--link link] [--saved] [--submitted] [--upvoted]
-                 [--subreddit SUBREDDIT [SUBREDDIT ...]]
+                 [--log LOG FILE] [--subreddit SUBREDDIT [SUBREDDIT ...]]
                  [--multireddit MULTIREDDIT] [--user redditor]
                  [--search query] [--sort SORT TYPE] [--limit Limit]
-                 [--time TIME_LIMIT] [--NoDownload]
+                 [--time TIME_LIMIT]
 
 This program downloads media from reddit posts
 
 optional arguments:
   -h, --help            show this help message and exit
-  --directory DIRECTORY
+  --directory DIRECTORY, -d DIRECTORY
                         Specifies the directory where posts will be downloaded
                         to
+  --NoDownload          Just gets the posts and stores them in a file for
+                        downloading later
+  --verbose, -v         Verbose Mode
+  --quit, -q            Auto quit afer the process finishes
   --link link, -l link  Get posts from link
   --saved               Triggers saved mode
   --submitted           Gets posts of --user
@@ -38,8 +42,6 @@ optional arguments:
   --limit Limit         default: unlimited
   --time TIME_LIMIT     Either hour, day, week, month, year or all. default:
                         all
-  --NoDownload          Just gets the posts and store them in a file for
-                        downloading later
 ```
 
 # Examples

docs/COMPILE_FROM_SOURCE.md

@@ -1,16 +1,14 @@
 # Compiling from source code
 ## Requirements
 ### Python 3 Interpreter
-Latest* version of **Python 3** is needed. See if it is already installed [here](#finding-the-correct-keyword-for-python). If not, download the matching release for your platform [here](https://www.python.org/downloads/) and install it. If you are a *Windows* user, selecting **Add Python 3 to PATH** option is mandatory.   
+Latest* version of **Python 3** is needed. See if it is already installed [here](#finding-the-correct-keyword-for-python). If not, download the matching release for your platform [here](https://www.python.org/downloads/) and install it. If you are a *Windows* user, selecting **Add Python 3 to PATH** option when installing the software is mandatory.   
   
 \* *Use Python 3.6.5 if you encounter an issue*
 ## Using terminal
 ### To open it...
-- **On Windows 8/8.1/10**: Press the File tab on **Windows Explorer**, click on **Open Windows PowerShell** or **Open Windows Command Prompt** or look for *Command Prompt* or *PowerShell* in *Start Menu*.
+-  **On Windows**: Press **Shift+Right Click**, select **Open Powershell window here** or **Open Command Prompt window here**
   
-- **On Windows 7**: Press **WindowsKey+R**, type **cmd** and hit Enter or look for *Command Prompt* or *PowerShell* in *Start Menu*.
+- **On Linux**: Right-click in a folder and select **Open Terminal** or press **Ctrl+Alt+T**.
-  
-- **On Linux**: Right-click in a folder and select **Open Terminal** or press **Ctrl+Alt+T** or look for **Terminal** in the programs.
   
 - **On MacOS**: Look for an app called **Terminal**.
   

script.py

@@ -22,7 +22,7 @@ from src.tools import (GLOBAL, createLogFile, jsonFile, nameCorrector,
 
 __author__ = "Ali Parlakci"
 __license__ = "GPL"
-__version__ = "1.4.0"
+__version__ = "1.6.0"
 __maintainer__ = "Ali Parlakci"
 __email__ = "parlakciali@gmail.com"
 
@@ -62,11 +62,27 @@ def parseArguments(arguments=[]):
                                      description="This program downloads " \
                                                  "media from reddit " \
                                                  "posts")
-    parser.add_argument("--directory",
+    parser.add_argument("--directory","-d",
                         help="Specifies the directory where posts will be " \
                         "downloaded to",
                         metavar="DIRECTORY")
         
+    parser.add_argument("--NoDownload",
+                        help="Just gets the posts and stores them in a file" \
+                             " for downloading later",
+                        action="store_true",
+                        default=False)
+    
+    parser.add_argument("--verbose","-v",
+                        help="Verbose Mode",
+                        action="store_true",
+                        default=False)
+    
+    parser.add_argument("--quit","-q",
+                        help="Auto quit afer the process finishes",
+                        action="store_true",
+                        default=False)
+
     parser.add_argument("--link","-l",
                         help="Get posts from link",
                         metavar="link")
@@ -138,13 +154,6 @@ def parseArguments(arguments=[]):
                         metavar="TIME_LIMIT",
                         type=str)
 
-    parser.add_argument("--NoDownload",
-                        help="Just gets the posts and store them in a file" \
-                             " for downloading later",
-                        action="store_true",
-                        default=False)
-    
-
     if arguments == []:
         return parser.parse_args()
     else:
@@ -423,21 +432,36 @@ def postFromLog(fileName):
 
     return posts
 
-def postExists(POST):
+def isPostExists(POST):
     """Figure out a file's name and checks if the file already exists"""
 
     title = nameCorrector(POST['postTitle'])
-    FILENAME = title + "_" + POST['postId']
     PATH = GLOBAL.directory / POST["postSubreddit"]
+
     possibleExtensions = [".jpg",".png",".mp4",".gif",".webm",".md"]
 
-    for i in range(2):
+    for extension in possibleExtensions:
-        for extension in possibleExtensions:
+
-            FILE_PATH = PATH / (FILENAME+extension)
+        OLD_FILE_PATH = PATH / (
-            if FILE_PATH.exists():
+            title
-                return True
+            + "_" + POST['postId']
-        else:
+            + extension
-            FILENAME = POST['postId']
+        )
+        FILE_PATH = PATH / (
+            POST["postSubmitter"] 
+            + "_" + title 
+            + "_" + POST['postId'] 
+            + extension
+        )
+
+        SHORT_FILE_PATH = PATH / (POST['postId']+extension)
+
+        if OLD_FILE_PATH.exists() or \
+           FILE_PATH.exists() or \
+           SHORT_FILE_PATH.exists():
+           
+            return True
+
     else:
         return False
 
@@ -450,13 +474,12 @@ def downloadPost(SUBMISSION):
         "imgur":Imgur,"gfycat":Gfycat,"erome":Erome,"direct":Direct,"self":Self
     }
 
+    print()
     if SUBMISSION['postType'] in downloaders:
 
-        print(SUBMISSION['postType'].upper())
-
         if SUBMISSION['postType'] == "imgur":
             
-            if int(time.time() - lastRequestTime) <= 2:
+            while int(time.time() - lastRequestTime) <= 2:
                 pass
 
             credit = Imgur.get_credits()
@@ -468,12 +491,17 @@ def downloadPost(SUBMISSION):
                             + str(int(IMGUR_RESET_TIME%60)) \
                             + " Seconds") 
             
+            if credit['ClientRemaining'] < 25 or credit['UserRemaining'] < 25:
+                printCredit = {"noPrint":False}
+            else:
+                printCredit = {"noPrint":True}
+
             print(
-                "Client: {} - User: {} - Reset {}".format(
+                "==> Client: {} - User: {} - Reset {}\n".format(
                     credit['ClientRemaining'],
                     credit['UserRemaining'],
                     USER_RESET
-                )
+                ),end="",**printCredit
             )
 
             if not (credit['UserRemaining'] == 0 or \
@@ -481,7 +509,7 @@ def downloadPost(SUBMISSION):
 
                 """This block of code is needed
                 """
-                if int(time.time() - lastRequestTime) <= 2:
+                while int(time.time() - lastRequestTime) <= 2:
                     pass
 
                 lastRequestTime = time.time()
@@ -515,17 +543,13 @@ def download(submissions):
     FAILED_FILE = createLogFile("FAILED")
 
     for i in range(subsLenght):
-        print("\n({}/{})".format(i+1,subsLenght))
         print(
-            "https://reddit.com/r/{subreddit}/comments/{id}".format(
+            f"\n({i+1}/{subsLenght}) – {submissions[i]['postType'].upper()} " \
-                subreddit=submissions[i]['postSubreddit'],
+            f"– r/{submissions[i]['postSubreddit']}",end=""
-                id=submissions[i]['postId']
-            )
         )
 
-        if postExists(submissions[i]):
+        if isPostExists(submissions[i]):
-            print(submissions[i]['postType'].upper())
+            print("\nIt already exists")
-            print("It already exists")
             duplicates += 1
             downloadedCount -= 1
             continue
@@ -546,12 +570,26 @@ def download(submissions):
             sys.exit()
 
         except ImgurLimitError as exception:
-            FAILED_FILE.add({int(i+1):[str(exception),submissions[i]]})
+            FAILED_FILE.add({int(i+1):[
+                "{class_name}: {info}".format(
+                    class_name=exception.__class__.__name__,info=str(exception)
+                ),
+                submissions[i]
+            ]})
             downloadedCount -= 1
 
         except NotADownloadableLinkError as exception:
-            print(exception)
+            print(
-            FAILED_FILE.add({int(i+1):[str(exception),submissions[i]]})
+                "{class_name}: {info}".format(
+                    class_name=exception.__class__.__name__,info=str(exception)
+                )
+            )
+            FAILED_FILE.add({int(i+1):[
+                "{class_name}: {info}".format(
+                    class_name=exception.__class__.__name__,info=str(exception)
+                ),
+                submissions[i]
+            ]})
             downloadedCount -= 1
 
         except NoSuitablePost:
@@ -560,8 +598,17 @@ def download(submissions):
         
         except Exception as exception:
             # raise exception
-            print(exception)
+            print(
-            FAILED_FILE.add({int(i+1):[str(exception),submissions[i]]})
+                "{class_name}: {info}".format(
+                    class_name=exception.__class__.__name__,info=str(exception)
+                )
+            )
+            FAILED_FILE.add({int(i+1):[
+                "{class_name}: {info}".format(
+                    class_name=exception.__class__.__name__,info=str(exception)
+                ),
+                submissions[i]
+            ]})
             downloadedCount -= 1
 
     if duplicates:
@@ -581,15 +628,12 @@ def main():
     else:
         GLOBAL.directory = Path(input("download directory: "))
 
-    print("\n"," ".join(sys.argv),"\n")
+    print("\n"," ".join(sys.argv),"\n",noPrint=True)
 
     try:
         checkConflicts()
     except ProgramModeError as err:
         PromptUser()
-    except Exception as err:
-        print(err)
-        sys.exit()
 
     if not Path(GLOBAL.configDirectory).is_dir():
         os.makedirs(GLOBAL.configDirectory)
@@ -648,7 +692,6 @@ if __name__ == "__main__":
     except KeyboardInterrupt:
         if GLOBAL.directory is None:
             GLOBAL.directory = Path(".\\")
-        print("\nQUITTING...")
         
     except Exception as exception:
         if GLOBAL.directory is None:
@@ -657,4 +700,4 @@ if __name__ == "__main__":
                       exc_info=full_exc_info(sys.exc_info()))
         print(log_stream.getvalue())
 
-    input("\nPress enter to quit\n")
+    if not GLOBAL.arguments.quit: input("\nPress enter to quit\n")

src/downloader.py

@@ -61,13 +61,14 @@ def getFile(fileDir,tempDir,imageURL,indent=0):
                                            tempDir,
                                            reporthook=dlProgress)
                 os.rename(tempDir,fileDir)
-                print(" "*indent+"Downloaded"+" "*10)
-                break
             except ConnectionResetError as exception:
                 print(" "*indent + str(exception))
                 print(" "*indent + "Trying again\n")
             except FileNotFoundError:
                 raise FileNameTooLong
+            else:
+                print(" "*indent+"Downloaded"+" "*10)
+                break
     else:
         raise FileAlreadyExistsError
 
@@ -87,13 +88,14 @@ class Erome:
             extension = getExtension(IMAGES[0])
 
             title = nameCorrector(post['postTitle'])
-            print(title+"_" +post['postId']+extension)
+            print(post["postSubmitter"]+"_"+title+"_"+post['postId']+extension)
 
-            fileDir = title + "_" + post['postId'] + extension
+            fileDir = directory / (
-            fileDir = directory / fileDir
+                post["postSubmitter"]+"_"+title+"_"+post['postId']+extension
-
+            )
-            tempDir = title + "_" + post['postId'] + '.tmp'
+            tempDir = directory / (
-            tempDir = directory / tempDir
+                post["postSubmitter"]+"_"+title+"_"+post['postId']+".tmp"
+            )
 
             imageURL = "https:" + IMAGES[0]
 
@@ -106,9 +108,11 @@ class Erome:
 
         else:
             title = nameCorrector(post['postTitle'])
-            print(title+"_"+post['postId'],end="\n\n")
+            print(post["postSubmitter"]+"_"+title+"_"+post['postId'],end="\n\n")
 
-            folderDir = directory / (title+"_"+post['postId'])
+            folderDir = directory / (
+                post["postSubmitter"] + "_" + title + "_" + post['postId']
+            )
 
             try:
                 if not os.path.exists(folderDir):
@@ -139,9 +143,16 @@ class Erome:
                     howManyDownloaded -= 1
 
                 except Exception as exception:
-                    raise exception
+                    # raise exception
                     print("\n  Could not get the file")
-                    print("  " + str(exception) + "\n")
+                    print(
+                        "  "
+                        + "{class_name}: {info}".format(
+                            class_name=exception.__class__.__name__,
+                            info=str(exception)
+                        )
+                        + "\n"
+                    )
                     exceptionType = exception
                     howManyDownloaded -= 1
 
@@ -213,13 +224,22 @@ class Imgur:
             post['postExt'] = getExtension(post['mediaURL'])
 
             title = nameCorrector(post['postTitle'])
-            print(title+"_" +post['postId']+post['postExt'])
+            print(post["postSubmitter"]+"_"+title+"_"+post['postId']+post['postExt'])
 
-            fileDir = title + "_" + post['postId'] + post['postExt']
+            fileDir = directory / (
-            fileDir = directory / fileDir
+                post["postSubmitter"]
+                + "_" + title
+                + "_" + post['postId'] 
+                + post['postExt']
+            )
+
+            tempDir = directory / (
+                post["postSubmitter"]
+                + "_" + title 
+                + "_" + post['postId'] 
+                + ".tmp"
+            )
 
-            tempDir = title + "_" + post['postId'] + '.tmp'
-            tempDir = directory / tempDir
             try:
                 getFile(fileDir,tempDir,post['mediaURL'])
             except FileNameTooLong:
@@ -235,9 +255,11 @@ class Imgur:
             duplicates = 0
 
             title = nameCorrector(post['postTitle'])
-            print(title+"_"+post['postId'],end="\n\n")
+            print(post["postSubmitter"]+"_"+title+"_"+post['postId'],end="\n\n")
 
-            folderDir = directory / (title+"_"+post['postId'])
+            folderDir = directory / (
+                post["postSubmitter"] + "_" + title + "_" + post['postId']
+            )
 
             try:
                 if not os.path.exists(folderDir):
@@ -290,7 +312,14 @@ class Imgur:
 
                 except Exception as exception:
                     print("\n  Could not get the file")
-                    print("  " + str(exception) + "\n")
+                    print(
+                        "  "
+                        + "{class_name}: {info}".format(
+                            class_name=exception.__class__.__name__,
+                            info=str(exception)
+                        )
+                        + "\n"
+                    )
                     exceptionType = exception
                     howManyDownloaded -= 1
 
@@ -355,10 +384,15 @@ class Gfycat:
 
         if not os.path.exists(directory): os.makedirs(directory)
         title = nameCorrector(POST['postTitle'])
-        print(title+"_"+POST['postId']+POST['postExt'])
+        print(POST["postSubmitter"]+"_"+title+"_"+POST['postId']+POST['postExt'])
+
+        fileDir = directory / (
+            POST["postSubmitter"]+"_"+title+"_"+POST['postId']+POST['postExt']
+        )
+        tempDir = directory / (
+            POST["postSubmitter"]+"_"+title+"_"+POST['postId']+".tmp"
+        )
         
-        fileDir = directory / (title+"_"+POST['postId']+POST['postExt'])
-        tempDir = directory / (title+"_"+POST['postId']+".tmp")
         try:
             getFile(fileDir,tempDir,POST['mediaURL'])
         except FileNameTooLong:
@@ -404,13 +438,14 @@ class Direct:
         POST['postExt'] = getExtension(POST['postURL'])
         if not os.path.exists(directory): os.makedirs(directory)
         title = nameCorrector(POST['postTitle'])
-        print(title+"_"+POST['postId']+POST['postExt'])
+        print(POST["postSubmitter"]+"_"+title+"_"+POST['postId']+POST['postExt'])
 
-        fileDir = title+"_"+POST['postId']+POST['postExt']
+        fileDir = directory / (
-        fileDir = directory / fileDir
+            POST["postSubmitter"]+"_"+title+"_"+POST['postId']+POST['postExt']
-
+        )
-        tempDir = title+"_"+POST['postId']+".tmp"
+        tempDir = directory / (
-        tempDir = directory / tempDir
+            POST["postSubmitter"]+"_"+title+"_"+POST['postId']+".tmp"
+        )
 
         try:
             getFile(fileDir,tempDir,POST['postURL'])
@@ -425,10 +460,11 @@ class Self:
         if not os.path.exists(directory): os.makedirs(directory)
 
         title = nameCorrector(post['postTitle'])
-        print(title+"_"+post['postId']+".md")
+        print(post["postSubmitter"]+"_"+title+"_"+post['postId']+".md")
 
-        fileDir = title+"_"+post['postId']+".md"
+        fileDir = directory / (
-        fileDir = directory / fileDir
+            post["postSubmitter"]+"_"+title+"_"+post['postId']+".md"
+        )
         
         if Path.is_file(fileDir):
             raise FileAlreadyExistsError
@@ -451,7 +487,11 @@ class Self:
                    + ")\n"
                    + post["postContent"]
                    + "\n\n---\n\n"
-                   + "submitted by [u/"
+                   + "submitted to [r/"
+                   + post["postSubreddit"]
+                   + "](https://www.reddit.com/r/"
+                   + post["postSubreddit"]
+                   + ") by [u/"
                    + post["postSubmitter"]
                    + "](https://www.reddit.com/user/"
                    + post["postSubmitter"]

src/searcher.py

@@ -1,4 +1,5 @@
 import os
+import sys
 import random
 import socket
 import webbrowser
@@ -14,60 +15,62 @@ from src.errors import (NoMatchingSubmissionFound, NoPrawSupport,
 
 print = printToFile
 
-class GetAuth:
-    def __init__(self,redditInstance,port):
-        self.redditInstance = redditInstance
-        self.PORT = int(port)
-
-    def recieve_connection(self):
-        """Wait for and then return a connected socket..
-        Opens a TCP connection on port 8080, and waits for a single client.
-        """
-        server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        server.bind(('localhost', self.PORT))
-        server.listen(1)
-        client = server.accept()[0]
-        server.close()
-        return client
-
-    def send_message(self, message):
-        """Send message to client and close the connection."""
-        self.client.send('HTTP/1.1 200 OK\r\n\r\n{}'.format(message).encode('utf-8'))
-        self.client.close()
-
-    def getRefreshToken(self,*scopes):
-        state = str(random.randint(0, 65000))
-        url = self.redditInstance.auth.url(scopes, state, 'permanent')
-        print("Go to this URL and login to reddit:\n\n",url)
-        webbrowser.open(url,new=2)
-
-        self.client = self.recieve_connection()
-        data = self.client.recv(1024).decode('utf-8')
-        param_tokens = data.split(' ', 2)[1].split('?', 1)[1].split('&')
-        params = {
-            key: value for (key, value) in [token.split('=') \
-            for token in param_tokens]
-        }
-        if state != params['state']:
-            self.send_message(
-                client, 'State mismatch. Expected: {} Received: {}'
-                .format(state, params['state'])
-            )
-            raise RedditLoginFailed
-        elif 'error' in params:
-            self.send_message(client, params['error'])
-            raise RedditLoginFailed
-        
-        refresh_token = self.redditInstance.auth.authorize(params['code'])
-        self.send_message(
-            "<script>" \
-            "alert(\"You can go back to terminal window now.\");" \
-            "</script>"
-        )
-        return (self.redditInstance,refresh_token)
-
 def beginPraw(config,user_agent = str(socket.gethostname())):
+    class GetAuth:
+        def __init__(self,redditInstance,port):
+            self.redditInstance = redditInstance
+            self.PORT = int(port)
+
+        def recieve_connection(self):
+            """Wait for and then return a connected socket..
+            Opens a TCP connection on port 8080, and waits for a single client.
+            """
+            server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            server.bind(('localhost', self.PORT))
+            server.listen(1)
+            client = server.accept()[0]
+            server.close()
+            return client
+
+        def send_message(self, message):
+            """Send message to client and close the connection."""
+            self.client.send(
+                'HTTP/1.1 200 OK\r\n\r\n{}'.format(message).encode('utf-8')
+            )
+            self.client.close()
+
+        def getRefreshToken(self,*scopes):
+            state = str(random.randint(0, 65000))
+            url = self.redditInstance.auth.url(scopes, state, 'permanent')
+            print("Go to this URL and login to reddit:\n\n",url)
+            webbrowser.open(url,new=2)
+
+            self.client = self.recieve_connection()
+            data = self.client.recv(1024).decode('utf-8')
+            param_tokens = data.split(' ', 2)[1].split('?', 1)[1].split('&')
+            params = {
+                key: value for (key, value) in [token.split('=') \
+                for token in param_tokens]
+            }
+            if state != params['state']:
+                self.send_message(
+                    client, 'State mismatch. Expected: {} Received: {}'
+                    .format(state, params['state'])
+                )
+                raise RedditLoginFailed
+            elif 'error' in params:
+                self.send_message(client, params['error'])
+                raise RedditLoginFailed
+            
+            refresh_token = self.redditInstance.auth.authorize(params['code'])
+            self.send_message(
+                "<script>" \
+                "alert(\"You can go back to terminal window now.\");" \
+                "</script>"
+            )
+            return (self.redditInstance,refresh_token)
+
     """Start reddit instance"""
     
     scopes = ['identity','history','read']
@@ -123,8 +126,6 @@ def getPosts(args):
         if args["user"] == "me":
             args["user"] = str(reddit.user.me())
 
-    print("\nGETTING POSTS\n.\n.\n.\n")
-
     if not "search" in args:
         if args["sort"] == "top" or args["sort"] == "controversial":
             keyword_params = {
@@ -156,7 +157,7 @@ def getPosts(args):
                     sort=args["sort"],
                     subreddit=args["subreddit"],
                     time=args["time"]
-                ).upper()
+                ).upper(),noPrint=True
             )            
             return redditSearcher(
                 reddit.subreddit(args["subreddit"]).search(
@@ -184,7 +185,7 @@ def getPosts(args):
             "saved posts\nuser:{username}\nlimit={limit}\n".format(
                 username=reddit.user.me(),
                 limit=args["limit"]
-            ).upper()
+            ).upper(),noPrint=True
         )
         return redditSearcher(reddit.user.me().saved(limit=args["limit"]))
 
@@ -199,7 +200,7 @@ def getPosts(args):
                     sort=args["sort"],
                     subreddit=args["subreddit"],
                     time=args["time"]
-                ).upper()
+                ).upper(),noPrint=True
             )
             return redditSearcher(
                 getattr(reddit.front,args["sort"]) (**keyword_params)
@@ -213,7 +214,7 @@ def getPosts(args):
                     sort=args["sort"],
                     subreddit=args["subreddit"],
                     time=args["time"]
-                ).upper()
+                ).upper(),noPrint=True
             )
             return redditSearcher(
                 getattr(
@@ -231,7 +232,7 @@ def getPosts(args):
                 sort=args["sort"],
                 multireddit=args["multireddit"],
                 time=args["time"]
-            ).upper()
+            ).upper(),noPrint=True
         )
         try:
             return redditSearcher(
@@ -245,8 +246,6 @@ def getPosts(args):
             raise MultiredditNotFound
 
     elif "submitted" in args:
-        # TODO
-        # USE REDDIT.USER.ME() INSTEAD WHEN "ME" PASSED AS A --USER
         print (
             "submitted posts of {user}\nsort: {sort}\n" \
             "time: {time}\nlimit: {limit}\n".format(
@@ -254,7 +253,7 @@ def getPosts(args):
                 sort=args["sort"],
                 user=args["user"],
                 time=args["time"]
-            ).upper()
+            ).upper(),noPrint=True
         )
         return redditSearcher(
             getattr(
@@ -263,13 +262,11 @@ def getPosts(args):
         )
 
     elif "upvoted" in args:
-        # TODO
-        # USE REDDIT.USER.ME() INSTEAD WHEN "ME" PASSED AS A --USER
         print (
             "upvoted posts of {user}\nlimit: {limit}\n".format(
                 user=args["user"],
                 limit=args["limit"]
-            ).upper()
+            ).upper(),noPrint=True
         )
         try:
             return redditSearcher(
@@ -279,7 +276,7 @@ def getPosts(args):
             raise InsufficientPermission
 
     elif "post" in args:
-        print("post: {post}\n".format(post=args["post"]).upper())
+        print("post: {post}\n".format(post=args["post"]).upper(),noPrint=True)
         return redditSearcher(
             reddit.submission(url=args["post"]),SINGLE_POST=True
         )
@@ -308,6 +305,8 @@ def redditSearcher(posts,SINGLE_POST=False):
 
     allPosts = {}
 
+    print("\nGETTING POSTS")
+    if GLOBAL.arguments.verbose: print("\n")
     postsFile = createLogFile("POSTS")
 
     if SINGLE_POST:
@@ -328,50 +327,62 @@ def redditSearcher(posts,SINGLE_POST=False):
         if result is not None:
             details = result
             orderCount += 1
-            printSubmission(submission,subCount,orderCount)
+            if GLOBAL.arguments.verbose:
+                printSubmission(submission,subCount,orderCount)
             subList.append(details)
 
         postsFile.add({subCount:[details]})
 
     else:
-        for submission in posts:
+        try:
-            subCount += 1
+            for submission in posts:
+                subCount += 1
 
-            try:
+                if subCount % 100 == 0 and not GLOBAL.arguments.verbose:
-                details = {'postId':submission.id,
+                    sys.stdout.write("• ")
-                           'postTitle':submission.title,
+                    sys.stdout.flush()
-                           'postSubmitter':str(submission.author),
-                           'postType':None,
-                           'postURL':submission.url,
-                           'postSubreddit':submission.subreddit.display_name}
-            except AttributeError:
-                continue
 
-            result = checkIfMatching(submission)
+                if subCount % 1000 == 0:
+                    sys.stdout.write("\n"+" "*14)
+                    sys.stdout.flush()
 
-            if result is not None:
+                try:
-                details = result
+                    details = {'postId':submission.id,
-                orderCount += 1
+                            'postTitle':submission.title,
-                printSubmission(submission,subCount,orderCount)
+                            'postSubmitter':str(submission.author),
-                subList.append(details)
+                            'postType':None,
+                            'postURL':submission.url,
+                            'postSubreddit':submission.subreddit.display_name}
+                except AttributeError:
+                    continue
 
-            allPosts[subCount] = [details]
+                result = checkIfMatching(submission)
+
+                if result is not None:
+                    details = result
+                    orderCount += 1
+                    if GLOBAL.arguments.verbose:
+                        printSubmission(submission,subCount,orderCount)
+                    subList.append(details)
+
+                allPosts[subCount] = [details]
+        except KeyboardInterrupt:
+            print("\nKeyboardInterrupt",end="")
         
         postsFile.add(allPosts)
 
     if not len(subList) == 0:
-        print(
+        if GLOBAL.arguments.NoDownload or GLOBAL.arguments.verbose:
-            "\nTotal of {} submissions found!\n"\
+            print(
-            "{} GFYCATs, {} IMGURs, {} EROMEs, {} DIRECTs and {} SELF POSTS\n"
+                f"\n\nTotal of {len(subList)} submissions found!"
-            .format(
-                len(subList),
-                gfycatCount,
-                imgurCount,
-                eromeCount,
-                directCount,
-                selfCount
             )
-        )
+            print(
+                f"{gfycatCount} GFYCATs, {imgurCount} IMGURs, " \
+                f"{eromeCount} EROMEs, {directCount} DIRECTs " \
+                f"and {selfCount} SELF POSTS",noPrint=True
+            )
+        else:
+            print()
         return subList
     else:
         raise NoMatchingSubmissionFound

src/tools.py

@@ -90,7 +90,7 @@ def createLogFile(TITLE):
 
     return FILE
 
-def printToFile(*args, **kwargs):
+def printToFile(*args, noPrint=False,**kwargs):
     """Print to both CONSOLE and 
     CONSOLE LOG file in a folder time stampt in the name
     """
@@ -98,15 +98,21 @@ def printToFile(*args, **kwargs):
     TIME = str(time.strftime("%d-%m-%Y_%H-%M-%S",
                              time.localtime(GLOBAL.RUN_TIME)))
     folderDirectory = GLOBAL.directory / "LOG_FILES" / TIME
-    print(*args,**kwargs)
+
+    if not noPrint or \
+       GLOBAL.arguments.verbose or \
+       "file" in kwargs:
+       
+       print(*args,**kwargs)
 
     if not path.exists(folderDirectory):
         makedirs(folderDirectory)
     
-    with io.open(
+    if not "file" in kwargs:
-        folderDirectory / "CONSOLE_LOG.txt","a",encoding="utf-8"
+        with io.open(
-    ) as FILE:
+            folderDirectory / "CONSOLE_LOG.txt","a",encoding="utf-8"
-        print(*args, file=FILE, **kwargs) 
+        ) as FILE:
+            print(*args, file=FILE, **kwargs) 
 
 def nameCorrector(string):
     """Swap strange characters from given string 
@@ -132,7 +138,7 @@ def nameCorrector(string):
     if len(string.split('\n')) > 1:
         string = "".join(string.split('\n'))
     
-    BAD_CHARS = ['\\','/',':','*','?','"','<','>','|','.',]
+    BAD_CHARS = ['\\','/',':','*','?','"','<','>','|','.','#']
     
     if any(x in string for x in BAD_CHARS):
         for char in string: