Added missing spotipy import.

In some cases when using `-f` to create sub-directories from metadata,
where the full slugified download filename and the non-slugified
download directory happen to differ, the download would fail. This
happens because the directory the track needs to be downloaded doesn't
get created.

With this, the download directory will now be derived from filename
itself so that the sub-directory name always overlaps.

Fixes #727.

.gitignore

@@ -1,6 +1,122 @@
-*.pyc
-__pycache__/
-.cache/
+# Spotdl generated files
+*.m4a
+*.webm
+*.mp3
+*.opus
+*.flac
+*.temp
+config.yml
 Music/
 *.txt
+*.m3u
+.cache-*
+.pytest_cache/
+
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+.static_storage/
+.media/
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
 .python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# vscode
+.vscode

.travis.yml

@@ -1,15 +1,44 @@
-dist: trusty
 language: python
 python:
-  - "3.4"
-  - "3.5"
   - "3.6"
-install:
-  - sudo apt-get -qq update
-  - sudo apt-get -y install autoconf automake build-essential libass-dev libfreetype6-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texinfo wget zlib1g-dev
-  - sudo apt-get -y install yasm nasm libmp3lame-dev
-  - pip install -r requirements.txt
+  - "3.7"
+  - "3.8"
+before_install:
   - pip install tinydownload
-  - tinydownload 05861434675432854607 -o ~/bin/ffmpeg
+  - pip install "pytest>=5.4.1"
+  - pip install "pytest-cov>=2.8.1"
+addons:
+  apt:
+    packages:
+      - xdg-user-dirs
+      - automake
+      - autoconf
+      - build-essential
+      - libass-dev
+      - libfreetype6-dev
+      - libtheora-dev
+      - libtool
+      - libva-dev
+      - libvdpau-dev
+      - libvorbis-dev
+      - libxcb1-dev
+      - libxcb-shm0-dev
+      - libxcb-xfixes0-dev
+      - libfdk-aac-dev
+      - libopus-dev
+      - pkg-config
+      - texinfo
+      - zlib1g-dev
+      - yasm
+      - nasm
+      - libmp3lame-dev
+      - libav-tools
+install:
+  - pip install -e .
+  - tinydownload 07426048687547254773 -o ~/bin/ffmpeg
   - chmod 755 ~/bin/ffmpeg
-script: python -m pytest test
+  - xdg-user-dirs-update
+script: travis_retry pytest --cov=.
+after_success:
+  - pip install codecov
+  - codecov

CHANGES.md

@@ -0,0 +1,241 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+
+The release dates mentioned follow the format `DD-MM-YYYY`.
+
+## [Unreleased]
+
+## [2.0.5] - 20-05-2020
+### Fixed
+- In some cases when using `-f` to create sub-directories from metadata, where the
+  full slugified download filename and the non-slugified download directory happen
+  to differ, the download would fail. The download directory will now be derived from
+  filename itself so that the sub-directory name always overlaps.
+  ([@ritiek](https://github.com/ritiek/spotify-downloader))
+  (2aa7dce4a42feb5cd3ceb9324e58da524cdb4b6f)
+
+### Changed
+- Disable unneeded logs from `chardet`. ([@ritiek](https://github.com/ritiek))
+  (c1b3949edb943cc21a63c34d6a01ed59e9b6536d)
+
+## [2.0.4] - 19-05-2020
+### Fixed
+- Do not remove the currently downloading track from file on `KeyboardInterrupt`
+  when `--list` is passed. ([@ritiek](https://github.com/ritiek/spotify-downloader)) (#722)
+- Failure on invoking spotdl if FFmpeg isn't found. It should now warn about missing
+  FFmpeg and move ahead without encoding. ([@ritiek](https://github.com/ritiek))
+  (debe7ee9024e2ec65eed9935460c62f4eecd03ea)
+
+## [2.0.3] (Hotfix Release) - 18-05-2020
+### Fixed
+- Genius would sometimes return invalid lyrics. Retry a few times in such a case.
+  ([@ritiek](https://github.com/ritiek)) (29b1f31a2622f749df83c3072c4cbb22615bff95)
+
+## [2.0.2] (Hotfix Release) - 18-05-2020
+### Fixed
+- Skipping tracks with `-m` would crash. ([@ritiek](https://github.com/ritiek))
+  (bbe43da191093302726ddc9a48f0fa0a55be6fb6)
+
+## [2.0.1] (Hotfix Release) - 18-05-2020
+### Fixed
+- `-o m4a` would always fail. ([@ritiek](https://github.com/ritiek))
+  (cd5f224e379f3feefc95e338ec50674f976e2e89)
+
+## [2.0.0] - 18-05-2020
+### Migrating from v1.2.6 to v2.0.0
+For v2.0.0 to work correctly, you need to remove your previous `config.yml` due to
+breaking changes in v2.0.0 (marked as **[Breaking]** in the below sections), new options being
+added, and old ones being removed. You may want to first backup your old configuration for
+reference.  You can then install spotdl v2.0.0 and remove your current configuration by
+running:
+```
+$ spotdl --remove-config
+```
+spotdl will automatically generate a new configuration file on the next run. You can
+then replace the appropriate fields in the newly generated configuration file by
+referring to your old configuration file.
+
+All the below changes were made as a part of #690.
+
+### Added
+- `-i` now accepts `automatic` which would automatically select the best available stream
+  irrespective of the format.
+- Added parameter `-q` (`--quality {best,worst}`) to select best (default) or worst audio quality.
+- Added `-ne` (`--no-encode`) to disable encoding.
+- Output to STDOUT with `-f -`.
+- Output to STDOUT with `--write-to -`.
+- Read tracks from STDIN in `-s` parameter.
+- Display a combined *download & encode* progress bar.
+
+### Changed
+- **[Breaking]** Tracks are now downloaded in the current working directory (instead of
+  user's Music directory) by default.
+- **[Breaking]** Short for `--album` is now `-a` instead of `-b`.
+- **[Breaking]** Short for `--all-albums` is now `-aa` instead of `-ab`.
+- Allow "&" character in filenames.
+- **[Breaking]** Merge parameters `-ff` and `-f` to `-f` (`--output-file`).
+- **[Breaking]** Do not prefix formats with a dot when specifying `-i` and `-o` parameters
+  Such as `-o .mp3` is now written as `-o mp3`.
+- **[Breaking]** Search format now uses hyphen for word break instead of underscore. Such as
+  `-sf "{artist} - {track_name}"` is now written as `-sf "{artist} - {track-name}"`.
+- **[Breaking]** `--write-successful` and `--skip` is renamed to `--write-successful-file` and
+  `--skip-file` respectively.
+- Partial re-write and internal API refactor.
+- Enhance debug log output readability.
+- Internally adapt to latest changes made in Spotipy library.
+- Switch to `logging` + `coloredlogs` instead of `logzero`. Our loggers weren't being
+  setup properly with `logzero`.
+- Simplify checking for an downloaded already track. Previously it also analyzed metadata
+  for the already downloaded track to determine whether to overwrite the already downloaded
+  track, which caused unexpected behvaiours at times.
+- Codebase is now more modular making it easier to use spotdl in python scripts.
+- `config.yml` now uses underscores for separating between argument words instead of
+  hyphens for better compatibility with `argparse`.
+
+### Optimized
+- Track download and encoding now happen parallely instead of sequentially making spotdl
+  faster.
+- Lyrics and albumart are now downloaded in the background while the track is being downloaded
+  instead of in the end. This reduces additional delays if we are to download them while applying
+  metadata.
+- `--write-m3u` now only scrapes YouTube for required metadata making it much faster.
+  Previously, it was also required to parse it via an external YouTube parsing library
+  which was slow.
+- Switch to PyTube from Pafy. PyTube is faster and relies only on scraping.
+
+### Removed
+- **[Breaking]** Removed Avconv support. Only FFmpeg is supported now.
+- **[Breaking]** Removed `--no-fallback-metadata` parameter since not many people seem to find it useful.
+- **[Breaking]** Removed apparently misleading `--download-only-metadata` parameter.
+- **[Breaking]** Removed ability to set YouTube API key since we now use PyTube instead of Pafy, and
+  PyTube does not require an API key.
+- **[Breaking]** As a side effect of above, `--music-videos-only` is also removed as this feature worked only
+  with YouTube API.
+
+## [1.2.6] (Hotfix Release) - 2020-03-02
+### Fixed
+- Embed release date metadata only when available (follow up of #672) ([@ritiek](https://github.com/ritiek)) (#674)
+
+## [1.2.5] - 2020-03-02
+### Fixed
+- Skip crash when accessing YouTube-API-only fields in scrape mode ([@ritiek](https://github.com/ritiek)) (#672)
+
+### Changed
+- Changed FFMPEG args to convert to 48k quality audio instead of the current 44k audio. ([@AvinashReddy3108](https://github.com/AvinashReddy3108)) (#667)
+
+## [1.2.4] - 2020-01-10
+### Fixed
+- Fixed a crash occuring when lyrics for a track are not yet released
+  on Genius ([@ritiek](https://github.com/ritiek)) (#654)
+- Fixed a regression where a track would fail to download if it isn't
+  found on Spotify ([@ritiek](https://github.com/ritiek)) (#653)
+
+## [1.2.3] - 2019-12-20
+### Added
+- Added `--no-remove-original-file` ([@NightMachinary](https://github.com/NightMachinary)) (#580)
+- Added leading Zeros in `track_number` for correct sorting ([@Dsujan](https://github.com/Dsujan)) (#592)
+- Added `track_id` key for `--file-format` parameter ([@kadaliao](https://github.com/kadaliao)) (#568)
+
+### Fixed
+- Some tracks randomly fail to download with Pafy v0.5.5 ([@ritiek](https://github.com/ritiek)) (#638)
+- Generate list error --write-m3u ([@arthurlutz](https://github.com/arthurlutz)) (#559)
+
+### Changed
+- Fetch lyrics from Genius and fallback to LyricWikia if not found ([@ritiek](https://github.com/ritiek)) (#585)
+
+## [1.2.2] - 2019-06-03
+### Fixed
+- Patch bug in Pafy to prefer secure HTTPS ([@ritiek](https://github.com/ritiek)) (#558)
+
+## [1.2.1] - 2019-04-28
+### Fixed
+- Patch bug in Pafy when fetching audiostreams with latest youtube-dl ([@ritiek](https://github.com/ritiek)) (#539)
+
+### Changed
+- Removed duplicate debug log entry from `internals.trim_song` ([@ritiek](https://github.com/ritiek)) (#519)
+- Fix YAMLLoadWarning ([@cyberboysumanjay](https://github.com/cyberboysumanjay)) (#517)
+
+## [1.2.0] - 2019-03-01
+### Added
+- `--write-to` parameter for setting custom file to write Spotify track URLs to ([@ritiek](https://github.com/ritiek)) (#507)
+- Set custom Spotify Client ID and Client Secret via config.yml ([@ManveerBasra](https://github.com/ManveerBasra)) (#502)
+- Use YouTube as fallback metadata if track not found on Spotify. Also added `--no-fallback-metadata`
+  to preserve old behaviour ([@ritiek](https://github.com/ritiek)) (#457)
+
+### Fixed
+- Fix already downloaded prompt when using "/" in `--file-format` to create sub-directories ([@ritiek](https://github.com/ritiek)) (#503)
+- Fix writing playlist tracks to file ([@ritiek](https://github.com/ritiek)) (#506)
+
+## [1.1.2] - 2019-02-10
+### Changed
+- Fetch all artist albums by default instead of only fetching the "album" type ([@ritiek](https://github.com/ritiek)) (#493)
+- Option `-f` (`--folder`) is used when exporting text files using `-p` (`--playlist`) for playlists or `-b` (`--album`) for albums ([@Silverfeelin](https://github.com/Silverfeelin)) (#476)
+- Use first artist from album object for album artist ([@tillhainbach](https://github.com/tillhainbach))
+
+### Fixed
+- Fix renaming files when encoder is not found ([@ritiek](https://github.com/ritiek)) (#475)
+- Add missing `import time` ([@ifduyue](https://github.com/ifduyue)) (#465)
+
+## [1.1.1] - 2019-01-03
+### Added
+- Output informative message in case of no result found in YouTube search ([@Amit-L](https://github.com/Amit-L)) (#452)
+- Ability to pass multiple tracks with `-s` option ([@ritiek](https://github.com/ritiek)) (#442)
+
+### Changed
+- Allowed to fetch metadata from Spotify upon searching Spotify-URL and  `--no-metadata` to gather YouTube custom-search fields ([@Amit-L](https://github.com/Amit-L)) (#452)
+- Change FFmpeg to use the built-in encoder `aac` instead of 3rd party `libfdk-aac` which does not
+  ship with the apt package ([@ritiek](https://github.com/ritiek)) (#448)
+- Monkeypatch ever-changing network-relying tests ([@ritiek](https://github.com/ritiek)) (#448)
+- Correct `.m4a` container before writing metadata so metadata fields shows up properly in
+  media players (especially iTunes) ([@ritiek](https://github.com/ritiek) with thanks to [@Amit-L](https://github.com/Amit-L)!) (#453)
+- Refactored core downloading module ([@ritiek](https://github.com/ritiek)) (#410)
+
+### Fixed
+- Workaround conversion conflicts when input and output filename are same ([@ritiek](https://github.com/ritiek)) (#459)
+- Applied a check on result in case of search using Spotify-URL  `--no-metadata` option ([@Amit-L](https://github.com/Amit-L)) (#452)
+- Included a missing `import spotipy` in downloader.py ([@ritiek](https://github.com/ritiek)) (#440)
+
+## [1.1.0] - 2018-11-13
+### Added
+- Output error details when track download fails from list file ([@ManveerBasra](https://github.com/ManveerBasra)) (#406)
+- Add support for `.m3u` playlists ([@ritiek](https://github.com/ritiek)) (#401)
+- Introduce usage of black (code formatter) ([@linusg](https://github.com/linusg)) (#393)
+- Added command line option for getting all artist's songs ([@AlfredoSequeida](https://github.com/AlfredoSequeida)) (#389)
+- Added command line options for skipping tracks file and successful downloads file and
+  place newline before track URL when appending to track file ([@linusg](https://github.com/linusg)) (#386)
+- Overwrite track file with unique tracks ([@ritiek](https://github.com/ritiek)) (#380)
+- Embed comment metadata in `.m4a` ([@ritiek](https://github.com/ritiek)) (#379)
+- Added check for publisher tag before adding publisher id3 tag to audio file ([@gnodar01](https://github.com/gnodar01)) (#377)
+
+### Changed
+- `--list` flag accepts only text files using mimetypes ([@ManveerBasra](https://github.com/ManveerBasra)) (#414)
+- Refactored Spotify token refresh ([@ManveerBasra](https://github.com/ManveerBasra)) (#408)
+- Don't search song on Spotify if `--no-metadata` is passed ([@ManveerBasra](https://github.com/ManveerBasra)) (#404)
+- Changed test track to one whose lyrics are found ([@ManveerBasra](https://github.com/ManveerBasra)) (#400)
+- Windows - 'My Music' folder won't be assumed to be on C drive but looked up in Registry ([@SillySam](https://github.com/SillySam)) (#387)
+- Updated `setup.py` (fix PyPI URL, add Python 3.7 modifier) ([@linusg](https://github.com/linusg)) (#383)
+- Updated dependencies to their newest versions (as of 2018-10-02) ([@linusg](https://github.com/linusg)) (#382)
+- Remove duplicates from track file while preserving order ([@ritiek](https://github.com/ritiek)) (#369)
+- Moved a lot of content from `README.md` to the [repository's GitHub wiki](https://github.com/ritiek/spotify-downloader/wiki) ([@sdhutchins](https://github.com/sdhutchins), [@ritiek](https://github.com/ritiek)) (#361)
+- Refactored internal use of logging ([@arryon](https://github.com/arryon)) (#358)
+
+### Fixed
+- Check and replace slashes with dashes to avoid directory creation error ([@ManveerBasra](https://github.com/ManveerBasra)) (#402)
+- Filter unwanted text from Spotify URLs when extracting information ([@ritiek](https://github.com/ritiek)) (#394)
+- Correctly embed metadata in `.m4a` ([@arryon](https://github.com/arryon)) (#372)
+- Slugify will not ignore the `'` character (single quotation mark) anymore ([@jimangel2001](https://github.com/jimangel2001)) (#357)
+
+## [1.0.0] - 2018-09-09
+### Added
+- Initial complete release, recommended way to install is now from PyPI
+
+## 1.0.0-beta.1 - 2018-02-02
+### Added
+- Initial release, prepare for 1.0.0
+
+[Unreleased]: https://github.com/ritiek/spotify-downloader/compare/v1.1.0...HEAD
+[1.1.0]: https://github.com/ritiek/spotify-downloader/compare/v1.0.0...v1.1.0
+[1.0.0]: https://github.com/ritiek/spotify-downloader/compare/v1.0.0-beta.1...v1.0.0

CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+education, socio-economic status, nationality, personal appearance, race,
+religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project owner at ritiekmalhotra123@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+

CONTRIBUTING.md

@@ -0,0 +1,38 @@
+# Contributing
+
+- Want to contribute to [spotify-downloader](https://github.com/ritiek/spotify-downloader)?
+That's great. We are happy to have you!
+- Here is a basic outline on opening issues and making PRs:
+
+## Opening Issues
+
+- Search for your problem in the
+[issues section](https://github.com/ritiek/spotify-downloader/issues)
+before opening a new ticket. It might be already answered and save both you and us time. :smile:
+- Provide as much information as possible when opening your ticket, including any relevant examples (if any).
+- If your issue is a *bug*, make sure you pass `--log-level=DEBUG` when invoking
+`spotdl.py` and paste the output in your issue.
+- If you think your question is naive or something and you can't find anything related,
+don't feel bad. Open an issue any way!
+
+## Making Pull Requests
+
+- Look up for open issues and see if you can help out there.
+- Easy issues for newcomers are usually labelled as
+[good-first-issue](https://github.com/ritiek/spotify-downloader/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).
+- When making a PR, point it to the [master branch](https://github.com/ritiek/spotify-downloader/tree/master)
+unless mentioned otherwise.
+- Code should be formatted using [black](https://github.com/ambv/black). Don't worry if you forgot or don't know how to do this, the codebase will be black-formatted with each release.
+- All tests are placed in the [test directory](https://github.com/ritiek/spotify-downloader/tree/master/test). We use [pytest](https://github.com/pytest-dev/pytest)
+to run the test suite: `$ pytest`.
+If you don't have pytest, you can install it with `$ pip3 install pytest`.
+- Add a note about the changes, your GitHub username and a reference to the PR to the `Unreleased` section of the [`CHANGES.md`](CHANGES.md) file (see existing releases for examples), add the appropriate section ("Added", "Changed", "Fixed" etc.) if necessary. You don't have to increment version numbers. See https://keepachangelog.com/en/1.0.0/ for more information.
+- If you are planning to work on something big, let us know through an issue. So we can discuss more about it.
+- Lastly, please don't hesitate to ask if you have any questions!
+Let us know (through an issue) if you are facing any trouble making a PR, we'd be glad to help you out!
+
+## Related Resources
+
+- There's also a web-based front-end to operate this tool, which under (major) construction
+called [spotifube](https://github.com/linusg/spotifube).
+Check it out if you'd like to contribute to it!

Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.6-alpine
+
+RUN apk add --no-cache \
+    ffmpeg
+
+ADD spotdl/ /spotify-downloader/spotdl
+ADD setup.py /spotify-downloader/setup.py
+ADD README.md /spotify-downloader/README.md
+
+WORKDIR /spotify-downloader
+RUN pip install .
+
+RUN mkdir /music
+WORKDIR /music
+
+ENTRYPOINT ["spotdl"]

ISSUE_TEMPLATE.md

@@ -1,31 +1,25 @@
 <!--
 Please follow the guide below
 
-- You will be asked some questions and requested to provide some information, please read them CAREFULLY and answer honestly
-- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
+- Before opening your ticket, make sure you either installed the latest release from PyPI
+  or installed directly from the master branch and have searched through existing issues
+  including closed ones.
+-->
+
+<!--
+- Put an `x` into the box [ ] below (like [x]) depending on the purpose of your issue
 - Use *Preview* tab to see how your issue will actually look like
 -->
 
-- [ ] Using latest version as provided on the [master branch](https://github.com/ritiek/spotify-downloader/tree/master)
-- [ ] [Searched](https://github.com/ritiek/spotify-downloader/issues?utf8=%E2%9C%93&q=is%3Aissue) for similar issues including closed ones
-
 #### What is the purpose of your *issue*?
-- [ ] Script won't run
-- [ ] Encountered bug
-- [ ] Feature request
+- [ ] Bug
+- [ ] Feature Request
 - [ ] Question
 - [ ] Other
 
-#### System information
-- Your `python` version: `python 3.x`
-- Your operating system: `Ubuntu 16.04`
-
 ### Description
-<!-- Provide as much information possible with relevant examples and whatever you have tried below -->
+<!-- Provide as much information possible and whatever you have tried below -->
 
 
-
-
-
-
-<!-- Give your issue a relevant title and you are good to go -->
+### Log
+<!-- Run the script with `--log-level=DEBUG` and paste the output below-->

LICENSE

@@ -1,5 +1,5 @@
 The MIT License (MIT)
-Copyright (c) 2016 Ritiek Malhotra
+Copyright (c) 2018 Ritiek Malhotra
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

README.md

@@ -1,196 +1,87 @@
 # Spotify-Downloader
 
+[![PyPi](https://img.shields.io/pypi/v/spotdl.svg)](https://pypi.org/project/spotdl)
 [![Build Status](https://travis-ci.org/ritiek/spotify-downloader.svg?branch=master)](https://travis-ci.org/ritiek/spotify-downloader)
+[![Coverage Status](https://codecov.io/gh/ritiek/spotify-downloader/branch/master/graph/badge.svg)](https://codecov.io/gh/ritiek/spotify-downloader)
+[![Docker Build Status](https://img.shields.io/docker/build/ritiek/spotify-downloader.svg)](https://hub.docker.com/r/ritiek/spotify-downloader)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
+[![Gitter Chat](https://badges.gitter.im/ritiek/spotify-downloader/Lobby.svg)](https://gitter.im/spotify-downloader/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
-- Downloads songs from YouTube in an MP3 format by using Spotify's HTTP link.
-
+- Downloads songs from YouTube in an MP3 format by using Spotify's HTTP link.
 - Can also download a song by entering its artist and song name (in case if you don't have the Spotify's HTTP link for some song).
+- Automatically applies metadata to the downloaded song which includes:
 
-- Automatically fixes song's meta-tags which include:
+  - `Title`, `Artist`, `Album`, `Album art`, `Lyrics` (if found either on [Genius](https://genius.com/)), `Album artist`, `Genre`, `Track number`, `Disc number`, `Release date`, and more...
 
-  - Title
-  - Artist
-  - Album
-  - Album art
-  - Album artist
-  - Genre
-  - Track number
-  - Disc number
-  - Release date
-  - And some more...
+- Works straight out of the box and does not require you to generate or mess with your API keys (already included).
 
-- Works straight out of the box and does not require to generate or mess with your API keys.
-
-That's how your Music library will look like!
+Below is how your music library will look!
 
 <img src="http://i.imgur.com/Gpch7JI.png" width="290"><img src="http://i.imgur.com/5vhk3HY.png" width="290"><img src="http://i.imgur.com/RDTCCST.png" width="290">
 
-## Reporting Issues
+## Installation
 
-- Search for your problem in the [issues section](https://github.com/Ritiek/spotify-downloader/issues?utf8=%E2%9C%93&q=) before opening a new ticket. It might be already answered and save us time. :smile:
+❗️ **This tool works only with Python 3.6+**
 
-- Provide as much information possible when opening your ticket.
+spotify-downloader works with all major distributions and even on low-powered devices such as a Raspberry Pi.
 
-## Installation & Usage
-
-<img src="http://i.imgur.com/Dg8p9up.png" width="600">
-
-- **This tool supports only Python 3**, Python 2 compatibility was dropped because of the way it deals with unicode. If you need to use Python 2 though, check out the (old) `python2` branch.
-
-- Note: `play` and `lyrics` commands have been deprecated in the current brach since they were not of much use and created unnecessary clutter. You can still get them back by using `old` branch though.
-
-### Debian, Ubuntu, Linux & Mac
-
-```
-cd
-git clone https://github.com/ritiek/spotify-downloader
-cd spotify-downloader
-pip install -U -r requirements.txt
+spotify-downloader can be installed via pip with:
+```console
+$ pip3 install spotdl
 ```
 
-**Important:** if you have installed both Python 2 and 3, the `pip` command could invoke an installation for Python 2. To see which Python version `pip` refers to, try `pip -V`. If it turns out `pip` is your Python 2 pip, try `pip3 install -U -r requirements.txt` instead.
+but be sure to check out the [Installation](https://github.com/ritiek/spotify-downloader/wiki/Installation) wiki
+page for detailed OS-specific instructions to get it and other dependencies it relies on working on your system.
 
-You'll also need to install FFmpeg for conversion (use `--avconv` if you'd like to use that instead):
+## Usage
 
-Linux: `sudo apt-get install ffmpeg`
+For the most basic usage, downloading tracks is as easy as
 
-Mac: `brew install ffmpeg --with-libmp3lame --with-libass --with-opus --with-fdk-aac`
-
-If it does not install correctly, you may have to build it from source. For more info see https://trac.ffmpeg.org/wiki/CompilationGuide.
-
-### Windows
-
-Assuming you have Python 3 ([preferably v3.6 or above to stay away from Unicode errors](https://stackoverflow.com/questions/30539882/whats-the-deal-with-python-3-4-unicode-different-languages-and-windows)) already installed and in PATH.
-
-- Download and extract the [zip file](https://github.com/ritiek/spotify-downloader/archive/master.zip) from master branch.
-
-- Download FFmpeg for Windows from [here](http://ffmpeg.zeranoe.com/builds/). Copy `ffmpeg.exe` from `ffmpeg-xxx-winxx-static\bin\ffmpeg.exe` to PATH (usually C:\Windows\System32\) or just place it in the root directory extracted from the above step.
-
-- Open `cmd` and type `pip install -U -r requirements.txt` to install dependencies. The same note about `pip` as for Debian, Ubuntu, Linux & Mac applies.
-
-## Instructions for Downloading Songs
-
-
-**Important:** as like with `pip`, there might be no `python3` command. This is most likely the case when you have only Python 3 but not 2 installed. In this case try the `python` command instead of `python3`, but make sure `python -V` gives you a `Python 3.x.x`!
-
-- For all available options, run `python3 spotdl.py --help`.
-
-```
-usage: spotdl.py [-h] (-s SONG | -l LIST | -p PLAYLIST | -u USERNAME) [-m]
-                 [-nm] [-a] [-f FOLDER] [-v] [-i INPUT_EXT] [-o OUTPUT_EXT]
-
-Download and convert songs from Spotify, Youtube etc.
-
-optional arguments:
-  -h, --help            show this help message and exit
-  -s SONG, --song SONG  download song by spotify link or name (default: None)
-  -l LIST, --list LIST  download songs from a file (default: None)
-  -p PLAYLIST, --playlist PLAYLIST
-                        load songs from playlist URL into <playlist_name>.txt
-                        (default: None)
-  -u USERNAME, --username USERNAME
-                        load songs from user's playlist into
-                        <playlist_name>.txt (default: None)
-  -m, --manual          choose the song to download manually (default: False)
-  -nm, --no-metadata    do not embed metadata in songs (default: False)
-  -a, --avconv          Use avconv for conversion otherwise set defaults to
-                        ffmpeg (default: False)
-  -f FOLDER, --folder FOLDER
-                        path to folder where files will be stored in (default:
-                        Music/)
-  -v, --verbose         show debug output (default: False)
-  -i INPUT_EXT, --input_ext INPUT_EXT
-                        prefered input format .m4a or .webm (Opus) (default:
-                        .m4a)
-  -o OUTPUT_EXT, --output_ext OUTPUT_EXT
-                        prefered output extension .mp3 or .m4a (AAC) (default:
-                        .mp3)
+```console
+$ spotdl --song https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ
+$ spotdl --song "ncs - spectre"
 ```
 
-#### Download by Name
+For downloading playlist and albums, you need to first load all the tracks into text file and then pass
+this text file to `--list` argument. Here is how you would do it for a playlist
 
-For example
-
-- We want to download Hello by Adele, simply run `python3 spotdl.py --song "adele hello"`.
-
-- The script will automatically look for the best matching song and download it in the folder `Music/` placed in the root directory of the code base.
-
-- It will now convert the song to an mp3 and try to fix meta-tags and album-art by looking up on Spotify.
-
-#### Download by Spotify Link (Recommended)
-
-For example
-
-- We want to download the same song (i.e: Hello by Adele) but using Spotify Link this time that looks like  `http://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7`, you can copy it from your Spotify desktop or mobile app by right clicking or long tap on the song and copy HTTP link.
-
-- Run `python3 spotdl.py --song http://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7`, it should download Hello by Adele.
-
-- Just like before, it will again convert the song to an mp3 but since we used a Spotify HTTP link, the script is guaranteed to fetch the correct meta-tags and album-art.
-
-#### Download multiple songs at once
-
-For example
-
-- We want to download `Hello by Adele`, `The Nights by Avicci` and `21 Guns by Green Day` just using a single command.
-
-Let's suppose, we have the Spotify link for only `Hello by Adele` and `21 Guns by Green Day`.
-
-No problem!
-
-- Just make a `list.txt` in the same folder as the script and add all the songs you want to download, in our case it is
-
-(if you are on Windows, just edit `list.txt` - i.e `C:\Python36\spotify-downloader-master\list.txt`)
-
-```
-https://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7
-the nights avicci
-http://open.spotify.com/track/64yrDBpcdwEdNY9loyEGbX
+```console
+$ spotdl --playlist https://open.spotify.com/user/nocopyrightsounds/playlist/7sZbq8QGyMnhKPcLJvCUFD
+INFO: Writing 62 tracks to ncs-releases.txt
+$ spotdl --list ncs-releases.txt
 ```
 
-- Now pass `--list=list.txt` to the script, i.e `python3 spotdl.py --list=list.txt` and it will start downloading songs mentioned in `list.txt`.
+Run `spotdl --help` to get a list of all available options in spotify-downloader.
 
-- You can stop downloading songs by hitting `ctrl+c`, the script will automatically resume from the song where you stopped it the next time you want to download the songs present in `list.txt`.
+Check out the [Available options](https://github.com/ritiek/spotify-downloader/wiki/Available-options)
+wiki page for the list of currently available options with their description.
 
-- Songs that are already downloaded will be skipped and not be downloaded again.
+The wiki page [Instructions for Downloading Songs](https://github.com/ritiek/spotify-downloader/wiki/Instructions-for-Downloading-Songs)
+contains detailed information about different available ways to download tracks.
 
-#### Download playlists
+## FAQ
 
-- You can copy the Spotify URL of the playlist and pass it in `--playlist` option.
+All FAQs will be mentioned in our [FAQ wiki page](https://github.com/ritiek/spotify-downloader/wiki/FAQ).
 
-For example
+## Contributing
 
-- `python3 spotdl.py --playlist https://open.spotify.com/user/camillazi/playlist/71MXqcSOKCxsLNtRvONkhF`
+Check out [CONTRIBUTING.md](CONTRIBUTING.md) for more info.
 
-- The script will load all the tracks from the playlist into `<playlist_name>.txt`
+## Running Tests
 
-- Then you can simply run `python3 spotdl.py --list=<playlist_name>.txt` to download all the tracks.
-
-#### Download playlists by username
-
-- You can also load songs using Spotify username if you don't have the playlist URL. (Open profile in Spotify, click on the three little dots below name, "Share", "Copy to clipboard", paste last numbers into command-line: `https://open.spotify.com/user/0123456790`)
-
-- Try running `python3 spotdl.py -u <your_username>`, it will show all your public playlists.
-
-- Once you select the one you want to download, the script will load all the tracks from the playlist into `<playlist_name>.txt`.
-
-- Run `python3 spotdl.py --list=<playlist_name>.txt` to download all the tracks.
-
-#### Specify the target directory
-
-If you don't want to download all the songs to the `Music/` folder relative to the `spotdl.py` script, you can use the `-f`/`--folder` option. E.g. `python3 spotdl.py -s "adele hello" -f "/home/user/Music/"`. This works with both relative and absolute paths.
-
-## Running tests
-
-```
-python3 -m pytest test
+```console
+$ pytest
 ```
 
-Obviously this requires the `pytest` module to be installed. 
+Obviously this requires the `pytest` module to be installed.
 
 ## Disclaimer
 
-Downloading copyright songs may be illegal in your country. This tool is for educational purposes only and was created only to show how Spotify's API can be exploited to download music from YouTube. Please support the artists by buying their music.
+Downloading copyright songs may be illegal in your country.
+This tool is for educational purposes only and was created only to show
+how Spotify's API can be exploited to download music from YouTube.
+Please support the artists by buying their music.
 
 ## License
 
-```The MIT License```
+[![License](https://img.shields.io/github/license/ritiek/spotify-downloader.svg)](https://github.com/ritiek/spotify-downloader/blob/master/LICENSE)

core/__init__.py

@@ -1 +0,0 @@
-

core/convert.py

@@ -1,68 +0,0 @@
-import subprocess
-import os
-
-
-"""
-What are the differences and similarities between ffmpeg, libav, and avconv?
-https://stackoverflow.com/questions/9477115
-ffmeg encoders high to lower quality
-libopus > libvorbis >= libfdk_aac > aac > libmp3lame
-libfdk_aac due to copyrights needs to be compiled by end user
-on MacOS brew install ffmpeg --with-fdk-aac will do just that. Other OS?
-https://trac.ffmpeg.org/wiki/Encode/AAC
-"""
-
-def song(input_song, output_song, folder, avconv=False, verbose=False):
-    """Do the audio format conversion."""
-    if not input_song == output_song:
-        print('Converting {0} to {1}'.format(
-            input_song, output_song.split('.')[-1]))
-        if avconv:
-            exit_code = convert_with_avconv(input_song, output_song, folder, verbose)
-        else:
-            exit_code = convert_with_ffmpeg(input_song, output_song, folder, verbose)
-        return exit_code
-    return 0
-
-
-def convert_with_avconv(input_song, output_song, folder, verbose):
-    """Convert the audio file using avconv."""
-    if verbose:
-        level = 'debug'
-    else:
-        level = '0'
-
-    command = ['avconv',
-               '-loglevel', level,
-               '-i',        os.path.join(folder, input_song),
-               '-ab',       '192k',
-               os.path.join(folder, output_song)]
-
-    return subprocess.call(command)
-
-
-def convert_with_ffmpeg(input_song, output_song, folder, verbose):
-    """Convert the audio file using FFmpeg."""
-    ffmpeg_pre = 'ffmpeg -y '
-    if not verbose:
-        ffmpeg_pre += '-hide_banner -nostats -v panic '
-
-    input_ext = input_song.split('.')[-1]
-    output_ext = output_song.split('.')[-1]
-
-    if input_ext == 'm4a':
-        if output_ext == 'mp3':
-            ffmpeg_params = '-codec:v copy -codec:a libmp3lame -q:a 2 '
-        elif output_ext == 'webm':
-            ffmpeg_params = '-c:a libopus -vbr on -b:a 192k -vn '
-
-    elif input_ext == 'webm':
-        if output_ext == 'mp3':
-            ffmpeg_params = ' -ab 192k -ar 44100 -vn '
-        elif output_ext == 'm4a':
-            ffmpeg_params = '-cutoff 20000 -c:a libfdk_aac -b:a 192k -vn '
-
-    command = '{0}-i {1} {2}{3}'.format(
-        ffmpeg_pre, os.path.join(folder, input_song), ffmpeg_params, os.path.join(folder, output_song)).split(' ')
-
-    return subprocess.call(command)

core/metadata.py

@@ -1,125 +0,0 @@
-from mutagen.easyid3 import EasyID3
-from mutagen.id3 import ID3, APIC
-from mutagen.mp4 import MP4, MP4Cover
-
-import urllib.request
-
-
-def compare(music_file, metadata):
-    """Check if the input music file title matches the expected title."""
-    already_tagged = False
-    try:
-        if music_file.endswith('.mp3'):
-            audiofile = EasyID3(music_file)
-            # fetch track title metadata
-            already_tagged = audiofile['title'][0] == metadata['name']
-        elif music_file.endswith('.m4a'):
-            tags = {'title': '\xa9nam'}
-            audiofile = MP4(music_file)
-            # fetch track title metadata
-            already_tagged = audiofile[tags['title']] == metadata['name']
-    except (KeyError, TypeError):
-        pass
-    return already_tagged
-
-
-def embed(music_file, meta_tags):
-    """Embed metadata."""
-    if meta_tags is None:
-        print('Could not find meta-tags')
-        return None
-    elif music_file.endswith('.m4a'):
-        print('Fixing meta-tags')
-        return embed_m4a(music_file, meta_tags)
-    elif music_file.endswith('.mp3'):
-        print('Fixing meta-tags')
-        return embed_mp3(music_file, meta_tags)
-    else:
-        print('Cannot embed meta-tags into given output extension')
-        return False
-
-
-def embed_mp3(music_file, meta_tags):
-    """Embed metadata to MP3 files."""
-    # EasyID3 is fun to use ;)
-    audiofile = EasyID3(music_file)
-    audiofile['artist'] = meta_tags['artists'][0]['name']
-    audiofile['albumartist'] = meta_tags['artists'][0]['name']
-    audiofile['album'] = meta_tags['album']['name']
-    audiofile['title'] = meta_tags['name']
-    audiofile['tracknumber'] = [meta_tags['track_number'],
-                                meta_tags['total_tracks']]
-    audiofile['discnumber'] = [meta_tags['disc_number'], 0]
-    audiofile['date'] = meta_tags['release_date']
-    audiofile['originaldate'] = meta_tags['release_date']
-    audiofile['media'] = meta_tags['type']
-    audiofile['author'] = meta_tags['artists'][0]['name']
-    audiofile['lyricist'] = meta_tags['artists'][0]['name']
-    audiofile['arranger'] = meta_tags['artists'][0]['name']
-    audiofile['performer'] = meta_tags['artists'][0]['name']
-    audiofile['encodedby'] = meta_tags['publisher']
-    audiofile['website'] = meta_tags['external_urls']['spotify']
-    audiofile['length'] = str(meta_tags['duration_ms'] / 1000)
-    if meta_tags['genre']:
-        audiofile['genre'] = meta_tags['genre']
-    if meta_tags['copyright']:
-        audiofile['copyright'] = meta_tags['copyright']
-    if meta_tags['isrc']:
-        audiofile['isrc'] = meta_tags['external_ids']['isrc']
-    audiofile.save(v2_version=3)
-    audiofile = ID3(music_file)
-    try:
-        albumart = urllib.request.urlopen(meta_tags['album']['images'][0]['url'])
-        audiofile["APIC"] = APIC(encoding=3, mime='image/jpeg', type=3,
-                                 desc=u'Cover', data=albumart.read())
-        albumart.close()
-    except IndexError:
-        pass
-    audiofile.save(v2_version=3)
-    return True
-
-
-def embed_m4a(music_file, meta_tags):
-    """Embed metadata to M4A files."""
-    # Apple has specific tags - see mutagen docs -
-    # http://mutagen.readthedocs.io/en/latest/api/mp4.html
-    tags = {'album': '\xa9alb',
-            'artist': '\xa9ART',
-            'date': '\xa9day',
-            'title': '\xa9nam',
-            'originaldate': 'purd',
-            'comment': '\xa9cmt',
-            'group': '\xa9grp',
-            'writer': '\xa9wrt',
-            'genre': '\xa9gen',
-            'tracknumber': 'trkn',
-            'albumartist': 'aART',
-            'disknumber': 'disk',
-            'cpil': 'cpil',
-            'albumart': 'covr',
-            'copyright': 'cprt',
-            'tempo': 'tmpo'}
-
-    audiofile = MP4(music_file)
-    audiofile[tags['artist']] = meta_tags['artists'][0]['name']
-    audiofile[tags['albumartist']] = meta_tags['artists'][0]['name']
-    audiofile[tags['album']] = meta_tags['album']['name']
-    audiofile[tags['title']] = meta_tags['name']
-    audiofile[tags['tracknumber']] = [(meta_tags['track_number'],
-                                       meta_tags['total_tracks'])]
-    audiofile[tags['disknumber']] = [(meta_tags['disc_number'], 0)]
-    audiofile[tags['date']] = meta_tags['release_date']
-    audiofile[tags['originaldate']] = meta_tags['release_date']
-    if meta_tags['genre']:
-        audiofile[tags['genre']] = meta_tags['genre']
-    if meta_tags['copyright']:
-        audiofile[tags['copyright']] = meta_tags['copyright']
-    try:
-        albumart = urllib.request.urlopen(meta_tags['album']['images'][0]['url'])
-        audiofile[tags['albumart']] = [MP4Cover(
-            albumart.read(), imageformat=MP4Cover.FORMAT_JPEG)]
-        albumart.close()
-    except IndexError:
-        pass
-    audiofile.save()
-    return True

core/misc.py

@@ -1,141 +0,0 @@
-import sys
-import os
-import argparse
-import spotipy.oauth2 as oauth2
-from urllib.request import quote
-from slugify import slugify
-
-
-def input_link(links):
-    """Let the user input a number."""
-    while True:
-        try:
-            the_chosen_one = int(input('>> Choose your number: '))
-            if 1 <= the_chosen_one <= len(links):
-                return links[the_chosen_one - 1]
-            elif the_chosen_one == 0:
-                return None
-            else:
-                print('Choose a valid number!')
-        except ValueError:
-            print('Choose a valid number!')
-
-
-def trim_song(file):
-    """Remove the first song from file."""
-    with open(file, 'r') as file_in:
-        data = file_in.read().splitlines(True)
-    with open(file, 'w') as file_out:
-        file_out.writelines(data[1:])
-
-
-def get_arguments():
-    parser = argparse.ArgumentParser(
-        description='Download and convert songs from Spotify, Youtube etc.',
-        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
-    group = parser.add_mutually_exclusive_group(required=True)
-
-    group.add_argument(
-        '-s', '--song', help='download song by spotify link or name')
-    group.add_argument(
-        '-l', '--list', help='download songs from a file')
-    group.add_argument(
-        '-p', '--playlist', help='load songs from playlist URL into <playlist_name>.txt')
-    group.add_argument(
-        '-u', '--username',
-        help="load songs from user's playlist into <playlist_name>.txt")
-    parser.add_argument(
-        '-m', '--manual', default=False,
-        help='choose the song to download manually', action='store_true')
-    parser.add_argument(
-        '-nm', '--no-metadata', default=False,
-        help='do not embed metadata in songs', action='store_true')
-    parser.add_argument(
-        '-a', '--avconv', default=False,
-        help='Use avconv for conversion otherwise set defaults to ffmpeg',
-        action='store_true')
-    parser.add_argument(
-        '-f', '--folder', default='Music/',
-        help='path to folder where files will be stored in')
-    parser.add_argument(
-        '-v', '--verbose', default=False, help='show debug output',
-        action='store_true')
-    parser.add_argument(
-        '-i', '--input_ext', default='.m4a',
-        help='prefered input format .m4a or .webm (Opus)')
-    parser.add_argument(
-        '-o', '--output_ext', default='.mp3',
-        help='prefered output extension .mp3 or .m4a (AAC)')
-
-    return parser.parse_args()
-
-
-def is_spotify(raw_song):
-    """Check if the input song is a Spotify link."""
-    status = len(raw_song) == 22 and raw_song.replace(" ", "%20") == raw_song
-    status = status or raw_song.find('spotify') > -1
-    return status
-
-def is_youtube(raw_song):
-    """Check if the input song is a YouTube link."""
-    status = len(raw_song) == 11 and raw_song.replace(" ", "%20") == raw_song
-    status = status and not raw_song.lower() == raw_song
-    status = status or 'youtube.com/watch?v=' in raw_song
-    return status
-
-
-def sanitize_title(title):
-    """Generate filename of the song to be downloaded."""
-    title = title.replace(' ', '_')
-    title = title.replace('/', '_')
-
-    # slugify removes any special characters
-    title = slugify(title, ok='-_()[]{}', lower=False)
-    return title
-
-
-def generate_token():
-    """Generate the token. Please respect these credentials :)"""
-    credentials = oauth2.SpotifyClientCredentials(
-        client_id='4fe3fecfe5334023a1472516cc99d805',
-        client_secret='0f02b7c483c04257984695007a4a8d5c')
-    token = credentials.get_access_token()
-    return token
-
-
-def generate_search_url(song, viewsort=False):
-    """Generate YouTube search URL for the given song."""
-    # urllib.request.quote() encodes URL with special characters
-    song = quote(song)
-    if viewsort:
-        url = u"https://www.youtube.com/results?q={0}".format(song)
-    else:
-        url = u"https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q={0}".format(song)
-
-    return url
-
-
-def filter_path(path):
-    os.chdir(sys.path[0])
-    if not os.path.exists(path):
-        os.makedirs(path)
-    for temp in os.listdir(path):
-        if temp.endswith('.temp'):
-            os.remove(os.path.join(path, temp))
-
-
-def grace_quit():
-    print('\n\nExiting.')
-    sys.exit()
-
-def get_sec(time_str):
-   v = time_str.split(':', 3)
-   v.reverse()
-   sec = 0
-   if len(v) > 0: #seconds
-       sec += int(v[0])
-   if len(v) > 1: # minutes
-       sec += int(v[1]) * 60
-   if len(v) > 2: # hours
-       sec += int(v[2]) * 3600
-   return sec

requirements.txt

@@ -1,8 +0,0 @@
-pathlib >= 1.0.1
-BeautifulSoup4 >= 0.4.13
-youtube_dl >= 2017.5.1
-pafy >= 0.5.3.1
-spotipy >= 2.4.4
-mutagen >= 1.37
-unicode-slugify >= 0.1.3
-titlecase >= 0.10.0

setup.cfg

@@ -0,0 +1,5 @@
+[tool:pytest]
+addopts = --strict-markers -m "not network"
+markers =
+	network: marks test which rely on external network resources (select with '-m network' or run all with '-m "network, not network"')
+

setup.py

@@ -0,0 +1,79 @@
+from setuptools import setup
+import os
+
+with open("README.md", "r", encoding="utf-8") as f:
+    long_description = f.read()
+
+# __version__ comes into namespace from here
+with open(os.path.join("spotdl", "version.py")) as version_file:
+    exec(version_file.read())
+
+setup(
+    # 'spotify-downloader' was already taken :/
+    name="spotdl",
+    # Tests are included automatically:
+    # https://docs.python.org/3.6/distutils/sourcedist.html#specifying-the-files-to-distribute
+    packages=[
+        "spotdl",
+        "spotdl.command_line",
+        "spotdl.lyrics",
+        "spotdl.lyrics.providers",
+        "spotdl.encode",
+        "spotdl.encode.encoders",
+        "spotdl.metadata",
+        "spotdl.metadata.embedders",
+        "spotdl.metadata.providers",
+        "spotdl.lyrics",
+        "spotdl.lyrics.providers",
+        "spotdl.authorize",
+        "spotdl.authorize.services",
+        "spotdl.helpers",
+    ],
+    version=__version__,
+    install_requires=[
+        "pathlib >= 1.0.1",
+        "youtube_dl >= 2017.9.26",
+        "pytube3 >= 9.5.5",
+        "spotipy >= 2.12.0",
+        "mutagen >= 1.41.1",
+        "beautifulsoup4 >= 4.6.3",
+        "unicode-slugify >= 0.1.3",
+        "coloredlogs >= 14.0",
+        "lyricwikia >= 0.1.8",
+        "PyYAML >= 3.13",
+        "appdirs >= 1.4.3",
+        "tqdm >= 4.45.0"
+    ],
+    description="Download songs from YouTube using Spotify song URLs or playlists with albumart and meta-tags.",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    author="Ritiek Malhotra",
+    author_email="ritiekmalhotra123@gmail.com",
+    license="MIT",
+    python_requires=">=3.6",
+    url="https://github.com/ritiek/spotify-downloader",
+    download_url="https://pypi.org/project/spotdl/",
+    keywords=[
+        "spotify",
+        "downloader",
+        "download",
+        "music",
+        "youtube",
+        "mp3",
+        "album",
+        "metadata",
+    ],
+    classifiers=[
+        "Intended Audience :: End Users/Desktop",
+        "License :: OSI Approved :: MIT License",
+        "Programming Language :: Python",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3 :: Only",
+        "Topic :: Multimedia",
+        "Topic :: Multimedia :: Sound/Audio",
+        "Topic :: Utilities",
+    ],
+    entry_points={"console_scripts": ["spotdl = spotdl:main"]},
+)

spotdl.py

@@ -1,417 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: UTF-8 -*-
-
-from core import metadata
-from core import convert
-from core import misc
-from bs4 import BeautifulSoup
-from titlecase import titlecase
-from slugify import slugify
-import spotipy
-import pafy
-import urllib.request
-import sys
-import os
-import time
-
-
-def generate_songname(tags):
-    """Generate a string of the format '[artist] - [song]' for the given spotify song."""
-    raw_song = u'{0} - {1}'.format(tags['artists'][0]['name'], tags['name'])
-    return raw_song
-
-
-def generate_metadata(raw_song):
-    """Fetch a song's metadata from Spotify."""
-    if misc.is_spotify(raw_song):
-        # fetch track information directly if it is spotify link
-        meta_tags = spotify.track(raw_song)
-    else:
-        # otherwise search on spotify and fetch information from first result
-        try:
-            meta_tags = spotify.search(raw_song, limit=1)['tracks']['items'][0]
-        except:
-            return None
-    artist = spotify.artist(meta_tags['artists'][0]['id'])
-    album = spotify.album(meta_tags['album']['id'])
-
-    try:
-        meta_tags[u'genre'] = titlecase(artist['genres'][0])
-    except IndexError:
-        meta_tags[u'genre'] = None
-    try:
-        meta_tags[u'copyright'] = album['copyrights'][0]['text']
-    except IndexError:
-        meta_tags[u'copyright'] = None
-    try:
-        meta_tags['isrc']
-    except KeyError:
-        meta_tags['isrc'] = None
-
-    meta_tags[u'release_date'] = album['release_date']
-    meta_tags[u'publisher'] = album['label']
-    meta_tags[u'total_tracks'] = album['tracks']['total']
-
-    return meta_tags
-
-
-def generate_youtube_url(raw_song, tries_remaining=5):
-    """Search for the song on YouTube and generate a URL to its video."""
-    # prevents an infinite loop but allows for a few retries
-    if tries_remaining == 0:
-        return
-
-    meta_tags = generate_metadata(raw_song)
-    if meta_tags is None:
-        song = raw_song
-        search_url = misc.generate_search_url(song, viewsort=False)
-    else:
-        song = generate_songname(meta_tags)
-        search_url = misc.generate_search_url(song, viewsort=True)
-
-    item = urllib.request.urlopen(search_url).read()
-    # item = unicode(item, 'utf-8')
-    items_parse = BeautifulSoup(item, "html.parser")
-
-    videos = []
-    for x in items_parse.find_all('div', {'class': 'yt-lockup-dismissable yt-uix-tile'}):
-        # ensure result is not a channel
-        if x.find('channel') is not None or 'yt-lockup-channel' in x.parent.attrs['class'] or 'yt-lockup-channel' in x.attrs['class']:
-            continue
-
-        # ensure result is not a mix/playlist
-        if 'yt-lockup-playlist' in x.parent.attrs['class']:
-            continue
-
-        # confirm the video result is not an advertisement
-        if x.find('googleads') is not None:
-            continue
-
-        y = x.find('div', class_='yt-lockup-content')
-        link = y.find('a')['href']
-        title = y.find('a')['title']
-        try:
-            videotime = x.find('span', class_="video-time").get_text()
-        except AttributeError:
-            return generate_youtube_url(raw_song, tries_remaining - 1)
-
-        youtubedetails = {'link': link, 'title': title, 'videotime': videotime, 'seconds':misc.get_sec(videotime)}
-        videos.append(youtubedetails)
-        if meta_tags is None:
-            break
-
-    if not videos:
-        return None
-
-    if args.manual:
-        print(song)
-        print('')
-        print('0. Skip downloading this song')
-        # fetch all video links on first page on YouTube
-        for i, v in enumerate(videos):
-          print(u'{0}. {1} {2} {3}'.format(i+1, v['title'], v['videotime'], "http://youtube.com"+v['link']))
-        print('')
-        # let user select the song to download
-        result = misc.input_link(videos)
-        if result is None:
-            return None
-    else:
-        if meta_tags is not None:
-            # filter out videos that do not have a similar length to the Spotify song
-            duration_tolerance = 10
-            max_duration_tolerance = 20
-            possible_videos_by_duration = list()
-
-            '''
-            start with a reasonable duration_tolerance, and increment duration_tolerance
-            until one of the Youtube results falls within the correct duration or
-            the duration_tolerance has reached the max_duration_tolerance
-            '''
-            while len(possible_videos_by_duration) == 0:
-                possible_videos_by_duration = list(filter(lambda x: abs(x['seconds'] - (int(meta_tags['duration_ms'])/1000)) <= duration_tolerance, videos))
-                duration_tolerance += 1
-                if duration_tolerance > max_duration_tolerance:
-                    print(meta_tags['name'], 'by', meta_tags['artists'][0]['name'], 'was not found')
-                    return None
-
-            result = possible_videos_by_duration[0]
-        else:
-            # if the metadata could not be acquired, take the first result from Youtube because the proper song length is unknown
-            result = videos[0]
-
-    full_link = None
-    if result:
-        full_link = u'youtube.com{0}'.format(result['link'])
-
-    return full_link
-
-
-def go_pafy(raw_song):
-    """Parse track from YouTube."""
-    if misc.is_youtube(raw_song):
-        track_info = pafy.new(raw_song)
-    else:
-        track_url = generate_youtube_url(raw_song)
-
-        if track_url is None:
-            track_info = None
-        else:
-            track_info = pafy.new(track_url)
-
-    return track_info
-
-
-def get_youtube_title(content, number=None):
-    """Get the YouTube video's title."""
-    title = content.title
-    if number is None:
-        return title
-    else:
-        return '{0}. {1}'.format(number, title)
-
-
-def feed_playlist(username):
-    """Fetch user playlists when using the -u option."""
-    playlists = spotify.user_playlists(username)
-    links = []
-    check = 1
-
-    while True:
-        for playlist in playlists['items']:
-            # in rare cases, playlists may not be found, so playlists['next']
-            # is None. Skip these. Also see Issue #91.
-            if playlist['name'] is not None:
-                print(u'{0:>5}. {1:<30}  ({2} tracks)'.format(
-                    check, playlist['name'],
-                    playlist['tracks']['total']))
-                links.append(playlist)
-                check += 1
-        if playlists['next']:
-            playlists = spotify.next(playlists)
-        else:
-            break
-
-    print('')
-    playlist = misc.input_link(links)
-    print('')
-    write_tracks(playlist)
-
-
-def write_tracks(playlist):
-    results = spotify.user_playlist(
-        playlist['owner']['id'], playlist['id'], fields='tracks,next')
-    text_file = u'{0}.txt'.format(slugify(playlist['name'], ok='-_()[]{}'))
-    print(u'Feeding {0} tracks to {1}'.format(playlist['tracks']['total'], text_file))
-
-    tracks = results['tracks']
-    with open(text_file, 'a') as file_out:
-        while True:
-            for item in tracks['items']:
-                track = item['track']
-                try:
-                    file_out.write(track['external_urls']['spotify'] + '\n')
-                except KeyError:
-                    print(u'Skipping track {0} by {1} (local only?)'.format(
-                        track['name'], track['artists'][0]['name']))
-            # 1 page = 50 results
-            # check if there are more pages
-            if tracks['next']:
-                tracks = spotify.next(tracks)
-            else:
-                break
-
-
-def download_song(file_name, content):
-    """Download the audio file from YouTube."""
-    if args.input_ext == '.webm':
-        link = content.getbestaudio(preftype='webm')
-    elif args.input_ext == '.m4a':
-        link = content.getbestaudio(preftype='m4a')
-    else:
-        return False
-
-    if link is None:
-        return False
-    else:
-        link.download(
-            filepath='{0}{1}'.format(os.path.join(args.folder, file_name), args.input_ext))
-        return True
-
-
-def check_exists(music_file, raw_song, islist=True):
-    """Check if the input song already exists in the given folder."""
-    songs = os.listdir(args.folder)
-    for song in songs:
-        if song.endswith('.temp'):
-            os.remove(os.path.join(args.folder, song))
-            continue
-        # check if any song with similar name is already present in the given folder
-        file_name = misc.sanitize_title(music_file)
-        if song.startswith(file_name):
-            # check if the already downloaded song has correct metadata
-            already_tagged = metadata.compare(os.path.join(args.folder, song), generate_metadata(raw_song))
-
-            # if not, remove it and download again without prompt
-            if misc.is_spotify(raw_song) and not already_tagged:
-                os.remove(os.path.join(args.folder, song))
-                return False
-
-            # do not prompt and skip the current song
-            # if already downloaded when using list
-            if islist:
-                print('Song already exists')
-                return True
-            # if downloading only single song, prompt to re-download
-            else:
-                prompt = input('Song with same name has already been downloaded. '
-                               'Re-download? (y/n): ').lower()
-                if prompt == 'y':
-                    os.remove(os.path.join(args.folder, song))
-                    return False
-                else:
-                    return True
-    return False
-
-
-def grab_list(text_file):
-    """Download all songs from the list."""
-    with open(text_file, 'r') as listed:
-        lines = (listed.read()).splitlines()
-    # ignore blank lines in text_file (if any)
-    try:
-        lines.remove('')
-    except ValueError:
-        pass
-    print(u'Total songs in list: {0} songs'.format(len(lines)))
-    print('')
-    # nth input song
-    number = 1
-    for raw_song in lines:
-        try:
-            grab_single(raw_song, number=number)
-        # token expires after 1 hour
-        except spotipy.oauth2.SpotifyOauthError:
-            # refresh token when it expires
-            new_token = misc.generate_token()
-            global spotify
-            spotify = spotipy.Spotify(auth=new_token)
-            grab_single(raw_song, number=number)
-        # detect network problems
-        except (urllib.request.URLError, TypeError, IOError):
-            lines.append(raw_song)
-            # remove the downloaded song from .txt
-            misc.trim_song(text_file)
-            # and append it to the last line in .txt
-            with open(text_file, 'a') as myfile:
-                myfile.write(raw_song + '\n')
-            print('Failed to download song. Will retry after other songs.')
-            # wait 0.5 sec to avoid infinite looping
-            time.sleep(0.5)
-            continue
-        except KeyboardInterrupt:
-            misc.grace_quit()
-        finally:
-            print('')
-        misc.trim_song(text_file)
-        number += 1
-
-
-def grab_playlist(playlist):
-    if '/' in playlist:
-        if playlist.endswith('/'):
-            playlist = playlist[:-1]
-        splits = playlist.split('/')
-    else:
-        splits = playlist.split(':')
-
-    username = splits[-3]
-    playlist_id = splits[-1]
-    playlists = spotify.user_playlists(username)
-
-    while True:
-        for playlist in playlists['items']:
-            if not playlist['name'] == None:
-                if playlist['id'] == playlist_id:
-                    playlists['next'] = None
-                    break
-        if playlists['next']:
-            playlists = spotify.next(playlists)
-        else:
-            break
-
-    write_tracks(playlist)
-
-
-def grab_single(raw_song, number=None):
-    """Logic behind downloading a song."""
-    if number:
-        islist = True
-    else:
-        islist = False
-
-    content = go_pafy(raw_song)
-    if content is None:
-        return
-
-    if misc.is_youtube(raw_song):
-        raw_song = slugify(content.title).replace('-', ' ')
-
-    # print '[number]. [artist] - [song]' if downloading from list
-    # otherwise print '[artist] - [song]'
-    print(get_youtube_title(content, number))
-
-    # generate file name of the song to download
-    meta_tags = generate_metadata(raw_song)
-    songname = content.title
-
-    if meta_tags is not None:
-        refined_songname = generate_songname(meta_tags)
-        if not refined_songname == ' - ':
-            songname = refined_songname
-
-    file_name = misc.sanitize_title(songname)
-
-    if not check_exists(file_name, raw_song, islist=islist):
-        if download_song(file_name, content):
-            print('')
-            input_song = file_name + args.input_ext
-            output_song = file_name + args.output_ext
-            convert.song(input_song, output_song, args.folder,
-                         avconv=args.avconv, verbose=args.verbose)
-            if not args.input_ext == args.output_ext:
-                os.remove(os.path.join(args.folder, input_song))
-
-            if not args.no_metadata:
-                metadata.embed(os.path.join(args.folder, output_song), meta_tags)
-        else:
-            print('No audio streams available')
-
-
-class TestArgs(object):
-    manual = False
-    input_ext = '.m4a'
-    output_ext = '.mp3'
-    folder = 'Music/'
-
-# token is mandatory when using Spotify's API
-# https://developer.spotify.com/news-stories/2017/01/27/removing-unauthenticated-calls-to-the-web-api/
-token = misc.generate_token()
-spotify = spotipy.Spotify(auth=token)
-
-if __name__ == '__main__':
-    os.chdir(sys.path[0])
-    args = misc.get_arguments()
-
-    misc.filter_path(args.folder)
-
-    if args.song:
-        grab_single(raw_song=args.song)
-    elif args.list:
-        grab_list(text_file=args.list)
-    elif args.playlist:
-        grab_playlist(playlist=args.playlist)
-    elif args.username:
-        feed_playlist(username=args.username)
-else:
-    misc.filter_path('Music')
-    args = TestArgs()

spotdl/__init__.py

@@ -0,0 +1,5 @@
+from spotdl.command_line.__main__ import main
+
+from spotdl.version import __version__
+from spotdl.command_line.core import Spotdl
+

spotdl/authorize/__init__.py

@@ -0,0 +1,6 @@
+from spotdl.authorize.authorize_base import AuthorizeBase
+
+from spotdl.authorize.exceptions import AuthorizationError
+from spotdl.authorize.exceptions import SpotifyAuthorizationError
+from spotdl.authorize.exceptions import YouTubeAuthorizationError
+

spotdl/authorize/authorize_base.py

@@ -0,0 +1,19 @@
+from abc import ABC
+from abc import abstractmethod
+
+class AuthorizeBase(ABC):
+    """
+    Defined service authenticators must inherit from this abstract
+    base class and implement their own functionality for the below
+    defined methods.
+    """
+
+    @abstractmethod
+    def authorize(self):
+        """
+        This method must authorize with the corresponding service
+        and return an object that can be utilized in making
+        authenticated requests.
+        """
+        pass
+

spotdl/authorize/exceptions.py

@@ -0,0 +1,20 @@
+class AuthorizationError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class SpotifyAuthorizationError(AuthorizationError):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class YouTubeAuthorizationError(AuthorizationError):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+

spotdl/authorize/services/__init__.py

@@ -0,0 +1,2 @@
+from spotdl.authorize.services.spotify import AuthorizeSpotify
+

spotdl/authorize/services/spotify.py

@@ -0,0 +1,45 @@
+from spotdl.authorize import AuthorizeBase
+from spotdl.authorize.exceptions import SpotifyAuthorizationError
+
+import spotipy
+import spotipy.oauth2 as oauth2
+
+import logging
+logger = logging.getLogger(__name__)
+
+# This masterclient is used to keep the last logged-in client
+# object in memory for for persistence. If credentials aren't
+# provided when creating further objects, the last authenticated
+# client object with correct credentials is returned when
+# `AuthorizeSpotify().authorize()` is called.
+masterclient = None
+
+class AuthorizeSpotify(spotipy.Spotify):
+    def __init__(self, client_id=None, client_secret=None):
+        global masterclient
+
+        credentials_provided = client_id is not None \
+                           and client_secret is not None
+        valid_input = credentials_provided or masterclient is not None
+
+        if not valid_input:
+            raise SpotifyAuthorizationError(
+                "You must pass in client_id and client_secret to this method "
+                "when authenticating for the first time."
+            )
+
+        if masterclient:
+            logger.debug("Reading cached master Spotify credentials.")
+            # Use cached client instead of authorizing again
+            # and thus wasting time.
+            self.__dict__.update(masterclient.__dict__)
+        else:
+            logger.debug("Setting master Spotify credentials.")
+            credential_manager = oauth2.SpotifyClientCredentials(
+                client_id=client_id,
+                client_secret=client_secret
+            )
+            super().__init__(client_credentials_manager=credential_manager)
+            # Cache current client
+            masterclient = self
+

spotdl/authorize/services/tests/test_spotify.py

@@ -0,0 +1,19 @@
+from spotdl.authorize.services import AuthorizeSpotify
+
+import pytest
+
+class TestSpotifyAuthorize:
+    # TODO: Test these once we a have config.py
+    #       storing pre-defined default credentials.
+    #
+    # We'll use these credentials to create
+    # a spotipy object via below tests
+
+    @pytest.mark.xfail
+    def test_generate_token(self):
+        raise NotImplementedError
+
+    @pytest.mark.xfail
+    def test_authorize(self):
+        raise NotImplementedError
+

spotdl/authorize/tests/test_authorize_base.py

@@ -0,0 +1,16 @@
+from spotdl.authorize import AuthorizeBase
+
+import pytest
+
+class TestAbstractBaseClass:
+    def test_error_abstract_base_class_authorizebase(self):
+        with pytest.raises(TypeError):
+            AuthorizeBase()
+
+    def test_inherit_abstract_base_class_authorizebase(self):
+        class AuthorizeKid(AuthorizeBase):
+            def authorize(self):
+                pass
+
+        AuthorizeKid()
+

spotdl/authorize/tests/test_authorize_exceptions.py

@@ -0,0 +1,15 @@
+from spotdl.authorize.exceptions import AuthorizationError
+from spotdl.authorize.exceptions import SpotifyAuthorizationError
+from spotdl.authorize.exceptions import YouTubeAuthorizationError
+
+
+class TestEncoderNotFoundSubclass:
+    def test_authozation_error_subclass(self):
+        assert issubclass(AuthorizationError, Exception)
+
+    def test_spotify_authorization_error_subclass(self):
+        assert issubclass(SpotifyAuthorizationError, AuthorizationError)
+
+    def test_youtube_authorization_error_subclass(self):
+        assert issubclass(YouTubeAuthorizationError, AuthorizationError)
+

spotdl/command_line/__main__.py

@@ -0,0 +1,57 @@
+import logging
+import coloredlogs
+
+import sys
+
+from spotdl.command_line.core import Spotdl
+from spotdl.command_line.arguments import get_arguments
+from spotdl.command_line.exceptions import ArgumentError
+
+# hardcode loglevel for dependencies so that they do not spew generic
+# log messages along with spotdl.
+for module in ("chardet", "urllib3", "spotipy", "pytube"):
+    logging.getLogger(module).setLevel(logging.CRITICAL)
+
+coloredlogs.DEFAULT_FIELD_STYLES = {
+    "levelname": {"bold": True, "color": "yellow"},
+    "name": {"color": "blue"},
+    "lineno": {"color": "magenta"},
+}
+
+
+def set_logger(level):
+    if level == logging.DEBUG:
+        fmt = "%(levelname)s:%(name)s:%(lineno)d:\n%(message)s\n"
+    else:
+        fmt = "%(levelname)s: %(message)s"
+    logging.basicConfig(format=fmt, level=level)
+    logger = logging.getLogger(name=__name__)
+    coloredlogs.install(level=level, fmt=fmt, logger=logger)
+    return logger
+
+
+def main():
+    try:
+        argument_handler = get_arguments()
+    except ArgumentError as e:
+        logger = set_logger(logging.INFO)
+        logger.info(e.args[0])
+        sys.exit(5)
+
+    logging_level = argument_handler.get_logging_level()
+    logger = set_logger(logging_level)
+    try:
+        spotdl = Spotdl(argument_handler)
+    except ArgumentError as e:
+        argument_handler.parser.error(e.args[0])
+    try:
+        spotdl.match_arguments()
+    except KeyboardInterrupt as e:
+        print("", file=sys.stderr)
+        logger.exception(e)
+        sys.exit(2)
+
+
+if __name__ == "__main__":
+    main()
+

spotdl/command_line/arguments.py

@@ -0,0 +1,362 @@
+import appdirs
+
+import argparse
+import mimetypes
+import os
+import sys
+import shutil
+
+from spotdl.command_line.exceptions import ArgumentError
+import spotdl.util
+import spotdl.config
+
+from collections.abc import Sequence
+import logging
+logger = logging.getLogger(__name__)
+
+
+_LOG_LEVELS = {
+    "INFO": logging.INFO,
+    "WARNING": logging.WARNING,
+    "ERROR": logging.ERROR,
+    "DEBUG": logging.DEBUG,
+}
+
+if os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
+    saved_config = spotdl.config.read_config(spotdl.config.DEFAULT_CONFIG_FILE)
+else:
+    saved_config = {"spotify-downloader": {}}
+
+_CONFIG_BASE = spotdl.util.merge_copy(
+    spotdl.config.DEFAULT_CONFIGURATION,
+    saved_config,
+)
+
+
+def get_arguments(config_base=_CONFIG_BASE):
+    parser = argparse.ArgumentParser(
+        description="Download and convert tracks from Spotify, Youtube, etc.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+
+    defaults = config_base["spotify-downloader"]
+
+    to_remove_config = "--remove-config" in sys.argv[1:]
+    if not to_remove_config and "download-only-metadata" in defaults:
+        raise ArgumentError(
+            "The default configuration file currently set is not suitable for spotdl>=2.0.0.\n"
+            "You need to remove your previous `config.yml` due to breaking changes\n"
+            "introduced in v2.0.0, new options being added, and old ones being removed\n"
+            "You may want to first backup your old configuration for reference. You can\n"
+            "then remove your current configuration by running:\n"
+            "```\n"
+            "$ spotdl --remove-config\n"
+            "```\n"
+            "spotdl will automatically generate a new configuration file on the next run.\n"
+            "You can then replace the appropriate fields in the newly generated\n"
+            "configuration file by referring to your old configuration file.\n\n"
+            "For the list of OTHER BREAKING CHANGES and release notes check out:\n"
+            "https://github.com/ritiek/spotify-downloader/releases/tag/v2.0.0"
+        )
+
+    possible_special_tags = (
+        "{track-name}",
+        "{artist}",
+        "{album}",
+        "{album-artist}",
+        "{genre}",
+        "{disc-number}",
+        "{duration}",
+        "{year}",
+        "{original-date}",
+        "{track-number}",
+        "{total-tracks}",
+        "{isrc}",
+        "{track-id}",
+        "{output-ext}",
+    )
+
+    # `--remove-config` does not require the any of the group arguments to be passed.
+    group = parser.add_mutually_exclusive_group(required=not to_remove_config)
+
+    group.add_argument(
+        "-s",
+        "--song",
+        nargs="+",
+        help="download track(s) by spotify link or name"
+    )
+    group.add_argument(
+        "-l",
+        "--list",
+        help="download tracks from a file (WARNING: this file will be modified!)"
+    )
+    group.add_argument(
+        "-p",
+        "--playlist",
+        help="load tracks from playlist URL into <playlist_name>.txt or "
+             "if `--write-to=<path/to/file.txt>` has been passed",
+    )
+    group.add_argument(
+        "-a",
+        "--album",
+        help="load tracks from album URL into <album_name>.txt or if "
+             "`--write-to=<path/to/file.txt>` has been passed"
+    )
+    group.add_argument(
+        "-aa",
+        "--all-albums",
+        help="load all tracks from artist URL into <artist_name>.txt "
+             "or if `--write-to=<path/to/file.txt>` has been passed"
+    )
+    group.add_argument(
+        "-u",
+        "--username",
+        help="load tracks from user's playlist into <playlist_name>.txt "
+             "or if `--write-to=<path/to/file.txt>` has been passed"
+    )
+
+    parser.add_argument(
+        "--write-m3u",
+        help="generate an .m3u playlist file with youtube links given "
+             "a text file containing tracks",
+        action="store_true",
+    )
+    parser.add_argument(
+        "-m",
+        "--manual",
+        default=defaults["manual"],
+        help="choose the track to download manually from a list of matching tracks",
+        action="store_true",
+    )
+    parser.add_argument(
+        "-nm",
+        "--no-metadata",
+        default=defaults["no_metadata"],
+        help="do not embed metadata in tracks",
+        action="store_true",
+    )
+    parser.add_argument(
+        "-ne",
+        "--no-encode",
+        default=defaults["no_encode"],
+        action="store_true",
+        help="do not encode media using FFmpeg",
+    )
+    parser.add_argument(
+        "--overwrite",
+        default=defaults["overwrite"],
+        choices={"prompt", "force", "skip"},
+        help="change the overwrite policy",
+    )
+    parser.add_argument(
+        "-q",
+        "--quality",
+        default=defaults["quality"],
+        choices={"best", "worst"},
+        help="preferred audio quality",
+    )
+    parser.add_argument(
+        "-i",
+        "--input-ext",
+        default=defaults["input_ext"],
+        choices={"automatic", "m4a", "opus"},
+        help="preferred input format",
+    )
+    parser.add_argument(
+        "-o",
+        "--output-ext",
+        default=defaults["output_ext"],
+        choices={"mp3", "m4a", "flac"},
+        help="preferred output format",
+    )
+    parser.add_argument(
+        "--write-to",
+        default=defaults["write_to"],
+        help="write tracks from Spotify playlist, album, etc. to this file",
+    )
+    parser.add_argument(
+        "-f",
+        "--output-file",
+        default=defaults["output_file"],
+        help="path where to write the downloaded track to, special tags "
+        "are to be surrounded by curly braces. Possible tags: {}".format(
+            possible_special_tags
+        )
+    )
+    parser.add_argument(
+        "--trim-silence",
+        default=defaults["trim_silence"],
+        help="remove silence from the start of the audio",
+        action="store_true",
+    )
+    parser.add_argument(
+        "-sf",
+        "--search-format",
+        default=defaults["search_format"],
+        help="search format to search for on YouTube, special tags "
+        "are to be surrounded by curly braces. Possible tags: {}".format(
+            possible_special_tags
+        )
+    )
+    parser.add_argument(
+        "-d",
+        "--dry-run",
+        default=defaults["dry_run"],
+        help="show only track title and YouTube URL, and then skip "
+        "to the next track (if any)",
+        action="store_true",
+    )
+    parser.add_argument(
+        "--processor",
+        default="synchronous",
+        choices={"synchronous", "threaded"},
+        # help='list downloading strategy: - "synchronous" downloads '
+        # 'tracks one-by-one. - "threaded" (highly experimental at the '
+        # 'moment! expect it to slash & burn) pre-fetches the next '
+        # 'track\'s metadata for more efficient downloading'
+        # XXX: Still very experimental to be exposed
+        help=argparse.SUPPRESS,
+    )
+    parser.add_argument(
+        "-ns",
+        "--no-spaces",
+        default=defaults["no_spaces"],
+        help="replace spaces in metadata values with underscores when "
+        "generating filenames",
+        action="store_true",
+    )
+    parser.add_argument(
+        "-sk",
+        "--skip-file",
+        default=defaults["skip_file"],
+        help="path to file containing tracks to skip",
+    )
+    parser.add_argument(
+        "-w",
+        "--write-successful-file",
+        default=defaults["write_successful_file"],
+        help="path to file to write successful tracks to",
+    )
+    parser.add_argument(
+        "--spotify-client-id",
+        default=defaults["spotify_client_id"],
+        help=argparse.SUPPRESS,
+    )
+    parser.add_argument(
+        "--spotify-client-secret",
+        default=defaults["spotify_client_secret"],
+        help=argparse.SUPPRESS,
+    )
+    parser.add_argument(
+        "-ll",
+        "--log-level",
+        default=defaults["log_level"],
+        choices=_LOG_LEVELS.keys(),
+        type=str.upper,
+        help="set log verbosity",
+    )
+    parser.add_argument(
+        "-c",
+        "--config",
+        default=spotdl.config.DEFAULT_CONFIG_FILE,
+        help="path to custom config.yml file"
+    )
+    parser.add_argument(
+        "--remove-config",
+        default=False,
+        action="store_true",
+        help="remove previously saved config"
+    )
+    parser.add_argument(
+        "-V",
+        "--version",
+        action="version",
+        version="%(prog)s {}".format(spotdl.__version__),
+    )
+
+    return ArgumentHandler(parser=parser)
+
+
+class ArgumentHandler:
+    def __init__(self, args=None, parser=argparse.ArgumentParser(""), config_base=_CONFIG_BASE):
+        args_were_passed = args is not None
+        if not args_were_passed:
+            args = parser.parse_args().__dict__
+
+        config_file = args.get("config")
+        configured_args = args.copy()
+        if config_file and os.path.isfile(config_file):
+            config = spotdl.config.read_config(config_file)
+            parser.set_defaults(**config["spotify-downloader"])
+            configured_args = parser.parse_args().__dict__
+
+        if args_were_passed:
+            parser.set_defaults(**args)
+            configured_args = parser.parse_args().__dict__
+
+        defaults = config_base["spotify-downloader"]
+        args = spotdl.util.merge_copy(defaults, args)
+
+        self.parser = parser
+        self.args = args
+        self.configured_args = configured_args
+
+    def get_configured_args(self):
+        return self.configured_args
+
+    def get_logging_level(self):
+        return _LOG_LEVELS[self.configured_args["log_level"]]
+
+    def run_errands(self):
+        args = self.get_configured_args()
+
+        if (args.get("list")
+            and not mimetypes.MimeTypes().guess_type(args["list"])[0] == "text/plain"
+        ):
+            raise ArgumentError(
+                "{0} is not of a valid argument to --list, argument must be plain text file.".format(
+                    args["list"]
+                )
+            )
+
+        if args.get("write_m3u") and not args.get("list"):
+            raise ArgumentError("--write-m3u can only be used with --list.")
+
+        if args["write_to"] and not (
+            args.get("playlist") or args.get("album") or args.get("all_albums") or args.get("username") or args.get("write_m3u")
+        ):
+            raise ArgumentError(
+                "--write-to can only be used with --playlist, --album, --all-albums, --username, or --write-m3u."
+            )
+
+        ffmpeg_exists = shutil.which("ffmpeg")
+        if not ffmpeg_exists:
+            logger.warn("FFmpeg was not found in PATH. Will not re-encode media to specified output format.")
+            args["no_encode"] = True
+
+        if args["no_encode"] and args["trim_silence"]:
+            logger.warn("--trim-silence can only be used when an encoder is set.")
+
+        if args["output_file"] == "-" and args["no_metadata"] is False:
+            logger.warn(
+                "Cannot write metadata when target is STDOUT. Pass "
+                "--no-metadata explicitly to hide this warning."
+            )
+            args["no_metadata"] = True
+        elif os.path.isdir(args["output_file"]):
+            adjusted_output_file = os.path.join(
+                args["output_file"],
+                self.parser.get_default("output_file")
+            )
+            logger.warn(
+                "Given output file is a directory. Will download tracks "
+                "in this directory with their filename as per the default "
+                "file format. Pass --output-file=\"{}\" to hide this "
+                "warning.".format(
+                    adjusted_output_file
+                )
+            )
+            args["output_file"] = adjusted_output_file
+
+        return args
+

spotdl/command_line/core.py

@@ -0,0 +1,442 @@
+from spotdl.metadata.providers import ProviderSpotify
+from spotdl.metadata.providers import ProviderYouTube
+from spotdl.metadata.providers import YouTubeSearch
+from spotdl.metadata.embedders import EmbedderDefault
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+import spotdl.metadata
+
+from spotdl.lyrics.providers import LyricWikia
+from spotdl.lyrics.providers import Genius
+from spotdl.lyrics.exceptions import LyricsNotFoundError
+
+from spotdl.encode.encoders import EncoderFFmpeg
+
+from spotdl.authorize.services import AuthorizeSpotify
+
+from spotdl.track import Track
+import spotdl.util
+import spotdl.config
+
+from spotdl.command_line.exceptions import NoYouTubeVideoFoundError
+from spotdl.command_line.exceptions import NoYouTubeVideoMatchError
+from spotdl.metadata_search import MetadataSearch
+
+from spotdl.helpers.spotify import SpotifyHelpers
+
+import sys
+import os
+import urllib.request
+
+import logging
+logger = logging.getLogger(__name__)
+
+
+class Spotdl:
+    def __init__(self, argument_handler):
+        self.arguments = argument_handler.run_errands()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, type, value, traceback):
+        del self
+
+    def match_arguments(self):
+        logger.debug("Received arguments:\n{}".format(self.arguments))
+
+        if self.arguments["remove_config"]:
+            self.remove_saved_config()
+            return
+        self.save_default_config()
+
+        AuthorizeSpotify(
+            client_id=self.arguments["spotify_client_id"],
+            client_secret=self.arguments["spotify_client_secret"]
+        )
+        spotify_tools = SpotifyHelpers()
+        if self.arguments["song"]:
+            for track in self.arguments["song"]:
+                if track == "-":
+                    for line in sys.stdin:
+                        self.download_track(
+                            line.strip(),
+                        )
+                else:
+                    self.download_track(track)
+        elif self.arguments["list"]:
+            if self.arguments["write_m3u"]:
+                self.write_m3u(
+                    self.arguments["list"],
+                    self.arguments["write_to"]
+                )
+            else:
+                list_download = {
+                    "synchronous": self.download_tracks_from_file,
+                    # "threaded"  : self.download_tracks_from_file_threaded,
+                }[self.arguments["processor"]]
+
+                list_download(
+                    self.arguments["list"],
+                )
+        elif self.arguments["playlist"]:
+            playlist = spotify_tools.fetch_playlist(self.arguments["playlist"])
+            spotify_tools.write_playlist_tracks(playlist, self.arguments["write_to"])
+        elif self.arguments["album"]:
+            album = spotify_tools.fetch_album(self.arguments["album"])
+            spotify_tools.write_album_tracks(album, self.arguments["write_to"])
+        elif self.arguments["all_albums"]:
+            albums = spotify_tools.fetch_albums_from_artist(self.arguments["all_albums"])
+            spotify_tools.write_all_albums(albums, self.arguments["write_to"])
+        elif self.arguments["username"]:
+            playlist_url = spotify_tools.prompt_for_user_playlist(self.arguments["username"])
+            playlist = spotify_tools.fetch_playlist(playlist_url)
+            spotify_tools.write_playlist_tracks(playlist, self.arguments["write_to"])
+
+    def save_config(self, config_file=spotdl.config.DEFAULT_CONFIG_FILE, config=spotdl.config.DEFAULT_CONFIGURATION):
+        config_dir = os.path.dirname(config_file)
+        os.makedirs(config_dir, exist_ok=True)
+        logger.info('Writing configuration to "{0}":'.format(config_file))
+        spotdl.config.dump_config(config_file=config_file, config=spotdl.config.DEFAULT_CONFIGURATION)
+        config = spotdl.config.dump_config(config=spotdl.config.DEFAULT_CONFIGURATION["spotify-downloader"])
+        for line in config.split("\n"):
+            if line.strip():
+                logger.info(line.strip())
+        logger.info(
+            "Please note that command line arguments have higher priority "
+            "than their equivalents in the configuration file.\n"
+        )
+
+    def save_default_config(self):
+        if not os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
+            self.save_config()
+
+    def remove_saved_config(self, config_file=spotdl.config.DEFAULT_CONFIG_FILE):
+        if os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
+            logger.info('Removing "{}".'.format(spotdl.config.DEFAULT_CONFIG_FILE))
+            os.remove(spotdl.config.DEFAULT_CONFIG_FILE)
+        else:
+            logger.info('File does not exist: "{}".'.format(spotdl.config.DEFAULT_CONFIG_FILE))
+
+    def write_m3u(self, track_file, target_file=None):
+        with open(track_file, "r") as fin:
+            tracks = fin.read().splitlines()
+
+        logger.info(
+            "Checking and removing any duplicate tracks in {}.".format(track_file)
+        )
+        # Remove duplicates and empty elements
+        # Also strip whitespaces from elements (if any)
+        tracks = spotdl.util.remove_duplicates(
+            tracks,
+            condition=lambda x: x,
+            operation=str.strip
+        )
+
+        if target_file is None:
+            target_file = "{}.m3u".format(track_file.split(".")[0])
+
+        total_tracks = len(tracks)
+        logger.info("Generating {0} from {1} YouTube URLs.".format(target_file, total_tracks))
+        write_to_stdout = target_file == "-"
+        m3u_headers = "#EXTM3U\n\n"
+        if write_to_stdout:
+            sys.stdout.write(m3u_headers)
+        else:
+            with open(target_file, "w") as output_file:
+                output_file.write(m3u_headers)
+
+        videos = []
+        for n, track in enumerate(tracks, 1):
+            try:
+                search_metadata = MetadataSearch(
+                    track,
+                    lyrics=not self.arguments["no_metadata"],
+                    yt_search_format=self.arguments["search_format"],
+                    yt_manual=self.arguments["manual"]
+                )
+                video = search_metadata.best_on_youtube_search()
+            except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
+                logger.error(e.args[0])
+            else:
+                logger.info(
+                    "Matched track {0}/{1} ({2})".format(
+                        str(n).zfill(len(str(total_tracks))),
+                        total_tracks,
+                        video["url"],
+                    )
+                )
+                m3u_key = "#EXTINF:{duration},{title}\n{youtube_url}\n".format(
+                    duration=spotdl.util.get_sec(video["duration"]),
+                    title=video["title"],
+                    youtube_url=video["url"],
+                )
+                logger.debug(m3u_key.strip())
+                if write_to_stdout:
+                    sys.stdout.write(m3u_key)
+                else:
+                    with open(target_file, "a") as output_file:
+                        output_file.write(m3u_key)
+
+    def download_track(self, track):
+        logger.info('Downloading "{}"'.format(track))
+        search_metadata = MetadataSearch(
+            track,
+            lyrics=not self.arguments["no_metadata"],
+            yt_search_format=self.arguments["search_format"],
+            yt_manual=self.arguments["manual"]
+        )
+        try:
+            if self.arguments["no_metadata"]:
+                metadata = search_metadata.on_youtube()
+            else:
+                metadata = search_metadata.on_youtube_and_spotify()
+        except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
+            logger.error(e.args[0])
+        else:
+            self.download_track_from_metadata(metadata)
+
+    def should_we_overwrite_existing_file(self, overwrite):
+        if overwrite == "force":
+            logger.info("Forcing overwrite on existing file.")
+            to_overwrite = True
+        elif overwrite == "prompt":
+            to_overwrite = input("Overwrite? (y/N): ").lower() == "y"
+        else:
+            logger.info("Not overwriting existing file.")
+            to_overwrite = False
+
+        return to_overwrite
+
+    def generate_temp_filename(self, filename, for_stdout=False):
+        if for_stdout:
+            return filename
+        return "{filename}.temp".format(filename=filename)
+
+    def output_filename_filter(self, allow_spaces):
+        replace_spaces_with_underscores = not allow_spaces
+        if replace_spaces_with_underscores:
+            return lambda s: s.replace(" ", "_")
+        return lambda s: s
+
+    def download_track_from_metadata(self, metadata):
+        track = Track(metadata, cache_albumart=(not self.arguments["no_metadata"]))
+        stream = metadata["streams"].get(
+            quality=self.arguments["quality"],
+            preftype=self.arguments["input_ext"],
+        )
+        if stream is None:
+            logger.error('No matching streams found for given input format: "{}".'.format(
+                self.arguments["input_ext"]
+            ))
+            return
+
+        if self.arguments["no_encode"]:
+            output_extension = stream["encoding"]
+        else:
+            output_extension = self.arguments["output_ext"]
+
+        filename = spotdl.metadata.format_string(
+            self.arguments["output_file"],
+            metadata,
+            output_extension=output_extension,
+            sanitizer=lambda s: spotdl.util.sanitize(
+                s, spaces_to_underscores=self.arguments["no_spaces"]
+            )
+        )
+        download_to_stdout = filename == "-"
+        temp_filename = self.generate_temp_filename(filename, for_stdout=download_to_stdout)
+
+        to_skip_download = self.arguments["dry_run"]
+        if os.path.isfile(filename):
+            logger.info('A file with name "{filename}" already exists.'.format(
+                filename=filename
+            ))
+            to_skip_download = to_skip_download \
+                or not self.should_we_overwrite_existing_file(self.arguments["overwrite"])
+
+        if to_skip_download:
+            logger.debug("Skip track download.")
+            return
+
+        if not self.arguments["no_metadata"]:
+            metadata["lyrics"].start()
+
+        os.makedirs(os.path.dirname(filename) or ".", exist_ok=True)
+
+        logger.info('Downloading to "{filename}"'.format(filename=filename))
+        if self.arguments["no_encode"]:
+            track.download(stream, temp_filename)
+        else:
+            encoder = EncoderFFmpeg()
+            if self.arguments["trim_silence"]:
+                encoder.set_trim_silence()
+            track.download_while_re_encoding(
+                stream,
+                temp_filename,
+                target_encoding=output_extension,
+                encoder=encoder,
+            )
+
+        if not self.arguments["no_metadata"]:
+            track.metadata["lyrics"] = track.metadata["lyrics"].join()
+            self.apply_metadata(track, temp_filename, output_extension)
+
+        if not download_to_stdout:
+            logger.debug("Renaming {temp_filename} to {filename}.".format(
+                temp_filename=temp_filename, filename=filename
+            ))
+            os.rename(temp_filename, filename)
+
+        return filename
+
+    def apply_metadata(self, track, filename, encoding):
+        logger.info("Applying metadata")
+        try:
+            track.apply_metadata(filename, encoding=encoding)
+        except TypeError:
+            logger.warning("Cannot apply metadata on provided output format.")
+
+    def strip_and_filter_duplicates(self, tracks):
+        filtered_tracks = spotdl.util.remove_duplicates(
+            tracks,
+            condition=lambda x: x,
+            operation=str.strip
+        )
+        return filtered_tracks
+
+    def filter_against_skip_file(self, items, skip_file):
+        skip_items = spotdl.util.readlines_from_nonbinary_file(skip_file)
+        filtered_skip_items = self.strip_and_filter_duplicates(skip_items)
+        filtered_items = [item for item in items if not item in filtered_skip_items]
+        return filtered_items
+
+    def download_tracks_from_file(self, path):
+        logger.info(
+            'Checking and removing any duplicate tracks in "{}".'.format(path)
+        )
+        tracks = spotdl.util.readlines_from_nonbinary_file(path)
+        tracks = self.strip_and_filter_duplicates(tracks)
+
+        if self.arguments["skip_file"]:
+            len_tracks_before = len(tracks)
+            tracks = self.filter_against_skip_file(tracks, self.arguments["skip_file"])
+            logger.info("Skipping {} tracks due to matches in skip file.".format(
+                len_tracks_before - len(tracks))
+            )
+        # Overwrite file
+        spotdl.util.writelines_to_nonbinary_file(path, tracks)
+
+        logger.info(
+            "Downloading {n} tracks.\n".format(n=len(tracks))
+        )
+
+        for position, track in enumerate(tracks, 1):
+            search_metadata = MetadataSearch(
+                track,
+                lyrics=True,
+                yt_search_format=self.arguments["search_format"],
+                yt_manual=self.arguments["manual"]
+            )
+            log_track_query = '{position}. Downloading "{track}"'.format(
+                position=position,
+                track=track
+            )
+            logger.info(log_track_query)
+            try:
+                metadata = search_metadata.on_youtube_and_spotify()
+                self.download_track_from_metadata(metadata)
+            except (urllib.request.URLError, TypeError, IOError) as e:
+                logger.exception(e.args[0])
+                logger.warning(
+                    "Failed to download current track due to possible network issue. "
+                    "Will retry after other songs."
+                )
+                tracks.append(track)
+            except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
+                logger.error("{err}".format(err=e.args[0]))
+            except KeyboardInterrupt:
+                # The current track hasn't been downloaded completely.
+                # Make sure we continue from here the next the program runs.
+                tracks.insert(0, track)
+                raise
+            else:
+                if self.arguments["write_successful_file"]:
+                    with open(self.arguments["write_successful_file"], "a") as fout:
+                        fout.write("{}\n".format(track))
+            finally:
+                spotdl.util.writelines_to_nonbinary_file(path, tracks[position:])
+                print("", file=sys.stderr)
+
+    """
+    def download_tracks_from_file_threaded(self, path):
+        # FIXME: Can we make this function cleaner?
+
+        logger.info(
+            "Checking and removing any duplicate tracks in {}.\n".format(path)
+        )
+        with open(path, "r") as fin:
+            # Read tracks into a list and remove any duplicates
+            tracks = fin.read().splitlines()
+
+        # Remove duplicates and empty elements
+        # Also strip whitespaces from elements (if any)
+        spotdl.util.remove_duplicates(
+            tracks,
+            condition=lambda x: x,
+            operation=str.strip
+        )
+
+        # Overwrite file
+        with open(path, "w") as fout:
+            fout.writelines(tracks)
+
+        tracks_count = len(tracks)
+        current_iteration = 1
+
+        next_track = tracks.pop(0)
+        metadata = {
+            "current_track": None,
+            "next_track": spotdl.util.ThreadWithReturnValue(
+                target=search_metadata,
+                args=(next_track, self.arguments["search_format"])
+            )
+        }
+        metadata["next_track"].start()
+        while tracks_count > 0:
+            metadata["current_track"] = metadata["next_track"].join()
+            metadata["next_track"] = None
+            try:
+                print(tracks_count, file=sys.stderr)
+                print(tracks, file=sys.stderr)
+                if tracks_count > 1:
+                    current_track = next_track
+                    next_track = tracks.pop(0)
+                    metadata["next_track"] = spotdl.util.ThreadWithReturnValue(
+                        target=search_metadata,
+                        args=(next_track, self.arguments["search_format"])
+                    )
+                    metadata["next_track"].start()
+
+                log_track_query = str(current_iteration) + ". {artist} - {track-name}"
+                logger.info(log_track_query)
+                if metadata["current_track"] is None:
+                    logger.warning("Something went wrong. Will retry after downloading remaining tracks.")
+                    pass
+                print(metadata["current_track"]["name"], file=sys.stderr)
+                # self.download_track_from_metadata(metadata["current_track"])
+            except (urllib.request.URLError, TypeError, IOError) as e:
+                print("", file=sys.stderr)
+                logger.exception(e.args[0])
+                logger.warning("Failed. Will retry after other songs\n")
+                tracks.append(current_track)
+            else:
+                tracks_count -= 1
+                if self.arguments["write_sucessful_file"]:
+                    with open(self.arguments["write_sucessful_file"], "a") as fout:
+                        fout.write(current_track)
+            finally:
+                current_iteration += 1
+                with open(path, "w") as fout:
+                    fout.writelines(tracks)
+    """
+

spotdl/command_line/exceptions.py

@@ -0,0 +1,20 @@
+class NoYouTubeVideoFoundError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class NoYouTubeVideoMatchError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class ArgumentError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+

spotdl/command_line/tests/test_arguments.py

@@ -0,0 +1,90 @@
+import spotdl.command_line.arguments
+from spotdl.command_line.exceptions import ArgumentError
+
+import logging
+import sys
+import pytest
+
+
+def test_logging_levels():
+    expect_logging_levels = {
+        "INFO": logging.INFO,
+        "WARNING": logging.WARNING,
+        "DEBUG": logging.DEBUG,
+        "ERROR": logging.ERROR,
+    }
+    assert spotdl.command_line.arguments._LOG_LEVELS == expect_logging_levels
+
+
+class TestBadArguments:
+    def test_error_m3u_without_list(self):
+        previous_argv = sys.argv
+        sys.argv[1:] = ["-s", "cool song", "--write-m3u"]
+        argument_handler = spotdl.command_line.arguments.get_arguments()
+        with pytest.raises(ArgumentError):
+            argument_handler.run_errands()
+        sys.argv[1:] = previous_argv[1:]
+
+    def test_write_to_error(self):
+        previous_argv = sys.argv
+        sys.argv[1:] = ["-s", "sekai all i had", "--write-to", "output.txt"]
+        argument_handler = spotdl.command_line.arguments.get_arguments()
+        with pytest.raises(ArgumentError):
+            argument_handler.run_errands()
+        sys.argv[1:] = previous_argv[1:]
+
+
+class TestArguments:
+    @pytest.mark.xfail
+    def test_general_arguments(self):
+        arguments = spotdl.command_line.arguments.get_arguments(argv=("-t", "elena coats - one last song"))
+        arguments = arguments.__dict__
+
+        assert isinstance(arguments["spotify_client_id"], str)
+        assert isinstance(arguments["spotify_client_secret"], str)
+
+        arguments["spotify_client_id"] = None
+        arguments["spotify_client_secret"] = None
+
+        expect_arguments = {
+            "song": ["elena coats - one last song"],
+            "song": None,
+            "list": None,
+            "playlist": None,
+            "album": None,
+            "all_albums": None,
+            "username": None,
+            "write_m3u": False,
+            "manual": False,
+            "no_remove_original": False,
+            "no_metadata": False,
+            "no_fallback_metadata": False,
+            "directory": "/home/ritiek/Music",
+            "overwrite": "prompt",
+            "input_ext": ".m4a",
+            "output_ext": ".mp3",
+            "write_to": None,
+            "file_format": "{artist} - {track_name}",
+            "trim_silence": False,
+            "search_format": "{artist} - {track_name} lyrics",
+            "download_only_metadata": False,
+            "dry_run": False,
+            "music_videos_only": False,
+            "no_spaces": False,
+            "log_level": 20,
+            "skip": None,
+            "write_successful": None,
+            "spotify_client_id": None,
+            "spotify_client_secret": None,
+            "config": None
+        }
+
+        assert arguments == expect_arguments
+
+    def test_grouped_arguments(self):
+        previous_argv = sys.argv
+        sys.argv[1:] = []
+        with pytest.raises(SystemExit):
+            argument_handler = spotdl.command_line.arguments.get_arguments()
+        sys.argv[1:] = previous_argv[1:]
+

spotdl/config.py

@@ -0,0 +1,52 @@
+import appdirs
+import yaml
+import os
+
+import spotdl.util
+import logging
+logger = logging.getLogger(__name__)
+
+DEFAULT_CONFIGURATION = {
+    "spotify-downloader": {
+        "manual": False,
+        "no_metadata": False,
+        "no_encode": False,
+        "overwrite": "prompt",
+        "quality": "best",
+        "input_ext": "automatic",
+        "output_ext": "mp3",
+        "write_to": None,
+        "trim_silence": False,
+        "search_format": "{artist} - {track-name} lyrics",
+        "dry_run": False,
+        "no_spaces": False,
+        # "processor": "synchronous",
+        "output_file": "{artist} - {track-name}.{output-ext}",
+        "skip_file": None,
+        "write_successful_file": None,
+        "spotify_client_id": "4fe3fecfe5334023a1472516cc99d805",
+        "spotify_client_secret": "0f02b7c483c04257984695007a4a8d5c",
+        "log_level": "INFO",
+    }
+}
+
+DEFAULT_CONFIG_FILE = os.path.join(
+    appdirs.user_config_dir(),
+    "spotdl",
+    "config.yml"
+)
+
+def read_config(config_file):
+    with open(config_file, "r") as ymlfile:
+        config = yaml.safe_load(ymlfile)
+    return config
+
+
+def dump_config(config_file=None, config=DEFAULT_CONFIGURATION):
+    if config_file is None:
+        config = yaml.dump(config, default_flow_style=False)
+        return config
+
+    with open(config_file, "w") as ymlfile:
+        yaml.dump(config, ymlfile, default_flow_style=False)
+

spotdl/encode/__init__.py

@@ -0,0 +1 @@
+from spotdl.encode.encode_base import EncoderBase

spotdl/encode/encode_base.py

@@ -0,0 +1,119 @@
+import shutil
+import os
+
+from abc import ABC
+from abc import abstractmethod
+
+from spotdl.encode.exceptions import EncoderNotFoundError
+
+"""
+  NOTE ON ENCODERS
+  ================
+
+* FFmeg encoders sorted in descending order based
+  on the quality of audio produced:
+  libopus > libvorbis >= libfdk_aac > aac > libmp3lame
+
+* libfdk_aac encoder, due to copyrights needs to be compiled
+  by end user on MacOS brew install ffmpeg --with-fdk-aac
+  will do just that. Other OS? See:
+  https://trac.ffmpeg.org/wiki/Encode/AAC
+
+"""
+
+_TARGET_FORMATS_FROM_ENCODING = {
+    "m4a": "mp4",
+    "mp3": "mp3",
+    "opus": "opus",
+    "flac": "flac"
+}
+
+
+class EncoderBase(ABC):
+    """
+    Defined encoders must inherit from this abstract base class
+    and implement their own functionality for the below defined
+    methods.
+    """
+
+    @abstractmethod
+    def __init__(self, encoder_path, must_exist, loglevel, additional_arguments=[]):
+        """
+        This method must make sure whether specified encoder
+        is available under PATH.
+        """
+        if must_exist and shutil.which(encoder_path) is None:
+            raise EncoderNotFoundError(
+                "{} executable does not exist or was not found in PATH.".format(
+                    encoder_path
+                )
+            )
+        self.encoder_path = encoder_path
+        self._loglevel = loglevel
+        self._additional_arguments = additional_arguments
+        self._target_formats_from_encoding = _TARGET_FORMATS_FROM_ENCODING
+
+    def set_argument(self, argument):
+        """
+        This method must be used to set any custom functionality
+        for the encoder by passing arguments to it.
+        """
+        self._additional_arguments += argument.split()
+
+    def get_encoding(self, path):
+        """
+        This method must determine the encoding for a local
+        audio file. Such as "mp3", "wav", "m4a", etc.
+        """
+        _, extension = os.path.splitext(path)
+        # Ignore the initial dot from file extension
+        return extension[1:]
+
+    @abstractmethod
+    def set_debuglog(self):
+        """
+        This method must enable verbose logging in the defined
+        encoder.
+        """
+        pass
+
+    @abstractmethod
+    def _generate_encode_command(self, input_path, target_path):
+        """
+        This method must the complete command for that would be
+        used to invoke the encoder and perform the encoding.
+        """
+        pass
+
+    @abstractmethod
+    def _generate_encoding_arguments(self, input_encoding, target_encoding):
+        """
+        This method must return the core arguments for the defined
+        encoder such as defining the sample rate, audio bitrate,
+        etc.
+        """
+        pass
+
+    @abstractmethod
+    def re_encode(self, input_path, target_path):
+        """
+        This method must invoke the encoder to encode a given input
+        file to a specified output file.
+        """
+        pass
+
+    def target_format_from_encoding(self, encoding):
+        """
+        This method generates the target stream format from given
+        input encoding.
+        """
+        target_format = self._target_formats_from_encoding[encoding]
+        return target_format
+
+    def re_encode_from_stdin(self, input_encoding, target_path):
+        """
+        This method must invoke the encoder to encode stdin to a
+        specified output file.
+        """
+        raise NotImplementedError
+

spotdl/encode/encoders/__init__.py

@@ -0,0 +1 @@
+from spotdl.encode.encoders.ffmpeg import EncoderFFmpeg

spotdl/encode/encoders/ffmpeg.py

@@ -0,0 +1,112 @@
+import subprocess
+import os
+
+from spotdl.encode import EncoderBase
+from spotdl.encode.exceptions import EncoderNotFoundError
+from spotdl.encode.exceptions import FFmpegNotFoundError
+
+import logging
+logger = logging.getLogger(__name__)
+
+
+# Key: from format
+# Subkey: to format
+RULES = {
+    "m4a": {
+        "mp3": "-codec:v copy -codec:a libmp3lame -ar 48000",
+        "opus": "-codec:a libopus -vbr on",
+        "m4a": "-acodec copy",
+        "flac": "-codec:a flac -ar 48000",
+    },
+    "opus": {
+        "mp3": "-codec:a libmp3lame -ar 48000",
+        "m4a": "-cutoff 20000 -codec:a aac -ar 48000",
+        "flac": "-codec:a flac -ar 48000",
+    },
+}
+
+
+class EncoderFFmpeg(EncoderBase):
+    def __init__(self, encoder_path="ffmpeg", must_exist=True):
+        _loglevel = "-hide_banner -nostats -v panic"
+        _additional_arguments = ["-b:a", "192k", "-vn"]
+        try:
+            super().__init__(encoder_path, must_exist, _loglevel, _additional_arguments)
+        except EncoderNotFoundError as e:
+            raise FFmpegNotFoundError(e.args[0])
+        self._rules = RULES
+
+    def set_trim_silence(self):
+        self.set_argument("-af silenceremove=start_periods=1")
+
+    def get_encoding(self, path):
+        return super().get_encoding(path)
+
+    def _generate_encoding_arguments(self, input_encoding, target_encoding):
+        initial_arguments = self._rules.get(input_encoding)
+        if initial_arguments is None:
+            raise TypeError(
+                'The input format ("{}") is not supported.'.format(
+                input_encoding,
+            ))
+        arguments = initial_arguments.get(target_encoding)
+        if arguments is None:
+            raise TypeError(
+                'The output format ("{}") is not supported.'.format(
+                target_encoding,
+            ))
+        return arguments
+
+    def set_debuglog(self):
+        self._loglevel = "-loglevel debug"
+
+    def _generate_encode_command(self, input_path, target_file,
+                                 input_encoding=None, target_encoding=None):
+        if input_encoding is None:
+            input_encoding = self.get_encoding(input_path)
+        if target_encoding is None:
+            target_encoding = self.get_encoding(target_file)
+        arguments = self._generate_encoding_arguments(
+            input_encoding,
+            target_encoding
+        )
+        command = [self.encoder_path] \
+            + ["-y", "-nostdin"] \
+            + self._loglevel.split() \
+            + ["-i", input_path] \
+            + arguments.split() \
+            + self._additional_arguments \
+            + ["-f", self.target_format_from_encoding(target_encoding)] \
+            + [target_file]
+
+        return command
+
+    def re_encode(self, input_path, target_file, target_encoding=None, delete_original=False):
+        encode_command = self._generate_encode_command(
+            input_path,
+            target_file,
+            target_encoding=target_encoding
+        )
+        logger.debug("Calling FFmpeg with:\n{command}".format(
+            command=encode_command,
+        ))
+        process = subprocess.Popen(encode_command)
+        process.wait()
+        encode_successful = process.returncode == 0
+        if encode_successful and delete_original:
+            os.remove(input_path)
+        return process
+
+    def re_encode_from_stdin(self, input_encoding, target_file, target_encoding=None):
+        encode_command = self._generate_encode_command(
+            "-",
+            target_file,
+            input_encoding=input_encoding,
+            target_encoding=target_encoding,
+        )
+        logger.debug("Calling FFmpeg with:\n{command}".format(
+            command=encode_command,
+        ))
+        process = subprocess.Popen(encode_command, stdin=subprocess.PIPE)
+        return process
+

spotdl/encode/encoders/tests/test_ffmpeg.py

@@ -0,0 +1,211 @@
+from spotdl.encode import EncoderBase
+from spotdl.encode.exceptions import FFmpegNotFoundError
+from spotdl.encode.encoders import EncoderFFmpeg
+
+import pytest
+
+class TestEncoderFFmpeg:
+    def test_subclass(self):
+        assert issubclass(EncoderFFmpeg, EncoderBase)
+
+    def test_ffmpeg_not_found_error(self):
+        with pytest.raises(FFmpegNotFoundError):
+            EncoderFFmpeg(encoder_path="/a/nonexistent/path")
+
+
+class TestEncodingDefaults:
+    def m4a_to_mp3_encoder(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:v', 'copy',
+            '-codec:a', 'libmp3lame',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'mp3',
+            target_path
+        ]
+        return command
+
+    def m4a_to_opus_encoder(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:a', 'libopus',
+            '-vbr', 'on',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'opus',
+            target_path
+        ]
+        return command
+
+    def m4a_to_m4a_encoder(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-acodec', 'copy',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'mp4',
+             target_path
+        ]
+        return command
+
+    def m4a_to_flac_encoder(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:a', 'flac',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'flac',
+            target_path
+        ]
+        return command
+
+    @pytest.mark.parametrize("files, expected_command", [
+        (("test.m4a", "test.mp3"), m4a_to_mp3_encoder("test.m4a", "test.mp3")),
+        (("abc.m4a", "cba.opus"), m4a_to_opus_encoder("abc.m4a", "cba.opus")),
+        (("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder("bla bla.m4a", "ble ble.m4a")),
+        (("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder("😛.m4a", "• tongue.flac")),
+    ])
+    def test_generate_encode_command(self, files, expected_command):
+        encoder = EncoderFFmpeg()
+        assert encoder._generate_encode_command(*files) == expected_command
+
+
+class TestEncodingInDebugMode:
+    def m4a_to_mp3_encoder_with_debug(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
+            '-i', input_path,
+            '-codec:v', 'copy',
+            '-codec:a', 'libmp3lame',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'mp3',
+            target_path
+        ]
+        return command
+
+    def m4a_to_opus_encoder_with_debug(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
+            '-i', input_path,
+            '-codec:a', 'libopus',
+            '-vbr', 'on',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'opus',
+             target_path
+        ]
+        return command
+
+    def m4a_to_m4a_encoder_with_debug(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
+            '-i', input_path,
+            '-acodec', 'copy',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'mp4',
+             target_path
+        ]
+        return command
+
+    def m4a_to_flac_encoder_with_debug(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
+            '-i', input_path,
+            '-codec:a', 'flac',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-f', 'flac',
+             target_path
+        ]
+        return command
+
+    @pytest.mark.parametrize("files, expected_command", [
+        (("test.m4a", "test.mp3"), m4a_to_mp3_encoder_with_debug("test.m4a", "test.mp3")),
+        (("abc.m4a", "cba.opus"), m4a_to_opus_encoder_with_debug("abc.m4a", "cba.opus")),
+        (("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder_with_debug("bla bla.m4a", "ble ble.m4a")),
+        (("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder_with_debug("😛.m4a", "• tongue.flac")),
+    ])
+    def test_generate_encode_command_with_debug(self, files, expected_command):
+        encoder = EncoderFFmpeg()
+        encoder.set_debuglog()
+        assert encoder._generate_encode_command(*files) == expected_command
+
+
+class TestEncodingAndTrimSilence:
+    def m4a_to_mp3_encoder_and_trim_silence(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:v', 'copy',
+            '-codec:a', 'libmp3lame',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-af', 'silenceremove=start_periods=1',
+            '-f', 'mp3',
+            target_path
+        ]
+        return command
+
+    def m4a_to_opus_encoder_and_trim_silence(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:a', 'libopus',
+            '-vbr', 'on',
+            '-b:a', '192k',
+            '-vn',
+            '-af', 'silenceremove=start_periods=1',
+            '-f', 'opus',
+            target_path
+        ]
+        return command
+
+    def m4a_to_m4a_encoder_and_trim_silence(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-acodec', 'copy',
+            '-b:a', '192k',
+            '-vn',
+            '-af', 'silenceremove=start_periods=1',
+            '-f', 'mp4',
+            target_path
+        ]
+        return command
+
+    def m4a_to_flac_encoder_and_trim_silence(input_path, target_path):
+        command = [
+            'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
+            '-i', input_path,
+            '-codec:a', 'flac',
+            '-ar', '48000',
+            '-b:a', '192k',
+            '-vn',
+            '-af', 'silenceremove=start_periods=1',
+            '-f', 'flac',
+            target_path
+        ]
+        return command
+
+    @pytest.mark.parametrize("files, expected_command", [
+        (("test.m4a", "test.mp3"), m4a_to_mp3_encoder_and_trim_silence("test.m4a", "test.mp3")),
+        (("abc.m4a", "cba.opus"), m4a_to_opus_encoder_and_trim_silence("abc.m4a", "cba.opus")),
+        (("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder_and_trim_silence("bla bla.m4a", "ble ble.m4a")),
+        (("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder_and_trim_silence("😛.m4a", "• tongue.flac")),
+    ])
+    def test_generate_encode_command_and_trim_silence(self, files, expected_command):
+        encoder = EncoderFFmpeg()
+        encoder.set_trim_silence()
+        assert encoder._generate_encode_command(*files) == expected_command

spotdl/encode/exceptions.py

@@ -0,0 +1,13 @@
+class EncoderNotFoundError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class FFmpegNotFoundError(EncoderNotFoundError):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+

spotdl/encode/tests/test_encode_base.py

@@ -0,0 +1,97 @@
+from spotdl.encode import EncoderBase
+from spotdl.encode.exceptions import EncoderNotFoundError
+
+import pytest
+
+class TestAbstractBaseClass:
+    def test_error_abstract_base_class_encoderbase(self):
+        encoder_path = "ffmpeg"
+        _loglevel = "-hide_banner -nostats -v panic"
+        _additional_arguments = ["-b:a", "192k", "-vn"]
+
+        with pytest.raises(TypeError):
+            # This abstract base class must be inherited from
+            # for instantiation
+            EncoderBase(encoder_path, _loglevel, _additional_arguments)
+
+
+    def test_inherit_abstract_base_class_encoderbase(self):
+        class EncoderKid(EncoderBase):
+            def __init__(self, encoder_path, _loglevel, _additional_arguments):
+                super().__init__(encoder_path, _loglevel, _additional_arguments)
+
+            def _generate_encode_command(self):
+                pass
+
+            def _generate_encoding_arguments(self):
+                pass
+
+            def re_encode(self):
+                pass
+
+            def set_debuglog(self):
+                pass
+
+
+        encoder_path = "ffmpeg"
+        _loglevel = "-hide_banner -nostats -v panic"
+        _additional_arguments = ["-b:a", "192k", "-vn"]
+
+        EncoderKid(encoder_path, _loglevel, _additional_arguments)
+
+
+class TestMethods:
+    class EncoderKid(EncoderBase):
+        def __init__(self, encoder_path, _loglevel, _additional_arguments):
+            super().__init__(encoder_path, _loglevel, _additional_arguments)
+
+        def _generate_encode_command(self, input_file, target_file):
+            pass
+
+        def _generate_encoding_arguments(self, input_encoding, target_encoding):
+            pass
+
+        def re_encode(self, input_encoding, target_encoding):
+            pass
+
+        def set_debuglog(self):
+            pass
+
+
+    @pytest.fixture(scope="module")
+    def encoderkid(self):
+        encoder_path = "ffmpeg"
+        _loglevel = "-hide_banner -nostats -v panic"
+        _additional_arguments = []
+
+        encoderkid = self.EncoderKid(encoder_path, _loglevel, _additional_arguments)
+        return encoderkid
+
+    def test_set_argument(self, encoderkid):
+        encoderkid.set_argument("-parameter argument")
+        assert encoderkid._additional_arguments == [
+            "-parameter",
+            "argument",
+        ]
+
+    @pytest.mark.parametrize("filename, encoding", [
+        ("example.m4a", "m4a"),
+        ("exampley.mp3", "mp3"),
+        ("test 123.opus", "opus"),
+        ("flakey.flac", "flac"),
+    ])
+    def test_get_encoding(self, encoderkid, filename, encoding):
+        assert encoderkid.get_encoding(filename) == encoding
+
+    def test_encoder_not_found_error(self):
+        with pytest.raises(EncoderNotFoundError):
+            self.EncoderKid("/a/nonexistent/path", "0", [])
+
+    @pytest.mark.parametrize("encoding, target_format", [
+        ("m4a", "mp4"),
+        ("mp3", "mp3"),
+        ("opus", "opus"),
+        ("flac", "flac"),
+    ])
+    def test_target_format_from_encoding(self, encoderkid, encoding, target_format):
+        assert encoderkid.target_format_from_encoding(encoding) == target_format

spotdl/encode/tests/test_encode_exceptions.py

@@ -0,0 +1,11 @@
+from spotdl.encode.exceptions import EncoderNotFoundError
+from spotdl.encode.exceptions import FFmpegNotFoundError
+
+
+class TestEncoderNotFoundSubclass:
+    def test_encoder_not_found_subclass(self):
+        assert issubclass(FFmpegNotFoundError, Exception)
+
+    def test_ffmpeg_not_found_subclass(self):
+        assert issubclass(FFmpegNotFoundError, EncoderNotFoundError)
+

spotdl/helpers/spotify.py

@@ -0,0 +1,166 @@
+from spotdl.authorize.services import AuthorizeSpotify
+import spotdl.util
+
+import sys
+import spotipy
+
+import logging
+logger = logging.getLogger(__name__)
+
+try:
+    from slugify import SLUG_OK, slugify
+except ImportError:
+    logger.error("Oops! `unicode-slugify` was not found.")
+    logger.info("Please remove any other slugify library and install `unicode-slugify`.")
+    raise
+
+
+class SpotifyHelpers:
+    def __init__(self, spotify=None):
+        if spotify is None:
+            spotify = AuthorizeSpotify()
+        self.spotify = spotify
+
+    def prompt_for_user_playlist(self, username):
+        """ Write user playlists to target_file """
+        playlists = self.fetch_user_playlist_urls(username)
+        for i, playlist in enumerate(playlists, 1):
+            playlist_details = "{0}. {1:<30}  ({2} tracks)".format(
+                i, playlist["name"], playlist["tracks"]["total"]
+            )
+            print(playlist_details, file=sys.stderr)
+        print("", file=sys.stderr)
+        playlist = spotdl.util.prompt_user_for_selection(playlists)
+        return playlist["external_urls"]["spotify"]
+
+    def fetch_user_playlist_urls(self, username):
+        """ Fetch user playlists when using the -u option. """
+        logger.debug('Fetching playlists for "{username}".'.format(username=username))
+        playlists = self.spotify.user_playlists(username)
+        collected_playlists = []
+        check = 1
+
+        while True:
+            for playlist in playlists["items"]:
+                # in rare cases, playlists may not be found, so playlists['next']
+                # is None. Skip these. Also see Issue #91.
+                if playlist["name"] is not None:
+                    collected_playlists.append(playlist)
+                    check += 1
+            if playlists["next"]:
+                playlists = self.spotify.next(playlists)
+            else:
+                break
+
+        return collected_playlists
+
+    def fetch_playlist(self, playlist_url):
+        logger.debug('Fetching playlist "{playlist}".'.format(playlist=playlist_url))
+        try:
+            results = self.spotify.playlist(playlist_url, fields="tracks,next,name")
+        except spotipy.client.SpotifyException:
+            logger.exception(
+                "Unable to find playlist. Make sure the playlist is set "
+                "to publicly visible and then try again."
+            )
+
+        return results
+
+    def write_playlist_tracks(self, playlist, target_file=None):
+        tracks = playlist["tracks"]
+        if not target_file:
+            target_file = u"{0}.txt".format(slugify(playlist["name"], ok="-_()[]{}"))
+        return self.write_tracks(tracks, target_file)
+
+    def fetch_album(self, album_uri):
+        logger.debug('Fetching album "{album}".'.format(album=album_uri))
+        album = self.spotify.album(album_uri)
+        return album
+
+    def write_album_tracks(self, album, target_file=None):
+        tracks = self.spotify.album_tracks(album["id"])
+        if not target_file:
+            target_file = u"{0}.txt".format(slugify(album["name"], ok="-_()[]{}"))
+        return self.write_tracks(tracks, target_file)
+
+    def fetch_albums_from_artist(self, artist_uri, album_type=None):
+        """
+        This function returns all the albums from a give artist_uri using the US
+        market
+        :param artist_uri - spotify artist uri
+        :param album_type - the type of album to fetch (ex: single) the default is
+                            all albums
+        :param return - the album from the artist
+        """
+
+        logger.debug('Fetching all albums for "{artist}".'.format(artist=artist_uri))
+        # fetching artist's albums limitting the results to the US to avoid duplicate
+        # albums from multiple markets
+        results = self.spotify.artist_albums(artist_uri, album_type=album_type, country="US")
+
+        albums = results["items"]
+
+        # indexing all pages of results
+        while results["next"]:
+            results = self.spotify.next(results)
+            albums.extend(results["items"])
+
+        return albums
+
+    def write_all_albums(self, albums, target_file=None):
+        """
+        This function gets all albums from an artist and writes it to a file in the
+        current working directory called [ARTIST].txt, where [ARTIST] is the artist
+        of the album
+        :param artist_uri - spotify artist uri
+        :param target_file - file to write albums to
+        """
+
+        # if no file if given, the default save file is in the current working
+        # directory with the name of the artist
+        if target_file is None:
+            target_file = albums[0]["artists"][0]["name"] + ".txt"
+
+        for album in albums:
+            logger.info('Fetching album "{album}".'.format(album=album["name"]))
+            self.write_album_tracks(album, target_file=target_file)
+
+    def write_tracks(self, tracks, target_file):
+        def writer(tracks, file_io):
+            track_urls = []
+            while True:
+                for item in tracks["items"]:
+                    if "track" in item:
+                        track = item["track"]
+                    else:
+                        track = item
+                    try:
+                        track_url = track["external_urls"]["spotify"]
+                        file_io.write(track_url + "\n")
+                        track_urls.append(track_url)
+                    except KeyError:
+                        # FIXME: Write "{artist} - {name}" instead of Spotify URI for
+                        #        "local only" tracks.
+                        logger.warning(
+                            'Skipping track "{0}" by "{1}" (local only?)'.format(
+                                track["name"], track["artists"][0]["name"]
+                            )
+                        )
+                # 1 page = 50 results
+                # check if there are more pages
+                if tracks["next"]:
+                    tracks = self.spotify.next(tracks)
+                else:
+                    break
+            return track_urls
+
+        logger.info(u"Writing {0} tracks to {1}.".format(tracks["total"], target_file))
+        write_to_stdout = target_file == "-"
+        if write_to_stdout:
+            file_out = sys.stdout
+            track_urls = writer(tracks, file_out)
+        else:
+            with open(target_file, "a") as file_out:
+                track_urls = writer(tracks, file_out)
+        return track_urls
+

spotdl/lyrics/__init__.py

@@ -0,0 +1 @@
+from spotdl.lyrics.lyric_base import LyricBase

spotdl/lyrics/exceptions.py

@@ -0,0 +1,5 @@
+class LyricsNotFoundError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)

spotdl/lyrics/lyric_base.py

@@ -0,0 +1,36 @@
+import lyricwikia
+
+from abc import ABC
+from abc import abstractmethod
+
+
+class LyricBase(ABC):
+    """
+    Defined lyric providers must inherit from this abstract base
+    class and implement their own functionality for the below
+    defined methods.
+    """
+
+    @abstractmethod
+    def from_url(self, url, linesep="\n", timeout=None):
+        """
+        This method must return the lyrics string for the
+        given track.
+        """
+        pass
+
+    @abstractmethod
+    def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
+        """
+        This method must return the lyrics string for the
+        given track.
+        """
+        pass
+
+    @abstractmethod
+    def from_query(self, query, linesep="\n", timeout=None):
+        """
+        This method must return the lyrics string for the
+        given track.
+        """
+        pass

spotdl/lyrics/providers/__init__.py

@@ -0,0 +1,4 @@
+from spotdl.lyrics.providers.genius import Genius
+from spotdl.lyrics.providers.lyricwikia_wrapper import LyricWikia
+
+LyricClasses = (Genius, LyricWikia)

spotdl/lyrics/providers/genius.py

@@ -0,0 +1,151 @@
+from bs4 import BeautifulSoup
+import urllib.request
+import json
+
+from spotdl.lyrics.lyric_base import LyricBase
+from spotdl.lyrics.exceptions import LyricsNotFoundError
+
+import logging
+logger = logging.getLogger(__name__)
+
+BASE_URL = "https://genius.com"
+BASE_SEARCH_URL = BASE_URL + "/api/search/multi?per_page=1&q="
+
+# FIXME: Make Genius a metadata provider instead of lyric provider
+#        Since, Genius parses additional metadata too (such as track
+#        name, artist name, albumart url). For example, fetch this URL:
+#        https://genius.com/api/search/multi?per_page=1&q=artist+trackname
+
+class Genius(LyricBase):
+    def __init__(self):
+        self.base_url = BASE_URL
+        self.base_search_url = BASE_SEARCH_URL
+
+    def guess_lyric_url_from_artist_and_track(self, artist, track):
+        """
+        Returns the possible lyric URL for the track available on
+        Genius. This may not always be a valid URL.
+        """
+        query = "/{} {} lyrics".format(artist, track)
+        query = query.replace(" ", "-")
+        encoded_query = urllib.request.quote(query)
+        lyric_url = self.base_url + encoded_query
+        return lyric_url
+
+    def _fetch_url_page(self, url, timeout=None):
+        """
+        Makes a GET request to the given lyrics page URL and returns
+        the HTML content in the case of a valid response.
+        """
+        request = urllib.request.Request(url)
+        request.add_header("User-Agent", "urllib")
+        try:
+            response = urllib.request.urlopen(request, timeout=timeout)
+        except urllib.request.HTTPError:
+            raise LyricsNotFoundError(
+                "Could not find Genius lyrics at URL: {}".format(url)
+            )
+        else:
+            return response.read()
+
+    def _get_lyrics_text(self, paragraph):
+        """
+        Extracts and returns the lyric content from the provided HTML.
+        """
+        if paragraph:
+            return paragraph.get_text()
+        else:
+            raise LyricsNotFoundError(
+                "The lyrics for this track are yet to be released on Genius."
+            )
+
+    def _fetch_search_page(self, url, timeout=None):
+        """
+        Returns search results from a given URL in JSON.
+        """
+        request = urllib.request.Request(url)
+        request.add_header("User-Agent", "urllib")
+        response = urllib.request.urlopen(request, timeout=timeout)
+        metadata = json.loads(response.read())
+        if len(metadata["response"]["sections"][0]["hits"]) == 0:
+            raise LyricsNotFoundError(
+                "Genius returned no lyric results for the search URL: {}".format(url)
+            )
+        return metadata
+
+    def best_matching_lyric_url_from_query(self, query):
+        """
+        Returns the best matching track's URL from a given query.
+        """
+        encoded_query = urllib.request.quote(query.replace(" ", "+"))
+        search_url = self.base_search_url + encoded_query
+        logger.debug('Fetching Genius search results from "{}".'.format(search_url))
+        metadata = self._fetch_search_page(search_url)
+
+        lyric_url = None
+        for section in metadata["response"]["sections"]:
+            result = section["hits"][0]["result"]
+            try:
+                lyric_url = result["path"]
+                break
+            except KeyError:
+                pass
+
+        if lyric_url is None:
+            raise LyricsNotFoundError(
+                "Could not find any valid lyric paths in Genius "
+                "lyrics API response for the query {}.".format(query)
+            )
+
+        return self.base_url + lyric_url
+
+    def from_query(self, query, linesep="\n", timeout=None):
+        """
+        Returns the lyric string for the track best matching the
+        given query.
+        """
+        logger.debug('Fetching lyrics for the search query on "{}".'.format(query))
+        try:
+            lyric_url = self.best_matching_lyric_url_from_query(query)
+        except LyricsNotFoundError:
+            raise LyricsNotFoundError(
+                'Genius returned no lyric results for the search query "{}".'.format(query)
+            )
+        else:
+            return self.from_url(lyric_url, linesep, timeout=timeout)
+
+    def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
+        """
+        Returns the lyric string for the given artist and track
+        by making scraping search results and fetching the first
+        result.
+        """
+        lyric_url = self.guess_lyric_url_from_artist_and_track(artist, track)
+        return self.from_url(lyric_url, linesep, timeout=timeout)
+
+    def from_url(self, url, linesep="\n", retries=5, timeout=None):
+        """
+        Returns the lyric string for the given URL.
+        """
+        logger.debug('Fetching lyric text from "{}".'.format(url))
+        lyric_html_page = self._fetch_url_page(url, timeout=timeout)
+        soup = BeautifulSoup(lyric_html_page, "html.parser")
+        paragraph = soup.find("p")
+        # If <p> has a class (like <p class="bla">), then we got an invalid
+        # response. Retry in such a case.
+        invalid_response = paragraph.get("class") is not None
+        to_retry = retries > 0 and invalid_response
+        if to_retry:
+            logger.debug(
+                "Retrying since Genius returned invalid response for search "
+                "results. Retries left: {retries}.".format(retries=retries)
+            )
+            return self.from_url(url, linesep=linesep, retries=retries-1, timeout=timeout)
+
+        if invalid_response:
+            raise LyricsNotFoundError(
+                'Genius returned invalid response for the search URL "{}".'.format(url)
+            )
+        lyrics = self._get_lyrics_text(paragraph)
+        return lyrics.replace("\n", linesep)
+

spotdl/lyrics/providers/lyricwikia_wrapper.py

@@ -0,0 +1,24 @@
+import lyricwikia
+
+from spotdl.lyrics.lyric_base import LyricBase
+from spotdl.lyrics.exceptions import LyricsNotFoundError
+
+
+class LyricWikia(LyricBase):
+    def from_query(self, query, linesep="\n", timeout=None):
+        raise NotImplementedError
+
+    def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
+        """
+        Returns the lyric string for the given artist and track.
+        """
+        try:
+            lyrics = lyricwikia.get_lyrics(artist, track, linesep, timeout)
+        except lyricwikia.LyricsNotFound as e:
+            raise LyricsNotFoundError(e.args[0])
+
+        return lyrics
+
+    def from_url(self, url, linesep="\n", timeout=None):
+        raise NotImplementedError
+

spotdl/lyrics/providers/tests/test_genius.py

@@ -0,0 +1,116 @@
+from spotdl.lyrics import LyricBase
+from spotdl.lyrics import exceptions
+from spotdl.lyrics.providers import Genius
+
+import urllib.request
+import json
+import pytest
+
+class TestGenius:
+    def test_subclass(self):
+        assert issubclass(Genius, LyricBase)
+
+    @pytest.fixture(scope="module")
+    def expect_lyrics_count(self):
+        # This is the number of characters in lyrics found
+        # for the track in `lyric_url` fixture below
+        return 1845
+
+    @pytest.fixture(scope="module")
+    def genius(self):
+        return Genius()
+
+    def test_base_url(self, genius):
+        assert genius.base_url == "https://genius.com"
+
+    @pytest.fixture(scope="module")
+    def artist(self):
+        return "selena gomez"
+
+    @pytest.fixture(scope="module")
+    def track(self):
+        return "wolves"
+
+    @pytest.fixture(scope="module")
+    def query(self, artist, track):
+        return "{} {}".format(artist, track)
+
+    @pytest.fixture(scope="module")
+    def guess_url(self, query):
+        return "https://genius.com/selena-gomez-wolves-lyrics"
+
+    @pytest.fixture(scope="module")
+    def lyric_url(self):
+        return "https://genius.com/Selena-gomez-and-marshmello-wolves-lyrics"
+
+    def test_guess_lyric_url_from_artist_and_track(self, genius, artist, track, guess_url):
+        url = genius.guess_lyric_url_from_artist_and_track(artist, track)
+        assert url == guess_url
+
+    class MockHTTPResponse:
+        expect_lyrics = ""
+
+        def __init__(self, request, timeout=None):
+            search_results_url = "https://genius.com/api/search/multi?per_page=1&q=selena%2Bgomez%2Bwolves"
+            if request._full_url == search_results_url:
+                read_method = lambda: json.dumps({
+                    "response": {"sections": [{"hits": [{"result": {
+                        "path": "/Selena-gomez-and-marshmello-wolves-lyrics"
+                    } }] }] }
+                })
+            else:
+                read_method = lambda: "<p>" + self.expect_lyrics + "</p>"
+
+            self.read = read_method
+
+    @pytest.mark.network
+    def test_best_matching_lyric_url_from_query(self, genius, query, lyric_url):
+        url = genius.best_matching_lyric_url_from_query(query)
+        assert url == lyric_url
+
+    def test_mock_best_matching_lyric_url_from_query(self, genius, query, lyric_url, monkeypatch):
+        monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
+        self.test_best_matching_lyric_url_from_query(genius, query, lyric_url)
+
+    @pytest.mark.network
+    def test_from_url(self, genius, lyric_url, expect_lyrics_count):
+        lyrics = genius.from_url(lyric_url)
+        assert len(lyrics) == expect_lyrics_count
+
+    def test_mock_from_url(self, genius, lyric_url, expect_lyrics_count, monkeypatch):
+        self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
+        monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
+        self.test_from_url(genius, lyric_url, expect_lyrics_count)
+
+    @pytest.mark.network
+    def test_from_artist_and_track(self, genius, artist, track, expect_lyrics_count):
+        lyrics = genius.from_artist_and_track(artist, track)
+        assert len(lyrics) == expect_lyrics_count
+
+    def test_mock_from_artist_and_track(self, genius, artist, track, expect_lyrics_count, monkeypatch):
+        self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
+        monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
+        self.test_from_artist_and_track(genius, artist, track, expect_lyrics_count)
+
+    @pytest.mark.network
+    def test_from_query(self, genius, query, expect_lyrics_count):
+        lyrics = genius.from_query(query)
+        assert len(lyrics) == expect_lyrics_count
+
+    def test_mock_from_query(self, genius, query, expect_lyrics_count, monkeypatch):
+        self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
+        monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
+        self.test_from_query(genius, query, expect_lyrics_count)
+
+    @pytest.mark.network
+    def test_lyrics_not_found_error(self, genius):
+        with pytest.raises(exceptions.LyricsNotFoundError):
+            genius.from_artist_and_track(self, "nonexistent_artist", "nonexistent_track")
+
+    def test_mock_lyrics_not_found_error(self, genius, monkeypatch):
+        def mock_urlopen(url, timeout=None):
+            raise urllib.request.HTTPError("", "", "", "", "")
+
+        monkeypatch.setattr("urllib.request.urlopen", mock_urlopen)
+        self.test_lyrics_not_found_error(genius)
+

spotdl/lyrics/providers/tests/test_lyricwikia_wrapper.py

@@ -0,0 +1,36 @@
+import lyricwikia
+
+from spotdl.lyrics import LyricBase
+from spotdl.lyrics import exceptions
+from spotdl.lyrics.providers import LyricWikia
+
+import pytest
+
+
+class TestLyricWikia:
+    def test_subclass(self):
+        assert issubclass(LyricWikia, LyricBase)
+
+    def test_from_artist_and_track(self, monkeypatch):
+        # `LyricWikia` class uses the 3rd party method `lyricwikia.get_lyrics`
+        # internally and there is no need to test a 3rd party library as they
+        # have their own implementation of tests.
+        monkeypatch.setattr(
+            "lyricwikia.get_lyrics", lambda a, b, c, d: "awesome lyrics!"
+        )
+        artist, track = "selena gomez", "wolves"
+        lyrics = LyricWikia().from_artist_and_track(artist, track)
+        assert lyrics == "awesome lyrics!"
+
+    def test_lyrics_not_found_error(self, monkeypatch):
+        def lyricwikia_lyrics_not_found(msg):
+            raise lyricwikia.LyricsNotFound(msg)
+
+        # Wrap `lyricwikia.LyricsNotFoundError` with `exceptions.LyricsNotFoundError` error.
+        monkeypatch.setattr(
+            "lyricwikia.get_lyrics",
+            lambda a, b, c, d: lyricwikia_lyrics_not_found("Nope, no lyrics."),
+        )
+        artist, track = "nonexistent_artist", "nonexistent_track"
+        with pytest.raises(exceptions.LyricsNotFoundError):
+            LyricWikia().from_artist_and_track(artist, track)

spotdl/lyrics/tests/test_lyric_base.py

@@ -0,0 +1,24 @@
+from spotdl.lyrics import LyricBase
+
+import pytest
+
+
+class TestAbstractBaseClass:
+    def test_error_abstract_base_class_lyricbase(self):
+        with pytest.raises(TypeError):
+            # This abstract base class must be inherited from
+            # for instantiation
+            LyricBase()
+
+    def test_inherit_abstract_base_class_encoderbase(self):
+        class LyricKid(LyricBase):
+            def from_query(self, query):
+                raise NotImplementedError
+
+            def from_artist_and_track(self, artist, track):
+                pass
+
+            def from_url(self, url):
+                raise NotImplementedError
+
+        LyricKid()

spotdl/lyrics/tests/test_lyrics_exceptions.py

@@ -0,0 +1,5 @@
+from spotdl.lyrics.exceptions import LyricsNotFoundError
+
+def test_lyrics_not_found_subclass():
+    assert issubclass(LyricsNotFoundError, Exception)
+

spotdl/metadata/__init__.py

@@ -0,0 +1,11 @@
+from spotdl.metadata.provider_base import ProviderBase
+from spotdl.metadata.provider_base import StreamsBase
+
+from spotdl.metadata.exceptions import MetadataNotFoundError
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
+
+from spotdl.metadata.embedder_base import EmbedderBase
+
+from spotdl.metadata.formatter import format_string
+

spotdl/metadata/embedder_base.py

@@ -0,0 +1,94 @@
+import os
+
+from abc import ABC
+from abc import abstractmethod
+
+import urllib.request
+
+class EmbedderBase(ABC):
+    """
+    The subclass must define the supported media file encoding
+    formats here using a static variable - such as:
+
+    >>> supported_formats = ("mp3", "m4a", "flac")
+    """
+    supported_formats = ()
+
+    @abstractmethod
+    def __init__(self):
+        """
+        For every supported format, there must be a corresponding
+        method that applies metadata on this format.
+
+        Such as if mp3 is supported, there must exist a method named
+        `as_mp3` on this class that applies metadata on mp3 files.
+        """
+        # self.targets = { fmt: eval(str("self.as_" + fmt))
+        #                  for fmt in self.supported_formats }
+        #
+        # TODO: The above code seems to fail for some reason
+        #       I do not know.
+        self.targets = {}
+        for fmt in self.supported_formats:
+            # FIXME: Calling `eval` is dangerous here!
+            self.targets[fmt] = eval("self.as_" + fmt)
+
+    def get_encoding(self, path):
+        """
+        This method must determine the encoding for a local
+        audio file. Such as "mp3", "wav", "m4a", etc.
+        """
+        _, extension = os.path.splitext(path)
+        # Ignore the initial dot from file extension
+        return extension[1:]
+
+    def apply_metadata(self, path, metadata, cached_albumart=None, encoding=None):
+        """
+        This method must automatically detect the media encoding
+        format from file path and embed the corresponding metadata
+        on the given file by calling an appropriate submethod.
+        """
+        if cached_albumart is None:
+            cached_albumart = urllib.request.urlopen(
+                metadata["album"]["images"][0]["url"],
+            ).read()
+        if encoding is None:
+            encoding = self.get_encoding(path)
+        if encoding not in self.supported_formats:
+            raise TypeError(
+                'The input format ("{}") is not supported.'.format(
+                encoding,
+            ))
+        embed_on_given_format = self.targets[encoding]
+        embed_on_given_format(path, metadata, cached_albumart=cached_albumart)
+
+    def as_mp3(self, path, metadata, cached_albumart=None):
+        """
+        Method for mp3 support. This method might be defined in
+        a subclass.
+
+        Other methods for additional supported formats must also
+        be declared here.
+        """
+        raise NotImplementedError
+
+    def as_m4a(self, path, metadata, cached_albumart=None):
+        """
+        Method for m4a support. This method might be defined in
+        a subclass.
+
+        Other methods for additional supported formats must also
+        be declared here.
+        """
+        raise NotImplementedError
+
+    def as_flac(self, path, metadata, cached_albumart=None):
+        """
+        Method for flac support. This method might be defined in
+        a subclass.
+
+        Other methods for additional supported formats must also
+        be declared here.
+        """
+        raise NotImplementedError
+

spotdl/metadata/embedders/__init__.py

@@ -0,0 +1,2 @@
+from spotdl.metadata.embedders.default_embedder import EmbedderDefault
+

spotdl/metadata/embedders/default_embedder.py

@@ -0,0 +1,189 @@
+from mutagen.easyid3 import EasyID3
+from mutagen.id3 import ID3, TORY, TYER, TPUB, APIC, USLT, COMM
+from mutagen.mp4 import MP4, MP4Cover
+from mutagen.flac import Picture, FLAC
+
+import urllib.request
+
+from spotdl.metadata import EmbedderBase
+
+import logging
+logger = logging.getLogger(__name__)
+
+# Apple has specific tags - see mutagen docs -
+# http://mutagen.readthedocs.io/en/latest/api/mp4.html
+M4A_TAG_PRESET = {
+    "album": "\xa9alb",
+    "artist": "\xa9ART",
+    "date": "\xa9day",
+    "title": "\xa9nam",
+    "year": "\xa9day",
+    "originaldate": "purd",
+    "comment": "\xa9cmt",
+    "group": "\xa9grp",
+    "writer": "\xa9wrt",
+    "genre": "\xa9gen",
+    "tracknumber": "trkn",
+    "albumartist": "aART",
+    "discnumber": "disk",
+    "cpil": "cpil",
+    "albumart": "covr",
+    "copyright": "cprt",
+    "tempo": "tmpo",
+    "lyrics": "\xa9lyr",
+    "comment": "\xa9cmt",
+}
+
+TAG_PRESET = {}
+for key in M4A_TAG_PRESET.keys():
+    TAG_PRESET[key] = key
+
+
+class EmbedderDefault(EmbedderBase):
+    supported_formats = ("mp3", "m4a", "flac")
+
+    def __init__(self):
+        super().__init__()
+        self._m4a_tag_preset = M4A_TAG_PRESET
+        self._tag_preset = TAG_PRESET
+        # self.provider = "spotify" if metadata["spotify_metadata"] else "youtube"
+
+    def as_mp3(self, path, metadata, cached_albumart=None):
+        """ Embed metadata to MP3 files. """
+        logger.debug('Writing MP3 metadata to "{path}".'.format(path=path))
+        # EasyID3 is fun to use ;)
+        # For supported easyid3 tags:
+        # https://github.com/quodlibet/mutagen/blob/master/mutagen/easyid3.py
+        # Check out somewhere at end of above linked file
+        audiofile = EasyID3(path)
+        self._embed_basic_metadata(audiofile, metadata, "mp3", preset=TAG_PRESET)
+        audiofile["media"] = metadata["type"]
+        audiofile["author"] = metadata["artists"][0]["name"]
+        audiofile["lyricist"] = metadata["artists"][0]["name"]
+        audiofile["arranger"] = metadata["artists"][0]["name"]
+        audiofile["performer"] = metadata["artists"][0]["name"]
+        provider = metadata["provider"]
+        audiofile["website"] = metadata["external_urls"][provider]
+        audiofile["length"] = str(metadata["duration"])
+        if metadata["publisher"]:
+            audiofile["encodedby"] = metadata["publisher"]
+        if metadata["external_ids"]["isrc"]:
+            audiofile["isrc"] = metadata["external_ids"]["isrc"]
+        audiofile.save(v2_version=3)
+
+        # For supported id3 tags:
+        # https://github.com/quodlibet/mutagen/blob/master/mutagen/id3/_frames.py
+        # Each class represents an id3 tag
+        audiofile = ID3(path)
+        if metadata["year"]:
+            audiofile["TORY"] = TORY(encoding=3, text=metadata["year"])
+            audiofile["TYER"] = TYER(encoding=3, text=metadata["year"])
+        if metadata["publisher"]:
+            audiofile["TPUB"] = TPUB(encoding=3, text=metadata["publisher"])
+        provider = metadata["provider"]
+        audiofile["COMM"] = COMM(
+            encoding=3, text=metadata["external_urls"][provider]
+        )
+        if metadata["lyrics"]:
+            audiofile["USLT"] = USLT(
+                encoding=3, desc=u"Lyrics", text=metadata["lyrics"]
+            )
+        if cached_albumart is None:
+            cached_albumart = urllib.request.urlopen(
+                metadata["album"]["images"][0]["url"]
+            ).read()
+            albumart.close()
+        try:
+            audiofile["APIC"] = APIC(
+                encoding=3,
+                mime="image/jpeg",
+                type=3,
+                desc=u"Cover",
+                data=cached_albumart,
+            )
+        except IndexError:
+            pass
+
+        audiofile.save(v2_version=3)
+
+    def as_m4a(self, path, metadata, cached_albumart=None):
+        """ Embed metadata to M4A files. """
+        logger.debug('Writing M4A metadata to "{path}".'.format(path=path))
+        audiofile = MP4(path)
+        self._embed_basic_metadata(audiofile, metadata, "m4a", preset=M4A_TAG_PRESET)
+        if metadata["year"]:
+            audiofile[M4A_TAG_PRESET["year"]] = metadata["year"]
+        provider = metadata["provider"]
+        audiofile[M4A_TAG_PRESET["comment"]] = metadata["external_urls"][provider]
+        if metadata["lyrics"]:
+            audiofile[M4A_TAG_PRESET["lyrics"]] = metadata["lyrics"]
+        try:
+            if cached_albumart is None:
+                cached_albumart = urllib.request.urlopen(
+                    metadata["album"]["images"][0]["url"]
+                ).read()
+                albumart.close()
+            audiofile[M4A_TAG_PRESET["albumart"]] = [
+                MP4Cover(cached_albumart, imageformat=MP4Cover.FORMAT_JPEG)
+            ]
+        except IndexError:
+            pass
+
+        audiofile.save()
+
+    def as_flac(self, path, metadata, cached_albumart=None):
+        logger.debug('Writing FLAC metadata to "{path}".'.format(path=path))
+        audiofile = FLAC(path)
+        self._embed_basic_metadata(audiofile, metadata, "flac")
+        if metadata["year"]:
+            audiofile["year"] = metadata["year"]
+        provider = metadata["provider"]
+        audiofile["comment"] = metadata["external_urls"][provider]
+        if metadata["lyrics"]:
+            audiofile["lyrics"] = metadata["lyrics"]
+
+        image = Picture()
+        image.type = 3
+        image.desc = "Cover"
+        image.mime = "image/jpeg"
+        if cached_albumart is None:
+            cached_albumart = urllib.request.urlopen(
+                metadata["album"]["images"][0]["url"]
+            ).read()
+            albumart.close()
+        image.data = cached_albumart
+        audiofile.add_picture(image)
+
+        audiofile.save()
+
+    def _embed_basic_metadata(self, audiofile, metadata, encoding, preset=TAG_PRESET):
+        audiofile[preset["artist"]] = metadata["artists"][0]["name"]
+        if metadata["album"]["artists"][0]["name"]:
+            audiofile[preset["albumartist"]] = metadata["album"]["artists"][0]["name"]
+        if metadata["album"]["name"]:
+            audiofile[preset["album"]] = metadata["album"]["name"]
+        audiofile[preset["title"]] = metadata["name"]
+        if metadata["release_date"]:
+            audiofile[preset["date"]] = metadata["release_date"]
+            audiofile[preset["originaldate"]] = metadata["release_date"]
+        if metadata["genre"]:
+            audiofile[preset["genre"]] = metadata["genre"]
+        if metadata["copyright"]:
+            audiofile[preset["copyright"]] = metadata["copyright"]
+        if encoding == "flac":
+            audiofile[preset["discnumber"]] = str(metadata["disc_number"])
+        else:
+            audiofile[preset["discnumber"]] = [(metadata["disc_number"], 0)]
+        zfilled_track_number = str(metadata["track_number"]).zfill(len(str(metadata["total_tracks"])))
+        if encoding == "flac":
+            audiofile[preset["tracknumber"]] = zfilled_track_number
+        else:
+            if preset["tracknumber"] == TAG_PRESET["tracknumber"]:
+                audiofile[preset["tracknumber"]] = "{}/{}".format(
+                    zfilled_track_number, metadata["total_tracks"]
+                )
+            else:
+                audiofile[preset["tracknumber"]] = [
+                    (metadata["track_number"], metadata["total_tracks"])
+                ]
+

spotdl/metadata/embedders/tests/test_default_embedder.py

@@ -0,0 +1,9 @@
+from spotdl.metadata.embedders import EmbedderDefault
+
+import pytest
+
+@pytest.mark.xfail
+def test_embedder():
+    # Do not forget to Write tests for this!
+    raise NotImplementedError
+

spotdl/metadata/exceptions.py

@@ -0,0 +1,20 @@
+class MetadataNotFoundError(Exception):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class SpotifyMetadataNotFoundError(MetadataNotFoundError):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+
+
+class YouTubeMetadataNotFoundError(MetadataNotFoundError):
+    __module__ = Exception.__module__
+
+    def __init__(self, message=None):
+        super().__init__(message)
+

spotdl/metadata/formatter.py

@@ -0,0 +1,23 @@
+def format_string(string, metadata, output_extension="", sanitizer=lambda s: s):
+    formats = {
+        "{track-name}"   : metadata["name"],
+        "{artist}"       : metadata["artists"][0]["name"],
+        "{album}"        : metadata["album"]["name"],
+        "{album-artist}" : metadata["artists"][0]["name"],
+        "{genre}"        : metadata["genre"],
+        "{disc-number}"  : metadata["disc_number"],
+        "{duration}"     : metadata["duration"],
+        "{year}"         : metadata["year"],
+        "{original-date}": metadata["release_date"],
+        "{track-number}" : str(metadata["track_number"]).zfill(len(str(metadata["total_tracks"]))),
+        "{total-tracks}" : metadata["total_tracks"],
+        "{isrc}"         : metadata["external_ids"]["isrc"],
+        "{track-id}"     : metadata.get("id", ""),
+        "{output-ext}"   : output_extension,
+    }
+
+    for key, value in formats.items():
+        string = string.replace(key, sanitizer(str(value)))
+
+    return string
+

spotdl/metadata/provider_base.py

@@ -0,0 +1,66 @@
+from abc import ABC
+from abc import abstractmethod
+
+class StreamsBase(ABC):
+    @abstractmethod
+    def __init__(self, streams):
+        """
+        This method must parse audio streams into a list of
+        dictionaries with the keys:
+        "bitrate", "download_url", "encoding", "filesize".
+
+        The list should typically be sorted in descending order
+        based on the audio stream's bitrate.
+
+        This sorted list must be assigned to `self.all`.
+        """
+        self.all = streams
+
+    def getbest(self):
+        """
+        This method must return the audio stream with the
+        highest bitrate.
+        """
+        return self.all[0]
+
+    def getworst(self):
+        """
+        This method must return the audio stream with the
+        lowest bitrate.
+        """
+        return self.all[-1]
+
+
+class ProviderBase(ABC):
+    def set_credentials(self, client_id, client_secret):
+        """
+        This method may or not be used depending on
+        whether the metadata provider requires authentication
+        or not.
+        """
+        pass
+
+    @abstractmethod
+    def from_url(self, url):
+        """
+        This method must return track metadata from the
+        corresponding Spotify URL.
+        """
+        pass
+
+    def from_query(self, query):
+        """
+        This method must return track metadata from the
+        corresponding search query.
+        """
+        raise NotImplementedError
+
+    @abstractmethod
+    def metadata_to_standard_form(self, metadata):
+        """
+        This method must transform the fetched metadata
+        into a format consistent with all other metadata
+        providers, for easy utilization.
+        """
+        pass
+

spotdl/metadata/providers/__init__.py

@@ -0,0 +1,4 @@
+from spotdl.metadata.providers.spotify import ProviderSpotify
+from spotdl.metadata.providers.youtube import ProviderYouTube
+from spotdl.metadata.providers.youtube import YouTubeSearch
+

spotdl/metadata/providers/spotify.py

@@ -0,0 +1,82 @@
+import spotipy
+import spotipy.oauth2 as oauth2
+
+from spotdl.metadata import ProviderBase
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+
+from spotdl.authorize.services import AuthorizeSpotify
+import spotdl.util
+
+import logging
+logger = logging.getLogger(__name__)
+
+
+class ProviderSpotify(ProviderBase):
+    def __init__(self, spotify=None):
+        if spotify is None:
+            spotify = AuthorizeSpotify()
+        self.spotify = spotify
+
+    def set_credentials(self, client_id, client_secret):
+        token = self._generate_token(client_id, client_secret)
+        self.spotify = spotipy.Spotify(auth=token)
+
+    def from_url(self, url):
+        logger.debug('Fetching Spotify metadata for "{url}".'.format(url=url))
+        metadata = self.spotify.track(url)
+        return self.metadata_to_standard_form(metadata)
+
+    def from_query(self, query):
+        tracks = self.search(query)["tracks"]["items"]
+        if not tracks:
+            raise SpotifyMetadataNotFoundError(
+                'Spotify returned no tracks for the search query "{}".'.format(
+                    query,
+                )
+            )
+        return self.metadata_to_standard_form(tracks[0])
+
+    def search(self, query):
+        return self.spotify.search(query)
+
+    def _generate_token(self, client_id, client_secret):
+        """ Generate the token. """
+        credentials = oauth2.SpotifyClientCredentials(
+            client_secret=client_secret,
+        )
+        token = credentials.get_access_token()
+        return token
+
+    def metadata_to_standard_form(self, metadata):
+        artist = self.spotify.artist(metadata["artists"][0]["id"])
+        album = self.spotify.album(metadata["album"]["id"])
+
+        try:
+            metadata[u"genre"] = spotdl.util.titlecase(artist["genres"][0])
+        except IndexError:
+            metadata[u"genre"] = None
+        try:
+            metadata[u"copyright"] = album["copyrights"][0]["text"]
+        except IndexError:
+            metadata[u"copyright"] = None
+        try:
+            metadata[u"external_ids"][u"isrc"]
+        except KeyError:
+            metadata[u"external_ids"][u"isrc"] = None
+
+        metadata[u"release_date"] = album["release_date"]
+        metadata[u"publisher"] = album["label"]
+        metadata[u"total_tracks"] = album["tracks"]["total"]
+
+        # Some sugar
+        metadata["year"], *_ = metadata["release_date"].split("-")
+        metadata["duration"] = metadata["duration_ms"] / 1000.0
+        metadata["provider"] = "spotify"
+
+        # Remove unwanted parameters
+        del metadata["duration_ms"]
+        del metadata["available_markets"]
+        del metadata["album"]["available_markets"]
+
+        return metadata
+

spotdl/metadata/providers/tests/test_spotify.py

@@ -0,0 +1,19 @@
+from spotdl.metadata import ProviderBase
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+from spotdl.metadata.providers import ProviderSpotify
+
+import pytest
+
+class TestProviderSpotify:
+    def test_subclass(self):
+        assert issubclass(ProviderSpotify, ProviderBase)
+
+    @pytest.mark.xfail
+    def test_spotify_stuff(self):
+        raise NotImplementedError
+
+    # def test_metadata_not_found_error(self):
+    #     provider = ProviderSpotify(spotify=spotify)
+    #     with pytest.raises(SpotifyMetadataNotFoundError):
+    #         provider.from_query("This track doesn't exist on Spotify.")
+

spotdl/metadata/providers/tests/test_youtube.py

@@ -0,0 +1,366 @@
+from spotdl.metadata.providers.youtube import YouTubeSearch
+from spotdl.metadata.providers.youtube import YouTubeStreams
+from spotdl.metadata.providers.youtube import YouTubeVideos
+from spotdl.metadata.providers import youtube
+from spotdl.metadata.providers import ProviderYouTube
+from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
+
+import pytube
+import urllib.request
+import pickle
+import sys
+import os
+import pytest
+
+
+@pytest.fixture(scope="module")
+def track():
+    """
+    This query is to be searched on YouTube for queries
+    that do return search results.
+    """
+    return "selena gomez wolves"
+
+
+@pytest.fixture(scope="module")
+def no_result_track():
+    """
+    This query is to be searched on YouTube for queries
+    that return no search results.
+    """
+    return "n0 v1d305 3x157 f0r 7h15 53arc4 qu3ry"
+
+
+@pytest.fixture(scope="module")
+def expect_search_results():
+    """
+    These are the expected search results for the "track"
+    query.
+    """
+    return YouTubeVideos([
+        {'duration': '3:33',
+         'title': 'Selena Gomez, Marshmello - Wolves',
+         'url': 'https://www.youtube.com/watch?v=cH4E_t3m3xM'},
+        {'duration': '3:18',
+         'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=xrbY9gDVms0'},
+        {'duration': '3:21',
+         'title': 'Wolves - Selena Gomez, Marshmello (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=jX0n2rSmDbE'},
+        {'duration': '6:26',
+         'title': 'Selena Gomez and Marshmello - Wolves (Official) Extended',
+         'url': 'https://www.youtube.com/watch?v=rQ6jcpwzQZU'},
+        {'duration': '3:43',
+         'title': 'Selena Gomez, Marshmello - Wolves (Vertical Video)',
+         'url': 'https://www.youtube.com/watch?v=nVzA1uWTydQ'},
+        {'duration': '3:18',
+         'title': 'Selena Gomez, Marshmello - Wolves (Visualizer)',
+         'url': 'https://www.youtube.com/watch?v=-grLLLTza6k'},
+        {'duration': '1:32',
+         'title': 'Wolves - Selena Gomez, Marshmello / Jun Liu Choreography',
+         'url': 'https://www.youtube.com/watch?v=zbWsb36U0uo'},
+        {'duration': '3:17',
+         'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=rykH1BkGwTo'},
+        {'duration': '3:16',
+         'title': 'Selena Gomez, Marshmello - Wolves (8D AUDIO)',
+         'url': 'https://www.youtube.com/watch?v=j0AxZ4V5WQw'},
+        {'duration': '3:47',
+         'title': 'Selena Gomez, Marshmello - Wolves (Vanrip Remix)',
+         'url': 'https://www.youtube.com/watch?v=RyxsaKfu-ZY'}
+    ])
+
+
+@pytest.fixture(scope="module")
+def expect_mock_search_results():
+    """
+    These are the expected mock search results for the
+    "track" query.
+    """
+    return YouTubeVideos([
+        {'duration': '3:33',
+         'title': 'Selena Gomez, Marshmello - Wolves',
+         'url': 'https://www.youtube.com/watch?v=cH4E_t3m3xM'},
+        {'duration': '3:18',
+         'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=xrbY9gDVms0'},
+        {'duration': '3:21',
+         'title': 'Wolves - Selena Gomez, Marshmello (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=jX0n2rSmDbE'},
+        {'duration': '6:26',
+         'title': 'Selena Gomez and Marshmello - Wolves (Official) Extended',
+         'url': 'https://www.youtube.com/watch?v=rQ6jcpwzQZU'},
+        {'duration': '3:43',
+         'title': 'Selena Gomez, Marshmello - Wolves (Vertical Video)',
+         'url': 'https://www.youtube.com/watch?v=nVzA1uWTydQ'},
+        {'duration': '3:18',
+         'title': 'Selena Gomez, Marshmello - Wolves (Visualizer)',
+         'url': 'https://www.youtube.com/watch?v=-grLLLTza6k'},
+        {'duration': '1:32',
+         'title': 'Wolves - Selena Gomez, Marshmello / Jun Liu Choreography',
+         'url': 'https://www.youtube.com/watch?v=zbWsb36U0uo'},
+        {'duration': '3:17',
+         'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
+         'url': 'https://www.youtube.com/watch?v=rykH1BkGwTo'},
+        {'duration': '3:16',
+         'title': 'Selena Gomez, Marshmello - Wolves (8D AUDIO)',
+         'url': 'https://www.youtube.com/watch?v=j0AxZ4V5WQw'},
+        {'duration': '3:47',
+         'title': 'Selena Gomez, Marshmello - Wolves (Vanrip Remix)',
+         'url': 'https://www.youtube.com/watch?v=RyxsaKfu-ZY'}
+    ])
+
+
+class MockHTTPResponse:
+    """
+    This mocks `urllib.request.urlopen` for custom response text.
+    """
+    response_file = ""
+
+    def __init__(self, request):
+        if isinstance(request, urllib.request.Request):
+            if request._full_url.endswith("ouVRL5arzUg=="):
+                self.headers = {"Content-Length": 3614184}
+            elif request._full_url.endswith("egl0iK2D-Bk="):
+                self.headers = {"Content-Length": 3444850}
+            elif request._full_url.endswith("J7VXJtoi3as="):
+                self.headers = {"Content-Length": 1847626}
+            elif request._full_url.endswith("_d5_ZthQdvtD"):
+                self.headers = {"Content-Length": 1407962}
+
+    def read(self):
+        module_directory = os.path.dirname(__file__)
+        mock_html = os.path.join(module_directory, "data", self.response_file)
+        with open(mock_html, "r") as fin:
+            html = fin.read()
+        return html
+
+
+class TestYouTubeSearch:
+    @pytest.fixture(scope="module")
+    def youtube_searcher(self):
+        return YouTubeSearch()
+
+    def test_generate_search_url(self, track, youtube_searcher):
+        url = youtube_searcher.generate_search_url(track)
+        expect_url = "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q=selena%20gomez%20wolves"
+        assert url == expect_url
+
+    @pytest.mark.network
+    def test_search(self, track, youtube_searcher, expect_search_results):
+        results = youtube_searcher.search(track)
+        assert results == expect_search_results
+
+    class MockHTTPResponse:
+        """
+        This mocks `urllib.request.urlopen` for custom response text.
+        """
+        response_file = ""
+
+        def __init__(self, url):
+            pass
+
+        def read(self):
+            module_directory = os.path.dirname(__file__)
+            mock_html = os.path.join(module_directory, "data", self.response_file)
+            with open(mock_html, "r") as fin:
+                html = fin.read()
+            return html
+
+    # @pytest.mark.mock
+    def test_mock_search(self, track, youtube_searcher, expect_mock_search_results, monkeypatch):
+        MockHTTPResponse.response_file = "youtube_search_results.html.test"
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_search(track, youtube_searcher, expect_mock_search_results)
+
+    @pytest.mark.network
+    def test_no_videos_search(self, no_result_track, youtube_searcher):
+        results = youtube_searcher.search(no_result_track)
+        assert results == YouTubeVideos([])
+
+    def test_mock_no_videos_search(self, no_result_track, youtube_searcher, monkeypatch):
+        MockHTTPResponse.response_file = "youtube_no_search_results.html.test"
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_no_videos_search(no_result_track, youtube_searcher)
+
+
+@pytest.fixture(scope="module")
+def content():
+    return pytube.YouTube("https://www.youtube.com/watch?v=cH4E_t3m3xM")
+
+
+class MockYouTube:
+    def __init__(self, url):
+        self.watch_html = '\\"category\\":\\"Music\\",\\"publishDate\\":\\"2017-11-18\\",\\"ownerChannelName\\":\\"SelenaGomezVEVO\\",'
+        self.title = "Selena Gomez, Marshmello - Wolves"
+        self.author = "SelenaGomezVEVO"
+        self.length = 213
+        self.watch_url = "https://youtube.com/watch?v=cH4E_t3m3xM"
+        self.thumbnail_url = "https://i.ytimg.com/vi/cH4E_t3m3xM/maxresdefault.jpg"
+
+    @property
+    def streams(self):
+        # For updating the test data:
+        # from spotdl.metadata.providers.youtube import YouTubeStreams
+        # import pytube
+        # import pickle
+        # content = pytube.YouTube("https://youtube.com/watch?v=cH4E_t3m3xM")
+        # with open("streams.dump", "wb") as fout:
+        #       pickle.dump(content.streams, fout)
+        module_directory = os.path.dirname(__file__)
+        mock_streams = os.path.join(module_directory, "data", "streams.dump")
+        with open(mock_streams, "rb") as fin:
+            streams_dump = pickle.load(fin)
+        return streams_dump
+
+
+@pytest.fixture(scope="module")
+def mock_content():
+    return MockYouTube("https://www.youtube.com/watch?v=cH4E_t3m3xM")
+
+
+@pytest.fixture(scope="module")
+def expect_formatted_streams():
+    """
+    Expected streams for the best matching video for "track" in
+    search results.
+
+    The `download_url` is expected as `None` since it's impossible
+    to predict its value before-hand.
+    """
+    return [
+        {"bitrate": 160, "content": None, "download_url": None, "encoding": "opus", "filesize": 3614184},
+        {"bitrate": 128, "content": None, "download_url": None, "encoding": "mp4a.40.2", "filesize": 3444850},
+        {"bitrate": 70, "content": None, "download_url": None, "encoding": "opus", "filesize": 1847626},
+        {"bitrate": 50, "content": None, "download_url": None, "encoding": "opus", "filesize": 1407962}
+    ]
+
+
+class TestYouTubeStreams:
+    @pytest.mark.network
+    def test_streams(self, content, expect_formatted_streams):
+        formatted_streams = YouTubeStreams(content.streams)
+        for index in range(len(formatted_streams.all)):
+            assert isinstance(formatted_streams.all[index]["download_url"], str)
+            assert formatted_streams.all[index]["connection"] is not None
+            # We `None` the `download_url` since it's impossible to
+            # predict its value before-hand.
+            formatted_streams.all[index]["download_url"] = None
+            formatted_streams.all[index]["connection"] = None
+
+        # assert formatted_streams.all == expect_formatted_streams
+        for f, e in zip(formatted_streams.all, expect_formatted_streams):
+            assert f["filesize"] == e["filesize"]
+
+    # @pytest.mark.mock
+    def test_mock_streams(self, mock_content, expect_formatted_streams, monkeypatch):
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_streams(mock_content, expect_formatted_streams)
+
+    @pytest.mark.network
+    def test_getbest(self, content):
+        formatted_streams = YouTubeStreams(content.streams)
+        best_stream = formatted_streams.getbest()
+        assert isinstance(best_stream["download_url"], str)
+        assert best_stream["connection"] is not None
+        # We `None` the `download_url` since it's impossible to
+        # predict its value before-hand.
+        best_stream["download_url"] = None
+        best_stream["connection"] = None
+        assert best_stream == {
+            "bitrate": 160,
+            "connection": None,
+            "download_url": None,
+            "encoding": "opus",
+            "filesize": 3614184
+        }
+
+    # @pytest.mark.mock
+    def test_mock_getbest(self, mock_content, monkeypatch):
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_getbest(mock_content)
+
+    @pytest.mark.network
+    def test_getworst(self, content):
+        formatted_streams = YouTubeStreams(content.streams)
+        worst_stream = formatted_streams.getworst()
+        assert isinstance(worst_stream["download_url"], str)
+        assert worst_stream["connection"] is not None
+        # We `None` the `download_url` since it's impossible to
+        # predict its value before-hand.
+        worst_stream["download_url"] = None
+        worst_stream["connection"] = None
+        assert worst_stream == {
+            "bitrate": 50,
+            "connection": None,
+            "download_url": None,
+            "encoding": 'opus',
+            "filesize": 1407962
+        }
+
+    # @pytest.mark.mock
+    def test_mock_getworst(self, mock_content, monkeypatch):
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_getworst(mock_content)
+
+
+class TestProviderYouTube:
+    @pytest.fixture(scope="module")
+    def youtube_provider(self):
+        return ProviderYouTube()
+
+    class MockYouTubeSearch:
+        watch_urls = []
+        def search(self, query):
+            return self.watch_urls
+
+    @pytest.mark.network
+    def test_from_query(self, track, youtube_provider):
+        metadata = youtube_provider.from_query(track)
+        assert isinstance(metadata["streams"], YouTubeStreams)
+        # We avoid testing each item for the `streams` key here
+        # again. It this has already been tested above.
+        metadata["streams"] = []
+        assert metadata == {
+          'album': {'artists': [{'name': None}],
+                    'images': [{'url': 'https://i.ytimg.com/vi/cH4E_t3m3xM/maxresdefault.jpg'}],
+                    'name': None},
+          'artists': [{'name': 'SelenaGomezVEVO'}],
+          'copyright': None,
+          'disc_number': 1,
+          'duration': 213,
+          'external_ids': {'isrc': None},
+          'external_urls': {'youtube': 'https://youtube.com/watch?v=cH4E_t3m3xM'},
+          'genre': None,
+          'lyrics': None,
+          'name': 'Selena Gomez, Marshmello - Wolves',
+          'provider': 'youtube',
+          'publisher': None,
+          'release_date': '2017-11-1',
+          'streams': [],
+          'total_tracks': 1,
+          'track_number': 1,
+          'type': 'track',
+          'year': '2017'
+       }
+
+
+
+    def test_mock_from_query(self, track, youtube_provider, expect_mock_search_results, monkeypatch):
+        self.MockYouTubeSearch.watch_urls = expect_mock_search_results
+        monkeypatch.setattr(youtube, "YouTubeSearch", self.MockYouTubeSearch)
+        monkeypatch.setattr(pytube, "YouTube", MockYouTube)
+        monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
+        self.test_from_query(track, youtube_provider)
+
+    @pytest.mark.network
+    def test_error_exception_from_query(self, no_result_track, youtube_provider):
+        with pytest.raises(YouTubeMetadataNotFoundError):
+            youtube_provider.from_query(no_result_track)
+
+    def test_mock_error_exception_from_query(self, no_result_track, youtube_provider, monkeypatch):
+        self.MockYouTubeSearch.watch_urls = []
+        monkeypatch.setattr(youtube, "YouTubeSearch", self.MockYouTubeSearch)
+        monkeypatch.setattr(pytube, "YouTube", MockYouTube)
+        self.test_error_exception_from_query(no_result_track, youtube_provider)
+

spotdl/metadata/providers/youtube.py

@@ -0,0 +1,283 @@
+import pytube
+from bs4 import BeautifulSoup
+
+import urllib.request
+import threading
+from collections.abc import Sequence
+
+from spotdl.metadata import StreamsBase
+from spotdl.metadata import ProviderBase
+from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
+
+import spotdl.util
+
+import logging
+logger = logging.getLogger(__name__)
+
+BASE_SEARCH_URL = "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q={}"
+HEADERS = [('Range', 'bytes=0-'),]
+
+
+class YouTubeVideos(Sequence):
+    def __init__(self, videos):
+        self.videos = videos
+        super().__init__()
+
+    def __repr__(self):
+        return "YouTubeVideos({})".format(self.videos)
+
+    def __len__(self):
+        return len(self.videos)
+
+    def __getitem__(self, index):
+        return self.videos[index]
+
+    def __eq__(self, instance):
+        return self.videos == instance.videos
+
+    def bestmatch(self):
+        video = self.videos[0]
+        logger.debug("Matched with: {title} ({url}) [{duration}]".format(
+            title=video["title"],
+            url=video["url"],
+            duration=video["duration"]
+        ))
+        return video
+
+
+class YouTubeSearch:
+    def __init__(self):
+        self.base_search_url = BASE_SEARCH_URL
+
+    def generate_search_url(self, query):
+        quoted_query = urllib.request.quote(query)
+        return self.base_search_url.format(quoted_query)
+
+    def _fetch_response_html(self, url):
+        response = urllib.request.urlopen(url)
+        soup = BeautifulSoup(response.read(), "html.parser")
+        return soup
+
+    def _extract_video_details_from_result(self, html):
+        video_time = html.find("span", class_="video-time").get_text()
+        inner_html = html.find("div", class_="yt-lockup-content")
+        video_id = inner_html.find("a")["href"][-11:]
+        video_title = inner_html.find("a")["title"]
+        video_details = {
+            "url": "https://www.youtube.com/watch?v=" + video_id,
+            "title": video_title,
+            "duration": video_time,
+        }
+        return video_details
+
+    def _fetch_search_results(self, html, limit=10):
+        result_source = html.find_all(
+            "div", {"class": "yt-lockup-dismissable yt-uix-tile"}
+        )
+        videos = []
+
+        for result in result_source:
+            if not self._is_video(result):
+                continue
+
+            video = self._extract_video_details_from_result(result)
+            videos.append(video)
+
+            if len(videos) >= limit:
+                break
+
+        return videos
+
+    def _is_video(self, result):
+        # ensure result is not a channel
+        not_video = (
+            result.find("channel") is not None
+            or "yt-lockup-channel" in result.parent.attrs["class"]
+            or "yt-lockup-channel" in result.attrs["class"]
+        )
+
+        # ensure result is not a mix/playlist
+        not_video = not_video or "yt-lockup-playlist" in result.parent.attrs["class"]
+
+        # ensure video result is not an advertisement
+        not_video = not_video or result.find("googleads") is not None
+
+        video = not not_video
+        return video
+
+    def _is_server_side_invalid_response(self, videos, html):
+        if videos:
+            return False
+        search_message = html.find("div", {"class":"search-message"})
+        return search_message is None
+
+    def search(self, query, limit=10, retries=5):
+        """ Search and scrape YouTube to return a list of matching videos. """
+        search_url = self.generate_search_url(query)
+        logger.debug('Fetching YouTube results for "{}" at "{}".'.format(query, search_url))
+        html = self._fetch_response_html(search_url)
+        videos = self._fetch_search_results(html, limit=limit)
+        to_retry = retries > 0 and self._is_server_side_invalid_response(videos, html)
+        if to_retry:
+            logger.debug(
+                "Retrying since YouTube returned invalid response for search "
+                "results. Retries left: {retries}.".format(retries=retries)
+            )
+            return self.search(query, limit=limit, retries=retries-1)
+        return YouTubeVideos(videos)
+
+
+class YouTubeStreams(StreamsBase):
+    def __init__(self, streams):
+        self.network_headers = HEADERS
+
+        audiostreams = streams.filter(only_audio=True).order_by("abr").desc()
+
+        thread_pool = []
+        self.all = []
+
+        for stream in audiostreams:
+            encoding = "m4a" if "mp4a" in stream.audio_codec else stream.audio_codec
+            standard_stream = {
+                # Store only the integer part for bitrate. For example
+                # the given bitrate would be "192kbps", we store only
+                # the integer part (192) here and drop the rest.
+                "bitrate": int(stream.abr[:-4]),
+                "connection": None,
+                "download_url": stream.url,
+                "encoding": encoding,
+                "filesize": None,
+            }
+            establish_connection = threading.Thread(
+               target=self._store_connection,
+               args=(standard_stream,),
+            )
+            thread_pool.append(establish_connection)
+            establish_connection.start()
+            self.all.append(standard_stream)
+
+        for thread in thread_pool:
+            thread.join()
+
+    def _store_connection(self, stream):
+        response = self._make_request(stream["download_url"])
+        stream["connection"] = response
+        stream["filesize"] = int(response.headers["Content-Length"])
+
+    def _make_request(self, url):
+        request = urllib.request.Request(url)
+        for header in self.network_headers:
+            request.add_header(*header)
+        return urllib.request.urlopen(request)
+
+    def get(self, quality="best", preftype="automatic"):
+        if quality == "best":
+            return self.getbest(preftype=preftype)
+        elif quality == "worst":
+            return self.getworst(preftype=preftype)
+        else:
+            return None
+
+    def getbest(self, preftype="automatic"):
+        selected_stream = None
+        if preftype == "automatic":
+            selected_stream = self.all[0]
+        else:
+            for stream in self.all:
+                if stream["encoding"] == preftype:
+                    selected_stream = stream
+                    break
+        logger.debug('Selected best quality stream for "{preftype}" format:\n{stream}'.format(
+            preftype=preftype,
+            stream=selected_stream,
+        ))
+        return selected_stream
+
+    def getworst(self, preftype="automatic"):
+        selected_stream = None
+        if preftype == "automatic":
+            selected_stream = self.all[-1]
+        else:
+            for stream in self.all[::-1]:
+                if stream["encoding"] == preftype:
+                    selected_stream = stream
+                    break
+        logger.debug('Selected worst quality stream for "{preftype}" format:\n{stream}'.format(
+            preftype=preftype,
+            stream=selected_stream,
+        ))
+        return selected_stream
+
+
+class ProviderYouTube(ProviderBase):
+    def from_query(self, query):
+        watch_urls = self.search(query)
+        if not watch_urls:
+            raise YouTubeMetadataNotFoundError(
+                'YouTube returned nothing for the given search '
+                'query ("{}")'.format(query)
+            )
+        return self.from_url(watch_urls[0])
+
+    def from_url(self, url, retries=5):
+        logger.debug('Fetching YouTube metadata for "{url}".'.format(url=url))
+        try:
+            content = pytube.YouTube(url)
+        except KeyError:
+            # Sometimes YouTube can return unexpected response, in such a case
+            # retry a few times before finally failing.
+            if retries > 0:
+                retries -= 1
+                logger.debug(
+                    "YouTube returned an unexpected response for "
+                    "`pytube.YouTube({url})`. Retries left: {retries}".format(
+                        url=url, retries=retries
+                    )
+                )
+                return self.from_url(url, retries=retries)
+            else:
+                raise
+        else:
+            return self.from_pytube_object(content)
+
+    def from_pytube_object(self, content):
+        return self.metadata_to_standard_form(content)
+
+    def search(self, query):
+        return YouTubeSearch().search(query)
+
+    def _fetch_publish_date(self, content):
+        # XXX: This needs to be supported in PyTube itself
+        # See https://github.com/nficano/pytube/issues/595
+        position = content.watch_html.find("publishDate")
+        publish_date = content.watch_html[position+16:position+25]
+        return publish_date
+
+    def metadata_to_standard_form(self, content):
+        """ Fetch a song's metadata from YouTube. """
+        publish_date = self._fetch_publish_date(content)
+        metadata = {
+            "name": content.title,
+            "artists": [{"name": content.author}],
+            "duration": content.length,
+            "external_urls": {"youtube": content.watch_url},
+            "album": {
+                "images": [{"url": content.thumbnail_url}],
+                "artists": [{"name": None}],
+                "name": None,
+            },
+            "year": publish_date.split("-")[0],
+            "release_date": publish_date,
+            "type": "track",
+            "disc_number": 1,
+            "track_number": 1,
+            "total_tracks": 1,
+            "publisher": None,
+            "external_ids": {"isrc": None},
+            "lyrics": None,
+            "copyright": None,
+            "genre": None,
+            "streams": YouTubeStreams(content.streams),
+            "provider": "youtube",
+        }
+        return metadata

spotdl/metadata/tests/test_embedder_base.py

@@ -0,0 +1,72 @@
+from spotdl.metadata import EmbedderBase
+
+import pytest
+
+class EmbedderKid(EmbedderBase):
+    def __init__(self):
+        super().__init__()
+
+
+class TestEmbedderBaseABC:
+    def test_error_base_class_embedderbase(self):
+        with pytest.raises(TypeError):
+            # This abstract base class must be inherited from
+            # for instantiation
+            EmbedderBase()
+
+    def test_inherit_abstract_base_class_streamsbase(self):
+        EmbedderKid()
+
+
+class TestMethods:
+    @pytest.fixture(scope="module")
+    def embedderkid(self):
+        return EmbedderKid()
+
+    def test_target_formats(self, embedderkid):
+        assert embedderkid.supported_formats == ()
+
+    @pytest.mark.parametrize("path, expect_encoding", (
+        ("/a/b/c/file.mp3", "mp3"),
+        ("music/pop/1.wav", "wav"),
+        ("/a path/with spaces/track.m4a", "m4a"),
+    ))
+    def test_get_encoding(self, embedderkid, path, expect_encoding):
+        assert embedderkid.get_encoding(path) == expect_encoding
+
+    def test_apply_metadata_with_explicit_encoding(self, embedderkid):
+        with pytest.raises(TypeError):
+            embedderkid.apply_metadata("/path/to/music.mp3", {}, cached_albumart="imagedata", encoding="mp3")
+
+    def test_apply_metadata_with_implicit_encoding(self, embedderkid):
+        with pytest.raises(TypeError):
+            embedderkid.apply_metadata("/path/to/music.wav", {}, cached_albumart="imagedata")
+
+    class MockHTTPResponse:
+        """
+        This mocks `urllib.request.urlopen` for custom response text.
+        """
+        response_file = ""
+
+        def __init__(self, url):
+            pass
+
+        def read(self):
+            pass
+
+    def test_apply_metadata_without_cached_image(self, embedderkid, monkeypatch):
+        monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
+        metadata = {"album": {"images": [{"url": "http://animageurl.com"},]}}
+        with pytest.raises(TypeError):
+            embedderkid.apply_metadata("/path/to/music.wav", metadata, cached_albumart=None)
+
+    @pytest.mark.parametrize("fmt_method_suffix", (
+        "as_mp3",
+        "as_m4a",
+        "as_flac",
+    ))
+    def test_embed_formats(self, fmt_method_suffix, embedderkid):
+        method = eval("embedderkid." + fmt_method_suffix)
+        with pytest.raises(NotImplementedError):
+            method("/a/random/path", {})
+

spotdl/metadata/tests/test_metadata_exceptions.py

@@ -0,0 +1,15 @@
+from spotdl.metadata.exceptions import MetadataNotFoundError
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
+
+
+class TestMetadataNotFoundSubclass:
+    def test_metadata_not_found_subclass(self):
+        assert issubclass(MetadataNotFoundError, Exception)
+
+    def test_spotify_metadata_not_found(self):
+        assert issubclass(SpotifyMetadataNotFoundError, MetadataNotFoundError)
+
+    def test_youtube_metadata_not_found(self):
+        assert issubclass(YouTubeMetadataNotFoundError, MetadataNotFoundError)
+

spotdl/metadata/tests/test_provider_base.py

@@ -0,0 +1,60 @@
+from spotdl.metadata import ProviderBase
+from spotdl.metadata import StreamsBase
+
+import pytest
+
+class TestStreamsBaseABC:
+    def test_error_abstract_base_class_streamsbase(self):
+        with pytest.raises(TypeError):
+            # This abstract base class must be inherited from
+            # for instantiation
+            StreamsBase()
+
+    def test_inherit_abstract_base_class_streamsbase(self):
+        class StreamsKid(StreamsBase):
+            def __init__(self, streams):
+                super().__init__(streams)
+
+        streams = ("stream1", "stream2", "stream3")
+        kid = StreamsKid(streams)
+        assert kid.all == streams
+
+
+class TestMethods:
+    class StreamsKid(StreamsBase):
+        def __init__(self, streams):
+            super().__init__(streams)
+
+
+    @pytest.fixture(scope="module")
+    def streamskid(self):
+        streams = ("stream1", "stream2", "stream3")
+        streamskid = self.StreamsKid(streams)
+        return streamskid
+
+    def test_getbest(self, streamskid):
+        best_stream = streamskid.getbest()
+        assert best_stream == "stream1"
+
+    def test_getworst(self, streamskid):
+        worst_stream = streamskid.getworst()
+        assert worst_stream == "stream3"
+
+
+class TestProviderBaseABC:
+    def test_error_abstract_base_class_providerbase(self):
+        with pytest.raises(TypeError):
+            # This abstract base class must be inherited from
+            # for instantiation
+            ProviderBase()
+
+    def test_inherit_abstract_base_class_providerbase(self):
+        class ProviderKid(ProviderBase):
+            def from_url(self, query):
+                pass
+
+            def metadata_to_standard_form(self, metadata):
+                pass
+
+        ProviderKid()
+

spotdl/metadata_search.py

@@ -0,0 +1,260 @@
+from spotdl.metadata.providers import ProviderSpotify
+from spotdl.metadata.providers import ProviderYouTube
+from spotdl.lyrics.providers import Genius
+from spotdl.lyrics.exceptions import LyricsNotFoundError
+
+import spotdl.metadata
+import spotdl.util
+from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
+
+from spotdl.command_line.exceptions import NoYouTubeVideoFoundError
+from spotdl.command_line.exceptions import NoYouTubeVideoMatchError
+
+import sys
+import logging
+logger = logging.getLogger(__name__)
+
+PROVIDERS = {
+    "spotify": ProviderSpotify,
+    "youtube": ProviderYouTube,
+}
+
+
+def prompt_for_youtube_search_result(videos):
+    max_index_length = len(str(len(videos)))
+    max_title_length = max(len(v["title"]) for v in videos)
+    msg = "{index:>{max_index}}. Skip downloading this track".format(
+        index=0,
+        max_index=max_index_length,
+    )
+    print(msg, file=sys.stderr)
+    for index, video in enumerate(videos, 1):
+        vid_details = "{index:>{max_index}}. {title:<{max_title}}\n{new_line_gap}  {url} [{duration}]".format(
+            index=index,
+            max_index=max_index_length,
+            title=video["title"],
+            max_title=max_title_length,
+            new_line_gap=" " * max_index_length,
+            url=video["url"],
+            duration=video["duration"],
+        )
+        print(vid_details, file=sys.stderr)
+    print("", file=sys.stderr)
+
+    selection = spotdl.util.prompt_user_for_selection(range(1, len(videos)+1))
+
+    if selection is None:
+        return None
+    return videos[selection-1]
+
+
+class MetadataSearch:
+    def __init__(self, track, lyrics=False, yt_search_format="{artist} - {track-name}", yt_manual=False, providers=PROVIDERS):
+        self.track = track
+        self.track_type = spotdl.util.track_type(track)
+        self.lyrics = lyrics
+        self.yt_search_format = yt_search_format
+        self.yt_manual = yt_manual
+        self.providers = {}
+        for provider, parent in providers.items():
+            self.providers[provider] = parent()
+        self.lyric_provider = Genius()
+
+    def get_lyrics(self, query):
+        try:
+            lyrics = self.lyric_provider.from_query(query)
+        except LyricsNotFoundError as e:
+            logger.warning(e.args[0])
+            lyrics = None
+        return lyrics
+
+    def _make_lyric_search_query(self, metadata):
+        if self.track_type == "query":
+            lyric_query = self.track
+        else:
+            lyric_search_format = "{artist} - {track-name}"
+            lyric_query = spotdl.metadata.format_string(
+                lyric_search_format,
+                metadata
+            )
+        return lyric_query
+
+    def on_youtube_and_spotify(self):
+        track_type_mapper = {
+            "spotify": self._on_youtube_and_spotify_for_type_spotify,
+            "youtube": self._on_youtube_and_spotify_for_type_youtube,
+            "query": self._on_youtube_and_spotify_for_type_query,
+        }
+        caller = track_type_mapper[self.track_type]
+        metadata = caller()
+
+        if not self.lyrics:
+            return metadata
+
+        lyric_query = self._make_lyric_search_query(metadata)
+        metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
+            target=self.get_lyrics,
+            args=(lyric_query,),
+        )
+
+        return metadata
+
+    def on_youtube(self):
+        track_type_mapper = {
+            "spotify": self._on_youtube_for_type_spotify,
+            "youtube": self._on_youtube_for_type_youtube,
+            "query": self._on_youtube_for_type_query,
+        }
+        caller = track_type_mapper[self.track_type]
+        metadata = caller(self.track)
+
+        if not self.lyrics:
+            return metadata
+
+        lyric_query = self._make_lyric_search_query(metadata)
+        metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
+            target=self.get_lyrics,
+            arguments=(lyric_query,),
+        )
+
+        return metadata
+
+    def on_spotify(self):
+        track_type_mapper = {
+            "spotify": self._on_spotify_for_type_spotify,
+            "youtube": self._on_spotify_for_type_youtube,
+            "query": self._on_spotify_for_type_query,
+        }
+        caller = track_type_mapper[self.track_type]
+        metadata = caller(self.track)
+
+        if not self.lyrics:
+            return metadata
+
+        lyric_query = self._make_lyric_search_query(metadata)
+        metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
+            target=self.get_lyrics,
+            arguments=(lyric_query,),
+        )
+
+        return metadata
+
+    def best_on_youtube_search(self):
+        track_type_mapper = {
+            "spotify": self._best_on_youtube_search_for_type_spotify,
+            "youtube": self._best_on_youtube_search_for_type_youtube,
+            "query": self._best_on_youtube_search_for_type_query,
+        }
+        caller = track_type_mapper[self.track_type]
+        video = caller(self.track)
+        return video
+
+    def _best_on_youtube_search_for_type_query(self, query):
+        videos = self.providers["youtube"].search(query)
+        if not videos:
+            raise NoYouTubeVideoFoundError(
+                'YouTube returned no videos for the search query "{}".'.format(query)
+            )
+        if self.yt_manual:
+            video = prompt_for_youtube_search_result(videos)
+        else:
+            video = videos.bestmatch()
+
+        if video is None:
+            raise NoYouTubeVideoMatchError(
+                'No matching videos found on YouTube for the search query "{}".'.format(
+                    query
+                )
+            )
+        return video
+
+    def _best_on_youtube_search_for_type_youtube(self, url):
+        video = self._best_on_youtube_search_for_type_query(url)
+        return video
+
+    def _best_on_youtube_search_for_type_spotify(self, url):
+        spotify_metadata = self._on_spotify_for_type_spotify(self.track)
+        search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
+        video = self._best_on_youtube_search_for_type_query(search_query)
+        return video
+
+    def _on_youtube_and_spotify_for_type_spotify(self):
+        logger.debug("Extracting YouTube and Spotify metadata for input Spotify URI.")
+        spotify_metadata = self._on_spotify_for_type_spotify(self.track)
+        search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
+        youtube_video = self._best_on_youtube_search_for_type_spotify(search_query)
+        youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
+        metadata = spotdl.util.merge_copy(
+            youtube_metadata,
+            spotify_metadata
+        )
+        return metadata
+
+    def _on_youtube_and_spotify_for_type_youtube(self):
+        logger.debug("Extracting YouTube and Spotify metadata for input YouTube URL.")
+        youtube_metadata = self._on_youtube_for_type_youtube(self.track)
+        search_query = spotdl.metadata.format_string("{track-name}", youtube_metadata)
+        spotify_metadata = self._on_spotify_for_type_query(search_query)
+        metadata = spotdl.util.merge_copy(
+            youtube_metadata,
+            spotify_metadata
+        )
+        return metadata
+
+    def _on_youtube_and_spotify_for_type_query(self):
+        logger.debug("Extracting YouTube and Spotify metadata for input track query.")
+        search_query = self.track
+        # Make use of threads here to search on both YouTube & Spotify
+        # at the same time.
+        spotify_metadata = spotdl.util.ThreadWithReturnValue(
+            target=self._on_spotify_for_type_query,
+            args=(search_query,)
+        )
+        spotify_metadata.start()
+        youtube_metadata = self._on_youtube_for_type_query(search_query)
+        metadata = spotdl.util.merge_copy(
+            youtube_metadata,
+            spotify_metadata.join()
+        )
+        return metadata
+
+    def _on_youtube_for_type_spotify(self):
+        logger.debug("Extracting YouTube metadata for input Spotify URI.")
+        spotify_metadata = self._on_spotify_for_type_spotify(self.track)
+        search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
+        youtube_video = self._best_on_youtube_search_for_type_spotify(search_query)
+        youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
+        return youtube_metadata
+
+    def _on_youtube_for_type_youtube(self, url):
+        logger.debug("Extracting YouTube metadata for input YouTube URL.")
+        youtube_metadata = self.providers["youtube"].from_url(url)
+        return youtube_metadata
+
+    def _on_youtube_for_type_query(self, query):
+        logger.debug("Extracting YouTube metadata for input track query.")
+        youtube_video = self._best_on_youtube_search_for_type_query(query)
+        youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
+        return youtube_metadata
+
+    def _on_spotify_for_type_youtube(self, url):
+        logger.debug("Extracting Spotify metadata for input YouTube URL.")
+        youtube_metadata = self.providers["youtube"].from_url(url)
+        search_query = spotdl.metadata.format_string("{track-name}", youtube_metadata)
+        spotify_metadata = self.providers["spotify"].from_query(search_query)
+        return spotify_metadata
+
+    def _on_spotify_for_type_spotify(self, url):
+        logger.debug("Extracting Spotify metadata for input Spotify URI.")
+        spotify_metadata = self.providers["spotify"].from_url(url)
+        return spotify_metadata
+
+    def _on_spotify_for_type_query(self, query):
+        logger.debug("Extracting Spotify metadata for input track query.")
+        try:
+            spotify_metadata = self.providers["spotify"].from_query(query)
+        except SpotifyMetadataNotFoundError as e:
+            logger.warn(e.args[0])
+            spotify_metadata = {}
+        return spotify_metadata
+

spotdl/tests/test_config.py

@@ -0,0 +1,71 @@
+import spotdl.config
+
+import argparse
+import os
+import sys
+import yaml
+import pytest
+
+
+@pytest.mark.xfail
+@pytest.fixture(scope="module")
+def config_path(tmpdir_factory):
+    config_path = os.path.join(str(tmpdir_factory.mktemp("config")), "config.yml")
+    return config_path
+
+
+@pytest.mark.xfail
+@pytest.fixture(scope="module")
+def modified_config():
+    modified_config = dict(spotdl.config.DEFAULT_CONFIGURATION)
+    return modified_config
+
+
+def test_dump_n_read_config(config_path):
+    expect_config = spotdl.config.DEFAULT_CONFIGURATION
+    spotdl.config.dump_config(
+        config_path,
+        config=expect_config,
+    )
+    config = spotdl.config.read_config(config_path)
+    assert config == expect_config
+
+
+class TestDefaultConfigFile:
+    @pytest.mark.skipif(not sys.platform == "linux", reason="Linux only")
+    def test_linux_default_config_file(self):
+        expect_default_config_file = os.path.expanduser("~/.config/spotdl/config.yml")
+        assert spotdl.config.DEFAULT_CONFIG_FILE == expect_default_config_file
+
+    @pytest.mark.xfail
+    @pytest.mark.skipif(not sys.platform == "darwin" and not sys.platform == "win32",
+                        reason="Windows only")
+    def test_windows_default_config_file(self):
+        raise NotImplementedError
+
+    @pytest.mark.xfail
+    @pytest.mark.skipif(not sys.platform == "darwin",
+                        reason="OS X only")
+    def test_osx_default_config_file(self):
+        raise NotImplementedError
+
+
+class TestConfig:
+    @pytest.mark.xfail
+    def test_custom_config_path(self, config_path, modified_config):
+        parser = argparse.ArgumentParser()
+        with open(config_path, "w") as config_file:
+            yaml.dump(modified_config, config_file, default_flow_style=False)
+        overridden_config = spotdl.config.override_config(
+            config_path, parser, raw_args=""
+        )
+        modified_values = [
+            str(value)
+            for value in modified_config["spotify-downloader"].values()
+        ]
+        overridden_config.folder = os.path.realpath(overridden_config.folder)
+        overridden_values = [
+            str(value) for value in overridden_config.__dict__.values()
+        ]
+        assert sorted(overridden_values) == sorted(modified_values)
+

spotdl/tests/test_util.py

@@ -0,0 +1,90 @@
+import sys
+import os
+import subprocess
+
+import spotdl.util
+
+import pytest
+
+
+@pytest.fixture(scope="module")
+def directory_fixture(tmpdir_factory):
+    dir_path = os.path.join(str(tmpdir_factory.mktemp("tmpdir")), "filter_this_directory")
+    return dir_path
+
+
+@pytest.mark.parametrize("value", [
+    5,
+    "string",
+    {"a": 1, "b": 2},
+    (10, 20, 30, "string"),
+    [2, 4, "sample"]
+])
+def test_thread_with_return_value(value):
+    returner = lambda x: x
+    thread = spotdl.util.ThreadWithReturnValue(
+        target=returner,
+        args=(value,)
+    )
+    thread.start()
+    assert value == thread.join()
+
+
+@pytest.mark.parametrize("track, track_type", [
+  ("https://open.spotify.com/track/3SipFlNddvL0XNZRLXvdZD", "spotify"),
+  ("spotify:track:3SipFlNddvL0XNZRLXvdZD", "spotify"),
+  ("3SipFlNddvL0XNZRLXvdZD", "spotify"),
+  ("https://www.youtube.com/watch?v=oMiNsd176NM", "youtube"),
+  ("oMiNsd176NM", "youtube"),
+  ("kodaline - saving grace", "query"),
+  ("or anything else", "query"),
+])
+def test_track_type(track, track_type):
+    assert spotdl.util.track_type(track) == track_type
+
+
+@pytest.mark.parametrize("str_duration, sec_duration", [
+    ("0:23", 23),
+	("0:45", 45),
+	("2:19", 139),
+	("3:33", 213),
+	("7:38", 458),
+	("1:30:05", 5405),
+])
+def test_get_seconds_from_video_time(str_duration, sec_duration):
+    secs = spotdl.util.get_sec(str_duration)
+    assert secs == sec_duration
+
+
+@pytest.mark.parametrize("duplicates, expected", [
+    (("https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
+      "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",),
+    ( "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",),),
+
+    (("https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
+      "",
+      "https://open.spotify.com/track/3SipFlNddvL0XNZRLXvdZD",),
+    ( "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
+      "https://open.spotify.com/track/3SipFlNddvL0XNZRLXvdZD",),),
+
+    (("ncs fade",
+      "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
+      "",
+      "ncs fade",),
+    ("ncs fade",
+     "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ"),),
+
+    (("ncs spectre ",
+      "  https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
+      ""),
+    ( "ncs spectre",
+      "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ"),),
+])
+def test_remove_duplicates(duplicates, expected):
+    uniques = spotdl.util.remove_duplicates(
+		duplicates,
+		condition=lambda x: x,
+		operation=str.strip,
+	)
+    assert tuple(uniques) == expected
+

spotdl/track.py

@@ -0,0 +1,111 @@
+import tqdm
+
+import urllib.request
+import subprocess
+import sys
+
+from spotdl.encode.encoders import EncoderFFmpeg
+from spotdl.metadata.embedders import EmbedderDefault
+
+import spotdl.util
+
+CHUNK_SIZE = 16 * 1024
+
+class Track:
+    def __init__(self, metadata, cache_albumart=False):
+        self.metadata = metadata
+        self._chunksize = CHUNK_SIZE
+
+        if cache_albumart:
+            self._albumart_thread = self._cache_albumart()
+
+        self._cache_albumart = cache_albumart
+
+    def _cache_albumart(self):
+        albumart_thread = spotdl.util.ThreadWithReturnValue(
+            target=lambda url: urllib.request.urlopen(url).read(),
+            args=(self.metadata["album"]["images"][0]["url"],)
+        )
+        albumart_thread.start()
+        return albumart_thread
+
+    def calculate_total_chunks(self, filesize):
+        return (filesize // self._chunksize) + 1
+
+    def make_progress_bar(self, total_chunks):
+        progress_bar = tqdm.trange(
+            total_chunks,
+            unit_scale=(self._chunksize // 1024),
+            unit="KiB",
+            dynamic_ncols=True,
+            bar_format='{desc}: {percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt}KiB '
+                '[{elapsed}<{remaining}, {rate_fmt}{postfix}]',
+        )
+        return progress_bar
+
+    def download_while_re_encoding(self, stream, target_path, target_encoding=None,
+                                   encoder=EncoderFFmpeg(must_exist=False), show_progress=True):
+        total_chunks = self.calculate_total_chunks(stream["filesize"])
+        process = encoder.re_encode_from_stdin(
+            stream["encoding"],
+            target_path,
+            target_encoding=target_encoding
+        )
+        response = stream["connection"]
+
+        progress_bar = self.make_progress_bar(total_chunks)
+        for _ in progress_bar:
+            chunk = response.read(self._chunksize)
+            process.stdin.write(chunk)
+
+        process.stdin.close()
+        process.wait()
+
+    def download(self, stream, target_path, show_progress=True):
+        total_chunks = self.calculate_total_chunks(stream["filesize"])
+        progress_bar = self.make_progress_bar(total_chunks)
+        response = stream["connection"]
+
+        def writer(response, progress_bar, file_io):
+            for _ in progress_bar:
+                chunk = response.read(self._chunksize)
+                file_io.write(chunk)
+
+        write_to_stdout = target_path == "-"
+        if write_to_stdout:
+            file_io = sys.stdout.buffer
+            writer(response, progress_bar, file_io)
+        else:
+            with open(target_path, "wb") as file_io:
+                writer(response, progress_bar, file_io)
+
+    def re_encode(self, input_path, target_path, target_encoding=None,
+                  encoder=EncoderFFmpeg(must_exist=False), show_progress=True):
+        stream = self.metadata["streams"].getbest()
+        total_chunks = self.calculate_total_chunks(stream["filesize"])
+        process = encoder.re_encode_from_stdin(
+            stream["encoding"],
+            target_path,
+            target_encoding=target_encoding
+        )
+        with open(input_path, "rb") as fin:
+            for _ in tqdm.trange(total_chunks):
+                chunk = fin.read(self._chunksize)
+                process.stdin.write(chunk)
+
+        process.stdin.close()
+        process.wait()
+
+    def apply_metadata(self, input_path, encoding=None, embedder=EmbedderDefault()):
+        if self._cache_albumart:
+            albumart = self._albumart_thread.join()
+        else:
+            albumart = None
+
+        embedder.apply_metadata(
+            input_path,
+            self.metadata,
+            cached_albumart=albumart,
+            encoding=encoding,
+        )
+

spotdl/util.py

@@ -0,0 +1,167 @@
+import os
+import sys
+import math
+import urllib.request
+import threading
+
+import logging
+logger = logging.getLogger(__name__)
+
+
+try:
+    import winreg
+except ImportError:
+    pass
+
+try:
+    from slugify import SLUG_OK, slugify
+except ImportError:
+    logger.error("Oops! `unicode-slugify` was not found.")
+    logger.info("Please remove any other slugify library and install `unicode-slugify`")
+    raise
+
+
+# This has been referred from
+# https://stackoverflow.com/a/6894023/6554943
+# It's because threaded functions do not return by default
+# Whereas this will return the value when `join` method
+# is called.
+class ThreadWithReturnValue(threading.Thread):
+    def __init__(self, target=lambda: None, args=()):
+        super().__init__(target=target, args=args)
+        self._return = None
+
+    def run(self):
+        if self._target is not None:
+            self._return = self._target(
+                *self._args,
+                **self._kwargs
+            )
+
+    def join(self, *args, **kwargs):
+        super().join(*args, **kwargs)
+        return self._return
+
+
+def merge_copy(base, overrider):
+    return merge(base.copy(), overrider)
+
+def merge(base, overrider):
+    """ Override base dict with an overrider dict, recursively. """
+    for key, value in overrider.items():
+        if isinstance(value, dict):
+            subitem = base.setdefault(key, {})
+            merge(subitem, value)
+        else:
+            base[key] = value
+
+    return base
+
+
+def prompt_user_for_selection(items):
+    """ Let the user input a choice. """
+    logger.info("Enter a number:")
+    while True:
+        try:
+            the_chosen_one = int(input("> "))
+            if 1 <= the_chosen_one <= len(items):
+                return items[the_chosen_one - 1]
+            elif the_chosen_one == 0:
+                return None
+            else:
+                logger.warning("Choose a valid number!")
+        except ValueError:
+            logger.warning("Choose a valid number!")
+
+
+def is_spotify(track):
+    """ Check if the input song is a Spotify link. """
+    status = len(track) == 22 and track.replace(" ", "%20") == track
+    status = status or track.find("spotify") > -1
+    return status
+
+
+def is_youtube(track):
+    """ Check if the input song is a YouTube link. """
+    status = len(track) == 11 and track.replace(" ", "%20") == track
+    status = status and not track.lower() == track
+    status = status or "youtube.com/watch?v=" in track
+    return status
+
+
+def track_type(track):
+    track_types = {
+        "spotify": is_spotify,
+        "youtube": is_youtube,
+    }
+    for provider, fn in track_types.items():
+        if fn(track):
+            return provider
+    return "query"
+
+
+def sanitize(string, ok="&-_()[]{}", spaces_to_underscores=False):
+    """ Generate filename of the song to be downloaded. """
+    if spaces_to_underscores:
+        string = string.replace(" ", "_")
+    # replace slashes with "-" to avoid directory creation errors
+    string = string.replace("/", "-").replace("\\", "-")
+    # slugify removes any special characters
+    string = slugify(string, ok=ok, lower=False, spaces=True)
+    return string
+
+
+def get_sec(time_str):
+    if ":" in time_str:
+        splitter = ":"
+    elif "." in time_str:
+        splitter = "."
+    else:
+        raise ValueError(
+            "No expected character found in {} to split" "time values.".format(time_str)
+        )
+    v = time_str.split(splitter, 3)
+    v.reverse()
+    sec = 0
+    if len(v) > 0:  # seconds
+        sec += int(v[0])
+    if len(v) > 1:  # minutes
+        sec += int(v[1]) * 60
+    if len(v) > 2:  # hours
+        sec += int(v[2]) * 3600
+    return sec
+
+
+def remove_duplicates(elements, condition=lambda _: True, operation=lambda x: x):
+    """
+    Removes duplicates from a list whilst preserving order.
+
+    We could directly call `set()` on the list but it changes
+    the order of elements.
+    """
+
+    local_set = set()
+    local_set_add = local_set.add
+    filtered_list = []
+    for x in elements:
+        if condition(x) and not (x in local_set or local_set_add(x)):
+            operated = operation(x)
+            filtered_list.append(operated)
+            local_set_add(operated)
+    return filtered_list
+
+
+def titlecase(string):
+    return " ".join(word.capitalize() for word in string.split())
+
+
+def readlines_from_nonbinary_file(path):
+    with open(path, "r") as fin:
+        lines = fin.read().splitlines()
+    return lines
+
+
+def writelines_to_nonbinary_file(path, lines):
+    with open(path, "w") as fout:
+        fout.writelines(map(lambda x: x + "\n", lines))
+

spotdl/version.py

@@ -0,0 +1,2 @@
+__version__ = "2.0.5"
+

test/test_simple.py

@@ -1,72 +0,0 @@
-# -*- coding: UTF-8 -*-
-
-import spotdl
-import os
-
-raw_song = "Tony's Videos VERY SHORT VIDEO 28.10.2016"
-
-for x in os.listdir(spotdl.args.folder):
-    os.remove(os.path.join(spotdl.args.folder, x))
-
-def test_youtube_url():
-    expect_url = 'youtube.com/watch?v=qOOcy2-tmbk'
-    url = spotdl.generate_youtube_url(raw_song)
-    assert url == expect_url
-
-
-def test_youtube_title():
-    expect_title = "Tony's Videos VERY SHORT VIDEO 28.10.2016"
-    global content
-    content = spotdl.go_pafy(raw_song)
-    global title
-    title = spotdl.get_youtube_title(content)
-    assert title == expect_title
-
-def test_check_exists():
-    expect_check = False
-    # prerequisites for determining filename
-    file_name = spotdl.misc.sanitize_title(title)
-    check = spotdl.check_exists(file_name, raw_song, islist=True)
-    assert check == expect_check
-
-
-def test_download():
-    expect_download = True
-    # prerequisites for determining filename
-    file_name = spotdl.misc.sanitize_title(title)
-    download = spotdl.download_song(file_name, content)
-    assert download == expect_download
-
-
-def test_convert():
-    # exit code 0 = success
-    expect_convert = 0
-    # prerequisites for determining filename
-    file_name = spotdl.misc.sanitize_title(title)
-    input_song = file_name + spotdl.args.input_ext
-    output_song = file_name + spotdl.args.output_ext
-    convert = spotdl.convert.song(input_song, output_song, spotdl.args.folder)
-    assert convert == expect_convert
-
-
-def test_metadata():
-    expect_metadata = None
-    # prerequisites for determining filename
-    meta_tags = spotdl.generate_metadata(raw_song)
-    meta_tags = spotdl.generate_metadata(raw_song)
-    file_name = spotdl.misc.sanitize_title(title)
-    output_song = file_name + spotdl.args.output_ext
-    metadata_output = spotdl.metadata.embed(os.path.join(spotdl.args.folder, output_song), meta_tags)
-    input_song = file_name + spotdl.args.input_ext
-    metadata_input = spotdl.metadata.embed(os.path.join(spotdl.args.folder, input_song), meta_tags)
-    assert (metadata_output == expect_metadata) and (metadata_input == expect_metadata)
-
-
-def test_check_exists2():
-    expect_check = True
-    # prerequisites for determining filename
-    file_name = spotdl.misc.sanitize_title(title)
-    input_song = file_name + spotdl.args.input_ext
-    os.remove(os.path.join(spotdl.args.folder, input_song))
-    check = spotdl.check_exists(file_name, raw_song, islist=True)
-    assert check == expect_check

test/test_spotify.py

@@ -1,77 +0,0 @@
-# -*- coding: UTF-8 -*-
-
-import spotdl
-import os
-
-raw_song = 'http://open.spotify.com/track/0JlS7BXXD07hRmevDnbPDU'
-
-for x in os.listdir(spotdl.args.folder):
-    os.remove(os.path.join(spotdl.args.folder, x))
-
-
-def test_spotify_title():
-    expect_title = 'David André Østby - Intro'
-    global meta_tags
-    meta_tags = spotdl.generate_metadata(raw_song)
-    title = spotdl.generate_songname(meta_tags)
-    assert title == expect_title
-
-
-def youtube_url():
-    expect_url = 'youtube.com/watch?v=rg1wfcty0BA'
-    url = spotdl.generate_youtube_url(raw_song)
-    assert url == expect_url
-
-
-def youtube_title():
-    expect_title = 'Intro - David André Østby'
-    content = spotdl.go_pafy(raw_song)
-    title = spotdl.get_youtube_title(content)
-    assert title == expect_title
-
-
-def test_check_exists():
-    expect_check = False
-    # prerequisites for determining filename
-    songname = spotdl.generate_songname(meta_tags)
-    global file_name
-    file_name = spotdl.misc.sanitize_title(songname)
-    check = spotdl.check_exists(file_name, raw_song, islist=True)
-    assert check == expect_check
-
-
-def test_download():
-    expect_download = True
-    # prerequisites for determining filename
-    content = spotdl.go_pafy(raw_song)
-    download = spotdl.download_song(file_name, content)
-    assert download == expect_download
-
-
-def test_convert():
-    # exit code 0 = success
-    expect_convert = 0
-    # prerequisites for determining filename
-    input_song = file_name + spotdl.args.input_ext
-    output_song = file_name + spotdl.args.output_ext
-    convert = spotdl.convert.song(input_song, output_song, spotdl.args.folder)
-    assert convert == expect_convert
-
-
-def test_metadata():
-    expect_metadata = True
-    # prerequisites for determining filename
-    output_song = file_name + spotdl.args.output_ext
-    metadata_output = spotdl.metadata.embed(os.path.join(spotdl.args.folder, output_song), meta_tags)
-    input_song = file_name + spotdl.args.input_ext
-    metadata_input = spotdl.metadata.embed(os.path.join(spotdl.args.folder, input_song), meta_tags)
-    assert metadata_output == (metadata_input == expect_metadata)
-
-
-def test_check_exists2():
-    expect_check = True
-    # prerequisites for determining filename
-    input_song = file_name + spotdl.args.input_ext
-    os.remove(os.path.join(spotdl.args.folder, input_song))
-    check = spotdl.check_exists(file_name, raw_song, islist=True)
-    assert check == expect_check

test/test_username.py

@@ -1,57 +0,0 @@
-# -*- coding: UTF-8 -*-
-
-import spotdl
-
-username = 'alex'
-
-
-def test_user():
-    expect_playlists = 7
-    playlists = spotdl.spotify.user_playlists(username)
-    playlists = len(playlists['items'])
-    assert playlists == expect_playlists
-
-
-def test_playlist():
-    expect_tracks = 14
-    playlist = spotdl.spotify.user_playlists(username)['items'][0]
-    tracks = playlist['tracks']['total']
-    assert tracks == expect_tracks
-
-
-def test_tracks():
-    playlist = spotdl.spotify.user_playlists(username)['items'][0]
-    expect_lines = playlist['tracks']['total']
-    result = spotdl.spotify.user_playlist(
-        playlist['owner']['id'], playlist['id'], fields='tracks,next')
-    tracks = result['tracks']
-
-    with open('list.txt', 'w') as fout:
-        while True:
-            for item in tracks['items']:
-                track = item['track']
-                try:
-                    fout.write(track['external_urls']['spotify'] + '\n')
-                except KeyError:
-                    title = track['name'] + ' by '+ track['artists'][0]['name']
-                    print('Skipping track ' + title + ' (local only?)')
-            # 1 page = 50 results
-            # check if there are more pages
-            if tracks['next']:
-                tracks = spotify.next(tracks)
-            else:
-                break
-
-    with open('list.txt', 'r') as listed:
-        expect_song = (listed.read()).splitlines()[0]
-
-    spotdl.misc.trim_song('list.txt')
-    with open('list.txt', 'a') as myfile:
-        myfile.write(expect_song)
-
-    with open('list.txt', 'r') as listed:
-        songs = (listed.read()).splitlines()
-
-    lines = len(songs)
-    song = songs[-1]
-    assert (expect_lines == lines and expect_song == song)