KevinMidboe/spotify-downloader
1
0
Fork 0
mirror of https://github.com/KevinMidboe/spotify-downloader.git synced 2025-10-29 18:00:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: KevinMidboe:v2.0.3
Branches Tags
KevinMidboe:master KevinMidboe:threading KevinMidboe:python2
KevinMidboe:v2.0.5 KevinMidboe:v2.0.4 KevinMidboe:v2.0.3 KevinMidboe:v2.0.2 KevinMidboe:v2.0.1 KevinMidboe:v2.0.0 KevinMidboe:v1.2.6 KevinMidboe:v1.2.5 KevinMidboe:v1.2.4 KevinMidboe:v1.2.3 KevinMidboe:v1.2.2 KevinMidboe:v1.2.1 KevinMidboe:v1.2.0 KevinMidboe:v1.1.2 KevinMidboe:v1.1.1 KevinMidboe:v1.1.0 KevinMidboe:v1.0.0
..
compare: KevinMidboe:threading
Branches Tags
KevinMidboe:master KevinMidboe:threading KevinMidboe:python2
KevinMidboe:v2.0.5 KevinMidboe:v2.0.4 KevinMidboe:v2.0.3 KevinMidboe:v2.0.2 KevinMidboe:v2.0.1 KevinMidboe:v2.0.0 KevinMidboe:v1.2.6 KevinMidboe:v1.2.5 KevinMidboe:v1.2.4 KevinMidboe:v1.2.3 KevinMidboe:v1.2.2 KevinMidboe:v1.2.1 KevinMidboe:v1.2.0 KevinMidboe:v1.1.2 KevinMidboe:v1.1.1 KevinMidboe:v1.1.0 KevinMidboe:v1.0.0

2 Commits
v2.0.3 ... threading

Author SHA1 Message Date
ritiek
d18663c0e7 Join threads before exitting 2017-10-08 13:06:51 +05:30
ritiek
4db1dcc9b8 Implement basic threading 2017-10-08 12:30:43 +05:30
86 changed files with 1165 additions and 4941 deletions
Expand all files Collapse all files

122
.gitignore vendored Normal file → Executable file
View File

@@ -1,122 +1,6 @@
# Spotdl generated files
*.m4a
*.webm
*.mp3
*.opus
*.flac
*.temp
config.yml
*.pyc
__pycache__/
.cache/
Music/
*.txt
*.m3u
.cache-*
.pytest_cache/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
.static_storage/
.media/
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
# vscode
.vscode

49
.travis.yml
View File

@@ -1,44 +1,15 @@
dist: trusty
language: python
python:
- "3.4"
- "3.5"
- "3.6"
- "3.7"
- "3.8"
before_install:
- pip install tinydownload
- pip install "pytest>=5.4.1"
- pip install "pytest-cov>=2.8.1"
addons:
apt:
packages:
- xdg-user-dirs
- automake
- autoconf
- build-essential
- libass-dev
- libfreetype6-dev
- libtheora-dev
- libtool
- libva-dev
- libvdpau-dev
- libvorbis-dev
- libxcb1-dev
- libxcb-shm0-dev
- libxcb-xfixes0-dev
- libfdk-aac-dev
- libopus-dev
- pkg-config
- texinfo
- zlib1g-dev
- yasm
- nasm
- libmp3lame-dev
- libav-tools
install:
- pip install -e .
- tinydownload 07426048687547254773 -o ~/bin/ffmpeg
- sudo apt-get -qq update
- sudo apt-get -y install autoconf automake build-essential libass-dev libfreetype6-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texinfo wget zlib1g-dev
- sudo apt-get -y install yasm nasm libmp3lame-dev
- pip install -r requirements.txt
- pip install tinydownload
- tinydownload 05861434675432854607 -o ~/bin/ffmpeg
- chmod 755 ~/bin/ffmpeg
- xdg-user-dirs-update
script: travis_retry pytest --cov=.
after_success:
- pip install codecov
- codecov
script: python -m pytest test

217
CHANGES.md
View File

@@ -1,217 +0,0 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
The release dates mentioned follow the format `DD-MM-YYYY`.
## [Unreleased]
## [2.0.3] (Hotfix Release) - 18-05-2020
### Fixed
- Genius would sometimes return invalid lyrics. Retry a few times in such a case.
## [2.0.2] (Hotfix Release) - 18-05-2020
### Fixed
- Skipping tracks with `-m` would crash.
## [2.0.1] (Hotfix Release) - 18-05-2020
### Fixed
- `-o m4a` would always fail.
## [2.0.0] - 18-05-2020
### Migrating from v1.2.6 to v2.0.0
For v2.0.0 to work correctly, you need to remove your previous `config.yml` due to
breaking changes in v2.0.0 (marked as **[Breaking]** in the below sections), new options being
added, and old ones being removed. You may want to first backup your old configuration for
reference. You can then install spotdl v2.0.0 and remove your current configuration by
running:
```
$ spotdl --remove-config
```
spotdl will automatically generate a new configuration file on the next run. You can
then replace the appropriate fields in the newly generated configuration file by
referring to your old configuration file.
All the below changes were made as a part of #690.
### Added
- `-i` now accepts `automatic` which would automatically select the best available stream
irrespective of the format.
- Added parameter `-q` (`--quality {best,worst}`) to select best (default) or worst audio quality.
- Added `-ne` (`--no-encode`) to disable encoding.
- Output to STDOUT with `-f -`.
- Output to STDOUT with `--write-to -`.
- Read tracks from STDIN in `-s` parameter.
- Display a combined *download & encode* progress bar.
### Changed
- **[Breaking]** Tracks are now downloaded in the current working directory (instead of
user's Music directory) by default.
- **[Breaking]** Short for `--album` is now `-a` instead of `-b`.
- **[Breaking]** Short for `--all-albums` is now `-aa` instead of `-ab`.
- Allow "&" character in filenames.
- **[Breaking]** Merge parameters `-ff` and `-f` to `-f` (`--output-file`).
- **[Breaking]** Do not prefix formats with a dot when specifying `-i` and `-o` parameters
Such as `-o .mp3` is now written as `-o mp3`.
- **[Breaking]** Search format now uses hyphen for word break instead of underscore. Such as
`-sf "{artist} - {track_name}"` is now written as `-sf "{artist} - {track-name}"`.
- **[Breaking]** `--write-successful` and `--skip` is renamed to `--write-successful-file` and
`--skip-file` respectively.
- Partial re-write and internal API refactor.
- Enhance debug log output readability.
- Internally adapt to latest changes made in Spotipy library.
- Switch to `logging` + `coloredlogs` instead of `logzero`. Our loggers weren't being
setup properly with `logzero`.
- Simplify checking for an downloaded already track. Previously it also analyzed metadata
for the already downloaded track to determine whether to overwrite the already downloaded
track, which caused unexpected behvaiours at times.
- Codebase is now more modular making it easier to use spotdl in python scripts.
- `config.yml` now uses underscores for separating between argument words instead of
hyphens for better compatibility with `argparse`.
### Optimized
- Track download and encoding now happen parallely instead of sequentially making spotdl
faster.
- Lyrics and albumart are now downloaded in the background while the track is being downloaded
instead of in the end. This reduces additional delays if we are to download them while applying
metadata.
- `--write-m3u` now only scrapes YouTube for required metadata making it much faster.
Previously, it was also required to parse it via an external YouTube parsing library
which was slow.
- Switch to PyTube from Pafy. PyTube is faster and relies only on scraping.
### Removed
- **[Breaking]** Removed Avconv support. Only FFmpeg is supported now.
- **[Breaking]** Removed `--no-fallback-metadata` parameter since not many people seem to find it useful.
- **[Breaking]** Removed apparently misleading `--download-only-metadata` parameter.
- **[Breaking]** Removed ability to set YouTube API key since we now use PyTube instead of Pafy, and
PyTube does not require an API key.
- **[Breaking]** As a side effect of above, `--music-videos-only` is also removed as this feature worked only
with YouTube API.
## [1.2.6] (Hotfix Release) - 2020-03-02
### Fixed
- Embed release date metadata only when available (follow up of #672) ([@ritiek](https://github.com/ritiek)) (#674)
## [1.2.5] - 2020-03-02
### Fixed
- Skip crash when accessing YouTube-API-only fields in scrape mode ([@ritiek](https://github.com/ritiek)) (#672)
### Changed
- Changed FFMPEG args to convert to 48k quality audio instead of the current 44k audio. ([@AvinashReddy3108](https://github.com/AvinashReddy3108)) (#667)
## [1.2.4] - 2020-01-10
### Fixed
- Fixed a crash occuring when lyrics for a track are not yet released
on Genius ([@ritiek](https://github.com/ritiek)) (#654)
- Fixed a regression where a track would fail to download if it isn't
found on Spotify ([@ritiek](https://github.com/ritiek)) (#653)
## [1.2.3] - 2019-12-20
### Added
- Added `--no-remove-original-file` ([@NightMachinary](https://github.com/NightMachinary)) (#580)
- Added leading Zeros in `track_number` for correct sorting ([@Dsujan](https://github.com/Dsujan)) (#592)
- Added `track_id` key for `--file-format` parameter ([@kadaliao](https://github.com/kadaliao)) (#568)
### Fixed
- Some tracks randomly fail to download with Pafy v0.5.5 ([@ritiek](https://github.com/ritiek)) (#638)
- Generate list error --write-m3u ([@arthurlutz](https://github.com/arthurlutz)) (#559)
### Changed
- Fetch lyrics from Genius and fallback to LyricWikia if not found ([@ritiek](https://github.com/ritiek)) (#585)
## [1.2.2] - 2019-06-03
### Fixed
- Patch bug in Pafy to prefer secure HTTPS ([@ritiek](https://github.com/ritiek)) (#558)
## [1.2.1] - 2019-04-28
### Fixed
- Patch bug in Pafy when fetching audiostreams with latest youtube-dl ([@ritiek](https://github.com/ritiek)) (#539)
### Changed
- Removed duplicate debug log entry from `internals.trim_song` ([@ritiek](https://github.com/ritiek)) (#519)
- Fix YAMLLoadWarning ([@cyberboysumanjay](https://github.com/cyberboysumanjay)) (#517)
## [1.2.0] - 2019-03-01
### Added
- `--write-to` parameter for setting custom file to write Spotify track URLs to ([@ritiek](https://github.com/ritiek)) (#507)
- Set custom Spotify Client ID and Client Secret via config.yml ([@ManveerBasra](https://github.com/ManveerBasra)) (#502)
- Use YouTube as fallback metadata if track not found on Spotify. Also added `--no-fallback-metadata`
to preserve old behaviour ([@ritiek](https://github.com/ritiek)) (#457)
### Fixed
- Fix already downloaded prompt when using "/" in `--file-format` to create sub-directories ([@ritiek](https://github.com/ritiek)) (#503)
- Fix writing playlist tracks to file ([@ritiek](https://github.com/ritiek)) (#506)
## [1.1.2] - 2019-02-10
### Changed
- Fetch all artist albums by default instead of only fetching the "album" type ([@ritiek](https://github.com/ritiek)) (#493)
- Option `-f` (`--folder`) is used when exporting text files using `-p` (`--playlist`) for playlists or `-b` (`--album`) for albums ([@Silverfeelin](https://github.com/Silverfeelin)) (#476)
- Use first artist from album object for album artist ([@tillhainbach](https://github.com/tillhainbach))
### Fixed
- Fix renaming files when encoder is not found ([@ritiek](https://github.com/ritiek)) (#475)
- Add missing `import time` ([@ifduyue](https://github.com/ifduyue)) (#465)
## [1.1.1] - 2019-01-03
### Added
- Output informative message in case of no result found in YouTube search ([@Amit-L](https://github.com/Amit-L)) (#452)
- Ability to pass multiple tracks with `-s` option ([@ritiek](https://github.com/ritiek)) (#442)
### Changed
- Allowed to fetch metadata from Spotify upon searching Spotify-URL and `--no-metadata` to gather YouTube custom-search fields ([@Amit-L](https://github.com/Amit-L)) (#452)
- Change FFmpeg to use the built-in encoder `aac` instead of 3rd party `libfdk-aac` which does not
ship with the apt package ([@ritiek](https://github.com/ritiek)) (#448)
- Monkeypatch ever-changing network-relying tests ([@ritiek](https://github.com/ritiek)) (#448)
- Correct `.m4a` container before writing metadata so metadata fields shows up properly in
media players (especially iTunes) ([@ritiek](https://github.com/ritiek) with thanks to [@Amit-L](https://github.com/Amit-L)!) (#453)
- Refactored core downloading module ([@ritiek](https://github.com/ritiek)) (#410)
### Fixed
- Workaround conversion conflicts when input and output filename are same ([@ritiek](https://github.com/ritiek)) (#459)
- Applied a check on result in case of search using Spotify-URL `--no-metadata` option ([@Amit-L](https://github.com/Amit-L)) (#452)
- Included a missing `import spotipy` in downloader.py ([@ritiek](https://github.com/ritiek)) (#440)
## [1.1.0] - 2018-11-13
### Added
- Output error details when track download fails from list file ([@ManveerBasra](https://github.com/ManveerBasra)) (#406)
- Add support for `.m3u` playlists ([@ritiek](https://github.com/ritiek)) (#401)
- Introduce usage of black (code formatter) ([@linusg](https://github.com/linusg)) (#393)
- Added command line option for getting all artist's songs ([@AlfredoSequeida](https://github.com/AlfredoSequeida)) (#389)
- Added command line options for skipping tracks file and successful downloads file and
place newline before track URL when appending to track file ([@linusg](https://github.com/linusg)) (#386)
- Overwrite track file with unique tracks ([@ritiek](https://github.com/ritiek)) (#380)
- Embed comment metadata in `.m4a` ([@ritiek](https://github.com/ritiek)) (#379)
- Added check for publisher tag before adding publisher id3 tag to audio file ([@gnodar01](https://github.com/gnodar01)) (#377)
### Changed
- `--list` flag accepts only text files using mimetypes ([@ManveerBasra](https://github.com/ManveerBasra)) (#414)
- Refactored Spotify token refresh ([@ManveerBasra](https://github.com/ManveerBasra)) (#408)
- Don't search song on Spotify if `--no-metadata` is passed ([@ManveerBasra](https://github.com/ManveerBasra)) (#404)
- Changed test track to one whose lyrics are found ([@ManveerBasra](https://github.com/ManveerBasra)) (#400)
- Windows - 'My Music' folder won't be assumed to be on C drive but looked up in Registry ([@SillySam](https://github.com/SillySam)) (#387)
- Updated `setup.py` (fix PyPI URL, add Python 3.7 modifier) ([@linusg](https://github.com/linusg)) (#383)
- Updated dependencies to their newest versions (as of 2018-10-02) ([@linusg](https://github.com/linusg)) (#382)
- Remove duplicates from track file while preserving order ([@ritiek](https://github.com/ritiek)) (#369)
- Moved a lot of content from `README.md` to the [repository's GitHub wiki](https://github.com/ritiek/spotify-downloader/wiki) ([@sdhutchins](https://github.com/sdhutchins), [@ritiek](https://github.com/ritiek)) (#361)
- Refactored internal use of logging ([@arryon](https://github.com/arryon)) (#358)
### Fixed
- Check and replace slashes with dashes to avoid directory creation error ([@ManveerBasra](https://github.com/ManveerBasra)) (#402)
- Filter unwanted text from Spotify URLs when extracting information ([@ritiek](https://github.com/ritiek)) (#394)
- Correctly embed metadata in `.m4a` ([@arryon](https://github.com/arryon)) (#372)
- Slugify will not ignore the `'` character (single quotation mark) anymore ([@jimangel2001](https://github.com/jimangel2001)) (#357)
## [1.0.0] - 2018-09-09
### Added
- Initial complete release, recommended way to install is now from PyPI
## 1.0.0-beta.1 - 2018-02-02
### Added
- Initial release, prepare for 1.0.0
[Unreleased]: https://github.com/ritiek/spotify-downloader/compare/v1.1.0...HEAD
[1.1.0]: https://github.com/ritiek/spotify-downloader/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/ritiek/spotify-downloader/compare/v1.0.0-beta.1...v1.0.0

74
CODE_OF_CONDUCT.md
View File

@@ -1,74 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
education, socio-economic status, nationality, personal appearance, race,
religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project owner at ritiekmalhotra123@gmail.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org

38
CONTRIBUTING.md
View File

@@ -1,38 +0,0 @@
# Contributing
- Want to contribute to [spotify-downloader](https://github.com/ritiek/spotify-downloader)?
That's great. We are happy to have you!
- Here is a basic outline on opening issues and making PRs:
## Opening Issues
- Search for your problem in the
[issues section](https://github.com/ritiek/spotify-downloader/issues)
before opening a new ticket. It might be already answered and save both you and us time. :smile:
- Provide as much information as possible when opening your ticket, including any relevant examples (if any).
- If your issue is a *bug*, make sure you pass `--log-level=DEBUG` when invoking
`spotdl.py` and paste the output in your issue.
- If you think your question is naive or something and you can't find anything related,
don't feel bad. Open an issue any way!
## Making Pull Requests
- Look up for open issues and see if you can help out there.
- Easy issues for newcomers are usually labelled as
[good-first-issue](https://github.com/ritiek/spotify-downloader/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).
- When making a PR, point it to the [master branch](https://github.com/ritiek/spotify-downloader/tree/master)
unless mentioned otherwise.
- Code should be formatted using [black](https://github.com/ambv/black). Don't worry if you forgot or don't know how to do this, the codebase will be black-formatted with each release.
- All tests are placed in the [test directory](https://github.com/ritiek/spotify-downloader/tree/master/test). We use [pytest](https://github.com/pytest-dev/pytest)
to run the test suite: `$ pytest`.
If you don't have pytest, you can install it with `$ pip3 install pytest`.
- Add a note about the changes, your GitHub username and a reference to the PR to the `Unreleased` section of the [`CHANGES.md`](CHANGES.md) file (see existing releases for examples), add the appropriate section ("Added", "Changed", "Fixed" etc.) if necessary. You don't have to increment version numbers. See https://keepachangelog.com/en/1.0.0/ for more information.
- If you are planning to work on something big, let us know through an issue. So we can discuss more about it.
- Lastly, please don't hesitate to ask if you have any questions!
Let us know (through an issue) if you are facing any trouble making a PR, we'd be glad to help you out!
## Related Resources
- There's also a web-based front-end to operate this tool, which under (major) construction
called [spotifube](https://github.com/linusg/spotifube).
Check it out if you'd like to contribute to it!

16
Dockerfile
View File

@@ -1,16 +0,0 @@
FROM python:3.6-alpine
RUN apk add --no-cache \
ffmpeg
ADD spotdl/ /spotify-downloader/spotdl
ADD setup.py /spotify-downloader/setup.py
ADD README.md /spotify-downloader/README.md
WORKDIR /spotify-downloader
RUN pip install .
RUN mkdir /music
WORKDIR /music
ENTRYPOINT ["spotdl"]

30
ISSUE_TEMPLATE.md
View File

@@ -1,25 +1,31 @@
<!--
Please follow the guide below
- Before opening your ticket, make sure you either installed the latest release from PyPI
or installed directly from the master branch and have searched through existing issues
including closed ones.
-->
<!--
- Put an `x` into the box [ ] below (like [x]) depending on the purpose of your issue
- You will be asked some questions and requested to provide some information, please read them CAREFULLY and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
- Use *Preview* tab to see how your issue will actually look like
-->
- [ ] Using latest version as provided on the [master branch](https://github.com/ritiek/spotify-downloader/tree/master)
- [ ] [Searched](https://github.com/ritiek/spotify-downloader/issues?utf8=%E2%9C%93&q=is%3Aissue) for similar issues including closed ones
#### What is the purpose of your *issue*?
- [ ] Bug
- [ ] Feature Request
- [ ] Script won't run
- [ ] Encountered bug
- [ ] Feature request
- [ ] Question
- [ ] Other
#### System information
- Your `python` version: `python 3.x`
- Your operating system: `Ubuntu 16.04`
### Description
<!-- Provide as much information possible and whatever you have tried below -->
<!-- Provide as much information possible with relevant examples and whatever you have tried below -->
### Log
<!-- Run the script with `--log-level=DEBUG` and paste the output below-->
<!-- Give your issue a relevant title and you are good to go -->

2
LICENSE → LICENSE.txt Normal file → Executable file
View File

@@ -1,5 +1,5 @@
The MIT License (MIT)
Copyright (c) 2018 Ritiek Malhotra
Copyright (c) 2016 Ritiek Malhotra
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

203
README.md Normal file → Executable file
View File

@@ -1,87 +1,196 @@
# Spotify-Downloader
[![PyPi](https://img.shields.io/pypi/v/spotdl.svg)](https://pypi.org/project/spotdl)
[![Build Status](https://travis-ci.org/ritiek/spotify-downloader.svg?branch=master)](https://travis-ci.org/ritiek/spotify-downloader)
[![Coverage Status](https://codecov.io/gh/ritiek/spotify-downloader/branch/master/graph/badge.svg)](https://codecov.io/gh/ritiek/spotify-downloader)
[![Docker Build Status](https://img.shields.io/docker/build/ritiek/spotify-downloader.svg)](https://hub.docker.com/r/ritiek/spotify-downloader)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
[![Gitter Chat](https://badges.gitter.im/ritiek/spotify-downloader/Lobby.svg)](https://gitter.im/spotify-downloader/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
- Downloads songs from YouTube in an MP3 format by using Spotify's HTTP link.
- Downloads songs from YouTube in an MP3 format by using Spotify's HTTP link.
- Can also download a song by entering its artist and song name (in case if you don't have the Spotify's HTTP link for some song).
- Automatically applies metadata to the downloaded song which includes:
- `Title`, `Artist`, `Album`, `Album art`, `Lyrics` (if found either on [Genius](https://genius.com/)), `Album artist`, `Genre`, `Track number`, `Disc number`, `Release date`, and more...
- Automatically fixes song's meta-tags which include:
- Works straight out of the box and does not require you to generate or mess with your API keys (already included).
- Title
- Artist
- Album
- Album art
- Album artist
- Genre
- Track number
- Disc number
- Release date
- And some more...
Below is how your music library will look!
- Works straight out of the box and does not require to generate or mess with your API keys.
That's how your Music library will look like!
<img src="http://i.imgur.com/Gpch7JI.png" width="290"><img src="http://i.imgur.com/5vhk3HY.png" width="290"><img src="http://i.imgur.com/RDTCCST.png" width="290">
## Installation
## Reporting Issues
❗️ **This tool works only with Python 3.6+**
- Search for your problem in the [issues section](https://github.com/Ritiek/spotify-downloader/issues?utf8=%E2%9C%93&q=) before opening a new ticket. It might be already answered and save us time. :smile:
spotify-downloader works with all major distributions and even on low-powered devices such as a Raspberry Pi.
- Provide as much information possible when opening your ticket.
spotify-downloader can be installed via pip with:
```console
$ pip3 install spotdl
## Installation & Usage
<img src="http://i.imgur.com/Dg8p9up.png" width="600">
- **This tool supports only Python 3**, Python 2 compatibility was dropped because of the way it deals with unicode. If you need to use Python 2 though, check out the (old) `python2` branch.
- Note: `play` and `lyrics` commands have been deprecated in the current brach since they were not of much use and created unnecessary clutter. You can still get them back by using `old` branch though.
### Debian, Ubuntu, Linux & Mac
```
cd
git clone https://github.com/ritiek/spotify-downloader
cd spotify-downloader
pip install -U -r requirements.txt
```
but be sure to check out the [Installation](https://github.com/ritiek/spotify-downloader/wiki/Installation) wiki
page for detailed OS-specific instructions to get it and other dependencies it relies on working on your system.
**Important:** if you have installed both Python 2 and 3, the `pip` command could invoke an installation for Python 2. To see which Python version `pip` refers to, try `pip -V`. If it turns out `pip` is your Python 2 pip, try `pip3 install -U -r requirements.txt` instead.
## Usage
You'll also need to install FFmpeg for conversion (use `--avconv` if you'd like to use that instead):
For the most basic usage, downloading tracks is as easy as
Linux: `sudo apt-get install ffmpeg`
```console
$ spotdl --song https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ
$ spotdl --song "ncs - spectre"
Mac: `brew install ffmpeg --with-libmp3lame --with-libass --with-opus --with-fdk-aac`
If it does not install correctly, you may have to build it from source. For more info see https://trac.ffmpeg.org/wiki/CompilationGuide.
### Windows
Assuming you have Python 3 ([preferably v3.6 or above to stay away from Unicode errors](https://stackoverflow.com/questions/30539882/whats-the-deal-with-python-3-4-unicode-different-languages-and-windows)) already installed and in PATH.
- Download and extract the [zip file](https://github.com/ritiek/spotify-downloader/archive/master.zip) from master branch.
- Download FFmpeg for Windows from [here](http://ffmpeg.zeranoe.com/builds/). Copy `ffmpeg.exe` from `ffmpeg-xxx-winxx-static\bin\ffmpeg.exe` to PATH (usually C:\Windows\System32\) or just place it in the root directory extracted from the above step.
- Open `cmd` and type `pip install -U -r requirements.txt` to install dependencies. The same note about `pip` as for Debian, Ubuntu, Linux & Mac applies.
## Instructions for Downloading Songs
**Important:** as like with `pip`, there might be no `python3` command. This is most likely the case when you have only Python 3 but not 2 installed. In this case try the `python` command instead of `python3`, but make sure `python -V` gives you a `Python 3.x.x`!
- For all available options, run `python3 spotdl.py --help`.
```
usage: spotdl.py [-h] (-s SONG | -l LIST | -p PLAYLIST | -u USERNAME) [-m]
[-nm] [-a] [-f FOLDER] [-v] [-i INPUT_EXT] [-o OUTPUT_EXT]
Download and convert songs from Spotify, Youtube etc.
optional arguments:
-h, --help show this help message and exit
-s SONG, --song SONG download song by spotify link or name (default: None)
-l LIST, --list LIST download songs from a file (default: None)
-p PLAYLIST, --playlist PLAYLIST
load songs from playlist URL into <playlist_name>.txt
(default: None)
-u USERNAME, --username USERNAME
load songs from user's playlist into
<playlist_name>.txt (default: None)
-m, --manual choose the song to download manually (default: False)
-nm, --no-metadata do not embed metadata in songs (default: False)
-a, --avconv Use avconv for conversion otherwise set defaults to
ffmpeg (default: False)
-f FOLDER, --folder FOLDER
path to folder where files will be stored in (default:
Music/)
-v, --verbose show debug output (default: False)
-i INPUT_EXT, --input_ext INPUT_EXT
prefered input format .m4a or .webm (Opus) (default:
.m4a)
-o OUTPUT_EXT, --output_ext OUTPUT_EXT
prefered output extension .mp3 or .m4a (AAC) (default:
.mp3)
```
For downloading playlist and albums, you need to first load all the tracks into text file and then pass
this text file to `--list` argument. Here is how you would do it for a playlist
#### Download by Name
```console
$ spotdl --playlist https://open.spotify.com/user/nocopyrightsounds/playlist/7sZbq8QGyMnhKPcLJvCUFD
INFO: Writing 62 tracks to ncs-releases.txt
$ spotdl --list ncs-releases.txt
For example
- We want to download Hello by Adele, simply run `python3 spotdl.py --song "adele hello"`.
- The script will automatically look for the best matching song and download it in the folder `Music/` placed in the root directory of the code base.
- It will now convert the song to an mp3 and try to fix meta-tags and album-art by looking up on Spotify.
#### Download by Spotify Link (Recommended)
For example
- We want to download the same song (i.e: Hello by Adele) but using Spotify Link this time that looks like `http://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7`, you can copy it from your Spotify desktop or mobile app by right clicking or long tap on the song and copy HTTP link.
- Run `python3 spotdl.py --song http://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7`, it should download Hello by Adele.
- Just like before, it will again convert the song to an mp3 but since we used a Spotify HTTP link, the script is guaranteed to fetch the correct meta-tags and album-art.
#### Download multiple songs at once
For example
- We want to download `Hello by Adele`, `The Nights by Avicci` and `21 Guns by Green Day` just using a single command.
Let's suppose, we have the Spotify link for only `Hello by Adele` and `21 Guns by Green Day`.
No problem!
- Just make a `list.txt` in the same folder as the script and add all the songs you want to download, in our case it is
(if you are on Windows, just edit `list.txt` - i.e `C:\Python36\spotify-downloader-master\list.txt`)
```
https://open.spotify.com/track/1MDoll6jK4rrk2BcFRP5i7
the nights avicci
http://open.spotify.com/track/64yrDBpcdwEdNY9loyEGbX
```
Run `spotdl --help` to get a list of all available options in spotify-downloader.
- Now pass `--list=list.txt` to the script, i.e `python3 spotdl.py --list=list.txt` and it will start downloading songs mentioned in `list.txt`.
Check out the [Available options](https://github.com/ritiek/spotify-downloader/wiki/Available-options)
wiki page for the list of currently available options with their description.
- You can stop downloading songs by hitting `ctrl+c`, the script will automatically resume from the song where you stopped it the next time you want to download the songs present in `list.txt`.
The wiki page [Instructions for Downloading Songs](https://github.com/ritiek/spotify-downloader/wiki/Instructions-for-Downloading-Songs)
contains detailed information about different available ways to download tracks.
- Songs that are already downloaded will be skipped and not be downloaded again.
## FAQ
#### Download playlists
All FAQs will be mentioned in our [FAQ wiki page](https://github.com/ritiek/spotify-downloader/wiki/FAQ).
- You can copy the Spotify URL of the playlist and pass it in `--playlist` option.
## Contributing
For example
Check out [CONTRIBUTING.md](CONTRIBUTING.md) for more info.
- `python3 spotdl.py --playlist https://open.spotify.com/user/camillazi/playlist/71MXqcSOKCxsLNtRvONkhF`
## Running Tests
- The script will load all the tracks from the playlist into `<playlist_name>.txt`
```console
$ pytest
- Then you can simply run `python3 spotdl.py --list=<playlist_name>.txt` to download all the tracks.
#### Download playlists by username
- You can also load songs using Spotify username if you don't have the playlist URL. (Open profile in Spotify, click on the three little dots below name, "Share", "Copy to clipboard", paste last numbers into command-line: `https://open.spotify.com/user/0123456790`)
- Try running `python3 spotdl.py -u <your_username>`, it will show all your public playlists.
- Once you select the one you want to download, the script will load all the tracks from the playlist into `<playlist_name>.txt`.
- Run `python3 spotdl.py --list=<playlist_name>.txt` to download all the tracks.
#### Specify the target directory
If you don't want to download all the songs to the `Music/` folder relative to the `spotdl.py` script, you can use the `-f`/`--folder` option. E.g. `python3 spotdl.py -s "adele hello" -f "/home/user/Music/"`. This works with both relative and absolute paths.
## Running tests
```
python3 -m pytest test
```
Obviously this requires the `pytest` module to be installed.
Obviously this requires the `pytest` module to be installed.
## Disclaimer
Downloading copyright songs may be illegal in your country.
This tool is for educational purposes only and was created only to show
how Spotify's API can be exploited to download music from YouTube.
Please support the artists by buying their music.
Downloading copyright songs may be illegal in your country. This tool is for educational purposes only and was created only to show how Spotify's API can be exploited to download music from YouTube. Please support the artists by buying their music.
## License
[![License](https://img.shields.io/github/license/ritiek/spotify-downloader.svg)](https://github.com/ritiek/spotify-downloader/blob/master/LICENSE)
```The MIT License```

1
core/__init__.py Executable file
View File

@@ -0,0 +1 @@

68
core/convert.py Normal file
View File

@@ -0,0 +1,68 @@
import subprocess
import os
"""
What are the differences and similarities between ffmpeg, libav, and avconv?
https://stackoverflow.com/questions/9477115
ffmeg encoders high to lower quality
libopus > libvorbis >= libfdk_aac > aac > libmp3lame
libfdk_aac due to copyrights needs to be compiled by end user
on MacOS brew install ffmpeg --with-fdk-aac will do just that. Other OS?
https://trac.ffmpeg.org/wiki/Encode/AAC
"""
def song(input_song, output_song, folder, avconv=False, verbose=False):
"""Do the audio format conversion."""
if not input_song == output_song:
print('Converting {0} to {1}'.format(
input_song, output_song.split('.')[-1]))
if avconv:
exit_code = convert_with_avconv(input_song, output_song, folder, verbose)
else:
exit_code = convert_with_ffmpeg(input_song, output_song, folder, verbose)
return exit_code
return 0
def convert_with_avconv(input_song, output_song, folder, verbose):
"""Convert the audio file using avconv."""
if verbose:
level = 'debug'
else:
level = '0'
command = ['avconv',
'-loglevel', level,
'-i', os.path.join(folder, input_song),
'-ab', '192k',
os.path.join(folder, output_song)]
return subprocess.call(command)
def convert_with_ffmpeg(input_song, output_song, folder, verbose):
"""Convert the audio file using FFmpeg."""
ffmpeg_pre = 'ffmpeg -y '
if not verbose:
ffmpeg_pre += '-hide_banner -nostats -v panic '
input_ext = input_song.split('.')[-1]
output_ext = output_song.split('.')[-1]
if input_ext == 'm4a':
if output_ext == 'mp3':
ffmpeg_params = '-codec:v copy -codec:a libmp3lame -q:a 2 '
elif output_ext == 'webm':
ffmpeg_params = '-c:a libopus -vbr on -b:a 192k -vn '
elif input_ext == 'webm':
if output_ext == 'mp3':
ffmpeg_params = ' -ab 192k -ar 44100 -vn '
elif output_ext == 'm4a':
ffmpeg_params = '-cutoff 20000 -c:a libfdk_aac -b:a 192k -vn '
command = '{0}-i {1} {2}{3}'.format(
ffmpeg_pre, os.path.join(folder, input_song), ffmpeg_params, os.path.join(folder, output_song)).split(' ')
return subprocess.call(command)

125
core/metadata.py Executable file
View File

@@ -0,0 +1,125 @@
from mutagen.easyid3 import EasyID3
from mutagen.id3 import ID3, APIC
from mutagen.mp4 import MP4, MP4Cover
import urllib.request
def compare(music_file, metadata):
"""Check if the input music file title matches the expected title."""
already_tagged = False
try:
if music_file.endswith('.mp3'):
audiofile = EasyID3(music_file)
# fetch track title metadata
already_tagged = audiofile['title'][0] == metadata['name']
elif music_file.endswith('.m4a'):
tags = {'title': '\xa9nam'}
audiofile = MP4(music_file)
# fetch track title metadata
already_tagged = audiofile[tags['title']] == metadata['name']
except (KeyError, TypeError):
pass
return already_tagged
def embed(music_file, meta_tags):
"""Embed metadata."""
if meta_tags is None:
print('Could not find meta-tags')
return None
elif music_file.endswith('.m4a'):
print('Fixing meta-tags')
return embed_m4a(music_file, meta_tags)
elif music_file.endswith('.mp3'):
print('Fixing meta-tags')
return embed_mp3(music_file, meta_tags)
else:
print('Cannot embed meta-tags into given output extension')
return False
def embed_mp3(music_file, meta_tags):
"""Embed metadata to MP3 files."""
# EasyID3 is fun to use ;)
audiofile = EasyID3(music_file)
audiofile['artist'] = meta_tags['artists'][0]['name']
audiofile['albumartist'] = meta_tags['artists'][0]['name']
audiofile['album'] = meta_tags['album']['name']
audiofile['title'] = meta_tags['name']
audiofile['tracknumber'] = [meta_tags['track_number'],
meta_tags['total_tracks']]
audiofile['discnumber'] = [meta_tags['disc_number'], 0]
audiofile['date'] = meta_tags['release_date']
audiofile['originaldate'] = meta_tags['release_date']
audiofile['media'] = meta_tags['type']
audiofile['author'] = meta_tags['artists'][0]['name']
audiofile['lyricist'] = meta_tags['artists'][0]['name']
audiofile['arranger'] = meta_tags['artists'][0]['name']
audiofile['performer'] = meta_tags['artists'][0]['name']
audiofile['encodedby'] = meta_tags['publisher']
audiofile['website'] = meta_tags['external_urls']['spotify']
audiofile['length'] = str(meta_tags['duration_ms'] / 1000)
if meta_tags['genre']:
audiofile['genre'] = meta_tags['genre']
if meta_tags['copyright']:
audiofile['copyright'] = meta_tags['copyright']
if meta_tags['isrc']:
audiofile['isrc'] = meta_tags['external_ids']['isrc']
audiofile.save(v2_version=3)
audiofile = ID3(music_file)
try:
albumart = urllib.request.urlopen(meta_tags['album']['images'][0]['url'])
audiofile["APIC"] = APIC(encoding=3, mime='image/jpeg', type=3,
desc=u'Cover', data=albumart.read())
albumart.close()
except IndexError:
pass
audiofile.save(v2_version=3)
return True
def embed_m4a(music_file, meta_tags):
"""Embed metadata to M4A files."""
# Apple has specific tags - see mutagen docs -
# http://mutagen.readthedocs.io/en/latest/api/mp4.html
tags = {'album': '\xa9alb',
'artist': '\xa9ART',
'date': '\xa9day',
'title': '\xa9nam',
'originaldate': 'purd',
'comment': '\xa9cmt',
'group': '\xa9grp',
'writer': '\xa9wrt',
'genre': '\xa9gen',
'tracknumber': 'trkn',
'albumartist': 'aART',
'disknumber': 'disk',
'cpil': 'cpil',
'albumart': 'covr',
'copyright': 'cprt',
'tempo': 'tmpo'}
audiofile = MP4(music_file)
audiofile[tags['artist']] = meta_tags['artists'][0]['name']
audiofile[tags['albumartist']] = meta_tags['artists'][0]['name']
audiofile[tags['album']] = meta_tags['album']['name']
audiofile[tags['title']] = meta_tags['name']
audiofile[tags['tracknumber']] = [(meta_tags['track_number'],
meta_tags['total_tracks'])]
audiofile[tags['disknumber']] = [(meta_tags['disc_number'], 0)]
audiofile[tags['date']] = meta_tags['release_date']
audiofile[tags['originaldate']] = meta_tags['release_date']
if meta_tags['genre']:
audiofile[tags['genre']] = meta_tags['genre']
if meta_tags['copyright']:
audiofile[tags['copyright']] = meta_tags['copyright']
try:
albumart = urllib.request.urlopen(meta_tags['album']['images'][0]['url'])
audiofile[tags['albumart']] = [MP4Cover(
albumart.read(), imageformat=MP4Cover.FORMAT_JPEG)]
albumart.close()
except IndexError:
pass
audiofile.save()
return True

141
core/misc.py Executable file
View File

@@ -0,0 +1,141 @@
import sys
import os
import argparse
import spotipy.oauth2 as oauth2
from urllib.request import quote
from slugify import slugify
def input_link(links):
"""Let the user input a number."""
while True:
try:
the_chosen_one = int(input('>> Choose your number: '))
if 1 <= the_chosen_one <= len(links):
return links[the_chosen_one - 1]
elif the_chosen_one == 0:
return None
else:
print('Choose a valid number!')
except ValueError:
print('Choose a valid number!')
def trim_song(file):
"""Remove the first song from file."""
with open(file, 'r') as file_in:
data = file_in.read().splitlines(True)
with open(file, 'w') as file_out:
file_out.writelines(data[1:])
def get_arguments():
parser = argparse.ArgumentParser(
description='Download and convert songs from Spotify, Youtube etc.',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument(
'-s', '--song', help='download song by spotify link or name')
group.add_argument(
'-l', '--list', help='download songs from a file')
group.add_argument(
'-p', '--playlist', help='load songs from playlist URL into <playlist_name>.txt')
group.add_argument(
'-u', '--username',
help="load songs from user's playlist into <playlist_name>.txt")
parser.add_argument(
'-m', '--manual', default=False,
help='choose the song to download manually', action='store_true')
parser.add_argument(
'-nm', '--no-metadata', default=False,
help='do not embed metadata in songs', action='store_true')
parser.add_argument(
'-a', '--avconv', default=False,
help='Use avconv for conversion otherwise set defaults to ffmpeg',
action='store_true')
parser.add_argument(
'-f', '--folder', default='Music/',
help='path to folder where files will be stored in')
parser.add_argument(
'-v', '--verbose', default=False, help='show debug output',
action='store_true')
parser.add_argument(
'-i', '--input_ext', default='.m4a',
help='prefered input format .m4a or .webm (Opus)')
parser.add_argument(
'-o', '--output_ext', default='.mp3',
help='prefered output extension .mp3 or .m4a (AAC)')
return parser.parse_args()
def is_spotify(raw_song):
"""Check if the input song is a Spotify link."""
status = len(raw_song) == 22 and raw_song.replace(" ", "%20") == raw_song
status = status or raw_song.find('spotify') > -1
return status
def is_youtube(raw_song):
"""Check if the input song is a YouTube link."""
status = len(raw_song) == 11 and raw_song.replace(" ", "%20") == raw_song
status = status and not raw_song.lower() == raw_song
status = status or 'youtube.com/watch?v=' in raw_song
return status
def sanitize_title(title):
"""Generate filename of the song to be downloaded."""
title = title.replace(' ', '_')
title = title.replace('/', '_')
# slugify removes any special characters
title = slugify(title, ok='-_()[]{}', lower=False)
return title
def generate_token():
"""Generate the token. Please respect these credentials :)"""
credentials = oauth2.SpotifyClientCredentials(
client_id='4fe3fecfe5334023a1472516cc99d805',
client_secret='0f02b7c483c04257984695007a4a8d5c')
token = credentials.get_access_token()
return token
def generate_search_url(song, viewsort=False):
"""Generate YouTube search URL for the given song."""
# urllib.request.quote() encodes URL with special characters
song = quote(song)
if viewsort:
url = u"https://www.youtube.com/results?q={0}".format(song)
else:
url = u"https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q={0}".format(song)
return url
def filter_path(path):
os.chdir(sys.path[0])
if not os.path.exists(path):
os.makedirs(path)
for temp in os.listdir(path):
if temp.endswith('.temp'):
os.remove(os.path.join(path, temp))
def grace_quit():
print('\n\nExiting.')
sys.exit()
def get_sec(time_str):
v = time_str.split(':', 3)
v.reverse()
sec = 0
if len(v) > 0: #seconds
sec += int(v[0])
if len(v) > 1: # minutes
sec += int(v[1]) * 60
if len(v) > 2: # hours
sec += int(v[2]) * 3600
return sec

8
requirements.txt Executable file
View File

@@ -0,0 +1,8 @@
pathlib >= 1.0.1
BeautifulSoup4 >= 0.4.13
youtube_dl >= 2017.5.1
pafy >= 0.5.3.1
spotipy >= 2.4.4
mutagen >= 1.37
unicode-slugify >= 0.1.3
titlecase >= 0.10.0

5
setup.cfg
View File

@@ -1,5 +0,0 @@
[tool:pytest]
addopts = --strict-markers -m "not network"
markers =
network: marks test which rely on external network resources (select with '-m network' or run all with '-m "network, not network"')

79
setup.py
View File

@@ -1,79 +0,0 @@
from setuptools import setup
import os
with open("README.md", "r", encoding="utf-8") as f:
long_description = f.read()
# __version__ comes into namespace from here
with open(os.path.join("spotdl", "version.py")) as version_file:
exec(version_file.read())
setup(
# 'spotify-downloader' was already taken :/
name="spotdl",
# Tests are included automatically:
# https://docs.python.org/3.6/distutils/sourcedist.html#specifying-the-files-to-distribute
packages=[
"spotdl",
"spotdl.command_line",
"spotdl.lyrics",
"spotdl.lyrics.providers",
"spotdl.encode",
"spotdl.encode.encoders",
"spotdl.metadata",
"spotdl.metadata.embedders",
"spotdl.metadata.providers",
"spotdl.lyrics",
"spotdl.lyrics.providers",
"spotdl.authorize",
"spotdl.authorize.services",
"spotdl.helpers",
],
version=__version__,
install_requires=[
"pathlib >= 1.0.1",
"youtube_dl >= 2017.9.26",
"pytube3 >= 9.5.5",
"spotipy >= 2.12.0",
"mutagen >= 1.41.1",
"beautifulsoup4 >= 4.6.3",
"unicode-slugify >= 0.1.3",
"coloredlogs >= 14.0",
"lyricwikia >= 0.1.8",
"PyYAML >= 3.13",
"appdirs >= 1.4.3",
"tqdm >= 4.45.0"
],
description="Download songs from YouTube using Spotify song URLs or playlists with albumart and meta-tags.",
long_description=long_description,
long_description_content_type="text/markdown",
author="Ritiek Malhotra",
author_email="ritiekmalhotra123@gmail.com",
license="MIT",
python_requires=">=3.6",
url="https://github.com/ritiek/spotify-downloader",
download_url="https://pypi.org/project/spotdl/",
keywords=[
"spotify",
"downloader",
"download",
"music",
"youtube",
"mp3",
"album",
"metadata",
],
classifiers=[
"Intended Audience :: End Users/Desktop",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3 :: Only",
"Topic :: Multimedia",
"Topic :: Multimedia :: Sound/Audio",
"Topic :: Utilities",
],
entry_points={"console_scripts": ["spotdl = spotdl:main"]},
)

428
spotdl.py Executable file
View File

@@ -0,0 +1,428 @@
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
from core import metadata
from core import convert
from core import misc
from bs4 import BeautifulSoup
from titlecase import titlecase
from slugify import slugify
import spotipy
import pafy
import urllib.request
import sys
import os
import time
import threading
def generate_songname(tags):
"""Generate a string of the format '[artist] - [song]' for the given spotify song."""
raw_song = u'{0} - {1}'.format(tags['artists'][0]['name'], tags['name'])
return raw_song
def generate_metadata(raw_song):
"""Fetch a song's metadata from Spotify."""
if misc.is_spotify(raw_song):
# fetch track information directly if it is spotify link
meta_tags = spotify.track(raw_song)
else:
# otherwise search on spotify and fetch information from first result
try:
meta_tags = spotify.search(raw_song, limit=1)['tracks']['items'][0]
except:
return None
artist = spotify.artist(meta_tags['artists'][0]['id'])
album = spotify.album(meta_tags['album']['id'])
try:
meta_tags[u'genre'] = titlecase(artist['genres'][0])
except IndexError:
meta_tags[u'genre'] = None
try:
meta_tags[u'copyright'] = album['copyrights'][0]['text']
except IndexError:
meta_tags[u'copyright'] = None
try:
meta_tags['isrc']
except KeyError:
meta_tags['isrc'] = None
meta_tags[u'release_date'] = album['release_date']
meta_tags[u'publisher'] = album['label']
meta_tags[u'total_tracks'] = album['tracks']['total']
return meta_tags
def generate_youtube_url(raw_song, tries_remaining=5):
"""Search for the song on YouTube and generate a URL to its video."""
# prevents an infinite loop but allows for a few retries
if tries_remaining == 0:
return
meta_tags = generate_metadata(raw_song)
if meta_tags is None:
song = raw_song
search_url = misc.generate_search_url(song, viewsort=False)
else:
song = generate_songname(meta_tags)
search_url = misc.generate_search_url(song, viewsort=True)
item = urllib.request.urlopen(search_url).read()
# item = unicode(item, 'utf-8')
items_parse = BeautifulSoup(item, "html.parser")
videos = []
for x in items_parse.find_all('div', {'class': 'yt-lockup-dismissable yt-uix-tile'}):
# ensure result is not a channel
if x.find('channel') is not None or 'yt-lockup-channel' in x.parent.attrs['class'] or 'yt-lockup-channel' in x.attrs['class']:
continue
# ensure result is not a mix/playlist
if 'yt-lockup-playlist' in x.parent.attrs['class']:
continue
# confirm the video result is not an advertisement
if x.find('googleads') is not None:
continue
y = x.find('div', class_='yt-lockup-content')
link = y.find('a')['href']
title = y.find('a')['title']
try:
videotime = x.find('span', class_="video-time").get_text()
except AttributeError:
return generate_youtube_url(raw_song, tries_remaining - 1)
youtubedetails = {'link': link, 'title': title, 'videotime': videotime, 'seconds':misc.get_sec(videotime)}
videos.append(youtubedetails)
if meta_tags is None:
break
if not videos:
return None
if args.manual:
print(song)
print('')
print('0. Skip downloading this song')
# fetch all video links on first page on YouTube
for i, v in enumerate(videos):
print(u'{0}. {1} {2} {3}'.format(i+1, v['title'], v['videotime'], "http://youtube.com"+v['link']))
print('')
# let user select the song to download
result = misc.input_link(videos)
if result is None:
return None
else:
if meta_tags is not None:
# filter out videos that do not have a similar length to the Spotify song
duration_tolerance = 10
max_duration_tolerance = 20
possible_videos_by_duration = list()
'''
start with a reasonable duration_tolerance, and increment duration_tolerance
until one of the Youtube results falls within the correct duration or
the duration_tolerance has reached the max_duration_tolerance
'''
while len(possible_videos_by_duration) == 0:
possible_videos_by_duration = list(filter(lambda x: abs(x['seconds'] - (int(meta_tags['duration_ms'])/1000)) <= duration_tolerance, videos))
duration_tolerance += 1
if duration_tolerance > max_duration_tolerance:
print(meta_tags['name'], 'by', meta_tags['artists'][0]['name'], 'was not found')
return None
result = possible_videos_by_duration[0]
else:
# if the metadata could not be acquired, take the first result from Youtube because the proper song length is unknown
result = videos[0]
full_link = None
if result:
full_link = u'youtube.com{0}'.format(result['link'])
return full_link
def go_pafy(raw_song):
"""Parse track from YouTube."""
if misc.is_youtube(raw_song):
track_info = pafy.new(raw_song)
else:
track_url = generate_youtube_url(raw_song)
if track_url is None:
track_info = None
else:
track_info = pafy.new(track_url)
return track_info
def get_youtube_title(content, number=None):
"""Get the YouTube video's title."""
title = content.title
if number is None:
return title
else:
return '{0}. {1}'.format(number, title)
def feed_playlist(username):
"""Fetch user playlists when using the -u option."""
playlists = spotify.user_playlists(username)
links = []
check = 1
while True:
for playlist in playlists['items']:
# in rare cases, playlists may not be found, so playlists['next']
# is None. Skip these. Also see Issue #91.
if playlist['name'] is not None:
print(u'{0:>5}. {1:<30} ({2} tracks)'.format(
check, playlist['name'],
playlist['tracks']['total']))
links.append(playlist)
check += 1
if playlists['next']:
playlists = spotify.next(playlists)
else:
break
print('')
playlist = misc.input_link(links)
print('')
write_tracks(playlist)
def write_tracks(playlist):
results = spotify.user_playlist(
playlist['owner']['id'], playlist['id'], fields='tracks,next')
text_file = u'{0}.txt'.format(slugify(playlist['name'], ok='-_()[]{}'))
print(u'Feeding {0} tracks to {1}'.format(playlist['tracks']['total'], text_file))
tracks = results['tracks']
with open(text_file, 'a') as file_out:
while True:
for item in tracks['items']:
track = item['track']
try:
file_out.write(track['external_urls']['spotify'] + '\n')
except KeyError:
print(u'Skipping track {0} by {1} (local only?)'.format(
track['name'], track['artists'][0]['name']))
# 1 page = 50 results
# check if there are more pages
if tracks['next']:
tracks = spotify.next(tracks)
else:
break
def download_song(file_name, content):
"""Download the audio file from YouTube."""
if args.input_ext == '.webm':
link = content.getbestaudio(preftype='webm')
elif args.input_ext == '.m4a':
link = content.getbestaudio(preftype='m4a')
else:
return False
if link is None:
return False
else:
link.download(
filepath='{0}{1}'.format(os.path.join(args.folder, file_name), args.input_ext))
return True
def check_exists(music_file, raw_song, islist=True):
"""Check if the input song already exists in the given folder."""
songs = os.listdir(args.folder)
for song in songs:
if song.endswith('.temp'):
os.remove(os.path.join(args.folder, song))
continue
# check if any song with similar name is already present in the given folder
file_name = misc.sanitize_title(music_file)
if song.startswith(file_name):
# check if the already downloaded song has correct metadata
already_tagged = metadata.compare(os.path.join(args.folder, song), generate_metadata(raw_song))
# if not, remove it and download again without prompt
if misc.is_spotify(raw_song) and not already_tagged:
os.remove(os.path.join(args.folder, song))
return False
# do not prompt and skip the current song
# if already downloaded when using list
if islist:
print('Song already exists')
return True
# if downloading only single song, prompt to re-download
else:
prompt = input('Song with same name has already been downloaded. '
'Re-download? (y/n): ').lower()
if prompt == 'y':
os.remove(os.path.join(args.folder, song))
return False
else:
return True
return False
def grab_list(text_file):
"""Download all songs from the list."""
with open(text_file, 'r') as listed:
lines = (listed.read()).splitlines()
# ignore blank lines in text_file (if any)
try:
lines.remove('')
except ValueError:
pass
print(u'Total songs in list: {0} songs'.format(len(lines)))
print('')
# nth input song
number = 1
for raw_song in lines:
try:
grab_single(raw_song, number=number)
# token expires after 1 hour
except spotipy.oauth2.SpotifyOauthError:
# refresh token when it expires
new_token = misc.generate_token()
global spotify
spotify = spotipy.Spotify(auth=new_token)
grab_single(raw_song, number=number)
# detect network problems
except (urllib.request.URLError, TypeError, IOError):
lines.append(raw_song)
# remove the downloaded song from .txt
misc.trim_song(text_file)
# and append it to the last line in .txt
with open(text_file, 'a') as myfile:
myfile.write(raw_song + '\n')
print('Failed to download song. Will retry after other songs.')
# wait 0.5 sec to avoid infinite looping
time.sleep(0.5)
continue
except KeyboardInterrupt:
misc.grace_quit()
finally:
print('')
misc.trim_song(text_file)
number += 1
def grab_playlist(playlist):
if '/' in playlist:
if playlist.endswith('/'):
playlist = playlist[:-1]
splits = playlist.split('/')
else:
splits = playlist.split(':')
username = splits[-3]
playlist_id = splits[-1]
playlists = spotify.user_playlists(username)
while True:
for playlist in playlists['items']:
if not playlist['name'] == None:
if playlist['id'] == playlist_id:
playlists['next'] = None
break
if playlists['next']:
playlists = spotify.next(playlists)
else:
break
write_tracks(playlist)
def grab_single(raw_song, number=None):
"""Logic behind downloading a song."""
if number:
islist = True
else:
islist = False
content = go_pafy(raw_song)
if content is None:
return
if misc.is_youtube(raw_song):
raw_song = slugify(content.title).replace('-', ' ')
# print '[number]. [artist] - [song]' if downloading from list
# otherwise print '[artist] - [song]'
print(get_youtube_title(content, number))
# generate file name of the song to download
meta_tags = generate_metadata(raw_song)
songname = content.title
if meta_tags is not None:
refined_songname = generate_songname(meta_tags)
if not refined_songname == ' - ':
songname = refined_songname
file_name = misc.sanitize_title(songname)
if not check_exists(file_name, raw_song, islist=islist):
if download_song(file_name, content):
print('')
thread = threading.Thread(target=finalize, args=(file_name, meta_tags))
threads.append(thread)
thread.start()
else:
print('No audio streams available')
def finalize(file_name, meta_tags):
input_song = file_name + args.input_ext
output_song = file_name + args.output_ext
convert.song(input_song, output_song, args.folder,
avconv=args.avconv, verbose=args.verbose)
if not args.input_ext == args.output_ext:
os.remove(os.path.join(args.folder, input_song))
if not args.no_metadata:
metadata.embed(os.path.join(args.folder, output_song), meta_tags)
class TestArgs(object):
manual = False
input_ext = '.m4a'
output_ext = '.mp3'
folder = 'Music/'
# token is mandatory when using Spotify's API
# https://developer.spotify.com/news-stories/2017/01/27/removing-unauthenticated-calls-to-the-web-api/
token = misc.generate_token()
spotify = spotipy.Spotify(auth=token)
if __name__ == '__main__':
os.chdir(sys.path[0])
args = misc.get_arguments()
misc.filter_path(args.folder)
threads = []
if args.song:
grab_single(raw_song=args.song)
elif args.list:
grab_list(text_file=args.list)
elif args.playlist:
grab_playlist(playlist=args.playlist)
elif args.username:
feed_playlist(username=args.username)
for thread in threads:
thread.join()
else:
misc.filter_path('Music')
args = TestArgs()

5
spotdl/__init__.py
View File

@@ -1,5 +0,0 @@
from spotdl.command_line.__main__ import main
from spotdl.version import __version__
from spotdl.command_line.core import Spotdl

6
spotdl/authorize/__init__.py
View File

@@ -1,6 +0,0 @@
from spotdl.authorize.authorize_base import AuthorizeBase
from spotdl.authorize.exceptions import AuthorizationError
from spotdl.authorize.exceptions import SpotifyAuthorizationError
from spotdl.authorize.exceptions import YouTubeAuthorizationError

19
spotdl/authorize/authorize_base.py
View File

@@ -1,19 +0,0 @@
from abc import ABC
from abc import abstractmethod
class AuthorizeBase(ABC):
"""
Defined service authenticators must inherit from this abstract
base class and implement their own functionality for the below
defined methods.
"""
@abstractmethod
def authorize(self):
"""
This method must authorize with the corresponding service
and return an object that can be utilized in making
authenticated requests.
"""
pass

20
spotdl/authorize/exceptions.py
View File

@@ -1,20 +0,0 @@
class AuthorizationError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class SpotifyAuthorizationError(AuthorizationError):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class YouTubeAuthorizationError(AuthorizationError):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)

2
spotdl/authorize/services/__init__.py
View File

@@ -1,2 +0,0 @@
from spotdl.authorize.services.spotify import AuthorizeSpotify

45
spotdl/authorize/services/spotify.py
View File

@@ -1,45 +0,0 @@
from spotdl.authorize import AuthorizeBase
from spotdl.authorize.exceptions import SpotifyAuthorizationError
import spotipy
import spotipy.oauth2 as oauth2
import logging
logger = logging.getLogger(__name__)
# This masterclient is used to keep the last logged-in client
# object in memory for for persistence. If credentials aren't
# provided when creating further objects, the last authenticated
# client object with correct credentials is returned when
# `AuthorizeSpotify().authorize()` is called.
masterclient = None
class AuthorizeSpotify(spotipy.Spotify):
def __init__(self, client_id=None, client_secret=None):
global masterclient
credentials_provided = client_id is not None \
and client_secret is not None
valid_input = credentials_provided or masterclient is not None
if not valid_input:
raise SpotifyAuthorizationError(
"You must pass in client_id and client_secret to this method "
"when authenticating for the first time."
)
if masterclient:
logger.debug("Reading cached master Spotify credentials.")
# Use cached client instead of authorizing again
# and thus wasting time.
self.__dict__.update(masterclient.__dict__)
else:
logger.debug("Setting master Spotify credentials.")
credential_manager = oauth2.SpotifyClientCredentials(
client_id=client_id,
client_secret=client_secret
)
super().__init__(client_credentials_manager=credential_manager)
# Cache current client
masterclient = self

19
spotdl/authorize/services/tests/test_spotify.py
View File

@@ -1,19 +0,0 @@
from spotdl.authorize.services import AuthorizeSpotify
import pytest
class TestSpotifyAuthorize:
# TODO: Test these once we a have config.py
# storing pre-defined default credentials.
#
# We'll use these credentials to create
# a spotipy object via below tests
@pytest.mark.xfail
def test_generate_token(self):
raise NotImplementedError
@pytest.mark.xfail
def test_authorize(self):
raise NotImplementedError

16
spotdl/authorize/tests/test_authorize_base.py
View File

@@ -1,16 +0,0 @@
from spotdl.authorize import AuthorizeBase
import pytest
class TestAbstractBaseClass:
def test_error_abstract_base_class_authorizebase(self):
with pytest.raises(TypeError):
AuthorizeBase()
def test_inherit_abstract_base_class_authorizebase(self):
class AuthorizeKid(AuthorizeBase):
def authorize(self):
pass
AuthorizeKid()

15
spotdl/authorize/tests/test_authorize_exceptions.py
View File

@@ -1,15 +0,0 @@
from spotdl.authorize.exceptions import AuthorizationError
from spotdl.authorize.exceptions import SpotifyAuthorizationError
from spotdl.authorize.exceptions import YouTubeAuthorizationError
class TestEncoderNotFoundSubclass:
def test_authozation_error_subclass(self):
assert issubclass(AuthorizationError, Exception)
def test_spotify_authorization_error_subclass(self):
assert issubclass(SpotifyAuthorizationError, AuthorizationError)
def test_youtube_authorization_error_subclass(self):
assert issubclass(YouTubeAuthorizationError, AuthorizationError)

0
spotdl/command_line/__init__.py
View File

57
spotdl/command_line/__main__.py
View File

@@ -1,57 +0,0 @@
import logging
import coloredlogs
import sys
from spotdl.command_line.core import Spotdl
from spotdl.command_line.arguments import get_arguments
from spotdl.command_line.exceptions import ArgumentError
# hardcode loglevel for dependencies so that they do not spew generic
# log messages along with spotdl.
for module in ("urllib3", "spotipy", "pytube"):
logging.getLogger(module).setLevel(logging.CRITICAL)
coloredlogs.DEFAULT_FIELD_STYLES = {
"levelname": {"bold": True, "color": "yellow"},
"name": {"color": "blue"},
"lineno": {"color": "magenta"},
}
def set_logger(level):
if level == logging.DEBUG:
fmt = "%(levelname)s:%(name)s:%(lineno)d:\n%(message)s\n"
else:
fmt = "%(levelname)s: %(message)s"
logging.basicConfig(format=fmt, level=level)
logger = logging.getLogger(name=__name__)
coloredlogs.install(level=level, fmt=fmt, logger=logger)
return logger
def main():
try:
argument_handler = get_arguments()
except ArgumentError as e:
logger = set_logger(logging.INFO)
logger.info(e.args[0])
sys.exit(5)
logging_level = argument_handler.get_logging_level()
logger = set_logger(logging_level)
try:
spotdl = Spotdl(argument_handler)
except ArgumentError as e:
argument_handler.parser.error(e.args[0])
try:
spotdl.match_arguments()
except KeyboardInterrupt as e:
print("", file=sys.stderr)
logger.exception(e)
sys.exit(2)
if __name__ == "__main__":
main()

362
spotdl/command_line/arguments.py
View File

@@ -1,362 +0,0 @@
import appdirs
import argparse
import mimetypes
import os
import sys
import shutil
from spotdl.command_line.exceptions import ArgumentError
import spotdl.util
import spotdl.config
from collections.abc import Sequence
import logging
logger = logging.getLogger(__name__)
_LOG_LEVELS = {
"INFO": logging.INFO,
"WARNING": logging.WARNING,
"ERROR": logging.ERROR,
"DEBUG": logging.DEBUG,
}
if os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
saved_config = spotdl.config.read_config(spotdl.config.DEFAULT_CONFIG_FILE)
else:
saved_config = {"spotify-downloader": {}}
_CONFIG_BASE = spotdl.util.merge_copy(
spotdl.config.DEFAULT_CONFIGURATION,
saved_config,
)
def get_arguments(config_base=_CONFIG_BASE):
parser = argparse.ArgumentParser(
description="Download and convert tracks from Spotify, Youtube, etc.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
defaults = config_base["spotify-downloader"]
to_remove_config = "--remove-config" in sys.argv[1:]
if not to_remove_config and "download-only-metadata" in defaults:
raise ArgumentError(
"The default configuration file currently set is not suitable for spotdl>=2.0.0.\n"
"You need to remove your previous `config.yml` due to breaking changes\n"
"introduced in v2.0.0, new options being added, and old ones being removed\n"
"You may want to first backup your old configuration for reference. You can\n"
"then remove your current configuration by running:\n"
"```\n"
"$ spotdl --remove-config\n"
"```\n"
"spotdl will automatically generate a new configuration file on the next run.\n"
"You can then replace the appropriate fields in the newly generated\n"
"configuration file by referring to your old configuration file.\n\n"
"For the list of OTHER BREAKING CHANGES and release notes check out:\n"
"https://github.com/ritiek/spotify-downloader/releases/tag/v2.0.0"
)
possible_special_tags = (
"{track-name}",
"{artist}",
"{album}",
"{album-artist}",
"{genre}",
"{disc-number}",
"{duration}",
"{year}",
"{original-date}",
"{track-number}",
"{total-tracks}",
"{isrc}",
"{track-id}",
"{output-ext}",
)
# `--remove-config` does not require the any of the group arguments to be passed.
group = parser.add_mutually_exclusive_group(required=not to_remove_config)
group.add_argument(
"-s",
"--song",
nargs="+",
help="download track(s) by spotify link or name"
)
group.add_argument(
"-l",
"--list",
help="download tracks from a file (WARNING: this file will be modified!)"
)
group.add_argument(
"-p",
"--playlist",
help="load tracks from playlist URL into <playlist_name>.txt or "
"if `--write-to=<path/to/file.txt>` has been passed",
)
group.add_argument(
"-a",
"--album",
help="load tracks from album URL into <album_name>.txt or if "
"`--write-to=<path/to/file.txt>` has been passed"
)
group.add_argument(
"-aa",
"--all-albums",
help="load all tracks from artist URL into <artist_name>.txt "
"or if `--write-to=<path/to/file.txt>` has been passed"
)
group.add_argument(
"-u",
"--username",
help="load tracks from user's playlist into <playlist_name>.txt "
"or if `--write-to=<path/to/file.txt>` has been passed"
)
parser.add_argument(
"--write-m3u",
help="generate an .m3u playlist file with youtube links given "
"a text file containing tracks",
action="store_true",
)
parser.add_argument(
"-m",
"--manual",
default=defaults["manual"],
help="choose the track to download manually from a list of matching tracks",
action="store_true",
)
parser.add_argument(
"-nm",
"--no-metadata",
default=defaults["no_metadata"],
help="do not embed metadata in tracks",
action="store_true",
)
parser.add_argument(
"-ne",
"--no-encode",
default=defaults["no_encode"],
action="store_true",
help="do not encode media using FFmpeg",
)
parser.add_argument(
"--overwrite",
default=defaults["overwrite"],
choices={"prompt", "force", "skip"},
help="change the overwrite policy",
)
parser.add_argument(
"-q",
"--quality",
default=defaults["quality"],
choices={"best", "worst"},
help="preferred audio quality",
)
parser.add_argument(
"-i",
"--input-ext",
default=defaults["input_ext"],
choices={"automatic", "m4a", "opus"},
help="preferred input format",
)
parser.add_argument(
"-o",
"--output-ext",
default=defaults["output_ext"],
choices={"mp3", "m4a", "flac"},
help="preferred output format",
)
parser.add_argument(
"--write-to",
default=defaults["write_to"],
help="write tracks from Spotify playlist, album, etc. to this file",
)
parser.add_argument(
"-f",
"--output-file",
default=defaults["output_file"],
help="path where to write the downloaded track to, special tags "
"are to be surrounded by curly braces. Possible tags: {}".format(
possible_special_tags
)
)
parser.add_argument(
"--trim-silence",
default=defaults["trim_silence"],
help="remove silence from the start of the audio",
action="store_true",
)
parser.add_argument(
"-sf",
"--search-format",
default=defaults["search_format"],
help="search format to search for on YouTube, special tags "
"are to be surrounded by curly braces. Possible tags: {}".format(
possible_special_tags
)
)
parser.add_argument(
"-d",
"--dry-run",
default=defaults["dry_run"],
help="show only track title and YouTube URL, and then skip "
"to the next track (if any)",
action="store_true",
)
parser.add_argument(
"--processor",
default="synchronous",
choices={"synchronous", "threaded"},
# help='list downloading strategy: - "synchronous" downloads '
# 'tracks one-by-one. - "threaded" (highly experimental at the '
# 'moment! expect it to slash & burn) pre-fetches the next '
# 'track\'s metadata for more efficient downloading'
# XXX: Still very experimental to be exposed
help=argparse.SUPPRESS,
)
parser.add_argument(
"-ns",
"--no-spaces",
default=defaults["no_spaces"],
help="replace spaces in metadata values with underscores when "
"generating filenames",
action="store_true",
)
parser.add_argument(
"-sk",
"--skip-file",
default=defaults["skip_file"],
help="path to file containing tracks to skip",
)
parser.add_argument(
"-w",
"--write-successful-file",
default=defaults["write_successful_file"],
help="path to file to write successful tracks to",
)
parser.add_argument(
"--spotify-client-id",
default=defaults["spotify_client_id"],
help=argparse.SUPPRESS,
)
parser.add_argument(
"--spotify-client-secret",
default=defaults["spotify_client_secret"],
help=argparse.SUPPRESS,
)
parser.add_argument(
"-ll",
"--log-level",
default=defaults["log_level"],
choices=_LOG_LEVELS.keys(),
type=str.upper,
help="set log verbosity",
)
parser.add_argument(
"-c",
"--config",
default=spotdl.config.DEFAULT_CONFIG_FILE,
help="path to custom config.yml file"
)
parser.add_argument(
"--remove-config",
default=False,
action="store_true",
help="remove previously saved config"
)
parser.add_argument(
"-V",
"--version",
action="version",
version="%(prog)s {}".format(spotdl.__version__),
)
return ArgumentHandler(parser=parser)
class ArgumentHandler:
def __init__(self, args=None, parser=argparse.ArgumentParser(""), config_base=_CONFIG_BASE):
args_were_passed = args is not None
if not args_were_passed:
args = parser.parse_args().__dict__
config_file = args.get("config")
configured_args = args.copy()
if config_file and os.path.isfile(config_file):
config = spotdl.config.read_config(config_file)
parser.set_defaults(**config["spotify-downloader"])
configured_args = parser.parse_args().__dict__
if args_were_passed:
parser.set_defaults(**args)
configured_args = parser.parse_args().__dict__
defaults = config_base["spotify-downloader"]
args = spotdl.util.merge_copy(defaults, args)
self.parser = parser
self.args = args
self.configured_args = configured_args
def get_configured_args(self):
return self.configured_args
def get_logging_level(self):
return _LOG_LEVELS[self.configured_args["log_level"]]
def run_errands(self):
args = self.get_configured_args()
if (args.get("list")
and not mimetypes.MimeTypes().guess_type(args["list"])[0] == "text/plain"
):
raise ArgumentError(
"{0} is not of a valid argument to --list, argument must be plain text file.".format(
args["list"]
)
)
if args.get("write_m3u") and not args.get("list"):
raise ArgumentError("--write-m3u can only be used with --list.")
if args["write_to"] and not (
args.get("playlist") or args.get("album") or args.get("all_albums") or args.get("username") or args.get("write_m3u")
):
raise ArgumentError(
"--write-to can only be used with --playlist, --album, --all-albums, --username, or --write-m3u."
)
ffmpeg_exists = shutil.which("ffmpeg")
if not ffmpeg_exists:
logger.warn("FFmpeg was not found in PATH. Will not re-encode media to specified output format.")
args["no_encode"] = True
if args["no_encode"] and args["trim_silence"]:
logger.warn("--trim-silence can only be used when an encoder is set.")
if args["output_file"] == "-" and args["no_metadata"] is False:
logger.warn(
"Cannot write metadata when target is STDOUT. Pass "
"--no-metadata explicitly to hide this warning."
)
args["no_metadata"] = True
elif os.path.isdir(args["output_file"]):
adjusted_output_file = os.path.join(
args["output_file"],
self.parser.get_default("output_file")
)
logger.warn(
"Given output file is a directory. Will download tracks "
"in this directory with their filename as per the default "
"file format. Pass --output-file=\"{}\" to hide this "
"warning.".format(
adjusted_output_file
)
)
args["output_file"] = adjusted_output_file
return args

446
spotdl/command_line/core.py
View File

@@ -1,446 +0,0 @@
from spotdl.metadata.providers import ProviderSpotify
from spotdl.metadata.providers import ProviderYouTube
from spotdl.metadata.providers import YouTubeSearch
from spotdl.metadata.embedders import EmbedderDefault
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
import spotdl.metadata
from spotdl.lyrics.providers import LyricWikia
from spotdl.lyrics.providers import Genius
from spotdl.lyrics.exceptions import LyricsNotFoundError
from spotdl.encode.encoders import EncoderFFmpeg
from spotdl.authorize.services import AuthorizeSpotify
from spotdl.track import Track
import spotdl.util
import spotdl.config
from spotdl.command_line.exceptions import NoYouTubeVideoFoundError
from spotdl.command_line.exceptions import NoYouTubeVideoMatchError
from spotdl.metadata_search import MetadataSearch
from spotdl.helpers.spotify import SpotifyHelpers
import sys
import os
import urllib.request
import logging
logger = logging.getLogger(__name__)
class Spotdl:
def __init__(self, argument_handler):
self.arguments = argument_handler.run_errands()
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
del self
def match_arguments(self):
logger.debug("Received arguments:\n{}".format(self.arguments))
if self.arguments["remove_config"]:
self.remove_saved_config()
return
self.save_default_config()
AuthorizeSpotify(
client_id=self.arguments["spotify_client_id"],
client_secret=self.arguments["spotify_client_secret"]
)
spotify_tools = SpotifyHelpers()
if self.arguments["song"]:
for track in self.arguments["song"]:
if track == "-":
for line in sys.stdin:
self.download_track(
line.strip(),
)
else:
self.download_track(track)
elif self.arguments["list"]:
if self.arguments["write_m3u"]:
self.write_m3u(
self.arguments["list"],
self.arguments["write_to"]
)
else:
list_download = {
"synchronous": self.download_tracks_from_file,
# "threaded" : self.download_tracks_from_file_threaded,
}[self.arguments["processor"]]
list_download(
self.arguments["list"],
)
elif self.arguments["playlist"]:
playlist = spotify_tools.fetch_playlist(self.arguments["playlist"])
spotify_tools.write_playlist_tracks(playlist, self.arguments["write_to"])
elif self.arguments["album"]:
album = spotify_tools.fetch_album(self.arguments["album"])
spotify_tools.write_album_tracks(album, self.arguments["write_to"])
elif self.arguments["all_albums"]:
albums = spotify_tools.fetch_albums_from_artist(self.arguments["all_albums"])
spotify_tools.write_all_albums(albums, self.arguments["write_to"])
elif self.arguments["username"]:
playlist_url = spotify_tools.prompt_for_user_playlist(self.arguments["username"])
playlist = spotify_tools.fetch_playlist(playlist_url)
spotify_tools.write_playlist_tracks(playlist, self.arguments["write_to"])
def save_config(self, config_file=spotdl.config.DEFAULT_CONFIG_FILE, config=spotdl.config.DEFAULT_CONFIGURATION):
config_dir = os.path.dirname(config_file)
os.makedirs(config_dir, exist_ok=True)
logger.info('Writing configuration to "{0}":'.format(config_file))
spotdl.config.dump_config(config_file=config_file, config=spotdl.config.DEFAULT_CONFIGURATION)
config = spotdl.config.dump_config(config=spotdl.config.DEFAULT_CONFIGURATION["spotify-downloader"])
for line in config.split("\n"):
if line.strip():
logger.info(line.strip())
logger.info(
"Please note that command line arguments have higher priority "
"than their equivalents in the configuration file.\n"
)
def save_default_config(self):
if not os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
self.save_config()
def remove_saved_config(self, config_file=spotdl.config.DEFAULT_CONFIG_FILE):
if os.path.isfile(spotdl.config.DEFAULT_CONFIG_FILE):
logger.info('Removing "{}".'.format(spotdl.config.DEFAULT_CONFIG_FILE))
os.remove(spotdl.config.DEFAULT_CONFIG_FILE)
else:
logger.info('File does not exist: "{}".'.format(spotdl.config.DEFAULT_CONFIG_FILE))
def write_m3u(self, track_file, target_file=None):
with open(track_file, "r") as fin:
tracks = fin.read().splitlines()
logger.info(
"Checking and removing any duplicate tracks in {}.".format(track_file)
)
# Remove duplicates and empty elements
# Also strip whitespaces from elements (if any)
tracks = spotdl.util.remove_duplicates(
tracks,
condition=lambda x: x,
operation=str.strip
)
if target_file is None:
target_file = "{}.m3u".format(track_file.split(".")[0])
total_tracks = len(tracks)
logger.info("Generating {0} from {1} YouTube URLs.".format(target_file, total_tracks))
write_to_stdout = target_file == "-"
m3u_headers = "#EXTM3U\n\n"
if write_to_stdout:
sys.stdout.write(m3u_headers)
else:
with open(target_file, "w") as output_file:
output_file.write(m3u_headers)
videos = []
for n, track in enumerate(tracks, 1):
try:
search_metadata = MetadataSearch(
track,
lyrics=not self.arguments["no_metadata"],
yt_search_format=self.arguments["search_format"],
yt_manual=self.arguments["manual"]
)
video = search_metadata.best_on_youtube_search()
except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
logger.error(e.args[0])
else:
logger.info(
"Matched track {0}/{1} ({2})".format(
str(n).zfill(len(str(total_tracks))),
total_tracks,
video["url"],
)
)
m3u_key = "#EXTINF:{duration},{title}\n{youtube_url}\n".format(
duration=spotdl.util.get_sec(video["duration"]),
title=video["title"],
youtube_url=video["url"],
)
logger.debug(m3u_key.strip())
if write_to_stdout:
sys.stdout.write(m3u_key)
else:
with open(target_file, "a") as output_file:
output_file.write(m3u_key)
def download_track(self, track):
logger.info('Downloading "{}"'.format(track))
search_metadata = MetadataSearch(
track,
lyrics=not self.arguments["no_metadata"],
yt_search_format=self.arguments["search_format"],
yt_manual=self.arguments["manual"]
)
try:
if self.arguments["no_metadata"]:
metadata = search_metadata.on_youtube()
else:
metadata = search_metadata.on_youtube_and_spotify()
except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
logger.error(e.args[0])
else:
self.download_track_from_metadata(metadata)
def should_we_overwrite_existing_file(self, overwrite):
if overwrite == "force":
logger.info("Forcing overwrite on existing file.")
to_overwrite = True
elif overwrite == "prompt":
to_overwrite = input("Overwrite? (y/N): ").lower() == "y"
else:
logger.info("Not overwriting existing file.")
to_overwrite = False
return to_overwrite
def generate_temp_filename(self, filename, for_stdout=False):
if for_stdout:
return filename
return "{filename}.temp".format(filename=filename)
def output_filename_filter(self, allow_spaces):
replace_spaces_with_underscores = not allow_spaces
if replace_spaces_with_underscores:
return lambda s: s.replace(" ", "_")
return lambda s: s
def download_track_from_metadata(self, metadata):
track = Track(metadata, cache_albumart=(not self.arguments["no_metadata"]))
stream = metadata["streams"].get(
quality=self.arguments["quality"],
preftype=self.arguments["input_ext"],
)
if stream is None:
logger.error('No matching streams found for given input format: "{}".'.format(
self.arguments["input_ext"]
))
return
if self.arguments["no_encode"]:
output_extension = stream["encoding"]
else:
output_extension = self.arguments["output_ext"]
filename = spotdl.metadata.format_string(
self.arguments["output_file"],
metadata,
output_extension=output_extension,
sanitizer=lambda s: spotdl.util.sanitize(
s, spaces_to_underscores=self.arguments["no_spaces"]
)
)
download_to_stdout = filename == "-"
temp_filename = self.generate_temp_filename(filename, for_stdout=download_to_stdout)
to_skip_download = self.arguments["dry_run"]
if os.path.isfile(filename):
logger.info('A file with name "{filename}" already exists.'.format(
filename=filename
))
to_skip_download = to_skip_download \
or not self.should_we_overwrite_existing_file(self.arguments["overwrite"])
if to_skip_download:
logger.debug("Skip track download.")
return
if not self.arguments["no_metadata"]:
metadata["lyrics"].start()
filter_space_chars = self.output_filename_filter(not self.arguments["no_spaces"])
directory = os.path.dirname(
spotdl.metadata.format_string(
self.arguments["output_file"],
metadata,
output_extension=output_extension,
sanitizer=filter_space_chars
)
)
os.makedirs(directory or ".", exist_ok=True)
logger.info('Downloading to "{filename}"'.format(filename=filename))
if self.arguments["no_encode"]:
track.download(stream, temp_filename)
else:
encoder = EncoderFFmpeg()
if self.arguments["trim_silence"]:
encoder.set_trim_silence()
track.download_while_re_encoding(
stream,
temp_filename,
target_encoding=output_extension,
encoder=encoder,
)
if not self.arguments["no_metadata"]:
track.metadata["lyrics"] = track.metadata["lyrics"].join()
self.apply_metadata(track, temp_filename, output_extension)
if not download_to_stdout:
logger.debug("Renaming {temp_filename} to {filename}.".format(
temp_filename=temp_filename, filename=filename
))
os.rename(temp_filename, filename)
return filename
def apply_metadata(self, track, filename, encoding):
logger.info("Applying metadata")
try:
track.apply_metadata(filename, encoding=encoding)
except TypeError:
logger.warning("Cannot apply metadata on provided output format.")
def strip_and_filter_duplicates(self, tracks):
filtered_tracks = spotdl.util.remove_duplicates(
tracks,
condition=lambda x: x,
operation=str.strip
)
return filtered_tracks
def filter_against_skip_file(self, items, skip_file):
skip_items = spotdl.util.readlines_from_nonbinary_file(skip_file)
filtered_skip_items = self.strip_and_filter_duplicates(skip_items)
filtered_items = [item for item in items if not item in filtered_skip_items]
return filtered_items
def download_tracks_from_file(self, path):
logger.info(
"Checking and removing any duplicate tracks in {}.".format(path)
)
tracks = spotdl.util.readlines_from_nonbinary_file(path)
tracks = self.strip_and_filter_duplicates(tracks)
if self.arguments["skip_file"]:
len_tracks_before = len(tracks)
tracks = self.filter_against_skip_file(tracks, self.arguments["skip_file"])
logger.info("Skipping {} tracks due to matches in skip file.".format(
len_tracks_before - len(tracks))
)
# Overwrite file
spotdl.util.writelines_to_nonbinary_file(path, tracks)
logger.info(
"Downloading {n} tracks.\n".format(n=len(tracks))
)
for position, track in enumerate(tracks, 1):
search_metadata = MetadataSearch(
track,
lyrics=True,
yt_search_format=self.arguments["search_format"],
yt_manual=self.arguments["manual"]
)
try:
log_track_query = '{position}. Downloading "{track}"'.format(
position=position,
track=track
)
logger.info(log_track_query)
metadata = search_metadata.on_youtube_and_spotify()
self.download_track_from_metadata(metadata)
except (urllib.request.URLError, TypeError, IOError) as e:
logger.exception(e.args[0])
logger.warning(
"Failed to download current track due to possible network issue. "
"Will retry after other songs."
)
tracks.append(track)
except (NoYouTubeVideoFoundError, NoYouTubeVideoMatchError) as e:
logger.error("{err}".format(err=e.args[0]))
else:
if self.arguments["write_successful_file"]:
with open(self.arguments["write_successful_file"], "a") as fout:
fout.write("{}\n".format(track))
finally:
spotdl.util.writelines_to_nonbinary_file(path, tracks[position:])
print("", file=sys.stderr)
"""
def download_tracks_from_file_threaded(self, path):
# FIXME: Can we make this function cleaner?
logger.info(
"Checking and removing any duplicate tracks in {}.\n".format(path)
)
with open(path, "r") as fin:
# Read tracks into a list and remove any duplicates
tracks = fin.read().splitlines()
# Remove duplicates and empty elements
# Also strip whitespaces from elements (if any)
spotdl.util.remove_duplicates(
tracks,
condition=lambda x: x,
operation=str.strip
)
# Overwrite file
with open(path, "w") as fout:
fout.writelines(tracks)
tracks_count = len(tracks)
current_iteration = 1
next_track = tracks.pop(0)
metadata = {
"current_track": None,
"next_track": spotdl.util.ThreadWithReturnValue(
target=search_metadata,
args=(next_track, self.arguments["search_format"])
)
}
metadata["next_track"].start()
while tracks_count > 0:
metadata["current_track"] = metadata["next_track"].join()
metadata["next_track"] = None
try:
print(tracks_count, file=sys.stderr)
print(tracks, file=sys.stderr)
if tracks_count > 1:
current_track = next_track
next_track = tracks.pop(0)
metadata["next_track"] = spotdl.util.ThreadWithReturnValue(
target=search_metadata,
args=(next_track, self.arguments["search_format"])
)
metadata["next_track"].start()
log_track_query = str(current_iteration) + ". {artist} - {track-name}"
logger.info(log_track_query)
if metadata["current_track"] is None:
logger.warning("Something went wrong. Will retry after downloading remaining tracks.")
pass
print(metadata["current_track"]["name"], file=sys.stderr)
# self.download_track_from_metadata(metadata["current_track"])
except (urllib.request.URLError, TypeError, IOError) as e:
print("", file=sys.stderr)
logger.exception(e.args[0])
logger.warning("Failed. Will retry after other songs\n")
tracks.append(current_track)
else:
tracks_count -= 1
if self.arguments["write_sucessful_file"]:
with open(self.arguments["write_sucessful_file"], "a") as fout:
fout.write(current_track)
finally:
current_iteration += 1
with open(path, "w") as fout:
fout.writelines(tracks)
"""

20
spotdl/command_line/exceptions.py
View File

@@ -1,20 +0,0 @@
class NoYouTubeVideoFoundError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class NoYouTubeVideoMatchError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class ArgumentError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)

90
spotdl/command_line/tests/test_arguments.py
View File

@@ -1,90 +0,0 @@
import spotdl.command_line.arguments
from spotdl.command_line.exceptions import ArgumentError
import logging
import sys
import pytest
def test_logging_levels():
expect_logging_levels = {
"INFO": logging.INFO,
"WARNING": logging.WARNING,
"DEBUG": logging.DEBUG,
"ERROR": logging.ERROR,
}
assert spotdl.command_line.arguments._LOG_LEVELS == expect_logging_levels
class TestBadArguments:
def test_error_m3u_without_list(self):
previous_argv = sys.argv
sys.argv[1:] = ["-s", "cool song", "--write-m3u"]
argument_handler = spotdl.command_line.arguments.get_arguments()
with pytest.raises(ArgumentError):
argument_handler.run_errands()
sys.argv[1:] = previous_argv[1:]
def test_write_to_error(self):
previous_argv = sys.argv
sys.argv[1:] = ["-s", "sekai all i had", "--write-to", "output.txt"]
argument_handler = spotdl.command_line.arguments.get_arguments()
with pytest.raises(ArgumentError):
argument_handler.run_errands()
sys.argv[1:] = previous_argv[1:]
class TestArguments:
@pytest.mark.xfail
def test_general_arguments(self):
arguments = spotdl.command_line.arguments.get_arguments(argv=("-t", "elena coats - one last song"))
arguments = arguments.__dict__
assert isinstance(arguments["spotify_client_id"], str)
assert isinstance(arguments["spotify_client_secret"], str)
arguments["spotify_client_id"] = None
arguments["spotify_client_secret"] = None
expect_arguments = {
"song": ["elena coats - one last song"],
"song": None,
"list": None,
"playlist": None,
"album": None,
"all_albums": None,
"username": None,
"write_m3u": False,
"manual": False,
"no_remove_original": False,
"no_metadata": False,
"no_fallback_metadata": False,
"directory": "/home/ritiek/Music",
"overwrite": "prompt",
"input_ext": ".m4a",
"output_ext": ".mp3",
"write_to": None,
"file_format": "{artist} - {track_name}",
"trim_silence": False,
"search_format": "{artist} - {track_name} lyrics",
"download_only_metadata": False,
"dry_run": False,
"music_videos_only": False,
"no_spaces": False,
"log_level": 20,
"skip": None,
"write_successful": None,
"spotify_client_id": None,
"spotify_client_secret": None,
"config": None
}
assert arguments == expect_arguments
def test_grouped_arguments(self):
previous_argv = sys.argv
sys.argv[1:] = []
with pytest.raises(SystemExit):
argument_handler = spotdl.command_line.arguments.get_arguments()
sys.argv[1:] = previous_argv[1:]

52
spotdl/config.py
View File

@@ -1,52 +0,0 @@
import appdirs
import yaml
import os
import spotdl.util
import logging
logger = logging.getLogger(__name__)
DEFAULT_CONFIGURATION = {
"spotify-downloader": {
"manual": False,
"no_metadata": False,
"no_encode": False,
"overwrite": "prompt",
"quality": "best",
"input_ext": "automatic",
"output_ext": "mp3",
"write_to": None,
"trim_silence": False,
"search_format": "{artist} - {track-name} lyrics",
"dry_run": False,
"no_spaces": False,
# "processor": "synchronous",
"output_file": "{artist} - {track-name}.{output-ext}",
"skip_file": None,
"write_successful_file": None,
"spotify_client_id": "4fe3fecfe5334023a1472516cc99d805",
"spotify_client_secret": "0f02b7c483c04257984695007a4a8d5c",
"log_level": "INFO",
}
}
DEFAULT_CONFIG_FILE = os.path.join(
appdirs.user_config_dir(),
"spotdl",
"config.yml"
)
def read_config(config_file):
with open(config_file, "r") as ymlfile:
config = yaml.safe_load(ymlfile)
return config
def dump_config(config_file=None, config=DEFAULT_CONFIGURATION):
if config_file is None:
config = yaml.dump(config, default_flow_style=False)
return config
with open(config_file, "w") as ymlfile:
yaml.dump(config, ymlfile, default_flow_style=False)

1
spotdl/encode/__init__.py
View File

@@ -1 +0,0 @@
from spotdl.encode.encode_base import EncoderBase

119
spotdl/encode/encode_base.py
View File

@@ -1,119 +0,0 @@
import shutil
import os
from abc import ABC
from abc import abstractmethod
from spotdl.encode.exceptions import EncoderNotFoundError
"""
NOTE ON ENCODERS
================
* FFmeg encoders sorted in descending order based
on the quality of audio produced:
libopus > libvorbis >= libfdk_aac > aac > libmp3lame
* libfdk_aac encoder, due to copyrights needs to be compiled
by end user on MacOS brew install ffmpeg --with-fdk-aac
will do just that. Other OS? See:
https://trac.ffmpeg.org/wiki/Encode/AAC
"""
_TARGET_FORMATS_FROM_ENCODING = {
"m4a": "mp4",
"mp3": "mp3",
"opus": "opus",
"flac": "flac"
}
class EncoderBase(ABC):
"""
Defined encoders must inherit from this abstract base class
and implement their own functionality for the below defined
methods.
"""
@abstractmethod
def __init__(self, encoder_path, loglevel, additional_arguments=[]):
"""
This method must make sure whether specified encoder
is available under PATH.
"""
if shutil.which(encoder_path) is None:
raise EncoderNotFoundError(
"{} executable does not exist or was not found in PATH.".format(
encoder_path
)
)
self.encoder_path = encoder_path
self._loglevel = loglevel
self._additional_arguments = additional_arguments
self._target_formats_from_encoding = _TARGET_FORMATS_FROM_ENCODING
def set_argument(self, argument):
"""
This method must be used to set any custom functionality
for the encoder by passing arguments to it.
"""
self._additional_arguments += argument.split()
def get_encoding(self, path):
"""
This method must determine the encoding for a local
audio file. Such as "mp3", "wav", "m4a", etc.
"""
_, extension = os.path.splitext(path)
# Ignore the initial dot from file extension
return extension[1:]
@abstractmethod
def set_debuglog(self):
"""
This method must enable verbose logging in the defined
encoder.
"""
pass
@abstractmethod
def _generate_encode_command(self, input_path, target_path):
"""
This method must the complete command for that would be
used to invoke the encoder and perform the encoding.
"""
pass
@abstractmethod
def _generate_encoding_arguments(self, input_encoding, target_encoding):
"""
This method must return the core arguments for the defined
encoder such as defining the sample rate, audio bitrate,
etc.
"""
pass
@abstractmethod
def re_encode(self, input_path, target_path):
"""
This method must invoke the encoder to encode a given input
file to a specified output file.
"""
pass
def target_format_from_encoding(self, encoding):
"""
This method generates the target stream format from given
input encoding.
"""
target_format = self._target_formats_from_encoding[encoding]
return target_format
def re_encode_from_stdin(self, input_encoding, target_path):
"""
This method must invoke the encoder to encode stdin to a
specified output file.
"""
raise NotImplementedError

1
spotdl/encode/encoders/__init__.py
View File

@@ -1 +0,0 @@
from spotdl.encode.encoders.ffmpeg import EncoderFFmpeg

112
spotdl/encode/encoders/ffmpeg.py
View File

@@ -1,112 +0,0 @@
import subprocess
import os
from spotdl.encode import EncoderBase
from spotdl.encode.exceptions import EncoderNotFoundError
from spotdl.encode.exceptions import FFmpegNotFoundError
import logging
logger = logging.getLogger(__name__)
# Key: from format
# Subkey: to format
RULES = {
"m4a": {
"mp3": "-codec:v copy -codec:a libmp3lame -ar 48000",
"opus": "-codec:a libopus -vbr on",
"m4a": "-acodec copy",
"flac": "-codec:a flac -ar 48000",
},
"opus": {
"mp3": "-codec:a libmp3lame -ar 48000",
"m4a": "-cutoff 20000 -codec:a aac -ar 48000",
"flac": "-codec:a flac -ar 48000",
},
}
class EncoderFFmpeg(EncoderBase):
def __init__(self, encoder_path="ffmpeg"):
_loglevel = "-hide_banner -nostats -v panic"
_additional_arguments = ["-b:a", "192k", "-vn"]
try:
super().__init__(encoder_path, _loglevel, _additional_arguments)
except EncoderNotFoundError as e:
raise FFmpegNotFoundError(e.args[0])
self._rules = RULES
def set_trim_silence(self):
self.set_argument("-af silenceremove=start_periods=1")
def get_encoding(self, path):
return super().get_encoding(path)
def _generate_encoding_arguments(self, input_encoding, target_encoding):
initial_arguments = self._rules.get(input_encoding)
if initial_arguments is None:
raise TypeError(
'The input format ("{}") is not supported.'.format(
input_encoding,
))
arguments = initial_arguments.get(target_encoding)
if arguments is None:
raise TypeError(
'The output format ("{}") is not supported.'.format(
target_encoding,
))
return arguments
def set_debuglog(self):
self._loglevel = "-loglevel debug"
def _generate_encode_command(self, input_path, target_file,
input_encoding=None, target_encoding=None):
if input_encoding is None:
input_encoding = self.get_encoding(input_path)
if target_encoding is None:
target_encoding = self.get_encoding(target_file)
arguments = self._generate_encoding_arguments(
input_encoding,
target_encoding
)
command = [self.encoder_path] \
+ ["-y", "-nostdin"] \
+ self._loglevel.split() \
+ ["-i", input_path] \
+ arguments.split() \
+ self._additional_arguments \
+ ["-f", self.target_format_from_encoding(target_encoding)] \
+ [target_file]
return command
def re_encode(self, input_path, target_file, target_encoding=None, delete_original=False):
encode_command = self._generate_encode_command(
input_path,
target_file,
target_encoding=target_encoding
)
logger.debug("Calling FFmpeg with:\n{command}".format(
command=encode_command,
))
process = subprocess.Popen(encode_command)
process.wait()
encode_successful = process.returncode == 0
if encode_successful and delete_original:
os.remove(input_path)
return process
def re_encode_from_stdin(self, input_encoding, target_file, target_encoding=None):
encode_command = self._generate_encode_command(
"-",
target_file,
input_encoding=input_encoding,
target_encoding=target_encoding,
)
logger.debug("Calling FFmpeg with:\n{command}".format(
command=encode_command,
))
process = subprocess.Popen(encode_command, stdin=subprocess.PIPE)
return process

0
spotdl/encode/encoders/tests/__init__.py
View File

211
spotdl/encode/encoders/tests/test_ffmpeg.py
View File

@@ -1,211 +0,0 @@
from spotdl.encode import EncoderBase
from spotdl.encode.exceptions import FFmpegNotFoundError
from spotdl.encode.encoders import EncoderFFmpeg
import pytest
class TestEncoderFFmpeg:
def test_subclass(self):
assert issubclass(EncoderFFmpeg, EncoderBase)
def test_ffmpeg_not_found_error(self):
with pytest.raises(FFmpegNotFoundError):
EncoderFFmpeg(encoder_path="/a/nonexistent/path")
class TestEncodingDefaults:
def m4a_to_mp3_encoder(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:v', 'copy',
'-codec:a', 'libmp3lame',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-f', 'mp3',
target_path
]
return command
def m4a_to_opus_encoder(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:a', 'libopus',
'-vbr', 'on',
'-b:a', '192k',
'-vn',
'-f', 'opus',
target_path
]
return command
def m4a_to_m4a_encoder(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-acodec', 'copy',
'-b:a', '192k',
'-vn',
'-f', 'mp4',
target_path
]
return command
def m4a_to_flac_encoder(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:a', 'flac',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-f', 'flac',
target_path
]
return command
@pytest.mark.parametrize("files, expected_command", [
(("test.m4a", "test.mp3"), m4a_to_mp3_encoder("test.m4a", "test.mp3")),
(("abc.m4a", "cba.opus"), m4a_to_opus_encoder("abc.m4a", "cba.opus")),
(("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder("bla bla.m4a", "ble ble.m4a")),
(("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder("😛.m4a", "• tongue.flac")),
])
def test_generate_encode_command(self, files, expected_command):
encoder = EncoderFFmpeg()
assert encoder._generate_encode_command(*files) == expected_command
class TestEncodingInDebugMode:
def m4a_to_mp3_encoder_with_debug(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
'-i', input_path,
'-codec:v', 'copy',
'-codec:a', 'libmp3lame',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-f', 'mp3',
target_path
]
return command
def m4a_to_opus_encoder_with_debug(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
'-i', input_path,
'-codec:a', 'libopus',
'-vbr', 'on',
'-b:a', '192k',
'-vn',
'-f', 'opus',
target_path
]
return command
def m4a_to_m4a_encoder_with_debug(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
'-i', input_path,
'-acodec', 'copy',
'-b:a', '192k',
'-vn',
'-f', 'mp4',
target_path
]
return command
def m4a_to_flac_encoder_with_debug(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-loglevel', 'debug',
'-i', input_path,
'-codec:a', 'flac',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-f', 'flac',
target_path
]
return command
@pytest.mark.parametrize("files, expected_command", [
(("test.m4a", "test.mp3"), m4a_to_mp3_encoder_with_debug("test.m4a", "test.mp3")),
(("abc.m4a", "cba.opus"), m4a_to_opus_encoder_with_debug("abc.m4a", "cba.opus")),
(("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder_with_debug("bla bla.m4a", "ble ble.m4a")),
(("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder_with_debug("😛.m4a", "• tongue.flac")),
])
def test_generate_encode_command_with_debug(self, files, expected_command):
encoder = EncoderFFmpeg()
encoder.set_debuglog()
assert encoder._generate_encode_command(*files) == expected_command
class TestEncodingAndTrimSilence:
def m4a_to_mp3_encoder_and_trim_silence(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:v', 'copy',
'-codec:a', 'libmp3lame',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-af', 'silenceremove=start_periods=1',
'-f', 'mp3',
target_path
]
return command
def m4a_to_opus_encoder_and_trim_silence(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:a', 'libopus',
'-vbr', 'on',
'-b:a', '192k',
'-vn',
'-af', 'silenceremove=start_periods=1',
'-f', 'opus',
target_path
]
return command
def m4a_to_m4a_encoder_and_trim_silence(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-acodec', 'copy',
'-b:a', '192k',
'-vn',
'-af', 'silenceremove=start_periods=1',
'-f', 'mp4',
target_path
]
return command
def m4a_to_flac_encoder_and_trim_silence(input_path, target_path):
command = [
'ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic',
'-i', input_path,
'-codec:a', 'flac',
'-ar', '48000',
'-b:a', '192k',
'-vn',
'-af', 'silenceremove=start_periods=1',
'-f', 'flac',
target_path
]
return command
@pytest.mark.parametrize("files, expected_command", [
(("test.m4a", "test.mp3"), m4a_to_mp3_encoder_and_trim_silence("test.m4a", "test.mp3")),
(("abc.m4a", "cba.opus"), m4a_to_opus_encoder_and_trim_silence("abc.m4a", "cba.opus")),
(("bla bla.m4a", "ble ble.m4a"), m4a_to_m4a_encoder_and_trim_silence("bla bla.m4a", "ble ble.m4a")),
(("😛.m4a", "• tongue.flac"), m4a_to_flac_encoder_and_trim_silence("😛.m4a", "• tongue.flac")),
])
def test_generate_encode_command_and_trim_silence(self, files, expected_command):
encoder = EncoderFFmpeg()
encoder.set_trim_silence()
assert encoder._generate_encode_command(*files) == expected_command

13
spotdl/encode/exceptions.py
View File

@@ -1,13 +0,0 @@
class EncoderNotFoundError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class FFmpegNotFoundError(EncoderNotFoundError):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)

0
spotdl/encode/tests/__init__.py
View File

97
spotdl/encode/tests/test_encode_base.py
View File

@@ -1,97 +0,0 @@
from spotdl.encode import EncoderBase
from spotdl.encode.exceptions import EncoderNotFoundError
import pytest
class TestAbstractBaseClass:
def test_error_abstract_base_class_encoderbase(self):
encoder_path = "ffmpeg"
_loglevel = "-hide_banner -nostats -v panic"
_additional_arguments = ["-b:a", "192k", "-vn"]
with pytest.raises(TypeError):
# This abstract base class must be inherited from
# for instantiation
EncoderBase(encoder_path, _loglevel, _additional_arguments)
def test_inherit_abstract_base_class_encoderbase(self):
class EncoderKid(EncoderBase):
def __init__(self, encoder_path, _loglevel, _additional_arguments):
super().__init__(encoder_path, _loglevel, _additional_arguments)
def _generate_encode_command(self):
pass
def _generate_encoding_arguments(self):
pass
def re_encode(self):
pass
def set_debuglog(self):
pass
encoder_path = "ffmpeg"
_loglevel = "-hide_banner -nostats -v panic"
_additional_arguments = ["-b:a", "192k", "-vn"]
EncoderKid(encoder_path, _loglevel, _additional_arguments)
class TestMethods:
class EncoderKid(EncoderBase):
def __init__(self, encoder_path, _loglevel, _additional_arguments):
super().__init__(encoder_path, _loglevel, _additional_arguments)
def _generate_encode_command(self, input_file, target_file):
pass
def _generate_encoding_arguments(self, input_encoding, target_encoding):
pass
def re_encode(self, input_encoding, target_encoding):
pass
def set_debuglog(self):
pass
@pytest.fixture(scope="module")
def encoderkid(self):
encoder_path = "ffmpeg"
_loglevel = "-hide_banner -nostats -v panic"
_additional_arguments = []
encoderkid = self.EncoderKid(encoder_path, _loglevel, _additional_arguments)
return encoderkid
def test_set_argument(self, encoderkid):
encoderkid.set_argument("-parameter argument")
assert encoderkid._additional_arguments == [
"-parameter",
"argument",
]
@pytest.mark.parametrize("filename, encoding", [
("example.m4a", "m4a"),
("exampley.mp3", "mp3"),
("test 123.opus", "opus"),
("flakey.flac", "flac"),
])
def test_get_encoding(self, encoderkid, filename, encoding):
assert encoderkid.get_encoding(filename) == encoding
def test_encoder_not_found_error(self):
with pytest.raises(EncoderNotFoundError):
self.EncoderKid("/a/nonexistent/path", "0", [])
@pytest.mark.parametrize("encoding, target_format", [
("m4a", "mp4"),
("mp3", "mp3"),
("opus", "opus"),
("flac", "flac"),
])
def test_target_format_from_encoding(self, encoderkid, encoding, target_format):
assert encoderkid.target_format_from_encoding(encoding) == target_format

11
spotdl/encode/tests/test_encode_exceptions.py
View File

@@ -1,11 +0,0 @@
from spotdl.encode.exceptions import EncoderNotFoundError
from spotdl.encode.exceptions import FFmpegNotFoundError
class TestEncoderNotFoundSubclass:
def test_encoder_not_found_subclass(self):
assert issubclass(FFmpegNotFoundError, Exception)
def test_ffmpeg_not_found_subclass(self):
assert issubclass(FFmpegNotFoundError, EncoderNotFoundError)

0
spotdl/helpers/__init__.py
View File

165
spotdl/helpers/spotify.py
View File

@@ -1,165 +0,0 @@
from spotdl.authorize.services import AuthorizeSpotify
import spotdl.util
import sys
import logging
logger = logging.getLogger(__name__)
try:
from slugify import SLUG_OK, slugify
except ImportError:
logger.error("Oops! `unicode-slugify` was not found.")
logger.info("Please remove any other slugify library and install `unicode-slugify`.")
raise
class SpotifyHelpers:
def __init__(self, spotify=None):
if spotify is None:
spotify = AuthorizeSpotify()
self.spotify = spotify
def prompt_for_user_playlist(self, username):
""" Write user playlists to target_file """
playlists = self.fetch_user_playlist_urls(username)
for i, playlist in enumerate(playlists, 1):
playlist_details = "{0}. {1:<30} ({2} tracks)".format(
i, playlist["name"], playlist["tracks"]["total"]
)
print(playlist_details, file=sys.stderr)
print("", file=sys.stderr)
playlist = spotdl.util.prompt_user_for_selection(playlists)
return playlist["external_urls"]["spotify"]
def fetch_user_playlist_urls(self, username):
""" Fetch user playlists when using the -u option. """
logger.debug('Fetching playlists for "{username}".'.format(username=username))
playlists = self.spotify.user_playlists(username)
collected_playlists = []
check = 1
while True:
for playlist in playlists["items"]:
# in rare cases, playlists may not be found, so playlists['next']
# is None. Skip these. Also see Issue #91.
if playlist["name"] is not None:
collected_playlists.append(playlist)
check += 1
if playlists["next"]:
playlists = self.spotify.next(playlists)
else:
break
return collected_playlists
def fetch_playlist(self, playlist_url):
logger.debug('Fetching playlist "{playlist}".'.format(playlist=playlist_url))
try:
results = self.spotify.playlist(playlist_url, fields="tracks,next,name")
except spotipy.client.SpotifyException:
logger.exception(
"Unable to find playlist. Make sure the playlist is set "
"to publicly visible and then try again."
)
return results
def write_playlist_tracks(self, playlist, target_file=None):
tracks = playlist["tracks"]
if not target_file:
target_file = u"{0}.txt".format(slugify(playlist["name"], ok="-_()[]{}"))
return self.write_tracks(tracks, target_file)
def fetch_album(self, album_uri):
logger.debug('Fetching album "{album}".'.format(album=album_uri))
album = self.spotify.album(album_uri)
return album
def write_album_tracks(self, album, target_file=None):
tracks = self.spotify.album_tracks(album["id"])
if not target_file:
target_file = u"{0}.txt".format(slugify(album["name"], ok="-_()[]{}"))
return self.write_tracks(tracks, target_file)
def fetch_albums_from_artist(self, artist_uri, album_type=None):
"""
This function returns all the albums from a give artist_uri using the US
market
:param artist_uri - spotify artist uri
:param album_type - the type of album to fetch (ex: single) the default is
all albums
:param return - the album from the artist
"""
logger.debug('Fetching all albums for "{artist}".'.format(artist=artist_uri))
# fetching artist's albums limitting the results to the US to avoid duplicate
# albums from multiple markets
results = self.spotify.artist_albums(artist_uri, album_type=album_type, country="US")
albums = results["items"]
# indexing all pages of results
while results["next"]:
results = self.spotify.next(results)
albums.extend(results["items"])
return albums
def write_all_albums(self, albums, target_file=None):
"""
This function gets all albums from an artist and writes it to a file in the
current working directory called [ARTIST].txt, where [ARTIST] is the artist
of the album
:param artist_uri - spotify artist uri
:param target_file - file to write albums to
"""
# if no file if given, the default save file is in the current working
# directory with the name of the artist
if target_file is None:
target_file = albums[0]["artists"][0]["name"] + ".txt"
for album in albums:
logger.info('Fetching album "{album}".'.format(album=album["name"]))
self.write_album_tracks(album, target_file=target_file)
def write_tracks(self, tracks, target_file):
def writer(tracks, file_io):
track_urls = []
while True:
for item in tracks["items"]:
if "track" in item:
track = item["track"]
else:
track = item
try:
track_url = track["external_urls"]["spotify"]
file_io.write(track_url + "\n")
track_urls.append(track_url)
except KeyError:
# FIXME: Write "{artist} - {name}" instead of Spotify URI for
# "local only" tracks.
logger.warning(
'Skipping track "{0}" by "{1}" (local only?)'.format(
track["name"], track["artists"][0]["name"]
)
)
# 1 page = 50 results
# check if there are more pages
if tracks["next"]:
tracks = self.spotify.next(tracks)
else:
break
return track_urls
logger.info(u"Writing {0} tracks to {1}.".format(tracks["total"], target_file))
write_to_stdout = target_file == "-"
if write_to_stdout:
file_out = sys.stdout
track_urls = writer(tracks, file_out)
else:
with open(target_file, "a") as file_out:
track_urls = writer(tracks, file_out)
return track_urls

1
spotdl/lyrics/__init__.py
View File

@@ -1 +0,0 @@
from spotdl.lyrics.lyric_base import LyricBase

5
spotdl/lyrics/exceptions.py
View File

@@ -1,5 +0,0 @@
class LyricsNotFoundError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)

36
spotdl/lyrics/lyric_base.py
View File

@@ -1,36 +0,0 @@
import lyricwikia
from abc import ABC
from abc import abstractmethod
class LyricBase(ABC):
"""
Defined lyric providers must inherit from this abstract base
class and implement their own functionality for the below
defined methods.
"""
@abstractmethod
def from_url(self, url, linesep="\n", timeout=None):
"""
This method must return the lyrics string for the
given track.
"""
pass
@abstractmethod
def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
"""
This method must return the lyrics string for the
given track.
"""
pass
@abstractmethod
def from_query(self, query, linesep="\n", timeout=None):
"""
This method must return the lyrics string for the
given track.
"""
pass

4
spotdl/lyrics/providers/__init__.py
View File

@@ -1,4 +0,0 @@
from spotdl.lyrics.providers.genius import Genius
from spotdl.lyrics.providers.lyricwikia_wrapper import LyricWikia
LyricClasses = (Genius, LyricWikia)

151
spotdl/lyrics/providers/genius.py
View File

@@ -1,151 +0,0 @@
from bs4 import BeautifulSoup
import urllib.request
import json
from spotdl.lyrics.lyric_base import LyricBase
from spotdl.lyrics.exceptions import LyricsNotFoundError
import logging
logger = logging.getLogger(__name__)
BASE_URL = "https://genius.com"
BASE_SEARCH_URL = BASE_URL + "/api/search/multi?per_page=1&q="
# FIXME: Make Genius a metadata provider instead of lyric provider
# Since, Genius parses additional metadata too (such as track
# name, artist name, albumart url). For example, fetch this URL:
# https://genius.com/api/search/multi?per_page=1&q=artist+trackname
class Genius(LyricBase):
def __init__(self):
self.base_url = BASE_URL
self.base_search_url = BASE_SEARCH_URL
def guess_lyric_url_from_artist_and_track(self, artist, track):
"""
Returns the possible lyric URL for the track available on
Genius. This may not always be a valid URL.
"""
query = "/{} {} lyrics".format(artist, track)
query = query.replace(" ", "-")
encoded_query = urllib.request.quote(query)
lyric_url = self.base_url + encoded_query
return lyric_url
def _fetch_url_page(self, url, timeout=None):
"""
Makes a GET request to the given lyrics page URL and returns
the HTML content in the case of a valid response.
"""
request = urllib.request.Request(url)
request.add_header("User-Agent", "urllib")
try:
response = urllib.request.urlopen(request, timeout=timeout)
except urllib.request.HTTPError:
raise LyricsNotFoundError(
"Could not find Genius lyrics at URL: {}".format(url)
)
else:
return response.read()
def _get_lyrics_text(self, paragraph):
"""
Extracts and returns the lyric content from the provided HTML.
"""
if paragraph:
return paragraph.get_text()
else:
raise LyricsNotFoundError(
"The lyrics for this track are yet to be released on Genius."
)
def _fetch_search_page(self, url, timeout=None):
"""
Returns search results from a given URL in JSON.
"""
request = urllib.request.Request(url)
request.add_header("User-Agent", "urllib")
response = urllib.request.urlopen(request, timeout=timeout)
metadata = json.loads(response.read())
if len(metadata["response"]["sections"][0]["hits"]) == 0:
raise LyricsNotFoundError(
"Genius returned no lyric results for the search URL: {}".format(url)
)
return metadata
def best_matching_lyric_url_from_query(self, query):
"""
Returns the best matching track's URL from a given query.
"""
encoded_query = urllib.request.quote(query.replace(" ", "+"))
search_url = self.base_search_url + encoded_query
logger.debug('Fetching Genius search results from "{}".'.format(search_url))
metadata = self._fetch_search_page(search_url)
lyric_url = None
for section in metadata["response"]["sections"]:
result = section["hits"][0]["result"]
try:
lyric_url = result["path"]
break
except KeyError:
pass
if lyric_url is None:
raise LyricsNotFoundError(
"Could not find any valid lyric paths in Genius "
"lyrics API response for the query {}.".format(query)
)
return self.base_url + lyric_url
def from_query(self, query, linesep="\n", timeout=None):
"""
Returns the lyric string for the track best matching the
given query.
"""
logger.debug('Fetching lyrics for the search query on "{}".'.format(query))
try:
lyric_url = self.best_matching_lyric_url_from_query(query)
except LyricsNotFoundError:
raise LyricsNotFoundError(
'Genius returned no lyric results for the search query "{}".'.format(query)
)
else:
return self.from_url(lyric_url, linesep, timeout=timeout)
def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
"""
Returns the lyric string for the given artist and track
by making scraping search results and fetching the first
result.
"""
lyric_url = self.guess_lyric_url_from_artist_and_track(artist, track)
return self.from_url(lyric_url, linesep, timeout)
def from_url(self, url, linesep="\n", retries=5, timeout=None):
"""
Returns the lyric string for the given URL.
"""
logger.debug('Fetching lyric text from "{}".'.format(url))
lyric_html_page = self._fetch_url_page(url, timeout=timeout)
soup = BeautifulSoup(lyric_html_page, "html.parser")
paragraph = soup.find("p")
# If <p> has a class (like <p class="bla">), then we got an invalid
# response. Retry in such a case.
invalid_response = paragraph.get("class") is not None
to_retry = retries > 0 and invalid_response
if to_retry:
logger.debug(
"Retrying since Genius returned invalid response for search "
"results. Retries left: {retries}.".format(retries=retries)
)
return self.from_url(url, linesep=linesep, retries=retries-1, timeout=timeout)
if invalid_response:
raise LyricsNotFoundError(
'Genius returned invalid response for the search URL "{}".'.format(url)
)
lyrics = self._get_lyrics_text(paragraph)
return lyrics.replace("\n", linesep)

24
spotdl/lyrics/providers/lyricwikia_wrapper.py
View File

@@ -1,24 +0,0 @@
import lyricwikia
from spotdl.lyrics.lyric_base import LyricBase
from spotdl.lyrics.exceptions import LyricsNotFoundError
class LyricWikia(LyricBase):
def from_query(self, query, linesep="\n", timeout=None):
raise NotImplementedError
def from_artist_and_track(self, artist, track, linesep="\n", timeout=None):
"""
Returns the lyric string for the given artist and track.
"""
try:
lyrics = lyricwikia.get_lyrics(artist, track, linesep, timeout)
except lyricwikia.LyricsNotFound as e:
raise LyricsNotFoundError(e.args[0])
return lyrics
def from_url(self, url, linesep="\n", timeout=None):
raise NotImplementedError

0
spotdl/lyrics/providers/tests/__init__.py
View File

116
spotdl/lyrics/providers/tests/test_genius.py
View File

@@ -1,116 +0,0 @@
from spotdl.lyrics import LyricBase
from spotdl.lyrics import exceptions
from spotdl.lyrics.providers import Genius
import urllib.request
import json
import pytest
class TestGenius:
def test_subclass(self):
assert issubclass(Genius, LyricBase)
@pytest.fixture(scope="module")
def expect_lyrics_count(self):
# This is the number of characters in lyrics found
# for the track in `lyric_url` fixture below
return 1845
@pytest.fixture(scope="module")
def genius(self):
return Genius()
def test_base_url(self, genius):
assert genius.base_url == "https://genius.com"
@pytest.fixture(scope="module")
def artist(self):
return "selena gomez"
@pytest.fixture(scope="module")
def track(self):
return "wolves"
@pytest.fixture(scope="module")
def query(self, artist, track):
return "{} {}".format(artist, track)
@pytest.fixture(scope="module")
def guess_url(self, query):
return "https://genius.com/selena-gomez-wolves-lyrics"
@pytest.fixture(scope="module")
def lyric_url(self):
return "https://genius.com/Selena-gomez-and-marshmello-wolves-lyrics"
def test_guess_lyric_url_from_artist_and_track(self, genius, artist, track, guess_url):
url = genius.guess_lyric_url_from_artist_and_track(artist, track)
assert url == guess_url
class MockHTTPResponse:
expect_lyrics = ""
def __init__(self, request, timeout=None):
search_results_url = "https://genius.com/api/search/multi?per_page=1&q=selena%2Bgomez%2Bwolves"
if request._full_url == search_results_url:
read_method = lambda: json.dumps({
"response": {"sections": [{"hits": [{"result": {
"path": "/Selena-gomez-and-marshmello-wolves-lyrics"
} }] }] }
})
else:
read_method = lambda: "<p>" + self.expect_lyrics + "</p>"
self.read = read_method
@pytest.mark.network
def test_best_matching_lyric_url_from_query(self, genius, query, lyric_url):
url = genius.best_matching_lyric_url_from_query(query)
assert url == lyric_url
def test_mock_best_matching_lyric_url_from_query(self, genius, query, lyric_url, monkeypatch):
monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
self.test_best_matching_lyric_url_from_query(genius, query, lyric_url)
@pytest.mark.network
def test_from_url(self, genius, lyric_url, expect_lyrics_count):
lyrics = genius.from_url(lyric_url)
assert len(lyrics) == expect_lyrics_count
def test_mock_from_url(self, genius, lyric_url, expect_lyrics_count, monkeypatch):
self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
self.test_from_url(genius, lyric_url, expect_lyrics_count)
@pytest.mark.network
def test_from_artist_and_track(self, genius, artist, track, expect_lyrics_count):
lyrics = genius.from_artist_and_track(artist, track)
assert len(lyrics) == expect_lyrics_count
def test_mock_from_artist_and_track(self, genius, artist, track, expect_lyrics_count, monkeypatch):
self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
self.test_from_artist_and_track(genius, artist, track, expect_lyrics_count)
@pytest.mark.network
def test_from_query(self, genius, query, expect_lyrics_count):
lyrics = genius.from_query(query)
assert len(lyrics) == expect_lyrics_count
def test_mock_from_query(self, genius, query, expect_lyrics_count, monkeypatch):
self.MockHTTPResponse.expect_lyrics = "a" * expect_lyrics_count
monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
self.test_from_query(genius, query, expect_lyrics_count)
@pytest.mark.network
def test_lyrics_not_found_error(self, genius):
with pytest.raises(exceptions.LyricsNotFoundError):
genius.from_artist_and_track(self, "nonexistent_artist", "nonexistent_track")
def test_mock_lyrics_not_found_error(self, genius, monkeypatch):
def mock_urlopen(url, timeout=None):
raise urllib.request.HTTPError("", "", "", "", "")
monkeypatch.setattr("urllib.request.urlopen", mock_urlopen)
self.test_lyrics_not_found_error(genius)

36
spotdl/lyrics/providers/tests/test_lyricwikia_wrapper.py
View File

@@ -1,36 +0,0 @@
import lyricwikia
from spotdl.lyrics import LyricBase
from spotdl.lyrics import exceptions
from spotdl.lyrics.providers import LyricWikia
import pytest
class TestLyricWikia:
def test_subclass(self):
assert issubclass(LyricWikia, LyricBase)
def test_from_artist_and_track(self, monkeypatch):
# `LyricWikia` class uses the 3rd party method `lyricwikia.get_lyrics`
# internally and there is no need to test a 3rd party library as they
# have their own implementation of tests.
monkeypatch.setattr(
"lyricwikia.get_lyrics", lambda a, b, c, d: "awesome lyrics!"
)
artist, track = "selena gomez", "wolves"
lyrics = LyricWikia().from_artist_and_track(artist, track)
assert lyrics == "awesome lyrics!"
def test_lyrics_not_found_error(self, monkeypatch):
def lyricwikia_lyrics_not_found(msg):
raise lyricwikia.LyricsNotFound(msg)
# Wrap `lyricwikia.LyricsNotFoundError` with `exceptions.LyricsNotFoundError` error.
monkeypatch.setattr(
"lyricwikia.get_lyrics",
lambda a, b, c, d: lyricwikia_lyrics_not_found("Nope, no lyrics."),
)
artist, track = "nonexistent_artist", "nonexistent_track"
with pytest.raises(exceptions.LyricsNotFoundError):
LyricWikia().from_artist_and_track(artist, track)

24
spotdl/lyrics/tests/test_lyric_base.py
View File

@@ -1,24 +0,0 @@
from spotdl.lyrics import LyricBase
import pytest
class TestAbstractBaseClass:
def test_error_abstract_base_class_lyricbase(self):
with pytest.raises(TypeError):
# This abstract base class must be inherited from
# for instantiation
LyricBase()
def test_inherit_abstract_base_class_encoderbase(self):
class LyricKid(LyricBase):
def from_query(self, query):
raise NotImplementedError
def from_artist_and_track(self, artist, track):
pass
def from_url(self, url):
raise NotImplementedError
LyricKid()

5
spotdl/lyrics/tests/test_lyrics_exceptions.py
View File

@@ -1,5 +0,0 @@
from spotdl.lyrics.exceptions import LyricsNotFoundError
def test_lyrics_not_found_subclass():
assert issubclass(LyricsNotFoundError, Exception)

11
spotdl/metadata/__init__.py
View File

@@ -1,11 +0,0 @@
from spotdl.metadata.provider_base import ProviderBase
from spotdl.metadata.provider_base import StreamsBase
from spotdl.metadata.exceptions import MetadataNotFoundError
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
from spotdl.metadata.embedder_base import EmbedderBase
from spotdl.metadata.formatter import format_string

94
spotdl/metadata/embedder_base.py
View File

@@ -1,94 +0,0 @@
import os
from abc import ABC
from abc import abstractmethod
import urllib.request
class EmbedderBase(ABC):
"""
The subclass must define the supported media file encoding
formats here using a static variable - such as:
>>> supported_formats = ("mp3", "m4a", "flac")
"""
supported_formats = ()
@abstractmethod
def __init__(self):
"""
For every supported format, there must be a corresponding
method that applies metadata on this format.
Such as if mp3 is supported, there must exist a method named
`as_mp3` on this class that applies metadata on mp3 files.
"""
# self.targets = { fmt: eval(str("self.as_" + fmt))
# for fmt in self.supported_formats }
#
# TODO: The above code seems to fail for some reason
# I do not know.
self.targets = {}
for fmt in self.supported_formats:
# FIXME: Calling `eval` is dangerous here!
self.targets[fmt] = eval("self.as_" + fmt)
def get_encoding(self, path):
"""
This method must determine the encoding for a local
audio file. Such as "mp3", "wav", "m4a", etc.
"""
_, extension = os.path.splitext(path)
# Ignore the initial dot from file extension
return extension[1:]
def apply_metadata(self, path, metadata, cached_albumart=None, encoding=None):
"""
This method must automatically detect the media encoding
format from file path and embed the corresponding metadata
on the given file by calling an appropriate submethod.
"""
if cached_albumart is None:
cached_albumart = urllib.request.urlopen(
metadata["album"]["images"][0]["url"],
).read()
if encoding is None:
encoding = self.get_encoding(path)
if encoding not in self.supported_formats:
raise TypeError(
'The input format ("{}") is not supported.'.format(
encoding,
))
embed_on_given_format = self.targets[encoding]
embed_on_given_format(path, metadata, cached_albumart=cached_albumart)
def as_mp3(self, path, metadata, cached_albumart=None):
"""
Method for mp3 support. This method might be defined in
a subclass.
Other methods for additional supported formats must also
be declared here.
"""
raise NotImplementedError
def as_m4a(self, path, metadata, cached_albumart=None):
"""
Method for m4a support. This method might be defined in
a subclass.
Other methods for additional supported formats must also
be declared here.
"""
raise NotImplementedError
def as_flac(self, path, metadata, cached_albumart=None):
"""
Method for flac support. This method might be defined in
a subclass.
Other methods for additional supported formats must also
be declared here.
"""
raise NotImplementedError

2
spotdl/metadata/embedders/__init__.py
View File

@@ -1,2 +0,0 @@
from spotdl.metadata.embedders.default_embedder import EmbedderDefault

189
spotdl/metadata/embedders/default_embedder.py
View File

@@ -1,189 +0,0 @@
from mutagen.easyid3 import EasyID3
from mutagen.id3 import ID3, TORY, TYER, TPUB, APIC, USLT, COMM
from mutagen.mp4 import MP4, MP4Cover
from mutagen.flac import Picture, FLAC
import urllib.request
from spotdl.metadata import EmbedderBase
import logging
logger = logging.getLogger(__name__)
# Apple has specific tags - see mutagen docs -
# http://mutagen.readthedocs.io/en/latest/api/mp4.html
M4A_TAG_PRESET = {
"album": "\xa9alb",
"artist": "\xa9ART",
"date": "\xa9day",
"title": "\xa9nam",
"year": "\xa9day",
"originaldate": "purd",
"comment": "\xa9cmt",
"group": "\xa9grp",
"writer": "\xa9wrt",
"genre": "\xa9gen",
"tracknumber": "trkn",
"albumartist": "aART",
"discnumber": "disk",
"cpil": "cpil",
"albumart": "covr",
"copyright": "cprt",
"tempo": "tmpo",
"lyrics": "\xa9lyr",
"comment": "\xa9cmt",
}
TAG_PRESET = {}
for key in M4A_TAG_PRESET.keys():
TAG_PRESET[key] = key
class EmbedderDefault(EmbedderBase):
supported_formats = ("mp3", "m4a", "flac")
def __init__(self):
super().__init__()
self._m4a_tag_preset = M4A_TAG_PRESET
self._tag_preset = TAG_PRESET
# self.provider = "spotify" if metadata["spotify_metadata"] else "youtube"
def as_mp3(self, path, metadata, cached_albumart=None):
""" Embed metadata to MP3 files. """
logger.debug('Writing MP3 metadata to "{path}".'.format(path=path))
# EasyID3 is fun to use ;)
# For supported easyid3 tags:
# https://github.com/quodlibet/mutagen/blob/master/mutagen/easyid3.py
# Check out somewhere at end of above linked file
audiofile = EasyID3(path)
self._embed_basic_metadata(audiofile, metadata, "mp3", preset=TAG_PRESET)
audiofile["media"] = metadata["type"]
audiofile["author"] = metadata["artists"][0]["name"]
audiofile["lyricist"] = metadata["artists"][0]["name"]
audiofile["arranger"] = metadata["artists"][0]["name"]
audiofile["performer"] = metadata["artists"][0]["name"]
provider = metadata["provider"]
audiofile["website"] = metadata["external_urls"][provider]
audiofile["length"] = str(metadata["duration"])
if metadata["publisher"]:
audiofile["encodedby"] = metadata["publisher"]
if metadata["external_ids"]["isrc"]:
audiofile["isrc"] = metadata["external_ids"]["isrc"]
audiofile.save(v2_version=3)
# For supported id3 tags:
# https://github.com/quodlibet/mutagen/blob/master/mutagen/id3/_frames.py
# Each class represents an id3 tag
audiofile = ID3(path)
if metadata["year"]:
audiofile["TORY"] = TORY(encoding=3, text=metadata["year"])
audiofile["TYER"] = TYER(encoding=3, text=metadata["year"])
if metadata["publisher"]:
audiofile["TPUB"] = TPUB(encoding=3, text=metadata["publisher"])
provider = metadata["provider"]
audiofile["COMM"] = COMM(
encoding=3, text=metadata["external_urls"][provider]
)
if metadata["lyrics"]:
audiofile["USLT"] = USLT(
encoding=3, desc=u"Lyrics", text=metadata["lyrics"]
)
if cached_albumart is None:
cached_albumart = urllib.request.urlopen(
metadata["album"]["images"][0]["url"]
).read()
albumart.close()
try:
audiofile["APIC"] = APIC(
encoding=3,
mime="image/jpeg",
type=3,
desc=u"Cover",
data=cached_albumart,
)
except IndexError:
pass
audiofile.save(v2_version=3)
def as_m4a(self, path, metadata, cached_albumart=None):
""" Embed metadata to M4A files. """
logger.debug('Writing M4A metadata to "{path}".'.format(path=path))
audiofile = MP4(path)
self._embed_basic_metadata(audiofile, metadata, "m4a", preset=M4A_TAG_PRESET)
if metadata["year"]:
audiofile[M4A_TAG_PRESET["year"]] = metadata["year"]
provider = metadata["provider"]
audiofile[M4A_TAG_PRESET["comment"]] = metadata["external_urls"][provider]
if metadata["lyrics"]:
audiofile[M4A_TAG_PRESET["lyrics"]] = metadata["lyrics"]
try:
if cached_albumart is None:
cached_albumart = urllib.request.urlopen(
metadata["album"]["images"][0]["url"]
).read()
albumart.close()
audiofile[M4A_TAG_PRESET["albumart"]] = [
MP4Cover(cached_albumart, imageformat=MP4Cover.FORMAT_JPEG)
]
except IndexError:
pass
audiofile.save()
def as_flac(self, path, metadata, cached_albumart=None):
logger.debug('Writing FLAC metadata to "{path}".'.format(path=path))
audiofile = FLAC(path)
self._embed_basic_metadata(audiofile, metadata, "flac")
if metadata["year"]:
audiofile["year"] = metadata["year"]
provider = metadata["provider"]
audiofile["comment"] = metadata["external_urls"][provider]
if metadata["lyrics"]:
audiofile["lyrics"] = metadata["lyrics"]
image = Picture()
image.type = 3
image.desc = "Cover"
image.mime = "image/jpeg"
if cached_albumart is None:
cached_albumart = urllib.request.urlopen(
metadata["album"]["images"][0]["url"]
).read()
albumart.close()
image.data = cached_albumart
audiofile.add_picture(image)
audiofile.save()
def _embed_basic_metadata(self, audiofile, metadata, encoding, preset=TAG_PRESET):
audiofile[preset["artist"]] = metadata["artists"][0]["name"]
if metadata["album"]["artists"][0]["name"]:
audiofile[preset["albumartist"]] = metadata["album"]["artists"][0]["name"]
if metadata["album"]["name"]:
audiofile[preset["album"]] = metadata["album"]["name"]
audiofile[preset["title"]] = metadata["name"]
if metadata["release_date"]:
audiofile[preset["date"]] = metadata["release_date"]
audiofile[preset["originaldate"]] = metadata["release_date"]
if metadata["genre"]:
audiofile[preset["genre"]] = metadata["genre"]
if metadata["copyright"]:
audiofile[preset["copyright"]] = metadata["copyright"]
if encoding == "flac":
audiofile[preset["discnumber"]] = str(metadata["disc_number"])
else:
audiofile[preset["discnumber"]] = [(metadata["disc_number"], 0)]
zfilled_track_number = str(metadata["track_number"]).zfill(len(str(metadata["total_tracks"])))
if encoding == "flac":
audiofile[preset["tracknumber"]] = zfilled_track_number
else:
if preset["tracknumber"] == TAG_PRESET["tracknumber"]:
audiofile[preset["tracknumber"]] = "{}/{}".format(
zfilled_track_number, metadata["total_tracks"]
)
else:
audiofile[preset["tracknumber"]] = [
(metadata["track_number"], metadata["total_tracks"])
]

9
spotdl/metadata/embedders/tests/test_default_embedder.py
View File

@@ -1,9 +0,0 @@
from spotdl.metadata.embedders import EmbedderDefault
import pytest
@pytest.mark.xfail
def test_embedder():
# Do not forget to Write tests for this!
raise NotImplementedError

20
spotdl/metadata/exceptions.py
View File

@@ -1,20 +0,0 @@
class MetadataNotFoundError(Exception):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class SpotifyMetadataNotFoundError(MetadataNotFoundError):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)
class YouTubeMetadataNotFoundError(MetadataNotFoundError):
__module__ = Exception.__module__
def __init__(self, message=None):
super().__init__(message)

23
spotdl/metadata/formatter.py
View File

@@ -1,23 +0,0 @@
def format_string(string, metadata, output_extension="", sanitizer=lambda s: s):
formats = {
"{track-name}" : metadata["name"],
"{artist}" : metadata["artists"][0]["name"],
"{album}" : metadata["album"]["name"],
"{album-artist}" : metadata["artists"][0]["name"],
"{genre}" : metadata["genre"],
"{disc-number}" : metadata["disc_number"],
"{duration}" : metadata["duration"],
"{year}" : metadata["year"],
"{original-date}": metadata["release_date"],
"{track-number}" : str(metadata["track_number"]).zfill(len(str(metadata["total_tracks"]))),
"{total-tracks}" : metadata["total_tracks"],
"{isrc}" : metadata["external_ids"]["isrc"],
"{track-id}" : metadata.get("id", ""),
"{output-ext}" : output_extension,
}
for key, value in formats.items():
string = string.replace(key, sanitizer(str(value)))
return string

66
spotdl/metadata/provider_base.py
View File

@@ -1,66 +0,0 @@
from abc import ABC
from abc import abstractmethod
class StreamsBase(ABC):
@abstractmethod
def __init__(self, streams):
"""
This method must parse audio streams into a list of
dictionaries with the keys:
"bitrate", "download_url", "encoding", "filesize".
The list should typically be sorted in descending order
based on the audio stream's bitrate.
This sorted list must be assigned to `self.all`.
"""
self.all = streams
def getbest(self):
"""
This method must return the audio stream with the
highest bitrate.
"""
return self.all[0]
def getworst(self):
"""
This method must return the audio stream with the
lowest bitrate.
"""
return self.all[-1]
class ProviderBase(ABC):
def set_credentials(self, client_id, client_secret):
"""
This method may or not be used depending on
whether the metadata provider requires authentication
or not.
"""
pass
@abstractmethod
def from_url(self, url):
"""
This method must return track metadata from the
corresponding Spotify URL.
"""
pass
def from_query(self, query):
"""
This method must return track metadata from the
corresponding search query.
"""
raise NotImplementedError
@abstractmethod
def metadata_to_standard_form(self, metadata):
"""
This method must transform the fetched metadata
into a format consistent with all other metadata
providers, for easy utilization.
"""
pass

4
spotdl/metadata/providers/__init__.py
View File

@@ -1,4 +0,0 @@
from spotdl.metadata.providers.spotify import ProviderSpotify
from spotdl.metadata.providers.youtube import ProviderYouTube
from spotdl.metadata.providers.youtube import YouTubeSearch

82
spotdl/metadata/providers/spotify.py
View File

@@ -1,82 +0,0 @@
import spotipy
import spotipy.oauth2 as oauth2
from spotdl.metadata import ProviderBase
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
from spotdl.authorize.services import AuthorizeSpotify
import spotdl.util
import logging
logger = logging.getLogger(__name__)
class ProviderSpotify(ProviderBase):
def __init__(self, spotify=None):
if spotify is None:
spotify = AuthorizeSpotify()
self.spotify = spotify
def set_credentials(self, client_id, client_secret):
token = self._generate_token(client_id, client_secret)
self.spotify = spotipy.Spotify(auth=token)
def from_url(self, url):
logger.debug('Fetching Spotify metadata for "{url}".'.format(url=url))
metadata = self.spotify.track(url)
return self.metadata_to_standard_form(metadata)
def from_query(self, query):
tracks = self.search(query)["tracks"]["items"]
if not tracks:
raise SpotifyMetadataNotFoundError(
'Spotify returned no tracks for the search query "{}".'.format(
query,
)
)
return self.metadata_to_standard_form(tracks[0])
def search(self, query):
return self.spotify.search(query)
def _generate_token(self, client_id, client_secret):
""" Generate the token. """
credentials = oauth2.SpotifyClientCredentials(
client_secret=client_secret,
)
token = credentials.get_access_token()
return token
def metadata_to_standard_form(self, metadata):
artist = self.spotify.artist(metadata["artists"][0]["id"])
album = self.spotify.album(metadata["album"]["id"])
try:
metadata[u"genre"] = spotdl.util.titlecase(artist["genres"][0])
except IndexError:
metadata[u"genre"] = None
try:
metadata[u"copyright"] = album["copyrights"][0]["text"]
except IndexError:
metadata[u"copyright"] = None
try:
metadata[u"external_ids"][u"isrc"]
except KeyError:
metadata[u"external_ids"][u"isrc"] = None
metadata[u"release_date"] = album["release_date"]
metadata[u"publisher"] = album["label"]
metadata[u"total_tracks"] = album["tracks"]["total"]
# Some sugar
metadata["year"], *_ = metadata["release_date"].split("-")
metadata["duration"] = metadata["duration_ms"] / 1000.0
metadata["provider"] = "spotify"
# Remove unwanted parameters
del metadata["duration_ms"]
del metadata["available_markets"]
del metadata["album"]["available_markets"]
return metadata

0
spotdl/metadata/providers/tests/__init__.py
View File

BIN
spotdl/metadata/providers/tests/data/streams.dump
View File

Binary file not shown.

1
spotdl/metadata/providers/tests/data/youtube_no_search_results.html.test
View File

File diff suppressed because one or more lines are too long

1
spotdl/metadata/providers/tests/data/youtube_search_results.html.test
View File

File diff suppressed because one or more lines are too long

19
spotdl/metadata/providers/tests/test_spotify.py
View File

@@ -1,19 +0,0 @@
from spotdl.metadata import ProviderBase
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
from spotdl.metadata.providers import ProviderSpotify
import pytest
class TestProviderSpotify:
def test_subclass(self):
assert issubclass(ProviderSpotify, ProviderBase)
@pytest.mark.xfail
def test_spotify_stuff(self):
raise NotImplementedError
# def test_metadata_not_found_error(self):
# provider = ProviderSpotify(spotify=spotify)
# with pytest.raises(SpotifyMetadataNotFoundError):
# provider.from_query("This track doesn't exist on Spotify.")

366
spotdl/metadata/providers/tests/test_youtube.py
View File

@@ -1,366 +0,0 @@
from spotdl.metadata.providers.youtube import YouTubeSearch
from spotdl.metadata.providers.youtube import YouTubeStreams
from spotdl.metadata.providers.youtube import YouTubeVideos
from spotdl.metadata.providers import youtube
from spotdl.metadata.providers import ProviderYouTube
from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
import pytube
import urllib.request
import pickle
import sys
import os
import pytest
@pytest.fixture(scope="module")
def track():
"""
This query is to be searched on YouTube for queries
that do return search results.
"""
return "selena gomez wolves"
@pytest.fixture(scope="module")
def no_result_track():
"""
This query is to be searched on YouTube for queries
that return no search results.
"""
return "n0 v1d305 3x157 f0r 7h15 53arc4 qu3ry"
@pytest.fixture(scope="module")
def expect_search_results():
"""
These are the expected search results for the "track"
query.
"""
return YouTubeVideos([
{'duration': '3:33',
'title': 'Selena Gomez, Marshmello - Wolves',
'url': 'https://www.youtube.com/watch?v=cH4E_t3m3xM'},
{'duration': '3:18',
'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
'url': 'https://www.youtube.com/watch?v=xrbY9gDVms0'},
{'duration': '3:21',
'title': 'Wolves - Selena Gomez, Marshmello (Lyrics)',
'url': 'https://www.youtube.com/watch?v=jX0n2rSmDbE'},
{'duration': '6:26',
'title': 'Selena Gomez and Marshmello - Wolves (Official) Extended',
'url': 'https://www.youtube.com/watch?v=rQ6jcpwzQZU'},
{'duration': '3:43',
'title': 'Selena Gomez, Marshmello - Wolves (Vertical Video)',
'url': 'https://www.youtube.com/watch?v=nVzA1uWTydQ'},
{'duration': '3:18',
'title': 'Selena Gomez, Marshmello - Wolves (Visualizer)',
'url': 'https://www.youtube.com/watch?v=-grLLLTza6k'},
{'duration': '1:32',
'title': 'Wolves - Selena Gomez, Marshmello / Jun Liu Choreography',
'url': 'https://www.youtube.com/watch?v=zbWsb36U0uo'},
{'duration': '3:17',
'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
'url': 'https://www.youtube.com/watch?v=rykH1BkGwTo'},
{'duration': '3:16',
'title': 'Selena Gomez, Marshmello - Wolves (8D AUDIO)',
'url': 'https://www.youtube.com/watch?v=j0AxZ4V5WQw'},
{'duration': '3:47',
'title': 'Selena Gomez, Marshmello - Wolves (Vanrip Remix)',
'url': 'https://www.youtube.com/watch?v=RyxsaKfu-ZY'}
])
@pytest.fixture(scope="module")
def expect_mock_search_results():
"""
These are the expected mock search results for the
"track" query.
"""
return YouTubeVideos([
{'duration': '3:33',
'title': 'Selena Gomez, Marshmello - Wolves',
'url': 'https://www.youtube.com/watch?v=cH4E_t3m3xM'},
{'duration': '3:18',
'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
'url': 'https://www.youtube.com/watch?v=xrbY9gDVms0'},
{'duration': '3:21',
'title': 'Wolves - Selena Gomez, Marshmello (Lyrics)',
'url': 'https://www.youtube.com/watch?v=jX0n2rSmDbE'},
{'duration': '6:26',
'title': 'Selena Gomez and Marshmello - Wolves (Official) Extended',
'url': 'https://www.youtube.com/watch?v=rQ6jcpwzQZU'},
{'duration': '3:43',
'title': 'Selena Gomez, Marshmello - Wolves (Vertical Video)',
'url': 'https://www.youtube.com/watch?v=nVzA1uWTydQ'},
{'duration': '3:18',
'title': 'Selena Gomez, Marshmello - Wolves (Visualizer)',
'url': 'https://www.youtube.com/watch?v=-grLLLTza6k'},
{'duration': '1:32',
'title': 'Wolves - Selena Gomez, Marshmello / Jun Liu Choreography',
'url': 'https://www.youtube.com/watch?v=zbWsb36U0uo'},
{'duration': '3:17',
'title': 'Selena Gomez, Marshmello - Wolves (Lyrics)',
'url': 'https://www.youtube.com/watch?v=rykH1BkGwTo'},
{'duration': '3:16',
'title': 'Selena Gomez, Marshmello - Wolves (8D AUDIO)',
'url': 'https://www.youtube.com/watch?v=j0AxZ4V5WQw'},
{'duration': '3:47',
'title': 'Selena Gomez, Marshmello - Wolves (Vanrip Remix)',
'url': 'https://www.youtube.com/watch?v=RyxsaKfu-ZY'}
])
class MockHTTPResponse:
"""
This mocks `urllib.request.urlopen` for custom response text.
"""
response_file = ""
def __init__(self, request):
if isinstance(request, urllib.request.Request):
if request._full_url.endswith("ouVRL5arzUg=="):
self.headers = {"Content-Length": 3614184}
elif request._full_url.endswith("egl0iK2D-Bk="):
self.headers = {"Content-Length": 3444850}
elif request._full_url.endswith("J7VXJtoi3as="):
self.headers = {"Content-Length": 1847626}
elif request._full_url.endswith("_d5_ZthQdvtD"):
self.headers = {"Content-Length": 1407962}
def read(self):
module_directory = os.path.dirname(__file__)
mock_html = os.path.join(module_directory, "data", self.response_file)
with open(mock_html, "r") as fin:
html = fin.read()
return html
class TestYouTubeSearch:
@pytest.fixture(scope="module")
def youtube_searcher(self):
return YouTubeSearch()
def test_generate_search_url(self, track, youtube_searcher):
url = youtube_searcher.generate_search_url(track)
expect_url = "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q=selena%20gomez%20wolves"
assert url == expect_url
@pytest.mark.network
def test_search(self, track, youtube_searcher, expect_search_results):
results = youtube_searcher.search(track)
assert results == expect_search_results
class MockHTTPResponse:
"""
This mocks `urllib.request.urlopen` for custom response text.
"""
response_file = ""
def __init__(self, url):
pass
def read(self):
module_directory = os.path.dirname(__file__)
mock_html = os.path.join(module_directory, "data", self.response_file)
with open(mock_html, "r") as fin:
html = fin.read()
return html
# @pytest.mark.mock
def test_mock_search(self, track, youtube_searcher, expect_mock_search_results, monkeypatch):
MockHTTPResponse.response_file = "youtube_search_results.html.test"
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_search(track, youtube_searcher, expect_mock_search_results)
@pytest.mark.network
def test_no_videos_search(self, no_result_track, youtube_searcher):
results = youtube_searcher.search(no_result_track)
assert results == YouTubeVideos([])
def test_mock_no_videos_search(self, no_result_track, youtube_searcher, monkeypatch):
MockHTTPResponse.response_file = "youtube_no_search_results.html.test"
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_no_videos_search(no_result_track, youtube_searcher)
@pytest.fixture(scope="module")
def content():
return pytube.YouTube("https://www.youtube.com/watch?v=cH4E_t3m3xM")
class MockYouTube:
def __init__(self, url):
self.watch_html = '\\"category\\":\\"Music\\",\\"publishDate\\":\\"2017-11-18\\",\\"ownerChannelName\\":\\"SelenaGomezVEVO\\",'
self.title = "Selena Gomez, Marshmello - Wolves"
self.author = "SelenaGomezVEVO"
self.length = 213
self.watch_url = "https://youtube.com/watch?v=cH4E_t3m3xM"
self.thumbnail_url = "https://i.ytimg.com/vi/cH4E_t3m3xM/maxresdefault.jpg"
@property
def streams(self):
# For updating the test data:
# from spotdl.metadata.providers.youtube import YouTubeStreams
# import pytube
# import pickle
# content = pytube.YouTube("https://youtube.com/watch?v=cH4E_t3m3xM")
# with open("streams.dump", "wb") as fout:
# pickle.dump(content.streams, fout)
module_directory = os.path.dirname(__file__)
mock_streams = os.path.join(module_directory, "data", "streams.dump")
with open(mock_streams, "rb") as fin:
streams_dump = pickle.load(fin)
return streams_dump
@pytest.fixture(scope="module")
def mock_content():
return MockYouTube("https://www.youtube.com/watch?v=cH4E_t3m3xM")
@pytest.fixture(scope="module")
def expect_formatted_streams():
"""
Expected streams for the best matching video for "track" in
search results.
The `download_url` is expected as `None` since it's impossible
to predict its value before-hand.
"""
return [
{"bitrate": 160, "content": None, "download_url": None, "encoding": "opus", "filesize": 3614184},
{"bitrate": 128, "content": None, "download_url": None, "encoding": "mp4a.40.2", "filesize": 3444850},
{"bitrate": 70, "content": None, "download_url": None, "encoding": "opus", "filesize": 1847626},
{"bitrate": 50, "content": None, "download_url": None, "encoding": "opus", "filesize": 1407962}
]
class TestYouTubeStreams:
@pytest.mark.network
def test_streams(self, content, expect_formatted_streams):
formatted_streams = YouTubeStreams(content.streams)
for index in range(len(formatted_streams.all)):
assert isinstance(formatted_streams.all[index]["download_url"], str)
assert formatted_streams.all[index]["connection"] is not None
# We `None` the `download_url` since it's impossible to
# predict its value before-hand.
formatted_streams.all[index]["download_url"] = None
formatted_streams.all[index]["connection"] = None
# assert formatted_streams.all == expect_formatted_streams
for f, e in zip(formatted_streams.all, expect_formatted_streams):
assert f["filesize"] == e["filesize"]
# @pytest.mark.mock
def test_mock_streams(self, mock_content, expect_formatted_streams, monkeypatch):
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_streams(mock_content, expect_formatted_streams)
@pytest.mark.network
def test_getbest(self, content):
formatted_streams = YouTubeStreams(content.streams)
best_stream = formatted_streams.getbest()
assert isinstance(best_stream["download_url"], str)
assert best_stream["connection"] is not None
# We `None` the `download_url` since it's impossible to
# predict its value before-hand.
best_stream["download_url"] = None
best_stream["connection"] = None
assert best_stream == {
"bitrate": 160,
"connection": None,
"download_url": None,
"encoding": "opus",
"filesize": 3614184
}
# @pytest.mark.mock
def test_mock_getbest(self, mock_content, monkeypatch):
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_getbest(mock_content)
@pytest.mark.network
def test_getworst(self, content):
formatted_streams = YouTubeStreams(content.streams)
worst_stream = formatted_streams.getworst()
assert isinstance(worst_stream["download_url"], str)
assert worst_stream["connection"] is not None
# We `None` the `download_url` since it's impossible to
# predict its value before-hand.
worst_stream["download_url"] = None
worst_stream["connection"] = None
assert worst_stream == {
"bitrate": 50,
"connection": None,
"download_url": None,
"encoding": 'opus',
"filesize": 1407962
}
# @pytest.mark.mock
def test_mock_getworst(self, mock_content, monkeypatch):
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_getworst(mock_content)
class TestProviderYouTube:
@pytest.fixture(scope="module")
def youtube_provider(self):
return ProviderYouTube()
class MockYouTubeSearch:
watch_urls = []
def search(self, query):
return self.watch_urls
@pytest.mark.network
def test_from_query(self, track, youtube_provider):
metadata = youtube_provider.from_query(track)
assert isinstance(metadata["streams"], YouTubeStreams)
# We avoid testing each item for the `streams` key here
# again. It this has already been tested above.
metadata["streams"] = []
assert metadata == {
'album': {'artists': [{'name': None}],
'images': [{'url': 'https://i.ytimg.com/vi/cH4E_t3m3xM/maxresdefault.jpg'}],
'name': None},
'artists': [{'name': 'SelenaGomezVEVO'}],
'copyright': None,
'disc_number': 1,
'duration': 213,
'external_ids': {'isrc': None},
'external_urls': {'youtube': 'https://youtube.com/watch?v=cH4E_t3m3xM'},
'genre': None,
'lyrics': None,
'name': 'Selena Gomez, Marshmello - Wolves',
'provider': 'youtube',
'publisher': None,
'release_date': '2017-11-1',
'streams': [],
'total_tracks': 1,
'track_number': 1,
'type': 'track',
'year': '2017'
}
def test_mock_from_query(self, track, youtube_provider, expect_mock_search_results, monkeypatch):
self.MockYouTubeSearch.watch_urls = expect_mock_search_results
monkeypatch.setattr(youtube, "YouTubeSearch", self.MockYouTubeSearch)
monkeypatch.setattr(pytube, "YouTube", MockYouTube)
monkeypatch.setattr(urllib.request, "urlopen", MockHTTPResponse)
self.test_from_query(track, youtube_provider)
@pytest.mark.network
def test_error_exception_from_query(self, no_result_track, youtube_provider):
with pytest.raises(YouTubeMetadataNotFoundError):
youtube_provider.from_query(no_result_track)
def test_mock_error_exception_from_query(self, no_result_track, youtube_provider, monkeypatch):
self.MockYouTubeSearch.watch_urls = []
monkeypatch.setattr(youtube, "YouTubeSearch", self.MockYouTubeSearch)
monkeypatch.setattr(pytube, "YouTube", MockYouTube)
self.test_error_exception_from_query(no_result_track, youtube_provider)

283
spotdl/metadata/providers/youtube.py
View File

@@ -1,283 +0,0 @@
import pytube
from bs4 import BeautifulSoup
import urllib.request
import threading
from collections.abc import Sequence
from spotdl.metadata import StreamsBase
from spotdl.metadata import ProviderBase
from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
import spotdl.util
import logging
logger = logging.getLogger(__name__)
BASE_SEARCH_URL = "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q={}"
HEADERS = [('Range', 'bytes=0-'),]
class YouTubeVideos(Sequence):
def __init__(self, videos):
self.videos = videos
super().__init__()
def __repr__(self):
return "YouTubeVideos({})".format(self.videos)
def __len__(self):
return len(self.videos)
def __getitem__(self, index):
return self.videos[index]
def __eq__(self, instance):
return self.videos == instance.videos
def bestmatch(self):
video = self.videos[0]
logger.debug("Matched with: {title} ({url}) [{duration}]".format(
title=video["title"],
url=video["url"],
duration=video["duration"]
))
return video
class YouTubeSearch:
def __init__(self):
self.base_search_url = BASE_SEARCH_URL
def generate_search_url(self, query):
quoted_query = urllib.request.quote(query)
return self.base_search_url.format(quoted_query)
def _fetch_response_html(self, url):
response = urllib.request.urlopen(url)
soup = BeautifulSoup(response.read(), "html.parser")
return soup
def _extract_video_details_from_result(self, html):
video_time = html.find("span", class_="video-time").get_text()
inner_html = html.find("div", class_="yt-lockup-content")
video_id = inner_html.find("a")["href"][-11:]
video_title = inner_html.find("a")["title"]
video_details = {
"url": "https://www.youtube.com/watch?v=" + video_id,
"title": video_title,
"duration": video_time,
}
return video_details
def _fetch_search_results(self, html, limit=10):
result_source = html.find_all(
"div", {"class": "yt-lockup-dismissable yt-uix-tile"}
)
videos = []
for result in result_source:
if not self._is_video(result):
continue
video = self._extract_video_details_from_result(result)
videos.append(video)
if len(videos) >= limit:
break
return videos
def _is_video(self, result):
# ensure result is not a channel
not_video = (
result.find("channel") is not None
or "yt-lockup-channel" in result.parent.attrs["class"]
or "yt-lockup-channel" in result.attrs["class"]
)
# ensure result is not a mix/playlist
not_video = not_video or "yt-lockup-playlist" in result.parent.attrs["class"]
# ensure video result is not an advertisement
not_video = not_video or result.find("googleads") is not None
video = not not_video
return video
def _is_server_side_invalid_response(self, videos, html):
if videos:
return False
search_message = html.find("div", {"class":"search-message"})
return search_message is None
def search(self, query, limit=10, retries=5):
""" Search and scrape YouTube to return a list of matching videos. """
search_url = self.generate_search_url(query)
logger.debug('Fetching YouTube results for "{}" at "{}".'.format(query, search_url))
html = self._fetch_response_html(search_url)
videos = self._fetch_search_results(html, limit=limit)
to_retry = retries > 0 and self._is_server_side_invalid_response(videos, html)
if to_retry:
logger.debug(
"Retrying since YouTube returned invalid response for search "
"results. Retries left: {retries}.".format(retries=retries)
)
return self.search(query, limit=limit, retries=retries-1)
return YouTubeVideos(videos)
class YouTubeStreams(StreamsBase):
def __init__(self, streams):
self.network_headers = HEADERS
audiostreams = streams.filter(only_audio=True).order_by("abr").desc()
thread_pool = []
self.all = []
for stream in audiostreams:
encoding = "m4a" if "mp4a" in stream.audio_codec else stream.audio_codec
standard_stream = {
# Store only the integer part for bitrate. For example
# the given bitrate would be "192kbps", we store only
# the integer part (192) here and drop the rest.
"bitrate": int(stream.abr[:-4]),
"connection": None,
"download_url": stream.url,
"encoding": encoding,
"filesize": None,
}
establish_connection = threading.Thread(
target=self._store_connection,
args=(standard_stream,),
)
thread_pool.append(establish_connection)
establish_connection.start()
self.all.append(standard_stream)
for thread in thread_pool:
thread.join()
def _store_connection(self, stream):
response = self._make_request(stream["download_url"])
stream["connection"] = response
stream["filesize"] = int(response.headers["Content-Length"])
def _make_request(self, url):
request = urllib.request.Request(url)
for header in self.network_headers:
request.add_header(*header)
return urllib.request.urlopen(request)
def get(self, quality="best", preftype="automatic"):
if quality == "best":
return self.getbest(preftype=preftype)
elif quality == "worst":
return self.getworst(preftype=preftype)
else:
return None
def getbest(self, preftype="automatic"):
selected_stream = None
if preftype == "automatic":
selected_stream = self.all[0]
else:
for stream in self.all:
if stream["encoding"] == preftype:
selected_stream = stream
break
logger.debug('Selected best quality stream for "{preftype}" format:\n{stream}'.format(
preftype=preftype,
stream=selected_stream,
))
return selected_stream
def getworst(self, preftype="automatic"):
selected_stream = None
if preftype == "automatic":
selected_stream = self.all[-1]
else:
for stream in self.all[::-1]:
if stream["encoding"] == preftype:
selected_stream = stream
break
logger.debug('Selected worst quality stream for "{preftype}" format:\n{stream}'.format(
preftype=preftype,
stream=selected_stream,
))
return selected_stream
class ProviderYouTube(ProviderBase):
def from_query(self, query):
watch_urls = self.search(query)
if not watch_urls:
raise YouTubeMetadataNotFoundError(
'YouTube returned nothing for the given search '
'query ("{}")'.format(query)
)
return self.from_url(watch_urls[0])
def from_url(self, url, retries=5):
logger.debug('Fetching YouTube metadata for "{url}".'.format(url=url))
try:
content = pytube.YouTube(url)
except KeyError:
# Sometimes YouTube can return unexpected response, in such a case
# retry a few times before finally failing.
if retries > 0:
retries -= 1
logger.debug(
"YouTube returned an unexpected response for "
"`pytube.YouTube({url})`. Retries left: {retries}".format(
url=url, retries=retries
)
)
return self.from_url(url, retries=retries)
else:
raise
else:
return self.from_pytube_object(content)
def from_pytube_object(self, content):
return self.metadata_to_standard_form(content)
def search(self, query):
return YouTubeSearch().search(query)
def _fetch_publish_date(self, content):
# XXX: This needs to be supported in PyTube itself
# See https://github.com/nficano/pytube/issues/595
position = content.watch_html.find("publishDate")
publish_date = content.watch_html[position+16:position+25]
return publish_date
def metadata_to_standard_form(self, content):
""" Fetch a song's metadata from YouTube. """
publish_date = self._fetch_publish_date(content)
metadata = {
"name": content.title,
"artists": [{"name": content.author}],
"duration": content.length,
"external_urls": {"youtube": content.watch_url},
"album": {
"images": [{"url": content.thumbnail_url}],
"artists": [{"name": None}],
"name": None,
},
"year": publish_date.split("-")[0],
"release_date": publish_date,
"type": "track",
"disc_number": 1,
"track_number": 1,
"total_tracks": 1,
"publisher": None,
"external_ids": {"isrc": None},
"lyrics": None,
"copyright": None,
"genre": None,
"streams": YouTubeStreams(content.streams),
"provider": "youtube",
}
return metadata

0
spotdl/metadata/tests/__init__.py
View File

72
spotdl/metadata/tests/test_embedder_base.py
View File

@@ -1,72 +0,0 @@
from spotdl.metadata import EmbedderBase
import pytest
class EmbedderKid(EmbedderBase):
def __init__(self):
super().__init__()
class TestEmbedderBaseABC:
def test_error_base_class_embedderbase(self):
with pytest.raises(TypeError):
# This abstract base class must be inherited from
# for instantiation
EmbedderBase()
def test_inherit_abstract_base_class_streamsbase(self):
EmbedderKid()
class TestMethods:
@pytest.fixture(scope="module")
def embedderkid(self):
return EmbedderKid()
def test_target_formats(self, embedderkid):
assert embedderkid.supported_formats == ()
@pytest.mark.parametrize("path, expect_encoding", (
("/a/b/c/file.mp3", "mp3"),
("music/pop/1.wav", "wav"),
("/a path/with spaces/track.m4a", "m4a"),
))
def test_get_encoding(self, embedderkid, path, expect_encoding):
assert embedderkid.get_encoding(path) == expect_encoding
def test_apply_metadata_with_explicit_encoding(self, embedderkid):
with pytest.raises(TypeError):
embedderkid.apply_metadata("/path/to/music.mp3", {}, cached_albumart="imagedata", encoding="mp3")
def test_apply_metadata_with_implicit_encoding(self, embedderkid):
with pytest.raises(TypeError):
embedderkid.apply_metadata("/path/to/music.wav", {}, cached_albumart="imagedata")
class MockHTTPResponse:
"""
This mocks `urllib.request.urlopen` for custom response text.
"""
response_file = ""
def __init__(self, url):
pass
def read(self):
pass
def test_apply_metadata_without_cached_image(self, embedderkid, monkeypatch):
monkeypatch.setattr("urllib.request.urlopen", self.MockHTTPResponse)
metadata = {"album": {"images": [{"url": "http://animageurl.com"},]}}
with pytest.raises(TypeError):
embedderkid.apply_metadata("/path/to/music.wav", metadata, cached_albumart=None)
@pytest.mark.parametrize("fmt_method_suffix", (
"as_mp3",
"as_m4a",
"as_flac",
))
def test_embed_formats(self, fmt_method_suffix, embedderkid):
method = eval("embedderkid." + fmt_method_suffix)
with pytest.raises(NotImplementedError):
method("/a/random/path", {})

15
spotdl/metadata/tests/test_metadata_exceptions.py
View File

@@ -1,15 +0,0 @@
from spotdl.metadata.exceptions import MetadataNotFoundError
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
from spotdl.metadata.exceptions import YouTubeMetadataNotFoundError
class TestMetadataNotFoundSubclass:
def test_metadata_not_found_subclass(self):
assert issubclass(MetadataNotFoundError, Exception)
def test_spotify_metadata_not_found(self):
assert issubclass(SpotifyMetadataNotFoundError, MetadataNotFoundError)
def test_youtube_metadata_not_found(self):
assert issubclass(YouTubeMetadataNotFoundError, MetadataNotFoundError)

60
spotdl/metadata/tests/test_provider_base.py
View File

@@ -1,60 +0,0 @@
from spotdl.metadata import ProviderBase
from spotdl.metadata import StreamsBase
import pytest
class TestStreamsBaseABC:
def test_error_abstract_base_class_streamsbase(self):
with pytest.raises(TypeError):
# This abstract base class must be inherited from
# for instantiation
StreamsBase()
def test_inherit_abstract_base_class_streamsbase(self):
class StreamsKid(StreamsBase):
def __init__(self, streams):
super().__init__(streams)
streams = ("stream1", "stream2", "stream3")
kid = StreamsKid(streams)
assert kid.all == streams
class TestMethods:
class StreamsKid(StreamsBase):
def __init__(self, streams):
super().__init__(streams)
@pytest.fixture(scope="module")
def streamskid(self):
streams = ("stream1", "stream2", "stream3")
streamskid = self.StreamsKid(streams)
return streamskid
def test_getbest(self, streamskid):
best_stream = streamskid.getbest()
assert best_stream == "stream1"
def test_getworst(self, streamskid):
worst_stream = streamskid.getworst()
assert worst_stream == "stream3"
class TestProviderBaseABC:
def test_error_abstract_base_class_providerbase(self):
with pytest.raises(TypeError):
# This abstract base class must be inherited from
# for instantiation
ProviderBase()
def test_inherit_abstract_base_class_providerbase(self):
class ProviderKid(ProviderBase):
def from_url(self, query):
pass
def metadata_to_standard_form(self, metadata):
pass
ProviderKid()

260
spotdl/metadata_search.py
View File

@@ -1,260 +0,0 @@
from spotdl.metadata.providers import ProviderSpotify
from spotdl.metadata.providers import ProviderYouTube
from spotdl.lyrics.providers import Genius
from spotdl.lyrics.exceptions import LyricsNotFoundError
import spotdl.metadata
import spotdl.util
from spotdl.metadata.exceptions import SpotifyMetadataNotFoundError
from spotdl.command_line.exceptions import NoYouTubeVideoFoundError
from spotdl.command_line.exceptions import NoYouTubeVideoMatchError
import sys
import logging
logger = logging.getLogger(__name__)
PROVIDERS = {
"spotify": ProviderSpotify,
"youtube": ProviderYouTube,
}
def prompt_for_youtube_search_result(videos):
max_index_length = len(str(len(videos)))
max_title_length = max(len(v["title"]) for v in videos)
msg = "{index:>{max_index}}. Skip downloading this track".format(
index=0,
max_index=max_index_length,
)
print(msg, file=sys.stderr)
for index, video in enumerate(videos, 1):
vid_details = "{index:>{max_index}}. {title:<{max_title}}\n{new_line_gap} {url} [{duration}]".format(
index=index,
max_index=max_index_length,
title=video["title"],
max_title=max_title_length,
new_line_gap=" " * max_index_length,
url=video["url"],
duration=video["duration"],
)
print(vid_details, file=sys.stderr)
print("", file=sys.stderr)
selection = spotdl.util.prompt_user_for_selection(range(1, len(videos)+1))
if selection is None:
return None
return videos[selection-1]
class MetadataSearch:
def __init__(self, track, lyrics=False, yt_search_format="{artist} - {track-name}", yt_manual=False, providers=PROVIDERS):
self.track = track
self.track_type = spotdl.util.track_type(track)
self.lyrics = lyrics
self.yt_search_format = yt_search_format
self.yt_manual = yt_manual
self.providers = {}
for provider, parent in providers.items():
self.providers[provider] = parent()
self.lyric_provider = Genius()
def get_lyrics(self, query):
try:
lyrics = self.lyric_provider.from_query(query)
except LyricsNotFoundError as e:
logger.warning(e.args[0])
lyrics = None
return lyrics
def _make_lyric_search_query(self, metadata):
if self.track_type == "query":
lyric_query = self.track
else:
lyric_search_format = "{artist} - {track-name}"
lyric_query = spotdl.metadata.format_string(
lyric_search_format,
metadata
)
return lyric_query
def on_youtube_and_spotify(self):
track_type_mapper = {
"spotify": self._on_youtube_and_spotify_for_type_spotify,
"youtube": self._on_youtube_and_spotify_for_type_youtube,
"query": self._on_youtube_and_spotify_for_type_query,
}
caller = track_type_mapper[self.track_type]
metadata = caller()
if not self.lyrics:
return metadata
lyric_query = self._make_lyric_search_query(metadata)
metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
target=self.get_lyrics,
args=(lyric_query,),
)
return metadata
def on_youtube(self):
track_type_mapper = {
"spotify": self._on_youtube_for_type_spotify,
"youtube": self._on_youtube_for_type_youtube,
"query": self._on_youtube_for_type_query,
}
caller = track_type_mapper[self.track_type]
metadata = caller(self.track)
if not self.lyrics:
return metadata
lyric_query = self._make_lyric_search_query(metadata)
metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
target=self.get_lyrics,
arguments=(lyric_query,),
)
return metadata
def on_spotify(self):
track_type_mapper = {
"spotify": self._on_spotify_for_type_spotify,
"youtube": self._on_spotify_for_type_youtube,
"query": self._on_spotify_for_type_query,
}
caller = track_type_mapper[self.track_type]
metadata = caller(self.track)
if not self.lyrics:
return metadata
lyric_query = self._make_lyric_search_query(metadata)
metadata["lyrics"] = spotdl.util.ThreadWithReturnValue(
target=self.get_lyrics,
arguments=(lyric_query,),
)
return metadata
def best_on_youtube_search(self):
track_type_mapper = {
"spotify": self._best_on_youtube_search_for_type_spotify,
"youtube": self._best_on_youtube_search_for_type_youtube,
"query": self._best_on_youtube_search_for_type_query,
}
caller = track_type_mapper[self.track_type]
video = caller(self.track)
return video
def _best_on_youtube_search_for_type_query(self, query):
videos = self.providers["youtube"].search(query)
if not videos:
raise NoYouTubeVideoFoundError(
'YouTube returned no videos for the search query "{}".'.format(query)
)
if self.yt_manual:
video = prompt_for_youtube_search_result(videos)
else:
video = videos.bestmatch()
if video is None:
raise NoYouTubeVideoMatchError(
'No matching videos found on YouTube for the search query "{}".'.format(
query
)
)
return video
def _best_on_youtube_search_for_type_youtube(self, url):
video = self._best_on_youtube_search_for_type_query(url)
return video
def _best_on_youtube_search_for_type_spotify(self, url):
spotify_metadata = self._on_spotify_for_type_spotify(self.track)
search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
video = self._best_on_youtube_search_for_type_query(search_query)
return video
def _on_youtube_and_spotify_for_type_spotify(self):
logger.debug("Extracting YouTube and Spotify metadata for input Spotify URI.")
spotify_metadata = self._on_spotify_for_type_spotify(self.track)
search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
youtube_video = self._best_on_youtube_search_for_type_spotify(search_query)
youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
metadata = spotdl.util.merge_copy(
youtube_metadata,
spotify_metadata
)
return metadata
def _on_youtube_and_spotify_for_type_youtube(self):
logger.debug("Extracting YouTube and Spotify metadata for input YouTube URL.")
youtube_metadata = self._on_youtube_for_type_youtube(self.track)
search_query = spotdl.metadata.format_string("{track-name}", youtube_metadata)
spotify_metadata = self._on_spotify_for_type_query(search_query)
metadata = spotdl.util.merge_copy(
youtube_metadata,
spotify_metadata
)
return metadata
def _on_youtube_and_spotify_for_type_query(self):
logger.debug("Extracting YouTube and Spotify metadata for input track query.")
search_query = self.track
# Make use of threads here to search on both YouTube & Spotify
# at the same time.
spotify_metadata = spotdl.util.ThreadWithReturnValue(
target=self._on_spotify_for_type_query,
args=(search_query,)
)
spotify_metadata.start()
youtube_metadata = self._on_youtube_for_type_query(search_query)
metadata = spotdl.util.merge_copy(
youtube_metadata,
spotify_metadata.join()
)
return metadata
def _on_youtube_for_type_spotify(self):
logger.debug("Extracting YouTube metadata for input Spotify URI.")
spotify_metadata = self._on_spotify_for_type_spotify(self.track)
search_query = spotdl.metadata.format_string(self.yt_search_format, spotify_metadata)
youtube_video = self._best_on_youtube_search_for_type_spotify(search_query)
youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
return youtube_metadata
def _on_youtube_for_type_youtube(self, url):
logger.debug("Extracting YouTube metadata for input YouTube URL.")
youtube_metadata = self.providers["youtube"].from_url(url)
return youtube_metadata
def _on_youtube_for_type_query(self, query):
logger.debug("Extracting YouTube metadata for input track query.")
youtube_video = self._best_on_youtube_search_for_type_query(query)
youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
return youtube_metadata
def _on_spotify_for_type_youtube(self, url):
logger.debug("Extracting Spotify metadata for input YouTube URL.")
youtube_metadata = self.providers["youtube"].from_url(url)
search_query = spotdl.metadata.format_string("{track-name}", youtube_metadata)
spotify_metadata = self.providers["spotify"].from_query(search_query)
return spotify_metadata
def _on_spotify_for_type_spotify(self, url):
logger.debug("Extracting Spotify metadata for input Spotify URI.")
spotify_metadata = self.providers["spotify"].from_url(url)
return spotify_metadata
def _on_spotify_for_type_query(self, query):
logger.debug("Extracting Spotify metadata for input track query.")
try:
spotify_metadata = self.providers["spotify"].from_query(query)
except SpotifyMetadataNotFoundError as e:
logger.warn(e.args[0])
spotify_metadata = {}
return spotify_metadata

71
spotdl/tests/test_config.py
View File

@@ -1,71 +0,0 @@
import spotdl.config
import argparse
import os
import sys
import yaml
import pytest
@pytest.mark.xfail
@pytest.fixture(scope="module")
def config_path(tmpdir_factory):
config_path = os.path.join(str(tmpdir_factory.mktemp("config")), "config.yml")
return config_path
@pytest.mark.xfail
@pytest.fixture(scope="module")
def modified_config():
modified_config = dict(spotdl.config.DEFAULT_CONFIGURATION)
return modified_config
def test_dump_n_read_config(config_path):
expect_config = spotdl.config.DEFAULT_CONFIGURATION
spotdl.config.dump_config(
config_path,
config=expect_config,
)
config = spotdl.config.read_config(config_path)
assert config == expect_config
class TestDefaultConfigFile:
@pytest.mark.skipif(not sys.platform == "linux", reason="Linux only")
def test_linux_default_config_file(self):
expect_default_config_file = os.path.expanduser("~/.config/spotdl/config.yml")
assert spotdl.config.DEFAULT_CONFIG_FILE == expect_default_config_file
@pytest.mark.xfail
@pytest.mark.skipif(not sys.platform == "darwin" and not sys.platform == "win32",
reason="Windows only")
def test_windows_default_config_file(self):
raise NotImplementedError
@pytest.mark.xfail
@pytest.mark.skipif(not sys.platform == "darwin",
reason="OS X only")
def test_osx_default_config_file(self):
raise NotImplementedError
class TestConfig:
@pytest.mark.xfail
def test_custom_config_path(self, config_path, modified_config):
parser = argparse.ArgumentParser()
with open(config_path, "w") as config_file:
yaml.dump(modified_config, config_file, default_flow_style=False)
overridden_config = spotdl.config.override_config(
config_path, parser, raw_args=""
)
modified_values = [
str(value)
for value in modified_config["spotify-downloader"].values()
]
overridden_config.folder = os.path.realpath(overridden_config.folder)
overridden_values = [
str(value) for value in overridden_config.__dict__.values()
]
assert sorted(overridden_values) == sorted(modified_values)

60
spotdl/tests/test_util.py
View File

@@ -1,60 +0,0 @@
import sys
import os
import subprocess
import spotdl.util
import pytest
@pytest.fixture(scope="module")
def directory_fixture(tmpdir_factory):
dir_path = os.path.join(str(tmpdir_factory.mktemp("tmpdir")), "filter_this_directory")
return dir_path
@pytest.mark.parametrize("str_duration, sec_duration", [
("0:23", 23),
("0:45", 45),
("2:19", 139),
("3:33", 213),
("7:38", 458),
("1:30:05", 5405),
])
def test_get_seconds_from_video_time(str_duration, sec_duration):
secs = spotdl.util.get_sec(str_duration)
assert secs == sec_duration
@pytest.mark.parametrize("duplicates, expected", [
(("https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
"https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",),
( "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",),),
(("https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
"",
"https://open.spotify.com/track/3SipFlNddvL0XNZRLXvdZD",),
( "https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
"https://open.spotify.com/track/3SipFlNddvL0XNZRLXvdZD",),),
(("ncs fade",
"https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
"",
"ncs fade",),
("ncs fade",
"https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ"),),
(("ncs spectre ",
" https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ",
""),
( "ncs spectre",
"https://open.spotify.com/track/2DGa7iaidT5s0qnINlwMjJ"),),
])
def test_remove_duplicates(duplicates, expected):
uniques = spotdl.util.remove_duplicates(
duplicates,
condition=lambda x: x,
operation=str.strip,
)
assert tuple(uniques) == expected

111
spotdl/track.py
View File

@@ -1,111 +0,0 @@
import tqdm
import urllib.request
import subprocess
import sys
from spotdl.encode.encoders import EncoderFFmpeg
from spotdl.metadata.embedders import EmbedderDefault
import spotdl.util
CHUNK_SIZE = 16 * 1024
class Track:
def __init__(self, metadata, cache_albumart=False):
self.metadata = metadata
self._chunksize = CHUNK_SIZE
if cache_albumart:
self._albumart_thread = self._cache_albumart()
self._cache_albumart = cache_albumart
def _cache_albumart(self):
albumart_thread = spotdl.util.ThreadWithReturnValue(
target=lambda url: urllib.request.urlopen(url).read(),
args=(self.metadata["album"]["images"][0]["url"],)
)
albumart_thread.start()
return albumart_thread
def calculate_total_chunks(self, filesize):
return (filesize // self._chunksize) + 1
def make_progress_bar(self, total_chunks):
progress_bar = tqdm.trange(
total_chunks,
unit_scale=(self._chunksize // 1024),
unit="KiB",
dynamic_ncols=True,
bar_format='{desc}: {percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt}KiB '
'[{elapsed}<{remaining}, {rate_fmt}{postfix}]',
)
return progress_bar
def download_while_re_encoding(self, stream, target_path, target_encoding=None,
encoder=EncoderFFmpeg(), show_progress=True):
total_chunks = self.calculate_total_chunks(stream["filesize"])
process = encoder.re_encode_from_stdin(
stream["encoding"],
target_path,
target_encoding=target_encoding
)
response = stream["connection"]
progress_bar = self.make_progress_bar(total_chunks)
for _ in progress_bar:
chunk = response.read(self._chunksize)
process.stdin.write(chunk)
process.stdin.close()
process.wait()
def download(self, stream, target_path, show_progress=True):
total_chunks = self.calculate_total_chunks(stream["filesize"])
progress_bar = self.make_progress_bar(total_chunks)
response = stream["connection"]
def writer(response, progress_bar, file_io):
for _ in progress_bar:
chunk = response.read(self._chunksize)
file_io.write(chunk)
write_to_stdout = target_path == "-"
if write_to_stdout:
file_io = sys.stdout.buffer
writer(response, progress_bar, file_io)
else:
with open(target_path, "wb") as file_io:
writer(response, progress_bar, file_io)
def re_encode(self, input_path, target_path, target_encoding=None,
encoder=EncoderFFmpeg(), show_progress=True):
stream = self.metadata["streams"].getbest()
total_chunks = self.calculate_total_chunks(stream["filesize"])
process = encoder.re_encode_from_stdin(
stream["encoding"],
target_path,
target_encoding=target_encoding
)
with open(input_path, "rb") as fin:
for _ in tqdm.trange(total_chunks):
chunk = fin.read(self._chunksize)
process.stdin.write(chunk)
process.stdin.close()
process.wait()
def apply_metadata(self, input_path, encoding=None, embedder=EmbedderDefault()):
if self._cache_albumart:
albumart = self._albumart_thread.join()
else:
albumart = None
embedder.apply_metadata(
input_path,
self.metadata,
cached_albumart=albumart,
encoding=encoding,
)

167
spotdl/util.py
View File

@@ -1,167 +0,0 @@
import os
import sys
import math
import urllib.request
import threading
import logging
logger = logging.getLogger(__name__)
try:
import winreg
except ImportError:
pass
try:
from slugify import SLUG_OK, slugify
except ImportError:
logger.error("Oops! `unicode-slugify` was not found.")
logger.info("Please remove any other slugify library and install `unicode-slugify`")
raise
# This has been referred from
# https://stackoverflow.com/a/6894023/6554943
# It's because threaded functions do not return by default
# Whereas this will return the value when `join` method
# is called.
class ThreadWithReturnValue(threading.Thread):
def __init__(self, target=lambda: None, args=()):
super().__init__(target=target, args=args)
self._return = None
def run(self):
if self._target is not None:
self._return = self._target(
*self._args,
**self._kwargs
)
def join(self, *args, **kwargs):
super().join(*args, **kwargs)
return self._return
def merge_copy(base, overrider):
return merge(base.copy(), overrider)
def merge(base, overrider):
""" Override base dict with an overrider dict, recursively. """
for key, value in overrider.items():
if isinstance(value, dict):
subitem = base.setdefault(key, {})
merge(subitem, value)
else:
base[key] = value
return base
def prompt_user_for_selection(items):
""" Let the user input a choice. """
logger.info("Enter a number:")
while True:
try:
the_chosen_one = int(input("> "))
if 1 <= the_chosen_one <= len(items):
return items[the_chosen_one - 1]
elif the_chosen_one == 0:
return None
else:
logger.warning("Choose a valid number!")
except ValueError:
logger.warning("Choose a valid number!")
def is_spotify(track):
""" Check if the input song is a Spotify link. """
status = len(track) == 22 and track.replace(" ", "%20") == track
status = status or track.find("spotify") > -1
return status
def is_youtube(track):
""" Check if the input song is a YouTube link. """
status = len(track) == 11 and track.replace(" ", "%20") == track
status = status and not track.lower() == track
status = status or "youtube.com/watch?v=" in track
return status
def track_type(track):
track_types = {
"spotify": is_spotify,
"youtube": is_youtube,
}
for provider, fn in track_types.items():
if fn(track):
return provider
return "query"
def sanitize(string, ok="&-_()[]{}", spaces_to_underscores=False):
""" Generate filename of the song to be downloaded. """
if spaces_to_underscores:
string = string.replace(" ", "_")
# replace slashes with "-" to avoid directory creation errors
string = string.replace("/", "-").replace("\\", "-")
# slugify removes any special characters
string = slugify(string, ok=ok, lower=False, spaces=True)
return string
def get_sec(time_str):
if ":" in time_str:
splitter = ":"
elif "." in time_str:
splitter = "."
else:
raise ValueError(
"No expected character found in {} to split" "time values.".format(time_str)
)
v = time_str.split(splitter, 3)
v.reverse()
sec = 0
if len(v) > 0: # seconds
sec += int(v[0])
if len(v) > 1: # minutes
sec += int(v[1]) * 60
if len(v) > 2: # hours
sec += int(v[2]) * 3600
return sec
def remove_duplicates(elements, condition=lambda _: True, operation=lambda x: x):
"""
Removes duplicates from a list whilst preserving order.
We could directly call `set()` on the list but it changes
the order of elements.
"""
local_set = set()
local_set_add = local_set.add
filtered_list = []
for x in elements:
if condition(x) and not (x in local_set or local_set_add(x)):
operated = operation(x)
filtered_list.append(operated)
local_set_add(operated)
return filtered_list
def titlecase(string):
return " ".join(word.capitalize() for word in string.split())
def readlines_from_nonbinary_file(path):
with open(path, "r") as fin:
lines = fin.read().splitlines()
return lines
def writelines_to_nonbinary_file(path, lines):
with open(path, "w") as fout:
fout.writelines(map(lambda x: x + "\n", lines))

2
spotdl/version.py
View File

@@ -1,2 +0,0 @@
__version__ = "2.0.3"

72
test/test_simple.py Normal file
View File

@@ -0,0 +1,72 @@
# -*- coding: UTF-8 -*-
import spotdl
import os
raw_song = "Tony's Videos VERY SHORT VIDEO 28.10.2016"
for x in os.listdir(spotdl.args.folder):
os.remove(os.path.join(spotdl.args.folder, x))
def test_youtube_url():
expect_url = 'youtube.com/watch?v=qOOcy2-tmbk'
url = spotdl.generate_youtube_url(raw_song)
assert url == expect_url
def test_youtube_title():
expect_title = "Tony's Videos VERY SHORT VIDEO 28.10.2016"
global content
content = spotdl.go_pafy(raw_song)
global title
title = spotdl.get_youtube_title(content)
assert title == expect_title
def test_check_exists():
expect_check = False
# prerequisites for determining filename
file_name = spotdl.misc.sanitize_title(title)
check = spotdl.check_exists(file_name, raw_song, islist=True)
assert check == expect_check
def test_download():
expect_download = True
# prerequisites for determining filename
file_name = spotdl.misc.sanitize_title(title)
download = spotdl.download_song(file_name, content)
assert download == expect_download
def test_convert():
# exit code 0 = success
expect_convert = 0
# prerequisites for determining filename
file_name = spotdl.misc.sanitize_title(title)
input_song = file_name + spotdl.args.input_ext
output_song = file_name + spotdl.args.output_ext
convert = spotdl.convert.song(input_song, output_song, spotdl.args.folder)
assert convert == expect_convert
def test_metadata():
expect_metadata = None
# prerequisites for determining filename
meta_tags = spotdl.generate_metadata(raw_song)
meta_tags = spotdl.generate_metadata(raw_song)
file_name = spotdl.misc.sanitize_title(title)
output_song = file_name + spotdl.args.output_ext
metadata_output = spotdl.metadata.embed(os.path.join(spotdl.args.folder, output_song), meta_tags)
input_song = file_name + spotdl.args.input_ext
metadata_input = spotdl.metadata.embed(os.path.join(spotdl.args.folder, input_song), meta_tags)
assert (metadata_output == expect_metadata) and (metadata_input == expect_metadata)
def test_check_exists2():
expect_check = True
# prerequisites for determining filename
file_name = spotdl.misc.sanitize_title(title)
input_song = file_name + spotdl.args.input_ext
os.remove(os.path.join(spotdl.args.folder, input_song))
check = spotdl.check_exists(file_name, raw_song, islist=True)
assert check == expect_check

77
test/test_spotify.py Normal file
View File

@@ -0,0 +1,77 @@
# -*- coding: UTF-8 -*-
import spotdl
import os
raw_song = 'http://open.spotify.com/track/0JlS7BXXD07hRmevDnbPDU'
for x in os.listdir(spotdl.args.folder):
os.remove(os.path.join(spotdl.args.folder, x))
def test_spotify_title():
expect_title = 'David André Østby - Intro'
global meta_tags
meta_tags = spotdl.generate_metadata(raw_song)
title = spotdl.generate_songname(meta_tags)
assert title == expect_title
def youtube_url():
expect_url = 'youtube.com/watch?v=rg1wfcty0BA'
url = spotdl.generate_youtube_url(raw_song)
assert url == expect_url
def youtube_title():
expect_title = 'Intro - David André Østby'
content = spotdl.go_pafy(raw_song)
title = spotdl.get_youtube_title(content)
assert title == expect_title
def test_check_exists():
expect_check = False
# prerequisites for determining filename
songname = spotdl.generate_songname(meta_tags)
global file_name
file_name = spotdl.misc.sanitize_title(songname)
check = spotdl.check_exists(file_name, raw_song, islist=True)
assert check == expect_check
def test_download():
expect_download = True
# prerequisites for determining filename
content = spotdl.go_pafy(raw_song)
download = spotdl.download_song(file_name, content)
assert download == expect_download
def test_convert():
# exit code 0 = success
expect_convert = 0
# prerequisites for determining filename
input_song = file_name + spotdl.args.input_ext
output_song = file_name + spotdl.args.output_ext
convert = spotdl.convert.song(input_song, output_song, spotdl.args.folder)
assert convert == expect_convert
def test_metadata():
expect_metadata = True
# prerequisites for determining filename
output_song = file_name + spotdl.args.output_ext
metadata_output = spotdl.metadata.embed(os.path.join(spotdl.args.folder, output_song), meta_tags)
input_song = file_name + spotdl.args.input_ext
metadata_input = spotdl.metadata.embed(os.path.join(spotdl.args.folder, input_song), meta_tags)
assert metadata_output == (metadata_input == expect_metadata)
def test_check_exists2():
expect_check = True
# prerequisites for determining filename
input_song = file_name + spotdl.args.input_ext
os.remove(os.path.join(spotdl.args.folder, input_song))
check = spotdl.check_exists(file_name, raw_song, islist=True)
assert check == expect_check

57
test/test_username.py Normal file
View File

@@ -0,0 +1,57 @@
# -*- coding: UTF-8 -*-
import spotdl
username = 'alex'
def test_user():
expect_playlists = 7
playlists = spotdl.spotify.user_playlists(username)
playlists = len(playlists['items'])
assert playlists == expect_playlists
def test_playlist():
expect_tracks = 14
playlist = spotdl.spotify.user_playlists(username)['items'][0]
tracks = playlist['tracks']['total']
assert tracks == expect_tracks
def test_tracks():
playlist = spotdl.spotify.user_playlists(username)['items'][0]
expect_lines = playlist['tracks']['total']
result = spotdl.spotify.user_playlist(
playlist['owner']['id'], playlist['id'], fields='tracks,next')
tracks = result['tracks']
with open('list.txt', 'w') as fout:
while True:
for item in tracks['items']:
track = item['track']
try:
fout.write(track['external_urls']['spotify'] + '\n')
except KeyError:
title = track['name'] + ' by '+ track['artists'][0]['name']
print('Skipping track ' + title + ' (local only?)')
# 1 page = 50 results
# check if there are more pages
if tracks['next']:
tracks = spotify.next(tracks)
else:
break
with open('list.txt', 'r') as listed:
expect_song = (listed.read()).splitlines()[0]
spotdl.misc.trim_song('list.txt')
with open('list.txt', 'a') as myfile:
myfile.write(expect_song)
with open('list.txt', 'r') as listed:
songs = (listed.read()).splitlines()
lines = len(songs)
song = songs[-1]
assert (expect_lines == lines and expect_song == song)