Enable auto-pushing watched info from emby to watcharr and add API keys in watcharr to allow programmatic communication

[ihaddy]

compressed_full_video.mp4

Hello! I'm a software engineer and i stumbled upon your project and loved it. It was missing one core feature for my use-case (that i've seen other users request as well) and that's the ability to "auto-scrobble" (i.e. push info automatically from emby to watcharr when you finish watching something in emby) so i decided to add it! Before i could do that, i had to figure out the authentication since watcharr uses expiring JWTs, so i had to add API key functionality to the app, so this is a large Feature PR! I'll cover what I added and how i tested it and add some screenshots and proposed changes to the documentation so users can make use of this feature.

I've added a lot of inline code comments that can be removed if you want but i figured it'd be useful to explain in line why i did what i did since this is a public collaborative project.

This will likely work with jellyfin too, but i haven't tested it. I do not use plex so it almost certainly won't work with plex, but with minimal tweaking it likely could.

Also, I only explicitly tested this with TV shows, it could work without changes for Movies in emby, but i don't have movies in my emby so i was unable to test it, hence i named the api route /api/watched/emby/show just to be safe. If pointing the emby webhook at that route doesn't work with movies out of the box, we could easily have /api/watched/emby/movies with no or minimal changes just to keep them separate in case they ever need to be separated so we don't have to refactor API routes later on and introduce breaking changes for clients

Total New Features and Changes Made:

1. The ability for an emby user to automatically push watched episodes after they're watched in emby to watcharr directly via emby's built in webhook feature.
   1. To get this feature to work, i added a TVDB api key to the server settings page so users can provide their own TVDB API key (required for emby auto-scrobbling as emby provides most all of its metadata with tvdb ids and not tmdb ids even if you configure tmdb to be a metadata provider).
   2. I added (decently) robust logic to disambiguate and convert from the TVDB data source that emby uses and provides to watcharr to the TMDB data source that watcharr uses. My implementation follows other programs like filebot, sonnarr/radarr, etc.
2. Additionally, i also enabled the ability for a user in emby to hit "mark as played" on a group of episodes, an entire season, or entire show, and that same webhook submits to watcharr all the same info in bulk so watcharr can bulk add in shows/seasons/episodes as watched.
   1. Because we can now make large bulk requests to TMDB/TVDB via this feature if a user marks a huge show as played in emby at once, i added configurable rate limiting to cap the outgoing requests from this feature at 10 requests per second. This is configurable in the server settings page. NOTE: this does not drop the requests if they exceed the rate-limit cap, it just staggers them so you can only send up to the cap per second.
   2. Additionally, I defaulted the "rating" of any episode/show sent over via emby automatically to 5 out of 10 before any rating scaling, but this is also configurable in the server settings page.
3. To facilitate programmatic communication with watcharr securely I added API keys to watcharr so not just my feature could work, but other devs could create more integrations into watcharr without having to get a JWT and code logic to refresh their JWT when it expires!
   1. The API keys scoped to an individual user so i added them to the GUI in the "profile" page and are stored in the database.
   2. These currently a user can create 5 API keys and they can be revoked at any time.
   3. Route authentication logic allows for a user to hit a route with a user-specific API key or the existing JWT to remain backwards compatibility across the app.

Testing That I Performed To Ensure it Works

1. I've tested the entire process by building a new docker image and pushing over my previous latest tag docker image to ensure the DB was migrated successfully and the functionality works when users upgrade from an existing instance
2. The only database addition was adding a new table for API keys, so this should not require any migrations as GORM will handle it, and i've tested by deploying my code in a new docker image overtop of my existing DB and it works.
3. The rate limiting feature works as expected. Marking a show with 100 episodes in emby with the rate limit set to 10 requests per second works as expected and the app does not exceed that rate-limit.
4. The API keys feature works as expected.
   1. There is no data leakage between different users different API keys, i.e. User 1 using their own api key cannot access User 2's data, and vice versa. They are properly scoped per user.
   2. API keys that are revoked immediately do not work and are not accepted by the application.
   3. Users can have up to 5 api keys at once and using API key 1 properly updates the "last-used" field in the database for API key 1 as opposed to API key 2-5.
   4. API keys can be added as query-params to the emby route correctly.
   5. API keys are generated to be URL safe so they do not use any illegal URL characters and can be plugged directly into a URL as a query paramater without any issues.
5. TVDB API keys are stored and used properly
6. The configurable rating and rate-limiting server config setting i added are working properly when updated in the GUI and have working default values.

Packages added

1. DayJS was added to the Frontend for properly formatted datetimestamps on API key creation,last-used and revocation timestamps.
2. For the backend, the go package golang.org/x/time/rate was added to implement the rate-limiting feature.

ScreenShots of UI changes

In the profile section you can see the ability to create API keys and list existing API keys. clicking the create API key in step 2 shows you the pop up with the newly generated API key that is displayed once in plaintext so the user can copy it and then hidden. When stored in the database it is irreversibly hashed. If the user loses this API key it's recommended to just revoke it and create another.

In the server settings page, I have added inputs for the 3 new configurable server settings with descriptions

the API rate-limit and the default emby rating are both numeric inputs that are limited to numbers and have appropriate validation. The user cannot input a number higher than 10 on the Default Emby rating.

Documentation

Attached at the top of the PR is a video walk through of how this feature is used in practice, I can also contribute more picture/text/video guides to the official watcharr documentation on how to configure this feature from scratch without any assumptions but i just wanted to get this PR approved first before doing that!

[ihaddy]

@IRHM Not sure if it github actually alerted you to the PR since i can't suggest reviewers so apologies if i'm double pinging you, at any rate, whenever you have time, please let me know your thoughts!