AZEN-SGG/Kavita
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Event Widget Update (#1098)

Browse source 
* Took care of some notes in the code

* Fixed an issue where Extra might get flagged as special too early, if in a word like Extraordinary

* Moved Tag cleanup code into Scanner service. Added a SplitQuery to another heavy API. Refactored Scan loop to remove parallelism and use async instead.

* Lots of rework on the codebase to support detailed messages and easier management of message sending. Need to take a break on this work.

* Progress is being made, but slowly. Code is broken in this commit.

* Progress is being made, but slowly. Code is broken in this commit.

* Fixed merge issue

* Fixed unit tests

* CoverUpdate is now hooked into new ProgressEvent structure

* Refactored code to remove custom observables and have everything use standard messages$

* Refactored a ton of instances to NotificationProgressEvent style and tons of the UI to respect that too. UI is still a bit buggy, but wholistically the work is done.

* Working much better. Sometimes events come in too fast. Currently cover update progress doesn't display on UI

* Fixed unit tests

* Removed SignalREvent to minimize internal event types. Updated the UI to use progress bars. Finished SiteThemeService.

* Merged metadata refresh progress events and changed library scan events to merge cleaner in the UI

* Changed RefreshMetadataProgress to CoverUpdateProgress to reflect the event better.

* Theme Cleanup (#1089)

* Fixed e-ink theme not properly applying correctly

* Fixed some seed changes. Changed card checkboxes to use our themed ones

* Fixed recently added carousel not going to recently-added page

* Fixed an issue where no results found would show when searching for a library name

* Cleaned up list a bit, typeahead dropdown still needs work

* Added a TODO to streamline series-card component

* Removed ng-lazyload-image module since we don't use it. We use lazysizes

* Darken card on hover

* Fixing accordion focus style

* ux pass updates

- Fixed typeahead width
- Fixed changelog download buttons
- Fixed a select
- Fixed various input box-shadows
- Fixed all anchors to only have underline on hover
- Added navtab hover and active effects

* more ux pass

- Fixed spacing on theme cards
- Fixed some light theme issues
- Exposed text-muted-color for theme card subtitle color

* UX pass fixes

- Changed back to bright green for primary on dark theme
- Changed fa icon to black on e-ink

* Merged changelog component

* Fixed anchor buttons text decoration

* Changed nav tabs to have a background color instead of open active state

* When user is not authenticated, make sure we set default theme (dark)

* Cleanup on carousel

* Updated Users tab to use small buttons with icons to align with Library tab

* Cleaned up brand to not underline, removed default link underline on hover in dropdown and pill tabs

* Fixed collection detail posters not rendering

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>

* Bump versions by dotnet-bump-version.

* Tweaked some of the emitting code

* Some css, but pretty bad. Robbie please save me

* Removed a todo

* styling update

* Only send filename on FileScanProgress

* Some console.log spam cleanup

* Various updates

* Show events widget activity based on activeEvents

* progress bar color updates

* Code cleanup

Co-authored-by: Robbie Davis <robbie@therobbiedavis.com>
This commit is contained in:
Joseph Milazzo 2022-02-18 18:57:37 -08:00 • committed by GitHub
parent d24620fd15
commit eddbb7ab18
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
49 changed files with 1022 additions and 463 deletions
Download patch file Download diff file Expand all files Collapse all files

44
API/SignalR/EventHub.cs Normal file
View file

@ -0,0 +1,44 @@
using System.Threading.Tasks;
using API.Data;
using API.SignalR.Presence;
using Microsoft.AspNetCore.SignalR;
namespace API.SignalR;
/// <summary>
/// Responsible for ushering events to the UI and allowing simple DI hook to send data
/// </summary>
public interface IEventHub
{
Task SendMessageAsync(string method, SignalRMessage message, bool onlyAdmins = true);
}
public class EventHub : IEventHub
{
private readonly IHubContext<MessageHub> _messageHub;
private readonly IPresenceTracker _presenceTracker;
private readonly IUnitOfWork _unitOfWork;
public EventHub(IHubContext<MessageHub> messageHub, IPresenceTracker presenceTracker, IUnitOfWork unitOfWork)
{
_messageHub = messageHub;
_presenceTracker = presenceTracker;
_unitOfWork = unitOfWork;
// TODO: When sending a message, queue the message up and on re-connect, reply the queued messages. Queue messages expire on a rolling basis (rolling array)
}
public async Task SendMessageAsync(string method, SignalRMessage message, bool onlyAdmins = true)
{
// TODO: If libraryId and NOT onlyAdmins, then perform RBS check before sending the event
var users = _messageHub.Clients.All;
if (onlyAdmins)
{
var admins = await _presenceTracker.GetOnlineAdmins();
_messageHub.Clients.Users(admins);
}
await users.SendAsync(method, message);
}
}

231
API/SignalR/MessageFactory.cs
View file

@ -1,16 +1,90 @@
using System;
using System.Diagnostics;
using System.IO;
using System.Threading;
using API.DTOs.Update;
using API.Entities;
namespace API.SignalR
{
public static class MessageFactory
{
/// <summary>
/// An update is available for the Kavita instance
/// </summary>
public const string UpdateAvailable = "UpdateAvailable";
/// <summary>
/// Used to tell when a scan series completes
/// </summary>
public const string ScanSeries = "ScanSeries";
/// <summary>
/// Event sent out during Refresh Metadata for progress tracking
/// </summary>
private const string CoverUpdateProgress = "CoverUpdateProgress";
/// <summary>
/// Series is added to server
/// </summary>
public const string SeriesAdded = "SeriesAdded";
/// <summary>
/// Series is removed from server
/// </summary>
public const string SeriesRemoved = "SeriesRemoved";
/// <summary>
/// When a user is connects/disconnects from server
/// </summary>
public const string OnlineUsers = "OnlineUsers";
/// <summary>
/// When a series is added to a collection
/// </summary>
public const string SeriesAddedToCollection = "SeriesAddedToCollection";
/// <summary>
/// When an error occurs during a scan library task
/// </summary>
public const string ScanLibraryError = "ScanLibraryError";
/// <summary>
/// Event sent out during backing up the database
/// </summary>
private const string BackupDatabaseProgress = "BackupDatabaseProgress";
/// <summary>
/// Event sent out during cleaning up temp and cache folders
/// </summary>
private const string CleanupProgress = "CleanupProgress";
/// <summary>
/// Event sent out during downloading of files
/// </summary>
private const string DownloadProgress = "DownloadProgress";
/// <summary>
/// A cover was updated
/// </summary>
public const string CoverUpdate = "CoverUpdate";
/// <summary>
/// A custom site theme was removed or added
/// </summary>
private const string SiteThemeProgress = "SiteThemeProgress";
/// <summary>
/// A type of event that has progress (determinate or indeterminate).
/// The underlying event will have a name to give details on how to handle.
/// </summary>
public const string NotificationProgress = "NotificationProgress";
/// <summary>
/// Event sent out when Scan Loop is parsing a file
/// </summary>
private const string FileScanProgress = "FileScanProgress";
/// <summary>
/// A generic error that can occur in background processing
/// </summary>
public const string Error = "Error";
/// <summary>
/// When DB updates are occuring during a library/series scan
/// </summary>
private const string ScanProgress = "ScanProgress";
public static SignalRMessage ScanSeriesEvent(int seriesId, string seriesName)
{
return new SignalRMessage()
{
Name = SignalREvents.ScanSeries,
Name = ScanSeries,
Body = new
{
SeriesId = seriesId,
@ -23,7 +97,7 @@ namespace API.SignalR
{
return new SignalRMessage()
{
Name = SignalREvents.SeriesAdded,
Name = SeriesAdded,
Body = new
{
SeriesId = seriesId,
@ -37,7 +111,7 @@ namespace API.SignalR
{
return new SignalRMessage()
{
Name = SignalREvents.SeriesRemoved,
Name = SeriesRemoved,
Body = new
{
SeriesId = seriesId,
@ -47,11 +121,15 @@ namespace API.SignalR
};
}
public static SignalRMessage ScanLibraryProgressEvent(int libraryId, float progress)
public static SignalRMessage CoverUpdateProgressEvent(int libraryId, float progress, string eventType, string subtitle = "")
{
return new SignalRMessage()
{
Name = SignalREvents.ScanLibraryProgress,
Name = CoverUpdateProgress,
Title = "Refreshing Covers",
SubTitle = subtitle,
EventType = eventType,
Progress = ProgressType.Determinate,
Body = new
{
LibraryId = libraryId,
@ -61,37 +139,40 @@ namespace API.SignalR
};
}
public static SignalRMessage RefreshMetadataProgressEvent(int libraryId, float progress)
public static SignalRMessage BackupDatabaseProgressEvent(float progress, string subtitle = "")
{
return new SignalRMessage()
{
Name = SignalREvents.RefreshMetadataProgress,
Body = new
Name = BackupDatabaseProgress,
Title = "Backing up Database",
SubTitle = subtitle,
EventType = progress switch
{
LibraryId = libraryId,
Progress = progress,
EventTime = DateTime.Now
}
};
}
public static SignalRMessage BackupDatabaseProgressEvent(float progress)
{
return new SignalRMessage()
{
Name = SignalREvents.BackupDatabaseProgress,
0f => "started",
1f => "ended",
_ => "updated"
},
Progress = ProgressType.Determinate,
Body = new
{
Progress = progress
}
};
}
public static SignalRMessage CleanupProgressEvent(float progress)
public static SignalRMessage CleanupProgressEvent(float progress, string subtitle = "")
{
return new SignalRMessage()
{
Name = SignalREvents.CleanupProgress,
Name = CleanupProgress,
Title = "Performing Cleanup",
SubTitle = subtitle,
EventType = progress switch
{
0f => "started",
1f => "ended",
_ => "updated"
},
Progress = ProgressType.Determinate,
Body = new
{
Progress = progress
@ -100,21 +181,26 @@ namespace API.SignalR
}
public static SignalRMessage UpdateVersionEvent(UpdateNotificationDto update)
{
return new SignalRMessage
{
Name = SignalREvents.UpdateAvailable,
Name = UpdateAvailable,
Title = "Update Available",
SubTitle = update.UpdateTitle,
EventType = ProgressEventType.Single,
Progress = ProgressType.None,
Body = update
};
}
public static SignalRMessage SeriesAddedToCollection(int tagId, int seriesId)
public static SignalRMessage SeriesAddedToCollectionEvent(int tagId, int seriesId)
{
return new SignalRMessage
{
Name = SignalREvents.UpdateAvailable,
Name = SeriesAddedToCollection,
Progress = ProgressType.None,
EventType = ProgressEventType.Single,
Body = new
{
TagId = tagId,
@ -123,11 +209,15 @@ namespace API.SignalR
};
}
public static SignalRMessage ScanLibraryError(int libraryId)
public static SignalRMessage ScanLibraryErrorEvent(int libraryId, string libraryName)
{
return new SignalRMessage
{
Name = SignalREvents.ScanLibraryError,
Name = ScanLibraryError,
Title = "Error",
SubTitle = $"Error Scanning {libraryName}",
Progress = ProgressType.None,
EventType = ProgressEventType.Single,
Body = new
{
LibraryId = libraryId,
@ -135,11 +225,15 @@ namespace API.SignalR
};
}
public static SignalRMessage DownloadProgressEvent(string username, string downloadName, float progress)
public static SignalRMessage DownloadProgressEvent(string username, string downloadName, float progress, string eventType = "updated")
{
return new SignalRMessage()
{
Name = SignalREvents.DownloadProgress,
Name = DownloadProgress,
Title = $"Downloading {downloadName}",
SubTitle = $"{username} is downloading {downloadName}",
EventType = eventType,
Progress = ProgressType.Determinate,
Body = new
{
UserName = username,
@ -149,11 +243,73 @@ namespace API.SignalR
};
}
/// <summary>
/// Represents a file being scanned by Kavita for processing and grouping
/// </summary>
/// <remarks>Does not have a progress as it's unknown how many files there are. Instead sends -1 to represent indeterminate</remarks>
/// <param name="filename"></param>
/// <param name="libraryName"></param>
/// <param name="eventType"></param>
/// <returns></returns>
public static SignalRMessage FileScanProgressEvent(string filename, string libraryName, string eventType)
{
return new SignalRMessage()
{
Name = FileScanProgress,
Title = $"Scanning {libraryName}",
SubTitle = Path.GetFileName(filename),
EventType = eventType,
Progress = ProgressType.Indeterminate,
Body = new
{
Title = $"Scanning {libraryName}",
Subtitle = filename,
Filename = filename,
EventTime = DateTime.Now,
}
};
}
public static SignalRMessage DbUpdateProgressEvent(Series series, string eventType)
{
// TODO: I want this as a detail of a Scanning Series and we can put more information like Volume or Chapter here
return new SignalRMessage()
{
Name = ScanProgress,
Title = $"Scanning {series.Library.Name}",
SubTitle = series.Name,
EventType = eventType,
Progress = ProgressType.Indeterminate,
Body = new
{
Title = "Updating Series",
SubTitle = series.Name
}
};
}
public static SignalRMessage LibraryScanProgressEvent(string libraryName, string eventType, string seriesName = "")
{
// TODO: I want this as a detail of a Scanning Series and we can put more information like Volume or Chapter here
return new SignalRMessage()
{
Name = ScanProgress,
Title = $"Scanning {libraryName}",
SubTitle = seriesName,
EventType = eventType,
Progress = ProgressType.Indeterminate,
Body = null
};
}
public static SignalRMessage CoverUpdateEvent(int id, string entityType)
{
return new SignalRMessage()
{
Name = SignalREvents.CoverUpdate,
Name = CoverUpdate,
Title = "Updating Cover",
//SubTitle = series.Name, // TODO: Refactor this
Progress = ProgressType.None,
Body = new
{
Id = id,
@ -162,17 +318,18 @@ namespace API.SignalR
};
}
public static SignalRMessage SiteThemeProgressEvent(int themeIteratedCount, int totalThemesToIterate, string themeName, float progress)
public static SignalRMessage SiteThemeProgressEvent(string subtitle, string themeName, string eventType)
{
return new SignalRMessage()
{
Name = SignalREvents.SiteThemeProgress,
Name = SiteThemeProgress,
Title = "Scanning Site Theme",
SubTitle = subtitle,
EventType = eventType,
Progress = ProgressType.Indeterminate,
Body = new
{
TotalUpdates = totalThemesToIterate,
CurrentCount = themeIteratedCount,
ThemeName = themeName,
Progress = progress
}
};
}

6
API/SignalR/MessageHub.cs
View file

@ -1,6 +1,7 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using API.Data;
using API.Extensions;
using API.SignalR.Presence;
using Microsoft.AspNetCore.Authorization;
@ -36,6 +37,7 @@ namespace API.SignalR
public override async Task OnConnectedAsync()
{
lock (Connections)
{
Connections.Add(Context.ConnectionId);
@ -44,7 +46,7 @@ namespace API.SignalR
await _tracker.UserConnected(Context.User.GetUsername(), Context.ConnectionId);
var currentUsers = await PresenceTracker.GetOnlineUsers();
await Clients.All.SendAsync(SignalREvents.OnlineUsers, currentUsers);
await Clients.All.SendAsync(MessageFactory.OnlineUsers, currentUsers);
await base.OnConnectedAsync();
@ -60,7 +62,7 @@ namespace API.SignalR
await _tracker.UserDisconnected(Context.User.GetUsername(), Context.ConnectionId);
var currentUsers = await PresenceTracker.GetOnlineUsers();
await Clients.All.SendAsync(SignalREvents.OnlineUsers, currentUsers);
await Clients.All.SendAsync(MessageFactory.OnlineUsers, currentUsers);
await base.OnDisconnectedAsync(exception);

34
API/SignalR/Presence/PresenceTracker.cs
View file

@ -15,13 +15,20 @@ namespace API.SignalR.Presence
}
internal class ConnectionDetail
{
public List<string> ConnectionIds { get; set; }
public bool IsAdmin { get; set; }
}
// TODO: This can respond to UserRoleUpdate events to handle online users
/// <summary>
/// This is a singleton service for tracking what users have a SignalR connection and their difference connectionIds
/// </summary>
public class PresenceTracker : IPresenceTracker
{
private readonly IUnitOfWork _unitOfWork;
private static readonly Dictionary<string, List<string>> OnlineUsers = new Dictionary<string, List<string>>();
private static readonly Dictionary<string, ConnectionDetail> OnlineUsers = new Dictionary<string, ConnectionDetail>();
public PresenceTracker(IUnitOfWork unitOfWork)
{
@ -30,20 +37,25 @@ namespace API.SignalR.Presence
public async Task UserConnected(string username, string connectionId)
{
var user = await _unitOfWork.UserRepository.GetUserByUsernameAsync(username);
var isAdmin = await _unitOfWork.UserRepository.IsUserAdminAsync(user);
lock (OnlineUsers)
{
if (OnlineUsers.ContainsKey(username))
{
OnlineUsers[username].Add(connectionId);
OnlineUsers[username].ConnectionIds.Add(connectionId);
}
else
{
OnlineUsers.Add(username, new List<string>() { connectionId });
OnlineUsers.Add(username, new ConnectionDetail()
{
ConnectionIds = new List<string>() {connectionId},
IsAdmin = isAdmin
});
}
}
// Update the last active for the user
var user = await _unitOfWork.UserRepository.GetUserByUsernameAsync(username);
user.LastActive = DateTime.Now;
await _unitOfWork.CommitAsync();
}
@ -54,9 +66,9 @@ namespace API.SignalR.Presence
{
if (!OnlineUsers.ContainsKey(username)) return Task.CompletedTask;
OnlineUsers[username].Remove(connectionId);
OnlineUsers[username].ConnectionIds.Remove(connectionId);
if (OnlineUsers[username].Count == 0)
if (OnlineUsers[username].ConnectionIds.Count == 0)
{
OnlineUsers.Remove(username);
}
@ -75,18 +87,16 @@ namespace API.SignalR.Presence
return Task.FromResult(onlineUsers);
}
public async Task<string[]> GetOnlineAdmins()
public Task<string[]> GetOnlineAdmins()
{
string[] onlineUsers;
lock (OnlineUsers)
{
onlineUsers = OnlineUsers.OrderBy(k => k.Key).Select(k => k.Key).ToArray();
onlineUsers = OnlineUsers.Where(pair => pair.Value.IsAdmin).OrderBy(k => k.Key).Select(k => k.Key).ToArray();
}
var admins = await _unitOfWork.UserRepository.GetAdminUsersAsync();
var result = admins.Select(a => a.UserName).Intersect(onlineUsers).ToArray();
return result;
return Task.FromResult(onlineUsers);
}
public Task<List<string>> GetConnectionsForUser(string username)
@ -94,7 +104,7 @@ namespace API.SignalR.Presence
List<string> connectionIds;
lock (OnlineUsers)
{
connectionIds = OnlineUsers.GetValueOrDefault(username);
connectionIds = OnlineUsers.GetValueOrDefault(username)?.ConnectionIds;
}
return Task.FromResult(connectionIds);

17
API/SignalR/ProgressEventType.cs Normal file
View file

@ -0,0 +1,17 @@
namespace API.SignalR;
public static class ProgressEventType
{
public const string Started = "started";
public const string Updated = "updated";
/// <summary>
/// End of the update chain
/// </summary>
public const string Ended = "ended";
/// <summary>
/// Represents a single update
/// </summary>
public const string Single = "started";
}

21
API/SignalR/ProgressType.cs Normal file
View file

@ -0,0 +1,21 @@
namespace API.SignalR;
/// <summary>
/// How progress should be represented on the UI
/// </summary>
public static class ProgressType
{
/// <summary>
/// Progress scales from 0F -> 1F
/// </summary>
public const string Determinate = "determinate";
/// <summary>
/// Progress has no understanding of quantity
/// </summary>
public const string Indeterminate = "indeterminate";
/// <summary>
/// No progress component to the event
/// </summary>
public const string None = "";
}

63
API/SignalR/SignalREvents.cs
View file

@ -1,63 +0,0 @@
namespace API.SignalR
{
public static class SignalREvents
{
/// <summary>
/// An update is available for the Kavita instance
/// </summary>
public const string UpdateAvailable = "UpdateAvailable";
/// <summary>
/// Used to tell when a scan series completes
/// </summary>
public const string ScanSeries = "ScanSeries";
/// <summary>
/// Event sent out during Refresh Metadata for progress tracking
/// </summary>
public const string RefreshMetadataProgress = "RefreshMetadataProgress";
/// <summary>
/// Series is added to server
/// </summary>
public const string SeriesAdded = "SeriesAdded";
/// <summary>
/// Series is removed from server
/// </summary>
public const string SeriesRemoved = "SeriesRemoved";
/// <summary>
/// Progress event for Scan library
/// </summary>
public const string ScanLibraryProgress = "ScanLibraryProgress";
/// <summary>
/// When a user is connects/disconnects from server
/// </summary>
public const string OnlineUsers = "OnlineUsers";
/// <summary>
/// When a series is added to a collection
/// </summary>
public const string SeriesAddedToCollection = "SeriesAddedToCollection";
/// <summary>
/// When an error occurs during a scan library task
/// </summary>
public const string ScanLibraryError = "ScanLibraryError";
/// <summary>
/// Event sent out during backing up the database
/// </summary>
public const string BackupDatabaseProgress = "BackupDatabaseProgress";
/// <summary>
/// Event sent out during cleaning up temp and cache folders
/// </summary>
public const string CleanupProgress = "CleanupProgress";
/// <summary>
/// Event sent out during downloading of files
/// </summary>
public const string DownloadProgress = "DownloadProgress";
/// <summary>
/// A cover was updated
/// </summary>
public const string CoverUpdate = "CoverUpdate";
/// <summary>
/// A custom site theme was removed or added
/// </summary>
public const string SiteThemeProgress = "SiteThemeProgress";
}
}

33
API/SignalR/SignalRMessage.cs
View file

@ -1,14 +1,39 @@
namespace API.SignalR
using System;
namespace API.SignalR
{
/// <summary>
/// Payload for SignalR messages to Frontend
/// </summary>
public class SignalRMessage
{
/// <summary>
/// Body of the event type
/// </summary>
public object Body { get; set; }
public string Name { get; set; }
//[JsonIgnore]
//public ModelAction Action { get; set; } // This will be for when we add new flows
/// <summary>
/// User friendly Title of the Event
/// </summary>
/// <example>Scanning Manga</example>
public string Title { get; set; } = string.Empty;
/// <summary>
/// User friendly subtitle. Should have extra info
/// </summary>
/// <example>C:/manga/Accel World V01.cbz</example>
public string SubTitle { get; set; } = string.Empty;
/// <summary>
/// Represents what this represents. started | updated | ended | single
/// <see cref="ProgressEventType"/>
/// </summary>
public string EventType { get; set; } = ProgressEventType.Updated;
/// <summary>
/// How should progress be represented. If Determinate, the Body MUST have a Progress float on it.
/// </summary>
public string Progress { get; set; } = ProgressType.None;
/// <summary>
/// When event took place
/// </summary>
public DateTime EventTime = DateTime.Now;
}
}