V15: Hybrid Caching (#16938)

* Update to dotnet 9 and update nuget packages * Update umbraco code version * Update Directory.Build.props Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com> * Include preview version in pipeline * update template projects * update global json with specific version * Update version.json to v15 * Rename TrimStart and TrimEnd to string specific * Rename to Exact * Update global.json Co-authored-by: Ronald Barendse <ronald@barend.se> * Remove includePreviewVersion * Rename to trim exact * Add new Hybridcache project * Add tests * Start implementing PublishedContent.cs * Implement repository for content * Refactor to use async everywhere * Add cache refresher * make public as needed for serialization * Use content type cache to get content type out * Refactor to use ContentCacheNode model, that goes in the memory cache * Remove content node kit as its not needed * Implement tests for ensuring caching * Implement better asserts * Implement published property * Refactor to use mapping * Rename to document tests * Update to test properties * Create more tests * Refactor mock tests into own file * Update property test * Fix published version of content * Change default cache level to elements * Refactor to always have draft * Refactor to not use PublishedModelFactory * Added tests * Added and updated tests * Fixed tests * Don't return empty object with id * More tests * Added key * Another key * Refactor CacheService to be responsible for using the hybrid cache * Use notification handler to remove deleted content from cache * Add more tests for missing functions * Implement missing methods * Remove HasContent as it pertains to routing * Fik up test * formatting * refactor variable names * Implement variant tests * Map all the published content properties * Get item out of cache first, to assert updated * Implement member cache * Add member test * Implement media cache * Implement property tests for media tests * Refactor tests to use extension method * Add more media tests * Refactor properties to no longer have element caching * Don't use property cache level * Start implementing seeding * Only seed when main * Add Immutable for performance * Implement permanent seeding of content * Implement cache settings * Implement tests for seeding * Update package version * start refactoring nurepo * Refactor so draft & published nodes are cached individually * Refactor RefreshContent to take node instead of IContent * Refactor media to also use cache nodes * Remove member from repo as it isn't cached * Refactor media to not include preview, as media has no draft * create new benchmark project * POC Integration benchmarks with custom api controllers * Start implementing content picker tests * Implement domain cache * Rework content cache to implement interface * Start implementing elements cache * Implement published snapshot service * Publish snapshot tests * Use snapshot for elements cache * Create test proving we don't clear cache when updating content picker * Clear entire elements cache * Remove properties from element cache, when content gets updated. * Rename methods to async * Refactor to use old cache interfaces instead of new ones * Remove snapshot, as it is no longer needed * Fix tests building * Refactor domaincache to not have snapshots * Delete benchmarks * Delete benchmarks * Add HybridCacheProject to Umbraco * Add comment to route value transformer * Implement is draft * remove snapshot from property * V15 updated the hybrid caching integration tests to use ContentEditingService (#16947) * Added builder extension withParentKey * Created builder with ContentEditingService * Added usage of the ContentEditingService to SETUP * Started using ContentEditingService builder in tests * Updated builder extensions * Fixed builder * Clean up * Clean up, not done * Added Ids * Remove entries from cache on delete * Fix up seeding logic * Don't register hybrid cache twice * Change seeded entry options * Update hybrid cache package * Fix up published property to work with delivery api again * Fix dependency injection to work with tests * Fix naming * Dont make caches nullable * Make content node sealed * Remove path and other unused from content node * Remove hacky 2 phase ctor * Refactor to actually set content templates * Remove umbraco context * Remove "HasBy" methods * rename property data * Delete obsolete legacy stuff * Add todo for making expiration configurable * Add todo in UmbracoContext * Add clarifying comment in content factory * Remove xml stuff from published property * Fix according to review * Make content type cache injectible * Make content type cache injectible * Rename to database cache repository * Rename to document cache * Add TODO * Refactor to async * Rename to async * Make everything async * Remove duplicate line from json schema * Move Hybrid cache project * Remove leftover file * Refactor to use keys * Refactor published content to no longer have content data, as it is on the node itself * Refactor to member to use proper content node ctor * Move tests to own folder * Add immutable objects to property and content data for performance * Make property data public * Fix member caching to be singleton * Obsolete GetContentType * Remove todo * Fix naming * Fix lots of exposed errors due to scope test * Add final scope tests * Rename to document cache service * Rename test files * Create new doc type tests * Add ignore to tests * Start implementing refresh for content type save * Clear contenttype cache when contenttype is updated * Fix test Teh contenttype is not upated unless the property is dirty * Use init for ContentSourceDto * Fix get by key in PublishedContentTypeCache * Remove ContentType from PublishedContentTypeCache when contenttype is deleted * Update to preview 7 * Fix versions * Increase timeout for sqlite integration tests * Undo timeout increase * Try and undo init change to ContentSourceDto * That wasn't it chief * Try and make DomainAndUrlsTests non NonParallelizable * Update versions * Only run cache tests on linux for now --------- Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com> Co-authored-by: Ronald Barendse <ronald@barend.se> Co-authored-by: Andreas Zerbst <andr317c@live.dk> Co-authored-by: Sven Geusens <sge@umbraco.dk> Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com> Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-10 00:49:18 +09:00
parent dcd6f1fbf4
commit 2704d4a34a
102 changed files with 5881 additions and 132 deletions
--- a/src/Umbraco.PublishedCache.HybridCache/NotificationHandlers/CacheRefreshingNotificationHandler.cs
+++ b/src/Umbraco.PublishedCache.HybridCache/NotificationHandlers/CacheRefreshingNotificationHandler.cs
@@ -0,0 +1,124 @@
+using Umbraco.Cms.Core.Events;
+using Umbraco.Cms.Core.Models;
+using Umbraco.Cms.Core.Models.Entities;
+using Umbraco.Cms.Core.Models.PublishedContent;
+using Umbraco.Cms.Core.Notifications;
+using Umbraco.Cms.Core.PropertyEditors;
+using Umbraco.Cms.Core.PublishedCache;
+using Umbraco.Cms.Core.Services;
+using Umbraco.Cms.Core.Services.Changes;
+using Umbraco.Cms.Infrastructure.HybridCache.Services;
+using Umbraco.Extensions;
+
+namespace Umbraco.Cms.Infrastructure.HybridCache.NotificationHandlers;
+
+internal sealed class CacheRefreshingNotificationHandler :
+    INotificationAsyncHandler<ContentRefreshNotification>,
+    INotificationAsyncHandler<ContentDeletedNotification>,
+    INotificationAsyncHandler<MediaRefreshNotification>,
+    INotificationAsyncHandler<MediaDeletedNotification>,
+    INotificationAsyncHandler<ContentTypeRefreshedNotification>
+{
+    private readonly IDocumentCacheService _documentCacheService;
+    private readonly IMediaCacheService _mediaCacheService;
+    private readonly IElementsCache _elementsCache;
+    private readonly IRelationService _relationService;
+    private readonly IPublishedContentTypeCache _publishedContentTypeCache;
+
+    public CacheRefreshingNotificationHandler(
+        IDocumentCacheService documentCacheService,
+        IMediaCacheService mediaCacheService,
+        IElementsCache elementsCache,
+        IRelationService relationService,
+        IPublishedContentTypeCache publishedContentTypeCache)
+    {
+        _documentCacheService = documentCacheService;
+        _mediaCacheService = mediaCacheService;
+        _elementsCache = elementsCache;
+        _relationService = relationService;
+        _publishedContentTypeCache = publishedContentTypeCache;
+    }
+
+    public async Task HandleAsync(ContentRefreshNotification notification, CancellationToken cancellationToken)
+    {
+        await RefreshElementsCacheAsync(notification.Entity);
+
+        await _documentCacheService.RefreshContentAsync(notification.Entity);
+    }
+
+    public async Task HandleAsync(ContentDeletedNotification notification, CancellationToken cancellationToken)
+    {
+        foreach (IContent deletedEntity in notification.DeletedEntities)
+        {
+            await RefreshElementsCacheAsync(deletedEntity);
+            await _documentCacheService.DeleteItemAsync(deletedEntity.Id);
+        }
+    }
+
+    public async Task HandleAsync(MediaRefreshNotification notification, CancellationToken cancellationToken)
+    {
+        await RefreshElementsCacheAsync(notification.Entity);
+        await _mediaCacheService.RefreshMediaAsync(notification.Entity);
+    }
+
+    public async Task HandleAsync(MediaDeletedNotification notification, CancellationToken cancellationToken)
+    {
+        foreach (IMedia deletedEntity in notification.DeletedEntities)
+        {
+            await RefreshElementsCacheAsync(deletedEntity);
+            await _mediaCacheService.DeleteItemAsync(deletedEntity.Id);
+        }
+    }
+
+    private async Task RefreshElementsCacheAsync(IUmbracoEntity content)
+    {
+        IEnumerable<IRelation> parentRelations = _relationService.GetByParent(content)!;
+        IEnumerable<IRelation> childRelations = _relationService.GetByChild(content);
+
+        var ids = parentRelations.Select(x => x.ChildId).Concat(childRelations.Select(x => x.ParentId)).ToHashSet();
+        foreach (var id in ids)
+        {
+            if (await _documentCacheService.HasContentByIdAsync(id) is false)
+            {
+                continue;
+            }
+
+            IPublishedContent? publishedContent = await _documentCacheService.GetByIdAsync(id);
+            if (publishedContent is null)
+            {
+                continue;
+            }
+
+            foreach (IPublishedProperty publishedProperty in publishedContent.Properties)
+            {
+                var property = (PublishedProperty) publishedProperty;
+                if (property.ReferenceCacheLevel != PropertyCacheLevel.Elements)
+                {
+                    continue;
+                }
+
+                _elementsCache.ClearByKey(property.ValuesCacheKey);
+            }
+        }
+    }
+
+    public Task HandleAsync(ContentTypeRefreshedNotification notification, CancellationToken cancellationToken)
+    {
+        const ContentTypeChangeTypes types // only for those that have been refreshed
+            = ContentTypeChangeTypes.RefreshMain | ContentTypeChangeTypes.RefreshOther | ContentTypeChangeTypes.Remove;
+        var contentTypeIds = notification.Changes.Where(x => x.ChangeTypes.HasTypesAny(types)).Select(x => x.Item.Id)
+            .ToArray();
+
+        if (contentTypeIds.Length != 0)
+        {
+            foreach (var contentTypeId in contentTypeIds)
+            {
+                _publishedContentTypeCache.ClearContentType(contentTypeId);
+            }
+
+            _documentCacheService.Rebuild(contentTypeIds);
+        }
+
+        return Task.CompletedTask;
+    }
+}