Commit Graph

24 Commits

Author SHA1 Message Date
Andy Butland
f0c5ecf086 Add request caching around published content factory (#19990)
* Add request caching around published content factory.

* Fixed ordering of log message parameters.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Invert if to reduce nesting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>
2025-08-27 11:01:44 +00:00
Andy Butland
4efe8f59b8 Optimize document and media seeding by looking up from database in batches (#19890)
* Optimize document and media seeding by looking up from database in batches.

* Ensure null values aren't stored in the cache when checking existance.

* Fixed failing integration tests.

* Resolved issue with not writing to the L1 cache on an L2 hit.

* Tidied up and populated XML header comments.

* Address issue raised in code review.
2025-08-12 11:58:41 +02:00
Kenn Jacobsen
2cf28271cd Service refactoring to "fully" enable segments (#19114)
* Refactor serverside content editing to support all variance combinations

* Fix build errors

* Reintroduce the tests ignored by #19060

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2025-04-23 14:54:51 +02:00
Andy Butland
825f791d01 Remove the non-controversial, straightforward obsoleted constructs for Umbraco 16 (#18661)
* Removed obsoletes from IConfigManipulator.

* Removed obsolete models builder extensions.

* Removed the obsolete ContentDashboardSettings.

* Removed the obsolete InstallMissingDatabase setting on GlobalSettings.

* Removed obsolete NuCache settings.

* Removed obsolete RuntimeMinificationSettings.

* Removed obsolete health check constant.

* Removed obsolete icon constant.

* Removed obsolete telemetry constant.

* Removed obsolete property and constructor on UmbracoBuilder.

* Removed obsolete constructor on AuditNotificationsHandler.

* Removed obsolete constructor on HTTP header health checks.

* Removed obsolete constructor on MediaFileManager.

* Removed obsolete GetDefaultFileContent on ViewHelper.

* Remove obsoleted methods on embed providers.

* Fix tests.

* Removed obsolete constructors on BlockEditorDataConverter.

* Removed obsolete SeedCacheDuration property on CacheSettings.

* Removed obsolete PublishCulture on ContentRepositoryExtensions.

* Removed obsolete MonitorLock.

* Removed obsolete synchronous HasSavedValues from IDataTypeUsageService and IDataTypeUsageRepository.

* Removed obsolete HasSavedPropertyValues from IPropertyTypeUsageService and IPropertyTypeUsageRepository.

* Removed obsolete methods in ITrackedReferencesService and ITrackedReferencesRepository.

* Removed obsolete DateValueEditor constructors.

* Removed obsolete GetAutomaticRelationTypesAliases.

* Removed obsolete constructor on TextOnlyValueEditor.

* Removed obsolete constructors on RegexValidator and RequiredValidator.

* Removed obsolete constructs on SliderValueConverter and TagsValueConverter.

* Removed obsolete GetContentType methods from IPublishedCache.

* Removed ContentFinderByIdPath.

* Removed obsolete constructor on DefaultMediaUrlProvider.

* Removed obsolete constructor on Domain.

* Removed obsolete constructor on PublishedRequest.

* Removed obsolete methods on CheckPermissions.

* Removed obsolete GetUserId from IBackOfficeSecurity.

* Removed obsolete methods on LegacyPasswordSecurity.

* Removed obsolete constructors on AuditService.

* Removed obsolete methods on IContentEditingService.

* Remove obsolete constructors and methods on ContentService/IContentService.

* Removed obsolete constructor in ContentTypeEditingService.

* Removed obsolete constructor in MediaTypeEditingService.

* Removed obsolete constructor in MemberTypeEditingService.

* Removed obsolete constructor in ContentTypeService.

* Removed obsolete constructors in ContentTypeServiceBase.

* Removed obsolete constructors and methods in ContentVersionService.

* Removed obsolete constructor in DataTypeUsageService.

* Removed obsolete constructor in DomainService.

* Removed obsolete constructor in FileService.

* Removes obsolete AttemptMove from IContentService.

* Removes obsolete SetPreventCleanup from IContentVersionService.

* Removes obsolete GetReferences from IDataTypeService.

* Removed obsolete SetConsentLevel from IMetricsConsentService.

* Removed obsolete methods from IPackageDataInstallation.

* Removed obsolete methods from IPackagingService.

* Removed obsolete methods on ITwoFactorLoginService.
Removed obsolete ITemporaryMediaService.

* Removed obsolete constructor from MediaService, MemberTypeService and MediaTypeService.

* More obsolete constructors.

* Removed obsoleted overloads on IPropertyValidationService.

* Fixed build for tests.

* Removed obsolete constructor for PublicAccessService, UserService and RelationService.

* Removed GetDefaultMemberType.

* Removed obsolete user group functionality from IUserService.

* Removed obsolete extension methods on IUserService.

* Removed obsolete method from ITelemetryService.

* Removed obsolete UdiParserServiceConnectors.

* Removed obsolete method on ICookieManager.

* Removed obsolete DynamicContext.

* Removed obsolete XmlHelper.

* Fixed failing integration tests.

* Removed obsoletes in Umbraco.Cms.Api.Common

* Removed obsoletes in Umbraco.Cms.Api.Delivery

* Removed obsoletes in Umbraco.Cms.Api.Management

* Removed obsoletes in Umbraco.Examine.Lucene

* Removed obsoletes in Umbraco.Infrastructure

* Fix failing delivery API contract integration test.

* Made integration tests internal.

* Removed obsoletes from web projects.

* Fix build.

* Removed Twitter OEmbed provider

* Removed obsolete constructor on PublishedDataType.

* Removed obsolete constructors on PublishedCacheBase.

* Removed the obsolete PropertyEditorTagsExtensions.

* Removed obsoletion properties on configuration response  models (#18697)

* Removed obsolete methods from server-side models.

* Update client-side types and sdk.

* Update client-side files.

* Removed obsoletion of Utf8ToAsciiConverter.ToAsciiString overload. (#18694)

* Removed obsolete method in UserService. (#18710)

* Removed obsoleted group alias keys from being publicly available. (#18682)

* Removed unneceessary ApiVersion attribute.

* Clean-up obsoletions on MemberService (#18703)

* Removed obsoleted method on MemberService, added future obsoletion to interface and updated all callers.

* Removed obsoletion on member service method that's not obsolete on the interface.
2025-03-21 17:02:31 +00:00
Mole
c76d764598 V15: Only cache items if all ancestors are published (#18337)
* Introduce IsDocumentPublishedInAnyCulture

Sometimes we don't care about culture

* Check ancestor path when resolving cache items

* Fix tests

* Rebuild NavigationService

* Only set node if it has a published ancestor path

* Remove branch when unpublished

* Add tests

* Add seed test

* Consider published ancestor path when seeding documents

* Introduce MediaBreadthFirstKeyProviderTests

This is needed since the logic of document and media is no longer the same

* Remove unused services

* Move assert page to helper

* Add variant tests

* Add tests

* Filter keys in ContentTypeSeedKeyProvider

* Fix tests

* Add failing test showing refreshing issue

* Don't blow up if we can't resolve the node from navigation cache

Turns out that this can actually happen :D Should be fine to just return false

* Refactor cache refresher check

* Make NavigationQueryService service protected

* Add comment on how to refactor breadth first key provider

* Refactor if statement
2025-02-17 12:51:33 +01:00
Nikolaj Geisle
1c859e75ca V15: Add custom serializer for hybrid cache (#17727)
* Add custom serializer

* Add migration to rebuild cache

* Rename migration namespace to 15.1

* Also clear media cache

* Remove failed cache items

* Refactor to only use keys for document cache repository

---------

Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-12-06 13:20:57 +01:00
Zeegaan
b8b96f05d1 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	Directory.Build.props
#	version.json
2024-11-14 09:03:28 +01:00
Nikolaj Geisle
a9636d5429 V15: Implement not-implemented methods for media cache (#17524)
* Implement not-implemented methods for media cache

* Fixed test

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-11-13 12:04:27 +01:00
Bjarke Berg
c9fa2a57d7 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
#	src/Umbraco.Core/Extensions/PublishedContentExtensions.cs
#	src/Umbraco.Web.UI.Client/src/packages/tiptap/components/input-tiptap/tiptap-toolbar.element.ts
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/ContentNavigationServiceBaseTests.cs
2024-11-12 12:14:50 +01:00
Bjarke Berg
87aff77054 Explicit cache entry settings and expose cache settings in json schema (#17480)
* Changed the cache entry settings to be explicit and exposed it in the schema file

* commit of tests

* Fix seed options

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
2024-11-11 12:52:11 +01:00
nikolajlauridsen
8f86ac1bd1 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/Services/DocumentUrlService.cs
2024-11-04 12:49:38 +01:00
Bjarke Berg
6417948a58 #17391 (#17413)
Fixes issue with the delivery api can find unpublished content
2024-11-04 12:27:47 +01:00
Mole
d1799ecdd2 V15: Refresh caches on load balanced environments (#17296)
* Move DocumentCacheService

* Add clear all documentws from memory cache

* Fix RedirectTracker

* Implement refresh node/branch/all/delete

* Only update databasecache in RefreshContentAsync

* Fix tests

* Skip blueprints in cache

* Clear caches when contenttype is updated

* Clear cache on data type update

* Refresh media

* Only update memory cache from refreshers

* Fix imports

* Add named options

* Use cache entry settings in media

* Obsolete nucache settings

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-28 15:31:39 +01:00
Bjarke Berg
6e0ab4dd7e Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.PublishedCache.HybridCache/Services/MediaCacheService.cs
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Common/Builders/MediaTypeEditingBuilder.cs
2024-10-10 11:11:08 +02:00
Bjarke Berg
537a657de3 V15: Fix previews (#17181)
* default to preview value from cookie instead of draft.

* Fix tests

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-10-03 15:35:17 +02:00
Sebastiaan Janssen
4090829fe8 Ignore warnings about using obsolete members in Umbraco.Tests.UnitTests.csproj and Umbraco.Core.csproj
LocalFileSystemTemporaryFileRepository: Don't pass null to a method that doesn't accept nullable argument - https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2264
Fix remaining warning marked as error
2024-10-02 14:37:37 +02:00
Jacob Overgaard
8a65f63434 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Integration/Umbraco.PublishedCache.HybridCache/DocumentHybridCacheTemplateTests.cs
#	version.json
2024-10-02 13:39:33 +02:00
Andreas Zerbst
04d71fa354 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:37:50 +02:00
Andreas Zerbst
e15b7f2214 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:36:26 +02:00
Bjarke Berg
668f4e15e3 Fixed issues with publishing in cache and reintroduced tests (#17168)
* Fixed issues with publishing in cache and reintroduced tests

* Fixed issue with republishing and templates
2024-10-02 09:10:18 +02:00
Mole
1258962429 V15: Remove Nucache (#17166)
* Remove nucache reference from Web.Common

* Get tests building-ish

* Move ReservedFieldNamesService to the right project

* Remove IPublishedSnapshotStatus

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Remove IPublishedSnapshot from UmbracoContext

* Begin removing usage of IPublishedSnapshot from PublishedContentExtensions

* Fix PublishedContentExtensions.cs

* Don't use snapshots in delivery media api

* Use IPublishedMediaCache in QueryMediaApiController

* Remove more usages of IPublishedSnapshotAccessor

* Comment out tests

* Remove more usages of PublishedSnapshotAccessor

* Remove PublishedSnapshot from property

* Fixed test build

* Fix errors

* Fix some tests

* Delete NuCache 🎉

* Implement DatabaseCacheRebuilder

* Remove usage of IPublishedSnapshotService

* Remove IPublishedSnapshotService

* Remove TestPublishedSnapshotAccessor and make tests build

* Don't test Snapshot cachelevel

It's no longer supported

* Fix BlockEditorConverter

Element != Element document type

* Remember to set cachemanager

* Fix RichTextParserTests

* Implement TryGetLevel on INavigationQueryService

* Fake level and obsolete it in PublishedContent

* Remove ChildrenForAllCultures

* Hack Path property on PublishedContent

* Remove usages of IPublishedSnapshot in tests

* More ConvertersTests

* Add hybrid cache to integration tests

We can actually do this now because we no longer save files on disk

* Rename IPublishedSnapshotRebuilder to ICacheRebuilder

* Comment out tests

* V15: Replacing the usages of Parent (navigation data) from IPublishedContent (#17125)

* Fix .Parent references in PublishedContentExtensions

* Add missing methods to FriendlyPublishedContentExtensions (ones that you were able to call on the content directly as they now require extra params)

* Fix references from the extension methods

* Fix dependencies in tests

* Replace IPublishedSnapshotAccessor with the content cache in tests

* Resolving more .Parent references

* Fix unit tests

* Obsolete and use extension methods

* Remove private method and use extension instead

* Moving code around

* Fix tests

* Fix more references

* Cleanup

* Fix more usages

* Resolve merge conflict

* Fix tests

* Cleanup

* Fix more tests

* Fixed unit tests

* Cleanup

* Replace last usages

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>

* Remove usage of IPublishedSnapshotAccessor from IRequestItemProvider

* Post merge fixup

* Remo IPublishedSnapshot

* Add HasAny to IDocumentUrlService

* Fix TextBuilder

* Fix modelsbuilder tests

* Use explicit types

* Implement GetByContentType

* Support element types in PublishedContentTypeCache

* Run enlistments before publishing notifications

* Fix elements cache refreshing

* Implement GetByUdi

* Implement GetAtRoot

* Implement GetByRoute

* Reimplement GetRouteById

* Fix blocks unit tests

* Initialize domain cache on boot

* Only return routes with domains on non default lanauges

* V15: Replacing the usages of `Children` (navigation data) from `IPublishedContent` (#17159)

* Update params in PublishedContentExtensions to the general interfaces for the published cache and navigation service, so that we can use the extension methods on both documents and media

* Introduce GetParent() which uses the right services

* Fix obsolete message on .Parent

* Obsolete .Children

* Fix usages of Children for ApiMediaQueryService

* Fix usage in internal

* Fix usages in views

* Fix indentation

* Fix issue with delete language

* Update nuget pacakges

* Clear elements cache when content is deleted

instead of trying to update it

* Reset publishedModelFactory

* Fixed publishing

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: kjac <kja@umbraco.dk>
2024-10-01 15:03:02 +02:00
Bjarke Berg
1b4cb87370 Fixed test build 2024-09-24 11:02:22 +02:00
Mole
e342e795dd V15: Cache Seeding (#17102)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-09-24 09:39:23 +02:00
Nikolaj Geisle
2704d4a34a V15: Hybrid Caching (#16938)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Use init for ContentSourceDto

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Update to preview 7

* Fix versions

* Increase timeout for sqlite integration tests

* Undo timeout increase

* Try and undo init change to ContentSourceDto

* That wasn't it chief

* Try and make DomainAndUrlsTests non NonParallelizable

* Update versions

* Only run cache tests on linux for now

---------

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-09 17:49:18 +02:00