Commit Graph

527 Commits

Author SHA1 Message Date
Nikolaj Geisle
e368710364 V15: Add custom serializer for hybrid cache (#17727)
* Add custom serializer

* Add migration to rebuild cache

* Rename migration namespace to 15.1

* Also clear media cache

* Remove failed cache items

* Refactor to only use keys for document cache repository

---------

Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-12-06 13:22:02 +01:00
Bjarke Berg
07d77aa465 Optimize Azure pipeline (#17674)
* Only fetch single commit

* Hopefully fixes Nerdbank.GitVersioning.GitException: Shallow clone lacks the objects required to calculate version height. Use full clones or clones with a history at least as deep as the last version height resetting change.

* Do not checkout again

* More test pipeline

* Another attempt

* yet another attempt

* more attempts

* Revert "more attempts"

This reverts commit 5694d97ba620e90fdeea287936f58002f2a5ddba.

* Test without building backoffice and login explicitly

* Fix mem leak in integration tests

* Fixes sqlserver lock test
2024-11-29 12:21:36 +01:00
Bjarke Berg
2846325a8a Fixes routing issues (#17572)
* Child of second root should also hide root path and backoffice needs to show all domains for each language.

* Fixes routing issues based on findings https://github.com/umbraco/Umbraco-CMS/pull/17572#issuecomment-2486219574

* Revert "Fixes routing issues based on findings https://github.com/umbraco/Umbraco-CMS/pull/17572#issuecomment-2486219574"

This reverts commit ba7fb5cc904fbe602450ac109e1821282b7f69d6.

* Fix urls of descendants of non-first roots do not show the root urlsegment when HideTopLevel is true
2024-11-26 09:59:09 +01:00
Elitsa Marinovska
4a19d899dd V15: Update navigation references in Partial Views (#17613)
* Inject and pass-in parameters to use the respective PublishedContentExtensions methods

* Fix view comments

* Other fixes
2024-11-25 10:30:43 +01:00
Jacob Overgaard
07b7c26fd9 Feature: Build static assets conditionally for Backoffice and Login (#17527)
* feat: conditionally install and build the 'login' and 'backoffice' targets depending on source files

* feat: remove the preconditions target, because we are now compiling a file list to check if we need to build

* feat: remove the 'clean' targets, because the project will be cleaned if any of the compiled file lists do not exist

* feat: remove the preserve.* files as they are no longer needed

* Enable default content items again

* Remove package.json from restore target inputs

* Include generated files as static assets after build

* Clean up project files

* Exclude CS0618 warning as error and fix CA2264

* Exclude CS0612 warning as error

* Suppress removal of test fixture

* Use separate property/item groups for backoffice and login project

---------

Co-authored-by: Ronald Barendse <ronald@barend.se>
2024-11-19 13:29:39 +01:00
Bjarke Berg
c846633ff9 Fixes to templates: Do not allow deletion of masters and ensure file names are pascal case (#17539)
* Make template file names start with capical character

* https://github.com/umbraco/Umbraco-CMS/issues/16443

Also ensure master templates cannot be deleted

* Fix test

* Do not check breaking changes in integration tests

---------

Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-11-16 20:30:43 +01:00
Bjarke Berg
ba8092fac7 Fixes https://github.com/umbraco/Umbraco-CMS/issues/17506 (#17536)
Now the validation of ClientId has a max of 255 characters
2024-11-15 09:40:26 +01:00
Zeegaan
b8b96f05d1 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	Directory.Build.props
#	version.json
2024-11-14 09:03:28 +01:00
Nikolaj Geisle
a9636d5429 V15: Implement not-implemented methods for media cache (#17524)
* Implement not-implemented methods for media cache

* Fixed test

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-11-13 12:04:27 +01:00
Zeegaan
b9b7cb6d85 Rename extension methods back 2024-11-13 09:27:29 +01:00
Bjarke Berg
c9fa2a57d7 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
#	src/Umbraco.Core/Extensions/PublishedContentExtensions.cs
#	src/Umbraco.Web.UI.Client/src/packages/tiptap/components/input-tiptap/tiptap-toolbar.element.ts
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/ContentNavigationServiceBaseTests.cs
2024-11-12 12:14:50 +01:00
Bjarke Berg
87aff77054 Explicit cache entry settings and expose cache settings in json schema (#17480)
* Changed the cache entry settings to be explicit and exposed it in the schema file

* commit of tests

* Fix seed options

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
2024-11-11 12:52:11 +01:00
Elitsa Marinovska
c26016b0b0 V15: Implement content type filtering for in-memory navigation structure (#17456)
* Adding contentType to navigation node

* Loading contentType from DB

* Considering contentTypeKey when adding a navigation node & fixing references

* Using IContentTypeBaseService to load content types

* Adding generics to ContentNavigationServiceBase and fixing references

* Adding TryGetChildrenKeysOfType and implementation

* Refactoring test data

* Adding unit tests for TryGetChildrenKeysOfType

* Update CreateContentCreateModel in tests to receive content type

* Fix references

* Cleanup

* Adding integration tests for TryGetChildrenKeysOfType

* Cleanup

* Cleanup

* Descendants of type implementation

* Descendants of type tests

* Interface updates

* Ancestors of type implementation and tests

* Siblings of type implementation and tests

* Cleanup

* Integration tests

* Adding root of type implementation and tests

* Fix Ancestors extension methods

* Fix descendants extension methods

* Fix children extension methods

* Fix siblings extension methods

* Add helper methods

* Fix a bug

* Fixed unit tests by setting up mocks

* Adding missing extension method

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-11-11 12:00:20 +01:00
Sven Geusens
e7eb14d310 V15/task/cleanup obsolete (#17433)
* Replace obsolete UserGroup Alias consts to key equivalent in tests

* Update use of usergroup alias consts to key equivalent in IsSystemgroup extension method

* Obsolete (internally) unused helper function which purpose doesn't even seem true

* Prepped EmbedProviders for proper removal of non async methods and unneeded proxy methods

* Remove obsoleted UmbracoPath and updated internal references

* Corrected  mistake and updated unittets

* Update usergroup tests that use aliases for "system" groups

* Replace more uses of globalsettings.UmbracoPath

* Remove GetDateType by key non async

* Cleanup some usages of hostingEnvironment.MapPathContentRoot

* More easy obsoletion cleanup

* Small Typeload cleanup

* More obsolete removal

* Deploy obsoletion cleanup

* Remove obsolete methods from OEmbedProviderBase.cs

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-11-07 12:20:22 +01:00
Kenn Jacobsen
669c585ac4 Validate client IDs before applying them (#17426)
* Validate client IDs before applying them

* Add operation result to return

* Update src/Umbraco.Cms.Api.Management/Controllers/User/ClientCredentials/ClientCredentialsUserControllerBase.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2024-11-07 10:42:40 +01:00
Bjarke Berg
336bd997f1 Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Infrastructure/Migrations/Upgrade/V_15_0_0/ConvertBlockEditorPropertiesBase.cs
#	src/Umbraco.Web.UI.Client
2024-11-05 09:14:08 +01:00
Kenn Jacobsen
2c04e37b42 Support limited language access at block level (#17322)
* Support limited language access at block level

* Account for AllowEditInvariantFromNonDefault when updating properties (#17333)

* Remove obsolete ctor

* Add explanatory comment

* Set AllowEditInvariantFromNonDefault to true on tests

* Refactor to account for merge and default language

* Merge invariant values on top of the already merged values

* Add integration test to prove invariant merging

---------

Co-authored-by: kjac <kja@umbraco.dk>

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
2024-11-05 06:32:34 +01:00
nikolajlauridsen
8f86ac1bd1 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/Services/DocumentUrlService.cs
2024-11-04 12:49:38 +01:00
Bjarke Berg
6417948a58 #17391 (#17413)
Fixes issue with the delivery api can find unpublished content
2024-11-04 12:27:47 +01:00
Kenn Jacobsen
1fe689e027 Add logging to RTE integration tests to trace the source of the failing tests (#17376)
* Add logging to RTE integration tests to trace the source of the failing tests

* Revert build host change

* Explicitly log to console

* Make the test assertions less fragile
2024-11-04 07:26:32 +01:00
Kenn Jacobsen
5fe91f837d Validation for block level variation (#17355)
* Validation for block level variation

* Make the "missing property value" JSON path expression valid

* Update src/Umbraco.Core/Services/ContentValidationServiceBase.cs

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>

* Update src/Umbraco.Core/Services/ContentValidationServiceBase.cs

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>

* Update src/Umbraco.Infrastructure/PropertyEditors/BlockEditorValidatorBase.cs

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
2024-10-31 10:04:54 +01:00
Mole
d1799ecdd2 V15: Refresh caches on load balanced environments (#17296)
* Move DocumentCacheService

* Add clear all documentws from memory cache

* Fix RedirectTracker

* Implement refresh node/branch/all/delete

* Only update databasecache in RefreshContentAsync

* Fix tests

* Skip blueprints in cache

* Clear caches when contenttype is updated

* Clear cache on data type update

* Refresh media

* Only update memory cache from refreshers

* Fix imports

* Add named options

* Use cache entry settings in media

* Obsolete nucache settings

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-28 15:31:39 +01:00
Bjarke Berg
9a80328224 Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Dictionary/Dictionary.spec.ts
#	version.json
2024-10-28 12:32:39 +01:00
Bjarke Berg
935c3b8e42 Initialize important services before unattended installs (#17366)
* Added new notification to hook in after the premigrations and use this to init different services.

* Force MaxDegreeOfParallelism to 1, while investigating scopes

* Tried some more workarounds

* Updated scopes and changed parallel to non parallel to ensure migration works

* Missing scope

* Make it parallel again - The secret is, the SuppressFlow needs to be when you create the task, but not on the await!.

* Fixed issue when DEBUG_SCOPES is not added to tests.

* Remove test exception

* Try build on ubuntu again, even that we know it can be stuck. Just a test to see if all tests pass

* Updated comment

---------

Co-authored-by: kjac <kja@umbraco.dk>
2024-10-28 12:10:38 +01:00
Nikolaj Geisle
5a5de18843 Format sql statement (#17354)
(cherry picked from commit aa9f194d76)
2024-10-24 15:07:40 +02:00
Nikolaj Geisle
226819de21 Format sql statement (#17354)
(cherry picked from commit aa9f194d76)
2024-10-24 15:07:04 +02:00
Kenn Jacobsen
bafcc2b945 Block level variants - search indexing (#17239)
* Support block level variance for search indexing

* Rename base class

---------

Co-authored-by: Elitsa <elm@umbraco.dk>
2024-10-20 15:42:13 +02:00
Andreas Zerbst
60ddff1ce2 Fixed build errors (#17298) 2024-10-17 10:49:33 +02:00
Jacob Overgaard
4f4f6787dc Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/Services/DocumentUrlService.cs
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/ContentNavigationServiceBaseTests.cs
2024-10-17 09:02:19 +02:00
Elitsa Marinovska
44ff8dc5e1 V15: Implement sorting for the in-memory navigation structures (document and media) (#17280)
* bump version to 15.1.0

* V15 Fixed the failing smoke tests in the pipeline v15 (#17158)

* Fixed the failing tests of Member Group due to UI changes

* Fixed the failing tests of Member due to UI changes

* Fixed the failing tests of User due to UI changes

* Fixed failing tests for Dictionary and Document Type

* Updated tests due to test helper changes

* Bumped version

* Updated assert steps due to the response changes

* Updated tests due to api helper changes

* Updated tests due to UI changes

* Fixed tests for delete partial view

* Fixed tests

* Added more waits

* Updated assert steps

* Fixed failing tests for Block Grid and Media

* Added more waits

* Added skip tests

* Removed waits time

* Updated assertion steps for User

* Added todo

* Updated tests due to api helper changes

* Bumped version

* Added skip tests

* Fetch sortOrder for each navigationNode

* Update NavigationNode to have sortOrder and change Parent and Children props to keys instead of NavigationNodes

* Consider sortOrder when building the navigation structures

* Renaming tests

* Adding tests for items being the last in structure when added, moved, etc.

* Updating names

* Cleanup

* Updating cache refreshers with changes due to sorting

* Refactoring due to sorting changes and resolving key to NavigationNode

* Removing sortOrder params from test as they are calculated automatically

* Adding content and media integration tests to test sorting functionality

* Adding sortOrder param for special case when adding new nodes

* Adding new UpdateSortOrder to INavigationManagementService

* Revert "V15 Fixed the failing smoke tests in the pipeline v15 (#17158)"

This reverts commit 31399c3b15.

* Revert "bump version to 15.1.0"

This reverts commit 5e4d15be

* Fix revert

* Add sort order when creating media

---------

Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-16 09:51:42 +02:00
Bjarke Berg
b477cf50f2 Bugfix: Do not allow routing content that is unpublished (#17251)
* Ensure routing respect publish status

* Check published status per culture

* Added PublishStatusService to get publish status for a given documentkey and culture

* Added tests and fixed bug with a static fields that should not have been static

* Make sure the write and read cache key is always the same no matter where the request comes from

There is an edge case where the incomming culure is fully capitalized while the read is camelcase

* Fixed review comments

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
2024-10-15 19:33:23 +02:00
Kenn Jacobsen
0e700af35d Support block level variant tags (#17268) 2024-10-15 12:03:05 +02:00
Bjarke Berg
6e0ab4dd7e Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.PublishedCache.HybridCache/Services/MediaCacheService.cs
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Common/Builders/MediaTypeEditingBuilder.cs
2024-10-10 11:11:08 +02:00
Mole
2b12eadd3d V15: Refresh IPublishedContentTypeFactory when data types change (#17180)
* Refresh IPublishedContentTypeFactory when data types change

* Add tests
2024-10-07 11:11:22 +02:00
Bjarke Berg
537a657de3 V15: Fix previews (#17181)
* default to preview value from cookie instead of draft.

* Fix tests

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-10-03 15:35:17 +02:00
Sebastiaan Janssen
4090829fe8 Ignore warnings about using obsolete members in Umbraco.Tests.UnitTests.csproj and Umbraco.Core.csproj
LocalFileSystemTemporaryFileRepository: Don't pass null to a method that doesn't accept nullable argument - https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2264
Fix remaining warning marked as error
2024-10-02 14:37:37 +02:00
Jacob Overgaard
8a65f63434 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Integration/Umbraco.PublishedCache.HybridCache/DocumentHybridCacheTemplateTests.cs
#	version.json
2024-10-02 13:39:33 +02:00
Andreas Zerbst
04d71fa354 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:37:50 +02:00
Andreas Zerbst
e15b7f2214 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:36:26 +02:00
Sebastiaan Janssen
c2d83808a5 Merge branch 'v15/dev' into contrib
# Conflicts:
#	src/Umbraco.Infrastructure/PublishedContentQuery.cs
#	src/Umbraco.PublishedCache.NuCache/ContentCache.cs
#	src/Umbraco.PublishedCache.NuCache/ContentStore.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentNodeKitSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.DictionaryOfPropertyDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/Umbraco.PublishedCache.NuCache.csproj
#	tests/Umbraco.Tests.UnitTests/Umbraco.Infrastructure/PublishedCache/PublishContentCacheTests.cs
2024-10-02 12:35:20 +02:00
Bjarke Berg
668f4e15e3 Fixed issues with publishing in cache and reintroduced tests (#17168)
* Fixed issues with publishing in cache and reintroduced tests

* Fixed issue with republishing and templates
2024-10-02 09:10:18 +02:00
Mole
1258962429 V15: Remove Nucache (#17166)
* Remove nucache reference from Web.Common

* Get tests building-ish

* Move ReservedFieldNamesService to the right project

* Remove IPublishedSnapshotStatus

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Remove IPublishedSnapshot from UmbracoContext

* Begin removing usage of IPublishedSnapshot from PublishedContentExtensions

* Fix PublishedContentExtensions.cs

* Don't use snapshots in delivery media api

* Use IPublishedMediaCache in QueryMediaApiController

* Remove more usages of IPublishedSnapshotAccessor

* Comment out tests

* Remove more usages of PublishedSnapshotAccessor

* Remove PublishedSnapshot from property

* Fixed test build

* Fix errors

* Fix some tests

* Delete NuCache 🎉

* Implement DatabaseCacheRebuilder

* Remove usage of IPublishedSnapshotService

* Remove IPublishedSnapshotService

* Remove TestPublishedSnapshotAccessor and make tests build

* Don't test Snapshot cachelevel

It's no longer supported

* Fix BlockEditorConverter

Element != Element document type

* Remember to set cachemanager

* Fix RichTextParserTests

* Implement TryGetLevel on INavigationQueryService

* Fake level and obsolete it in PublishedContent

* Remove ChildrenForAllCultures

* Hack Path property on PublishedContent

* Remove usages of IPublishedSnapshot in tests

* More ConvertersTests

* Add hybrid cache to integration tests

We can actually do this now because we no longer save files on disk

* Rename IPublishedSnapshotRebuilder to ICacheRebuilder

* Comment out tests

* V15: Replacing the usages of Parent (navigation data) from IPublishedContent (#17125)

* Fix .Parent references in PublishedContentExtensions

* Add missing methods to FriendlyPublishedContentExtensions (ones that you were able to call on the content directly as they now require extra params)

* Fix references from the extension methods

* Fix dependencies in tests

* Replace IPublishedSnapshotAccessor with the content cache in tests

* Resolving more .Parent references

* Fix unit tests

* Obsolete and use extension methods

* Remove private method and use extension instead

* Moving code around

* Fix tests

* Fix more references

* Cleanup

* Fix more usages

* Resolve merge conflict

* Fix tests

* Cleanup

* Fix more tests

* Fixed unit tests

* Cleanup

* Replace last usages

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>

* Remove usage of IPublishedSnapshotAccessor from IRequestItemProvider

* Post merge fixup

* Remo IPublishedSnapshot

* Add HasAny to IDocumentUrlService

* Fix TextBuilder

* Fix modelsbuilder tests

* Use explicit types

* Implement GetByContentType

* Support element types in PublishedContentTypeCache

* Run enlistments before publishing notifications

* Fix elements cache refreshing

* Implement GetByUdi

* Implement GetAtRoot

* Implement GetByRoute

* Reimplement GetRouteById

* Fix blocks unit tests

* Initialize domain cache on boot

* Only return routes with domains on non default lanauges

* V15: Replacing the usages of `Children` (navigation data) from `IPublishedContent` (#17159)

* Update params in PublishedContentExtensions to the general interfaces for the published cache and navigation service, so that we can use the extension methods on both documents and media

* Introduce GetParent() which uses the right services

* Fix obsolete message on .Parent

* Obsolete .Children

* Fix usages of Children for ApiMediaQueryService

* Fix usage in internal

* Fix usages in views

* Fix indentation

* Fix issue with delete language

* Update nuget pacakges

* Clear elements cache when content is deleted

instead of trying to update it

* Reset publishedModelFactory

* Fixed publishing

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: kjac <kja@umbraco.dk>
2024-10-01 15:03:02 +02:00
Kenn Jacobsen
2e563f70f3 Make RTE work without blocks in a culture variant context (#17163) 2024-10-01 14:22:49 +02:00
Bjarke Berg
04ba12297f Handle navigation updates in cache refeshers (#17161)
* Handle navigation updates in cache refeshers

* Same for media cache refreshers

* Clean up

* More clean up and renaming content to media

* Update src/Umbraco.Core/Services/Navigation/ContentNavigationServiceBase.cs

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

---------

Co-authored-by: Elitsa <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2024-09-30 16:43:05 +02:00
Kenn Jacobsen
1be503e71f Block level variance (#17120)
* Block level variance - initial commit

* Remove TODOs

* Only convert RTEs with blocks

* Fix JSON paths for block level property validation

* Rename Properties to Values

* Correct the JSON path of block level validation errors

* Make it possible to skip content migration + ensure backwards compat for the new block format

* Partial culture variance publishing at property level

* UDI to key conversion for block editors - draft, WIP, do NOT merge 😄  (#16970)

* Convert block UDIs to GUIDs

* Fix merge

* Fix merge issues

* Rework nested layout item key parsing for backwards compatibility

* Clean-up

* Reverse block layout item key calculation

* Review

* Use IOptions to skip content migrations

* Remove "published" from data editor feature naming, as it can be used in other contexts too

* Parallel migration

* Don't use deprecated constructor

* Ensure that layout follows structure for partial publishing

* Block Grid element level variance + tests (incl. refactor of element level variation tests)

* Rollback unintended changes to Program.cs

* Fix bad casing

* Minor formatting

* RTE element level variance + tests

* Remove obsoleted constructors

* Use Umbraco.RichText instead of Umbraco.TinyMCE as layout alias for blocks in the RTE

* Fix bad merge

* Temporary fix for new cache in integration tests

* Add EditorAlias to block level properties

* Remove the unintended PropertyEditorAlias output for block values

* Add EditorAlias to Datatype Item model

* Update OpenApi.json

* Introduce "expose" for blocks

* Strict (explicit) handling for Expose

* Improve handling of document and element level variance changes

* Refactor variance alignment for published rendering

* Block UDI to Key conversion should also register as a conversion

* Convert newly added RTE unit test to new RTE blocks format

* Minor review changes

* Run memory intensive tests on Linux only

* Add tests proving that AllowEditInvariantFromNonDefault has effect for block level variance too

* Fix the Platform annotations

* Removed Platform annotations for tests.

* Fix merge

* Obsolete old PublishCulture extension

* More fixing bad merge

---------

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-30 07:01:18 +02:00
Bjarke Berg
734b3cce2c Determine urls at save and publish time (#17033)
* Started work on service

* temp work

* temp commit

* Temp commit

* Added more routing logic

* Fixed tests

* Refactor and prepare for isdraft

* Work on drafts

* Fixed tests

* Move to enlistment to ensure caches is only updated on scope complete

* Clean up and handle null cultures

* Added functionality to the INavigationQueryService to get root keys

* Added migration

* Fixed issue with navigation

* Added migration

* Temp commit, move to cache refreshers.

* Fixed issues

* List urls

* fix build

* Fixed integration tests

* Refactor to create new content finder instead of changing the old

* rollback wrong commited line

* Clean up, and use docuemnt url service for index

* Fixed List endpoin

* Do not use Navigation service in methods intended by management api

* Fixed examine tests

* Make methods virtual

* Use domain from published request

* Use hybrid cache from new content finder

* Eliminate nucache usage

* Fixed issue with delivery api and url generation

* Fixed linux tests

* Added hybrid cache to all integration tests
2024-09-27 09:12:19 +02:00
Sebastiaan Janssen
8d81c70391 Merge branch 'v15/dev' into contrib 2024-09-27 09:09:18 +02:00
Bjarke Berg
1951ac96b5 Merge remote-tracking branch 'refs/remotes/origin/v14/dev' into v15/dev
# Conflicts:
#	build/nightly-E2E-test-pipelines.yml
#	src/Umbraco.Infrastructure/Security/BackOfficeIdentityUser.cs
#	src/Umbraco.Web.UI.Client
#	src/Umbraco.Web.UI.Login/package-lock.json
#	version.json
2024-09-26 08:56:43 +02:00
Kenn Jacobsen
548b5e4150 Enable validation of specific cultures only for document updates (#17087)
* Enable validation of specific cultures only for document updates

* Only validate explicitly sent cultures in the create validation endpoint

* Fix backwards compat (obsolete old method)

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2024-09-25 15:34:14 +02:00
Andreas Zerbst
71d6b45a95 V14 QA added navigation integration tests (#16973)
* Tests

* Remove props and use local vars

* Adding preliminary navigation service and content implementation

* Adding preliminary unit tests

* Change from async methods

* Refactor GetParentKey to TryGetParentKey

* Refactor GetChildrenKeys to TryGetChildrenKeys

* Refactor GetDescendantsKeys to TryGetDescendantsKeys

* Refactor GetAncestorsKeys to TryGetAncestorsKeys

* Refactor GetSiblingsKeys to TryGetSiblingsKeys

* Refactor TryGetChildrenKeys

* Initial integration tests

* Use ContentEditingService instead of ContentService

* Remove INavigationService.Copy implementation and unit tests

* Rename var

* Adding clarification

* Initial ContentNavigationRepository

* Initial NavigationFactory

* Remove filtering from factory

* NavigationRepository and implementation

* InitializationService responsible for seeding the in-memory structure

* Register repository and service

* Adding NavigationDto and NavigationNode

* Adding INavigationService dependency and Enlist updating navigation structure actions

* Documentation

* Adding tests for removing descendants as well

* Changed to ConcurrentDictionary

* Remove keys comments for tests

* Adding documentation

* Forgotten ConcurrentDictionary change

* Isolating the operations on the model

* Splitting the INavigationService to separate the querying from the managing functionality

* Introducing specific navigation services for document, document recycle bin, media and media recycle bin

* Making ContentNavigationService into a base as the functionality will be shared between the document, document recycle bin, media and media recycle bin services

* Adding the implementations of document, document recycle bin, media and media recycle bin navigation services

* Fixing comments

* Initializing all 4 collections

* Adapting the navigation unit tests to the base now

* Adapting integration tests to specific navigation service

* Adding test for rebuilding the structure

* Adding implementation for Adding and Getting a node - needed for moving to and restoring from the recycle bin + tests

* Updating the document navigation structure from the ContentService

* Fix typo

* Adding trashed items implementation in base - currently managing 2 structures

* Removing no longer relevant GetNavigationNode and AddNavigationNode

* Fix removing parent when child is removed supporting methods

* Added restoring functionality

* Adding Bin functionality to DocumentNavigationService

* Removing Move signature from IDocumentNavigationService

* Adding RecycleBin query and management services

* Re-adding Move and removing GetNavigationNode and AddNavigationNode signatures from interface

* Rebuilding bin structure using _documentNavigationService, instead of _documentRecycleBinNavigationService

* Fixing test name

* Adding more tests for remove

* Adding tests for restore and removing ones for GetNavigationNode and AddNavigationNode

* Remove comments

* Removing document and media RecycleBinNavigationService and their interfaces

* Adding media rebuild bin

* Fixing initialization with correct interfaces

* Removing RecycleBinNavigationServices' registration

* Remove IDocumentRecycleBinNavigationService dependency

* Updating in-memory nav structure when content updates happen

* Adding the rest of the integration tests

* Clean up IMediaNavigationService

* Fix comments

* Remove CustomTestSetup in integration tests as the structure is updated when content updates happen

* Adding and fixing comments

* Making RebuildBinAsync abstract as well

* Adding DocumentNavigationServiceTestsBase

* Splitting DocumentNavigationServiceTests into partial test classes

* Cleaning up DocumentNavigationServiceTests since tests have been moved to specific partial classes

* Reuse a method for creating content in tests

* Change type in test base

* Adding navigation structure updates in media service

* Adding MediaNavigationServiceTestsBase

* Adding integration tests for media nav str

* Remove services as we will have more concrete ones

* Add document and media IXNavigationQueryService and IXNavigationManagementService

* Inject ManagementService in ContentService.cs and MediaService.cs

* Change implementation to implement the new services + registration

* Make classes sealed

* Inject correct services in InitializationService

* Using the right services in integration tests

* Adding comments

* Removing bin interfaces from main navigation ones

* Rename Remove to MoveToBin

* Added tests for Copy

* Added additional tests

* Split test

* Added Media tests

* Updated and added content tests

* Cleaned up naming

* Cleaned up

* Rename initialization service to initialization hosted service

* Refactor repository to return a collection

* Add interface for the NavigationDto

* Add constants to bind property names between DTOs

* Move factory and fix input type

* Use constants for column names

* Use factory from base

* Fixed indentation

* Fix bug when rebuilding the recycle bin structure

* Fix comments

* Fix merged in code

* Fix bug again after merge

* Minor things

---------

Co-authored-by: Elitsa <elm@umbraco.dk>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-09-25 08:33:23 +02:00