Commit Graph

696 Commits

Author SHA1 Message Date
Bjarke Berg
6e0ab4dd7e Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.PublishedCache.HybridCache/Services/MediaCacheService.cs
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Common/Builders/MediaTypeEditingBuilder.cs
2024-10-10 11:11:08 +02:00
Mole
2b12eadd3d V15: Refresh IPublishedContentTypeFactory when data types change (#17180)
* Refresh IPublishedContentTypeFactory when data types change

* Add tests
2024-10-07 11:11:22 +02:00
Bjarke Berg
537a657de3 V15: Fix previews (#17181)
* default to preview value from cookie instead of draft.

* Fix tests

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-10-03 15:35:17 +02:00
Sebastiaan Janssen
4090829fe8 Ignore warnings about using obsolete members in Umbraco.Tests.UnitTests.csproj and Umbraco.Core.csproj
LocalFileSystemTemporaryFileRepository: Don't pass null to a method that doesn't accept nullable argument - https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2264
Fix remaining warning marked as error
2024-10-02 14:37:37 +02:00
Jacob Overgaard
8a65f63434 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Integration/Umbraco.PublishedCache.HybridCache/DocumentHybridCacheTemplateTests.cs
#	version.json
2024-10-02 13:39:33 +02:00
Andreas Zerbst
04d71fa354 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:37:50 +02:00
Andreas Zerbst
e15b7f2214 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:36:26 +02:00
Sebastiaan Janssen
c2d83808a5 Merge branch 'v15/dev' into contrib
# Conflicts:
#	src/Umbraco.Infrastructure/PublishedContentQuery.cs
#	src/Umbraco.PublishedCache.NuCache/ContentCache.cs
#	src/Umbraco.PublishedCache.NuCache/ContentStore.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentNodeKitSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.DictionaryOfPropertyDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/Umbraco.PublishedCache.NuCache.csproj
#	tests/Umbraco.Tests.UnitTests/Umbraco.Infrastructure/PublishedCache/PublishContentCacheTests.cs
2024-10-02 12:35:20 +02:00
Bjarke Berg
668f4e15e3 Fixed issues with publishing in cache and reintroduced tests (#17168)
* Fixed issues with publishing in cache and reintroduced tests

* Fixed issue with republishing and templates
2024-10-02 09:10:18 +02:00
Mole
1258962429 V15: Remove Nucache (#17166)
* Remove nucache reference from Web.Common

* Get tests building-ish

* Move ReservedFieldNamesService to the right project

* Remove IPublishedSnapshotStatus

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Remove IPublishedSnapshot from UmbracoContext

* Begin removing usage of IPublishedSnapshot from PublishedContentExtensions

* Fix PublishedContentExtensions.cs

* Don't use snapshots in delivery media api

* Use IPublishedMediaCache in QueryMediaApiController

* Remove more usages of IPublishedSnapshotAccessor

* Comment out tests

* Remove more usages of PublishedSnapshotAccessor

* Remove PublishedSnapshot from property

* Fixed test build

* Fix errors

* Fix some tests

* Delete NuCache 🎉

* Implement DatabaseCacheRebuilder

* Remove usage of IPublishedSnapshotService

* Remove IPublishedSnapshotService

* Remove TestPublishedSnapshotAccessor and make tests build

* Don't test Snapshot cachelevel

It's no longer supported

* Fix BlockEditorConverter

Element != Element document type

* Remember to set cachemanager

* Fix RichTextParserTests

* Implement TryGetLevel on INavigationQueryService

* Fake level and obsolete it in PublishedContent

* Remove ChildrenForAllCultures

* Hack Path property on PublishedContent

* Remove usages of IPublishedSnapshot in tests

* More ConvertersTests

* Add hybrid cache to integration tests

We can actually do this now because we no longer save files on disk

* Rename IPublishedSnapshotRebuilder to ICacheRebuilder

* Comment out tests

* V15: Replacing the usages of Parent (navigation data) from IPublishedContent (#17125)

* Fix .Parent references in PublishedContentExtensions

* Add missing methods to FriendlyPublishedContentExtensions (ones that you were able to call on the content directly as they now require extra params)

* Fix references from the extension methods

* Fix dependencies in tests

* Replace IPublishedSnapshotAccessor with the content cache in tests

* Resolving more .Parent references

* Fix unit tests

* Obsolete and use extension methods

* Remove private method and use extension instead

* Moving code around

* Fix tests

* Fix more references

* Cleanup

* Fix more usages

* Resolve merge conflict

* Fix tests

* Cleanup

* Fix more tests

* Fixed unit tests

* Cleanup

* Replace last usages

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>

* Remove usage of IPublishedSnapshotAccessor from IRequestItemProvider

* Post merge fixup

* Remo IPublishedSnapshot

* Add HasAny to IDocumentUrlService

* Fix TextBuilder

* Fix modelsbuilder tests

* Use explicit types

* Implement GetByContentType

* Support element types in PublishedContentTypeCache

* Run enlistments before publishing notifications

* Fix elements cache refreshing

* Implement GetByUdi

* Implement GetAtRoot

* Implement GetByRoute

* Reimplement GetRouteById

* Fix blocks unit tests

* Initialize domain cache on boot

* Only return routes with domains on non default lanauges

* V15: Replacing the usages of `Children` (navigation data) from `IPublishedContent` (#17159)

* Update params in PublishedContentExtensions to the general interfaces for the published cache and navigation service, so that we can use the extension methods on both documents and media

* Introduce GetParent() which uses the right services

* Fix obsolete message on .Parent

* Obsolete .Children

* Fix usages of Children for ApiMediaQueryService

* Fix usage in internal

* Fix usages in views

* Fix indentation

* Fix issue with delete language

* Update nuget pacakges

* Clear elements cache when content is deleted

instead of trying to update it

* Reset publishedModelFactory

* Fixed publishing

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: kjac <kja@umbraco.dk>
2024-10-01 15:03:02 +02:00
Kenn Jacobsen
2e563f70f3 Make RTE work without blocks in a culture variant context (#17163) 2024-10-01 14:22:49 +02:00
Bjarke Berg
04ba12297f Handle navigation updates in cache refeshers (#17161)
* Handle navigation updates in cache refeshers

* Same for media cache refreshers

* Clean up

* More clean up and renaming content to media

* Update src/Umbraco.Core/Services/Navigation/ContentNavigationServiceBase.cs

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

---------

Co-authored-by: Elitsa <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2024-09-30 16:43:05 +02:00
Kenn Jacobsen
1be503e71f Block level variance (#17120)
* Block level variance - initial commit

* Remove TODOs

* Only convert RTEs with blocks

* Fix JSON paths for block level property validation

* Rename Properties to Values

* Correct the JSON path of block level validation errors

* Make it possible to skip content migration + ensure backwards compat for the new block format

* Partial culture variance publishing at property level

* UDI to key conversion for block editors - draft, WIP, do NOT merge 😄  (#16970)

* Convert block UDIs to GUIDs

* Fix merge

* Fix merge issues

* Rework nested layout item key parsing for backwards compatibility

* Clean-up

* Reverse block layout item key calculation

* Review

* Use IOptions to skip content migrations

* Remove "published" from data editor feature naming, as it can be used in other contexts too

* Parallel migration

* Don't use deprecated constructor

* Ensure that layout follows structure for partial publishing

* Block Grid element level variance + tests (incl. refactor of element level variation tests)

* Rollback unintended changes to Program.cs

* Fix bad casing

* Minor formatting

* RTE element level variance + tests

* Remove obsoleted constructors

* Use Umbraco.RichText instead of Umbraco.TinyMCE as layout alias for blocks in the RTE

* Fix bad merge

* Temporary fix for new cache in integration tests

* Add EditorAlias to block level properties

* Remove the unintended PropertyEditorAlias output for block values

* Add EditorAlias to Datatype Item model

* Update OpenApi.json

* Introduce "expose" for blocks

* Strict (explicit) handling for Expose

* Improve handling of document and element level variance changes

* Refactor variance alignment for published rendering

* Block UDI to Key conversion should also register as a conversion

* Convert newly added RTE unit test to new RTE blocks format

* Minor review changes

* Run memory intensive tests on Linux only

* Add tests proving that AllowEditInvariantFromNonDefault has effect for block level variance too

* Fix the Platform annotations

* Removed Platform annotations for tests.

* Fix merge

* Obsolete old PublishCulture extension

* More fixing bad merge

---------

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-30 07:01:18 +02:00
Bjarke Berg
734b3cce2c Determine urls at save and publish time (#17033)
* Started work on service

* temp work

* temp commit

* Temp commit

* Added more routing logic

* Fixed tests

* Refactor and prepare for isdraft

* Work on drafts

* Fixed tests

* Move to enlistment to ensure caches is only updated on scope complete

* Clean up and handle null cultures

* Added functionality to the INavigationQueryService to get root keys

* Added migration

* Fixed issue with navigation

* Added migration

* Temp commit, move to cache refreshers.

* Fixed issues

* List urls

* fix build

* Fixed integration tests

* Refactor to create new content finder instead of changing the old

* rollback wrong commited line

* Clean up, and use docuemnt url service for index

* Fixed List endpoin

* Do not use Navigation service in methods intended by management api

* Fixed examine tests

* Make methods virtual

* Use domain from published request

* Use hybrid cache from new content finder

* Eliminate nucache usage

* Fixed issue with delivery api and url generation

* Fixed linux tests

* Added hybrid cache to all integration tests
2024-09-27 09:12:19 +02:00
Sebastiaan Janssen
8d81c70391 Merge branch 'v15/dev' into contrib 2024-09-27 09:09:18 +02:00
Bjarke Berg
1951ac96b5 Merge remote-tracking branch 'refs/remotes/origin/v14/dev' into v15/dev
# Conflicts:
#	build/nightly-E2E-test-pipelines.yml
#	src/Umbraco.Infrastructure/Security/BackOfficeIdentityUser.cs
#	src/Umbraco.Web.UI.Client
#	src/Umbraco.Web.UI.Login/package-lock.json
#	version.json
2024-09-26 08:56:43 +02:00
Kenn Jacobsen
548b5e4150 Enable validation of specific cultures only for document updates (#17087)
* Enable validation of specific cultures only for document updates

* Only validate explicitly sent cultures in the create validation endpoint

* Fix backwards compat (obsolete old method)

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2024-09-25 15:34:14 +02:00
Andreas Zerbst
71d6b45a95 V14 QA added navigation integration tests (#16973)
* Tests

* Remove props and use local vars

* Adding preliminary navigation service and content implementation

* Adding preliminary unit tests

* Change from async methods

* Refactor GetParentKey to TryGetParentKey

* Refactor GetChildrenKeys to TryGetChildrenKeys

* Refactor GetDescendantsKeys to TryGetDescendantsKeys

* Refactor GetAncestorsKeys to TryGetAncestorsKeys

* Refactor GetSiblingsKeys to TryGetSiblingsKeys

* Refactor TryGetChildrenKeys

* Initial integration tests

* Use ContentEditingService instead of ContentService

* Remove INavigationService.Copy implementation and unit tests

* Rename var

* Adding clarification

* Initial ContentNavigationRepository

* Initial NavigationFactory

* Remove filtering from factory

* NavigationRepository and implementation

* InitializationService responsible for seeding the in-memory structure

* Register repository and service

* Adding NavigationDto and NavigationNode

* Adding INavigationService dependency and Enlist updating navigation structure actions

* Documentation

* Adding tests for removing descendants as well

* Changed to ConcurrentDictionary

* Remove keys comments for tests

* Adding documentation

* Forgotten ConcurrentDictionary change

* Isolating the operations on the model

* Splitting the INavigationService to separate the querying from the managing functionality

* Introducing specific navigation services for document, document recycle bin, media and media recycle bin

* Making ContentNavigationService into a base as the functionality will be shared between the document, document recycle bin, media and media recycle bin services

* Adding the implementations of document, document recycle bin, media and media recycle bin navigation services

* Fixing comments

* Initializing all 4 collections

* Adapting the navigation unit tests to the base now

* Adapting integration tests to specific navigation service

* Adding test for rebuilding the structure

* Adding implementation for Adding and Getting a node - needed for moving to and restoring from the recycle bin + tests

* Updating the document navigation structure from the ContentService

* Fix typo

* Adding trashed items implementation in base - currently managing 2 structures

* Removing no longer relevant GetNavigationNode and AddNavigationNode

* Fix removing parent when child is removed supporting methods

* Added restoring functionality

* Adding Bin functionality to DocumentNavigationService

* Removing Move signature from IDocumentNavigationService

* Adding RecycleBin query and management services

* Re-adding Move and removing GetNavigationNode and AddNavigationNode signatures from interface

* Rebuilding bin structure using _documentNavigationService, instead of _documentRecycleBinNavigationService

* Fixing test name

* Adding more tests for remove

* Adding tests for restore and removing ones for GetNavigationNode and AddNavigationNode

* Remove comments

* Removing document and media RecycleBinNavigationService and their interfaces

* Adding media rebuild bin

* Fixing initialization with correct interfaces

* Removing RecycleBinNavigationServices' registration

* Remove IDocumentRecycleBinNavigationService dependency

* Updating in-memory nav structure when content updates happen

* Adding the rest of the integration tests

* Clean up IMediaNavigationService

* Fix comments

* Remove CustomTestSetup in integration tests as the structure is updated when content updates happen

* Adding and fixing comments

* Making RebuildBinAsync abstract as well

* Adding DocumentNavigationServiceTestsBase

* Splitting DocumentNavigationServiceTests into partial test classes

* Cleaning up DocumentNavigationServiceTests since tests have been moved to specific partial classes

* Reuse a method for creating content in tests

* Change type in test base

* Adding navigation structure updates in media service

* Adding MediaNavigationServiceTestsBase

* Adding integration tests for media nav str

* Remove services as we will have more concrete ones

* Add document and media IXNavigationQueryService and IXNavigationManagementService

* Inject ManagementService in ContentService.cs and MediaService.cs

* Change implementation to implement the new services + registration

* Make classes sealed

* Inject correct services in InitializationService

* Using the right services in integration tests

* Adding comments

* Removing bin interfaces from main navigation ones

* Rename Remove to MoveToBin

* Added tests for Copy

* Added additional tests

* Split test

* Added Media tests

* Updated and added content tests

* Cleaned up naming

* Cleaned up

* Rename initialization service to initialization hosted service

* Refactor repository to return a collection

* Add interface for the NavigationDto

* Add constants to bind property names between DTOs

* Move factory and fix input type

* Use constants for column names

* Use factory from base

* Fixed indentation

* Fix bug when rebuilding the recycle bin structure

* Fix comments

* Fix merged in code

* Fix bug again after merge

* Minor things

---------

Co-authored-by: Elitsa <elm@umbraco.dk>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-09-25 08:33:23 +02:00
Emma L Garland
ac575662ac Resolved more warnings, and marked more warning types as errors (#16991)
* Fix warnings SA1111, SA1028, SA1500, IDE1270  in Umbraco.Web.Website, and updated rules.

* Remove warnings: IDE0270: Null check can be simplified

* More SqlServer project warnings resolved

* CS0105 namespace appeared already

* Suppress warning until implementation:

#pragma warning disable CS0162 // Unreachable code detected
#pragma warning disable CS0618 // Type or member is obsolete

CS0162 remove unreachable code
SA1028 remove trailing whitespace
SA1106 no empty statements
CS1570 malformed XML
CS1572 corrected xml parameter
CS1573 param tag added
IDE0007 var not explicit
IDE0008 explicit not var
IDE0057 simplify substring
IDE0074 compound assignment
CA1825 array.empty

Down to 3479 warnings

* - SA1116, SA117 params on same line
- IDE0057 substring simplified

Specific warnings for Umbraco.Tests.Benchmarks

* Fixed IDE0074 compound assignment and added specific warnings for Umbraco.Tests.Common

* Specific warnings for Umbraco.Tests.Integration and Umbraco.Tests.Common

Fixed:

- SA1111, SA1116, SA117 params and line formatting (not all as there are many)
- SA1122 string.Empty
- IDE0057 simplify substring
- IDE0044,IDE0044 make field readonly
- IDE1006 naming rule violation (add _)
- SA1111 closing parenthesis on line of last parameter
- SA1649 filename match type name
- SA1312,SA1306 lowercase variable and field names

* Fixed various warnings where they are more straight-forward, including:

- SA1649 file name match type name
- SA111 parenthesis on line of last parameter
- IDE0028 simplify collection initializer
- SA1306 lower-case letter field
- IDE044 readonly field
- SA1122 string.Empty
- SA1116 params same line
- IDE1006 upper casing
- IDE0041 simplify null check

Updated the following projects to only list their remaining specific warning codes:

- Umbraco.Tests.UnitTests

Typo in `Umbraco.Web.Website` project

* Reverted test change

* Now 1556 warnings.

Fixed various warnings where they are more straight-forward, including:

- SA1111/SA1116/SA1119 parenthesis
- SA1117 params
- SA1312 lowercase variable
- SA1121 built-in type
- SA1500/SA1513/SA1503 formatting braces
- SA1400 declare access modifier
- SA1122 string.Empty
- SA1310 no underscore
- IDE0049 name simplified
- IDE0057 simplify substring
- IDE0074 compound assignment
- IDE0032 use auto-property
- IDE0037 simplify member name
- IDE0008 explicit type not var
- IDE0016/IDE0270/IDE0041 simplify null checks
- IDE0048/SA1407 clarity in arithmetic
- IDE1006 correct param names
- IDE0042 deconstruct variable
- IDE0044 readonly
- IDE0018 inline variable declarations
- IDE0074/IDE0054 compound assignment
- IDE1006 naming
- CS1573 param XML
- CS0168 unused variable

Comment formatting in project files for consistency.

Updated all projects to only list remaining specific warning codes as warnings instead of errors (errors is now default).

* Type not var, and more warning exceptions

* Tweaked merge issue, readded comment about rollback

* Readded comment re rollback.

* Readded comments

* Comment tweak

* Comment tweak
2024-09-24 12:56:28 +01:00
Bjarke Berg
1b4cb87370 Fixed test build 2024-09-24 11:02:22 +02:00
Mole
e342e795dd V15: Cache Seeding (#17102)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-09-24 09:39:23 +02:00
Bjarke Berg
f12aafd5be Updated to dotnet 9 RC.1 - and other nuget packages (#17053)
* Updated nuget packages and dotnet 9

* attempt to make pipelines happy

* Another attempt to make pipelines happy
2024-09-13 10:42:19 +02:00
Emma L Garland
862820c6a6 Resolved more warnings, and marked more warning types as errors (#16990)
* Fix warnings SA1111, SA1028, SA1500, IDE1270  in Umbraco.Web.Website, and updated rules.

* Remove warnings: IDE0270: Null check can be simplified

* More SqlServer project warnings resolved

* CS0105 namespace appeared already

* Suppress warning until implementation:

#pragma warning disable CS0162 // Unreachable code detected
#pragma warning disable CS0618 // Type or member is obsolete

CS0162 remove unreachable code
SA1028 remove trailing whitespace
SA1106 no empty statements
CS1570 malformed XML
CS1572 corrected xml parameter
CS1573 param tag added
IDE0007 var not explicit
IDE0008 explicit not var
IDE0057 simplify substring
IDE0074 compound assignment
CA1825 array.empty

Down to 3479 warnings
2024-09-10 15:17:29 +01:00
Nikolaj Geisle
2704d4a34a V15: Hybrid Caching (#16938)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Use init for ContentSourceDto

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Update to preview 7

* Fix versions

* Increase timeout for sqlite integration tests

* Undo timeout increase

* Try and undo init change to ContentSourceDto

* That wasn't it chief

* Try and make DomainAndUrlsTests non NonParallelizable

* Update versions

* Only run cache tests on linux for now

---------

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-09 17:49:18 +02:00
Elitsa Marinovska
5a7d563b8a Introduce INavigationService for in-memory navigation data (#16818)
* Tests

* Remove props and use local vars

* Adding preliminary navigation service and content implementation

* Adding preliminary unit tests

* Change from async methods

* Refactor GetParentKey to TryGetParentKey

* Refactor GetChildrenKeys to TryGetChildrenKeys

* Refactor GetDescendantsKeys to TryGetDescendantsKeys

* Refactor GetAncestorsKeys to TryGetAncestorsKeys

* Refactor GetSiblingsKeys to TryGetSiblingsKeys

* Refactor TryGetChildrenKeys

* Initial integration tests

* Use ContentEditingService instead of ContentService

* Remove INavigationService.Copy implementation and unit tests

* Rename var

* Adding clarification

* Initial ContentNavigationRepository

* Initial NavigationFactory

* Remove filtering from factory

* NavigationRepository and implementation

* InitializationService responsible for seeding the in-memory structure

* Register repository and service

* Adding NavigationDto and NavigationNode

* Adding INavigationService dependency and Enlist updating navigation structure actions

* Documentation

* Adding tests for removing descendants as well

* Changed to ConcurrentDictionary

* Remove keys comments for tests

* Adding documentation

* Forgotten ConcurrentDictionary change

* Isolating the operations on the model

* Splitting the INavigationService to separate the querying from the managing functionality

* Introducing specific navigation services for document, document recycle bin, media and media recycle bin

* Making ContentNavigationService into a base as the functionality will be shared between the document, document recycle bin, media and media recycle bin services

* Adding the implementations of document, document recycle bin, media and media recycle bin navigation services

* Fixing comments

* Initializing all 4 collections

* Adapting the navigation unit tests to the base now

* Adapting integration tests to specific navigation service

* Adding test for rebuilding the structure

* Adding implementation for Adding and Getting a node - needed for moving to and restoring from the recycle bin + tests

* Updating the document navigation structure from the ContentService

* Fix typo

* Adding trashed items implementation in base - currently managing 2 structures

* Removing no longer relevant GetNavigationNode and AddNavigationNode

* Fix removing parent when child is removed supporting methods

* Added restoring functionality

* Adding Bin functionality to DocumentNavigationService

* Removing Move signature from IDocumentNavigationService

* Adding RecycleBin query and management services

* Re-adding Move and removing GetNavigationNode and AddNavigationNode signatures from interface

* Rebuilding bin structure using _documentNavigationService, instead of _documentRecycleBinNavigationService

* Fixing test name

* Adding more tests for remove

* Adding tests for restore and removing ones for GetNavigationNode and AddNavigationNode

* Remove comments

* Removing document and media RecycleBinNavigationService and their interfaces

* Adding media rebuild bin

* Fixing initialization with correct interfaces

* Removing RecycleBinNavigationServices' registration

* Remove IDocumentRecycleBinNavigationService dependency

* Updating in-memory nav structure when content updates happen

* Adding the rest of the integration tests

* Clean up IMediaNavigationService

* Fix comments

* Remove CustomTestSetup in integration tests as the structure is updated when content updates happen

* Adding and fixing comments

* Making RebuildBinAsync abstract as well

* Adding DocumentNavigationServiceTestsBase

* Splitting DocumentNavigationServiceTests into partial test classes

* Cleaning up DocumentNavigationServiceTests since tests have been moved to specific partial classes

* Reuse a method for creating content in tests

* Change type in test base

* Adding navigation structure updates in media service

* Adding MediaNavigationServiceTestsBase

* Adding integration tests for media nav str

* Remove services as we will have more concrete ones

* Add document and media IXNavigationQueryService and IXNavigationManagementService

* Inject ManagementService in ContentService.cs and MediaService.cs

* Change implementation to implement the new services + registration

* Make classes sealed

* Inject correct services in InitializationService

* Using the right services in integration tests

* Adding comments

* Removing bin interfaces from main navigation ones

* Rename Remove to MoveToBin

* V14 QA added block list editor tests (#16862)

* Added tests for blocklistEditor

* Added more tets

* Removed faker

* Added blockTest

* Updates

* Added tests

* Removed dependencies

* Fixes

* Clean up

* Fixed naming

* Cleaned up

* Bumped version

* Added missing semicolons

* Added tags

* Only runs the new tests

* Updates

* Bumped version

* Fixed tests

* Cleaned up

* Updated version

* Fixes, not done

* Fixed tests

* Bumped helpers

* Bumped helpers

* Fixed conflict

* Fixed comment

* Reverted to run smokeTests

* Updated helpers

* improve missingProperties data returned for missing propertie values (#16910)

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>

* update backoffice submodule

* Rename initialization service to initialization hosted service

* Refactor repository to return a collection

* Add interface for the NavigationDto

* Add constants to bind property names between DTOs

* Move factory and fix input type

* Use constants for column names

* Use factory from base

---------

Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
2024-09-04 11:18:08 +02:00
Kenn Jacobsen
874055eeab Add member "kind" - and refactor user "type" to "kind" for consistency (#16979)
* Rename UserType to UserKind

* Add MemberKind to tell API members from regular ones

* Remove user kind from invite user endpoint

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2024-09-03 10:43:09 +02:00
Emma L Garland
98e9a300bc Treat warnings as errors for initial projects (#15019)
* Treat warnings as errors for Umbraco.Cms, Umbraco.Cms.Api.Common and Umbraco.Cms.Api.Delivery projects.

* Treat warnings as errors for Umbraco.Cms.Imaging.ImageSharp and Umbraco.Cms.Imaging.ImageSharp2 projects

* Treat warnings as errors for Umbraco.Cms.Persistence.EFCore.Sqlite, Umbraco.Cms.Persistence.EFCore.SqlServer and Umbraco.Cms.StaticAssets

* Treat warnings as errors for Umbraco.Cms.Targets

* Treat warnings as errors for templates/Umbraco.Templates and Umbraco.JsonSchema

* More projects with warnings as errors.

* Reverted warnings as errors due to some file changes since the update.

* Remove unwanted tag

* Removed warnings as errors on TestData project

* Implement warnings as errors in props file, and exclude specific warnings where appropriate as per PR review suggestions.

* Reverted spaces

* Revert "Reverted spaces"

This reverts commit 3734c45e2270c3324768b33e459aefcc6a8c4739.

* Update sdk version to fully support Umbraco.code 2.2.0 dependency on Microsoft.CodeAnalysis.CSharp.Workspaces 4.10.0

* Ignore PathToLongErrors for now in static assets

---------

Co-authored-by: Emma Garland <emma.garland@rocksolidknowledge.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
2024-08-27 18:56:09 +02:00
Bjarke Berg
9208980398 Merge remote-tracking branch 'origin/v14/dev' into v15/dev
# Conflicts:
#	Directory.Packages.props
#	src/Umbraco.Infrastructure/Migrations/Upgrade/UmbracoPlan.cs
#	tests/Directory.Packages.props
#	version.json
2024-08-27 13:42:21 +02:00
Sven Geusens
fd100602c2 V14/fix/element switch validation (#16421)
* Added Element <-> Document type switch validation

* Apply HasElementconfigured to block grid and block list

Fix smalle bug + optimization

* Moved some of the logic into warnings trough notifcationhandlers and eventmessages

* Cleanup

* Update openApi spec (merge changes)

* Add IsElement check between parent and child on creation

* Typos

* Transformed HasElementConfigured into HasElementConfigured

* Typo

Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>

* IsElement Validation refactor

Moved validation logic regarding doctype IsElement switch into its own service as it will be consumed by more things down the line

* commit missing services...

* Naming improvements

* Bugfix

* First batch of integration tests for ElementSwitchValidator

* More integration tests!

* Little reformatting

* Changed the default values of block based configuration to match expected values.

---------

Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
2024-08-15 07:11:17 +02:00
Bjarke Berg
58e515da11 Do not allow save of invalid domains (#16880) 2024-08-07 13:38:32 +02:00
Kenn Jacobsen
68db079700 API access with client credentials (core functionality) (#16817)
* First stab

* Delivery API client credentials + a little refactor to ensure unique client IDs

* Introduce user type

* Support user type in the Management API

* Clean up TODOs

* Update API user last login date when issuing a token

* Better error reporting for mismatched user types

* Do not allow password change or reset for API users

* Update OpenApi.json

* Revert change

* Remove obsolete comment

* Make applicable classes abstract or sealed

* Review changes

* Add endpoint for retrieving all user client IDs
2024-07-29 14:34:11 +02:00
nikolajlauridsen
88730ab481 Merge remote-tracking branch 'refs/remotes/origin/v14/dev' into v15/merge
# Conflicts:
#	Directory.Packages.props
#	tests/Directory.Packages.props
#	version.json
2024-07-19 11:28:38 +02:00
Nikolaj Geisle
a5b9fd1650 V15: Update to dotnet 9 (#16625)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

---------

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
2024-07-01 10:17:48 +03:00
Josh Brown
cb090353f4 Fix unguarded calls to ServiceDescriptor.ImplementationType for keyed services (#16604)
* Update integration test base class to verify that calls to ServiceDescriptor.ImplementationType are guarded for keyed services

* Fix unguarded calls to ServiceDescriptor.ImplementationType for keyed services
2024-06-22 10:20:12 +01:00
Lars-Erik Aabech
de74ae4c04 Added custom mvc setup method to testserver base class (#16545) 2024-06-21 16:43:30 +01:00
Nikolaj Geisle
ebf77024ce V14: Add clearer error for invalid property type (#16530)
* Add clearer error for invalid property type

* Update test
2024-06-06 08:54:47 +02:00
Nikolaj Geisle
4478d22390 Add retries to flaky tests (#16529) 2024-05-31 13:02:43 +02:00
Nikolaj Geisle
7eaaa8e2e6 Merge branch 'v14/dev' into release/14.0 2024-05-27 22:11:24 +02:00
Sven Geusens
e5e87c96db [v14] Fix multiple upgrader issues (#16339)
* Made previewhubupdater work with full cache refreshes

* Make signout_Async available on coreSignInManager

* Allow Migrations to signout the logged in user

* Adding a guid to a user requires resignin

* Added a token revoke mechanism during migrations

* Revert "Make signout_Async available on coreSignInManager"

This reverts commit b103cf119a505e61de659dc206f6c85c2a27f2d5.

* Revert add allRefreshed on preview hub

Clarified with a comment

* Updated failing test setups

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
2024-05-27 07:17:45 +02:00
Kenn Jacobsen
07711aac28 Expose and manage system media types (#16343)
* Expose and manage system media types

* Removed license :D
2024-05-24 08:59:47 +02:00
Kenn Jacobsen
2b79d24afa Add endpoint for retrieving "folder" media types. (#16345)
* Add endpoint for retrieving "folder" media types.

* The system Folder media type should always be considered a "folder"
2024-05-23 14:43:00 +02:00
Jacob Overgaard
9186d1de5e Merge branch 'v14/dev' into release/14.0 2024-05-22 13:54:28 +02:00
Nikolaj Geisle
a4a9a274bb V14: Merge 13 to 14 (#16340)
* Updates JSON schema for Umbraco 10 with latest references for Forms and Deploy (#15918)

* Ported over #15928 changes for 13.3 RC (#16023)

* Ported over #15928 changes for 13.3 RC

* Use GetOrAdd()

* Lock dictionary initialization

---------

Co-authored-by: Jason Elkin <jasonelkin86@gmail.com>

* Make the API content response builder extendable (#16056)

* Make the API content response builder extendable

* DeliveryApiJsonTypeResolver needs to be extendable too

* bump rc to regular

* Bump to next minor

* Add blocks in RTE telemetry (#16104)

* Add blocks telemetry

* Use constants and update tests

* V13: Add property type information to telemetry (#16109)

* Add property type counts to telemetry

* Use constants and fix tests

* Update description

* V10: Fix for fallback file upload (#14892) (#15868)

* Fix for fallback file upload (#14892)

* Added check for file type

* Removed unneeded null checks and fixed tabs

* Cleaning

* Cleanups, cleanups, and removal of unneeded null checks

* Reverted removal of relationshipservice

* Revert null check removals (too risky)

---------

Co-authored-by: Ambert van Unen <AvanUnen@ilionx.com>
Co-authored-by: Laura Neto <12862535+lauraneto@users.noreply.github.com>

(cherry picked from commit 0b5d1f8aa6)

* Fix up formatting

---------

Co-authored-by: Ambert van Unen <ambertvu@gmail.com>

* Implementors using Umbraco.Tests.Integration won't have to override GetLocalizedTextService

(cherry picked from commit b0016687eb)
(cherry picked from commit 2bb56f1b81)

* Fix logic for retrieving lastKnownElement

(cherry picked from commit cae106bfe8)

* bump version

* Bump version

* Bump version

* Since v13 properties can sometimes be of type IRichTextEditorIntermediateValue - this was unexpected in the XPath navigator code (#16121)

* Webhook log improvements (#16200)

* fix: include all headers in webhook log

* feat: return webhook log status from server

* feat: make webhook logs deep linkable

* feat: add webhook log pagination

* feat: improve webhook request/response body preview

* V13: Optimize custom MVC routing (#16218)

* Introduce EagerMatcherPolicy to conditionally bypass content routing

* Ensure that the candidate we disable dynamic routing for is valid

* Skip Umbraco endpoints

* Simplify logic a bit

* Move install logic to matcher

* Ensure that dynamic routing is still skipped when in upgrade state

* Fixup comments

* Reduce nesting a bit

* Don't show maintenance page when statically routed controllers are hít

* Remove excess check, since installer requests are statically routed

* V13: Optimize custom MVC routing (#16218)

* Introduce EagerMatcherPolicy to conditionally bypass content routing

* Ensure that the candidate we disable dynamic routing for is valid

* Skip Umbraco endpoints

* Simplify logic a bit

* Move install logic to matcher

* Ensure that dynamic routing is still skipped when in upgrade state

* Fixup comments

* Reduce nesting a bit

* Don't show maintenance page when statically routed controllers are hít

* Remove excess check, since installer requests are statically routed

(cherry picked from commit ba9ddd11da)

* Property source level variation should only be applied when configured (#16270)

* Property source level variation should only be applied when configured (#16270)

(cherry picked from commit ab32bac5d9)

* Merge pull request from GHSA-j74q-mv2c-rxmp

* Merge pull request from GHSA-j74q-mv2c-rxmp

* Merge pull request from GHSA-j74q-mv2c-rxmp

* Fix up after merge

* Remove obselete test

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
Co-authored-by: Jason Elkin <jasonelkin86@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>
Co-authored-by: Ambert van Unen <ambertvu@gmail.com>
Co-authored-by: Lars-Erik <lars-erik@aabech.no>
Co-authored-by: Joshua Daniel Pratt Nielsen <jdpnielsen@gmail.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Sebastiaan Janssen <sebastiaan@umbraco.com>
Co-authored-by: Rasmus John Pedersen <mail@rjp.dk>
2024-05-22 12:05:41 +02:00
Bjarke Berg
11e5257b56 V14: Untangle the preview functionality from the auth cookie (#16308)
* AB40660 - untangle the preview cookie from the auth cookie

* Clean up

* Allow anonymous to end preview sessions

* Some refinements

* update OpenApi.json

* Fix enter preview test

* correct tests to match new expectations of the preview cookie

* sync preview tests with correct expectations of access level

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
2024-05-17 16:06:26 +02:00
Sven Geusens
80794f3efd [V14] import/export media/document type endpoints (#16100)
* Working import/export media/document types

* WIP

* Refactoring of import doctype/media types

- added analyze endpoint to extract relevant data without fully processing the file
- split up import endpoints into POST & PUT
- removed availableAtAction as the new endpoint allows clients to call the POST/PUT endpoints with confidence
- Added a new service that is responsible for turning temp files into Import compatible XML and being able to extracty partial information from it

* Wrap persistance access in scopes

* Typos, formatting, clean-up

* PR feedback

* update openapi spec

* Changed deleteFile flag to _temporaryFileService.EnlistDeleteIfScopeCompletes

* Itty bitty typo

* Moved magic cleanup into its own method so orchestration can decide when.

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: kjac <kja@umbraco.dk>
2024-05-17 14:35:18 +02:00
Ronald Barendse
cbf9781ae8 DX: Add BlockValue constructors to force correct property editor alias and layout item type (#16266)
* Improve getting and initializing new block value layouts

* Remove unnecessary generic type constraints

* Add and use new block value/layout item constructors in tests

* Removed GetLayouts that did not make sense

* Added constructor to BlockItemData to simplify explicit usages

---------

Co-authored-by: kjac <kja@umbraco.dk>
2024-05-14 13:34:54 +02:00
Bjarke Berg
87c7347900 V14: Reintroduce umbraco api controller as obsolete (#16263)
* Revert "v14: Remove mentions of UmbracoApiController (#15863)"

This reverts commit 30e2dea57a.

* Obsolete UmbracoApiController

* Added a few more obsoletion messages

* Removed some of the reintroduced stuff again

* Add obsoletion to FrontEndRoutes controller

---------

Co-authored-by: kjac <kja@umbraco.dk>
2024-05-13 08:28:42 +02:00
Sven Geusens
f9c0235a35 [v14] Add missing alias and id to usergroup related api models (#16154)
* Added missing alias and Id to usergroup models

create/update/response/item

* Changed userGroup IsSystemGroup to more meaningfull fields

Also enforced the AliasCanBeChanged businessrule 🙈

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Mads Rasmussen <madsr@hey.com>
2024-05-03 10:24:09 +02:00
Kenn Jacobsen
8ad6c36038 Amend user start node handling (#16094)
* Amend user start node handling

* Add "has root access" to current user endpoint

* Add document and media root access to user response model

* Update OpenApi.json

* Applied API suggestions

---------

Co-authored-by: Sven Geusens <sge@umbraco.dk>
2024-05-03 08:47:10 +02:00
Kenn Jacobsen
374d699fd9 Move audit log endpoints to their respective silos and clean up (#16170)
* Move audit log endpoints to their respective silos and clean up

* Fix failing integration tests

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2024-05-01 12:07:06 +02:00