Commit Graph

161 Commits

Author SHA1 Message Date
Andy Butland
44aa5dcf1c Provides an abstraction for creating the JavaScriptEncoder used in SystemTextConfigurationEditorJsonSerializer (#19849)
* Provides an abstraction for creating the JavaScriptEncoder used in SystemTextConfigurationEditorJsonSerializer.

* Generalised JSON serialization encoder factory to work for all System.Tex.Json serializers.
Added the serializer's name as a parameter to allow for different encodings per serializer if required.

* Fixed tests by removing use of obsolete constructors.

* Removed name parameter and used a generic type instead.
2025-08-07 09:08:09 +02:00
Henrik
7f88013689 Make internal & private classes sealed where possible, to avoid code for virtual dispatch (#19719) 2025-07-21 08:32:54 +02:00
Laura Neto
5d968afb7e Fix failing integration tests in SQL Server (#19725)
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
2025-07-15 09:01:04 +02:00
Laura Neto
55506bac3a Simplify creating content from a blueprint programmatically (#19528)
* Rename `IContentService.CreateContentFromBlueprint` to `CreateBlueprintFromContent`

In reality, this method is used by the core to create a blueprint from content, and not the other way around, which doesn't need new ids. This was causing confusion, so the old name has been marked as deprecated in favor of the new name. If developers want to create content from blueprints they should use `IContentBlueprintEditingService.GetScaffoldedAsync()` instead, which is what is used by the management api.

* Added integration tests to verify that new block ids are generated when creating content from a blueprint

* Return copy of the blueprint in `ContentBlueprintEditingService.GetScaffoldedAsync` instead of the blueprint itself

* Update CreateContentFromBlueprint xml docs to mention both replacement methods

* Fix tests for rich text blocks

* Small re-organization

* Adjusted tests that were still referencing `ContentService.CreateContentFromBlueprint`

* Add default implementation to new CreateBlueprintFromContent method

* Update tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Services/ContentBlueprintEditingServiceTests.GetScaffold.cs

Co-authored-by: Andy Butland <abutland73@gmail.com>

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>
2025-06-24 13:43:34 +02:00
Kenn Jacobsen
8caf4ae3e0 Retrofit ContentEditingBaseBuilder to match new create/update models (#19129) 2025-04-24 12:28:05 +02:00
Kenn Jacobsen
2cf28271cd Service refactoring to "fully" enable segments (#19114)
* Refactor serverside content editing to support all variance combinations

* Fix build errors

* Reintroduce the tests ignored by #19060

---------

Co-authored-by: Mads Rasmussen <madsr@hey.com>
2025-04-23 14:54:51 +02:00
Andy Butland
ecbfc72323 Merge branch 'v15/dev' into v16/merge-from-15
# Conflicts:
#	src/Umbraco.Cms.Api.Management/Controllers/PublishedCache/RebuildPublishedCacheController.cs
#	src/Umbraco.Cms.Api.Management/Factories/UserPresentationFactory.cs
#	src/Umbraco.Core/Persistence/Repositories/ITrackedReferencesRepository.cs
#	src/Umbraco.Core/Services/ContentEditingService.cs
#	src/Umbraco.Core/Services/DataTypeService.cs
#	src/Umbraco.Core/Services/IContentEditingService.cs
#	src/Umbraco.Core/Services/IDataTypeService.cs
#	src/Umbraco.Core/Services/ITrackedReferencesService.cs
#	src/Umbraco.Core/Services/RelationService.cs
#	src/Umbraco.Core/Services/TrackedReferencesService.cs
#	src/Umbraco.Infrastructure/Examine/Deferred/DeliveryApiContentIndexHandleContentTypeChanges.cs
#	src/Umbraco.Infrastructure/Examine/DeliveryApiIndexingHandler.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/TrackedReferencesRepository.cs
#	src/Umbraco.Web.UI.Client/src/external/backend-api/src/sdk.gen.ts
#	src/Umbraco.Web.UI.Client/src/mocks/data/document-blueprint/document-blueprint.data.ts
#	src/Umbraco.Web.UI.Client/src/mocks/data/document/document.db.ts
#	src/Umbraco.Web.UI.Client/src/packages/core/router/modal-registration/modal-route-registration.controller.ts
#	src/Umbraco.Web.UI.Client/src/packages/core/router/route/route.context.ts
#	src/Umbraco.Web.UI.Client/src/packages/core/router/route/route.interface.ts
#	src/Umbraco.Web.UI.Client/src/packages/core/router/route/router-slot.element.ts
#	src/Umbraco.Web.UI.Client/src/packages/core/router/router-slot/model.ts
#	src/Umbraco.Web.UI.Client/src/packages/data-type/reference/repository/data-type-reference.server.data.ts
#	src/Umbraco.Web.UI.Client/src/packages/documents/documents/publishing/repository/document-publishing.server.data-source.ts
#	src/Umbraco.Web.UI.Client/src/packages/documents/documents/rollback/entity-action/rollback.action.ts
#	tests/Umbraco.Tests.AcceptanceTest/package-lock.json
#	tests/Umbraco.Tests.AcceptanceTest/package.json
#	tests/Umbraco.Tests.Common/Builders/UserGroupBuilder.cs
#	tests/Umbraco.Tests.Integration/Umbraco.Core/Services/TemporaryFileServiceTests.cs
2025-04-09 22:45:56 +02:00
Andy Butland
a0e3ca601e Aggregate document permissions for current user in API response (#18721)
* Create integration test verifying existing behaviour.

* Aggregate permissions per document for the current user response.

* Refactoring following Codescene warnings.
2025-04-09 10:57:50 +02:00
Kenn Jacobsen
3bd66b89e1 Merge branch 'v15/dev' into v16/dev (#18971)
* Only prevent the unpublish or delete of a related item when configured to do so if it is related as a child, not as a parent (#18886)

* Only prevent the unpubkish or delete of a related item when configured to do so if it is related as a child, not as a parent.

* Fixed incorect parameter names.

* Fixed failing integration tests.

* Use using variable instead to reduce nesting

* Applied suggestions from code review.

* Used simple using statement throughout RelationService for consistency.

* Applied XML header comments consistently.

---------

Co-authored-by: mole <nikolajlauridsen@protonmail.ch>

* Feature: highlight invariant doc with variant blocks is unsupported (#18806)

* mark variant blocks in invariant docs as invalid

* implement RTE Blocks

* Fix pagination for users restricted by start nodes (#18907)

* Fix pagination for users restricted by start nodes

* Default implementation to avoid breakage

* Review comments

* Fix failing test

* Add media start node tests

* Fix issue preventing blueprint derived values from being scaffolded (#18917)

* Fix issue preventing blueprint derived values from being scaffolded.

* fix manipulating frooen array

* compare with variantId as well

---------

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>

* ci: add Azure Static Web Apps workflow file
on-behalf-of: @Azure opensource@microsoft.com

* ci: add Azure Static Web Apps workflow file
on-behalf-of: @Azure opensource@microsoft.com

* ci: add Azure Static Web Apps workflow file
on-behalf-of: @Azure opensource@microsoft.com

* Remove admin permission on user configuration, allowing users with user section access only to manaage users and groups. (#18848)

* Tiptap RTE: Style Menu extension kind (#18918)

* Adds 'styleMenu' Tiptap toolbar extension kind

* Adds icons for `<h4>` and `<p>` tags

* Adds commands to HTML Global Attributes extension

for setting the `class` and `id` attributes.

* Renamed "default-tiptap-toolbar-element.api.ts" file

The "element" part was confusing.

* Toolbar Menu: uses correct `item` value

* Cascading Menu: adds localization for the label

* Adds `label` attribute to UUI components

for accessibility.

* Toolbar Menu: uses correct `appearance` value

* Removed unrequired `api` from Style Select

* Destructs the `item.data` object

* Ensure has children reflects only items with folder children when folders only are queried. (#18790)

* Ensure has children reflects only items with folder children when folders only are queried.

* Added supression for change to integration test public code.

---------

Co-authored-by: Migaroez <geusens@gmail.com>

* Only apply validation on content update to variant cultures where the editor has permission for the culture (#18778)

* Only apply validation on content update to variant cultures where the editor has permission for the culture.

* Remove inadvertent comment updates.

* Fixed failing integration test.

* Adds ancestor ID details on document tree and collection responses (#18909)

* Populate ancestor keys on document tree response items.

* Populate ancestor keys on document collection response items.

* Update OpenApi.json

* Use array of objects rather than Ids for the ancestor collection.

* Update OpenApi.json.

* Move publish with descendants to a background task with polling (#18497)

* Use background queue for database cache rebuild and track rebuilding status.

* Updated OpenApi.json and client-side types.

* Updated client to poll for completion of database rebuild.

* Move IBackgroundTaskQueue to core and prepare publish branch to run as background task.

* Endpoints for retrieval of status and result from branch publish operations.

* Poll and retrieve result for publish with descendants.

* Handled issues from testing.

* Rework to single controller for status and result.

* Updated client side sdk.

* OpenApi post dev merge gen

---------

Co-authored-by: Migaroez <geusens@gmail.com>

* Clear roots before rebuilding navigation dictionary (#18766)

* Clear roots before rebuilding navigation dictionary.

* Added tests to verify fix.

* Correct test implementation.

* Convert integration tests with method overloads into test cases.

* Integration test compatibility supressions.

* Fixes save of empty, invariant block list on variant content. (#18932)

* remove unnecessary code (#18927)

* V15/bugfix/fix route issue from 18859 (#18931)

* unique check

* unique for workspace empty path

* more unique routes

* Bump vite from 6.2.3 to 6.2.4 in /src/Umbraco.Web.UI.Client

Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.2.3 to 6.2.4.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.4/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.4/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.4
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* removes autogenerated workflows

* make getHasUnpersistedChanges public (#18929)

* Added management API endpoint, service and repository for retrieval of references from the recycle bin (#18882)

* Added management API endpoint, service and repository for retrieval of references from the recycle bin.

* Update src/Umbraco.Cms.Api.Management/Controllers/Document/RecycleBin/ReferencedByDocumentRecycleBinController.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Removed unused code.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Updated management API endpoint and model for data type references to align with that used for documents, media etc. (#18905)

* Updated management API endpoint and model for data type references to align with that used for documents, media etc.

* Refactoring.

* Update src/Umbraco.Core/Constants-ReferenceTypes.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fixed typos.

* Added id to tracked reference content type response.

* Updated OpenApi.json.

* Added missing updates.

* Renamed model and constants from code review feedback.

* Fix typo

* Fix multiple enumeration

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>

* Skip lock tests

* Look-up redirect in content finder for multi-lingual sites using path and legacy route prefixed with the integer ID of the node with domains defined (#18763)

* Look-up redirect in content finder for multi-lingual sites using path and legacy route prefixed with the integer ID of the node with domains defined.

* Added tests to verify functionality.

* Added reference to previous PR.

* Referenced second PR.

* Assemble URLs for all cultures, not just the default.

* Revert previous update.

* Display an original URL if we have one.

* Bump vite from 6.2.4 to 6.2.5 in /src/Umbraco.Web.UI.Client

Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.2.4 to 6.2.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.5/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.5
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add raw value validation to multiple text strings property editor (#18936)

* Add raw value validation to multiple text strings property editor

* Added additional assert on unit test and comment on validation logic.

* Don't remove items to obtain a valid value

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>

* Integration tests for content publishing with ancestor unpublished (#18941)

* Resolved warnings in test class.

* Refactor regions into partial classes.

* Aligned test names.

* Variable name refactoring.

* Added tests for unpublished paths.

* Adjust tests to verify current behaviour.

* Cleaned up project file.

* fix circular icon import (#18952)

* remove segment toggle for elements (#18949)

* Fix modal route registration circular import (#18953)

* fix modal route registration circular import

* Update modal-route-registration.controller.ts

* V15/fix/18595 (#18925)

* fix for #18595

* updates the en.ts

* Avoid unneeded Dictionary operations (#18890)

* Avoid some heap allocations

* Remove unneeded double seek

* Avoid allocating new empty arrays, reuse existing empty array

* Avoid allocating strings for parsing comma separated int values (#18199)

* Data type References UI: Workspace + Delete (#18914)

* Updated management API endpoint and model for data type references to align with that used for documents, media etc.

* Refactoring.

* Update src/Umbraco.Core/Constants-ReferenceTypes.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fixed typos.

* generate server models

* add extension slot

* register data type reference info app

* add reference data mappers

* Added id to tracked reference content type response.

* Updated OpenApi.json.

* Added missing updates.

* generate new models

* update models

* register ref item

* remove debugger

* render types

* register member type property type ref

* register media type property type ref

* Renamed model and constants from code review feedback.

* register reference workspace info app kind

* use kind for document references

* use kind for media references

* use kind for member references

* use deleteWithRelation kind when deleting data types

* fix manifest types

* fix types

* Update types.gen.ts

* update code to fit new server models

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Feature: discard changes for block workspace (#18930)

* make getHasUnpersistedChanges public

* Discard changes impl for Block Workspace

* fix 18367 (#18956)

* Merge commit from fork

* Prevent path traveral vulnerability with upload of temporary files.

* Used BadRequest instead of NotFound for invalid file name response.

* V15 QA Fixing the failing media acceptance tests (#18881)

* Fixed the function name due to test helper changes

* Updated assertion steps due to UI changes

* Added more waits

* Bumped version

* Increase timeout

* Reverted

---------

Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>

* V15 QA added clipboard test for not being able to copy to root when block is not allowed at root (#18937)

* Added clipboard test

* Bumped version

* Updated to use the name

* Run all tests on the pipeline

* Reverted command

* build: adjusts circular ref number to 4

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>
Co-authored-by: Niels Lyngsø <nsl@umbraco.dk>
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
Co-authored-by: Lee Kelleher <leekelleher@users.noreply.github.com>
Co-authored-by: Migaroez <geusens@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Mads Rasmussen <madsr@hey.com>
Co-authored-by: Jacob Welander Jensen <64834767+Welander1994@users.noreply.github.com>
Co-authored-by: Henrik <hg@impact.dk>
Co-authored-by: Sebastiaan Janssen <sebastiaan@umbraco.com>
Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
2025-04-09 09:58:01 +02:00
Andy Butland
3d1e17b07f Integration tests for content publishing with ancestor unpublished (#18941)
* Resolved warnings in test class.

* Refactor regions into partial classes.

* Aligned test names.

* Variable name refactoring.

* Added tests for unpublished paths.

* Adjust tests to verify current behaviour.

* Cleaned up project file.
2025-04-07 10:04:11 +02:00
Andy Butland
1f4c19d484 Updated management API endpoint and model for data type references to align with that used for documents, media etc. (#18905)
* Updated management API endpoint and model for data type references to align with that used for documents, media etc.

* Refactoring.

* Update src/Umbraco.Core/Constants-ReferenceTypes.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fixed typos.

* Added id to tracked reference content type response.

* Updated OpenApi.json.

* Added missing updates.

* Renamed model and constants from code review feedback.

* Fix typo

* Fix multiple enumeration

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>
2025-04-04 15:10:06 +02:00
Andy Butland
3e6b9313e5 Only apply validation on content update to variant cultures where the editor has permission for the culture (#18778)
* Only apply validation on content update to variant cultures where the editor has permission for the culture.

* Remove inadvertent comment updates.

* Fixed failing integration test.
2025-04-03 22:09:40 +02:00
Henrik
95e89f8481 Avoid a hash key generation and lookup when inserting in the LockingMechanism (#18243)
* Avoid a hash key generation and lookup when inserting in the LockingMechanism

* Added comments for CollectionsMarshal.GetValueRefOrAddDefault

* Added further comments and tests.

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>
2025-03-26 16:55:03 +01:00
Mole
6668f3a867 V15: Add authorization to create (#18169)
* Add failing tests

* Re-add auth
2025-01-30 16:00:33 +01:00
Mole
c9758edfb1 V15: Add authorization to saves (#18111)
* Re-add authorization

* Add test plumbing

* Add test helper

* Add happy path test

* Remove usage of negation

* Minor DRYup of test code.

---------

Co-authored-by: Andy Butland <abutland73@gmail.com>
2025-01-24 13:45:56 +01:00
Sebastiaan Janssen
37343b24bb Merge branch 'contrib' into v15/dev 2025-01-15 16:54:02 +01:00
Andy Butland
413398afc6 Use data type configuration to determine default value for empty toggle and slider property values (#17854)
* Use data type configuration to determine default value for empty toggle property values.

* Added/updated unit tests.

* Fixed failing integration tests.

* Applied similar default value display for the slider property editor and aligned implementation of true/false with this.

* Fixed unit tests.

* Removed "duplicate" JsonPropertyName attributes and added a custom TypeInfoResolver for data type configuration so we can re-use the existing ConfigurationField attributes.

* Minor cleanup

---------

Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2025-01-08 11:42:13 +01:00
Jeroen Breuer
09575599ad Fix Umbraco.Tests.Common warnings. (#17172)
* Fix async warning.

* Fix MediaFileManager warning.

* Fix TypeLoader warning.

* Fix CodeBase warning.

* Fix ContentTypeEditingBuilder warning.

* Fix _containerKey warning.

* Fix _key warning.

* Fix issues due to merge with contrib

---------

Co-authored-by: Laura Neto <12862535+lauraneto@users.noreply.github.com>
2025-01-03 17:59:50 +01:00
Jacob Overgaard
07b7c26fd9 Feature: Build static assets conditionally for Backoffice and Login (#17527)
* feat: conditionally install and build the 'login' and 'backoffice' targets depending on source files

* feat: remove the preconditions target, because we are now compiling a file list to check if we need to build

* feat: remove the 'clean' targets, because the project will be cleaned if any of the compiled file lists do not exist

* feat: remove the preserve.* files as they are no longer needed

* Enable default content items again

* Remove package.json from restore target inputs

* Include generated files as static assets after build

* Clean up project files

* Exclude CS0618 warning as error and fix CA2264

* Exclude CS0612 warning as error

* Suppress removal of test fixture

* Use separate property/item groups for backoffice and login project

---------

Co-authored-by: Ronald Barendse <ronald@barend.se>
2024-11-19 13:29:39 +01:00
Bjarke Berg
c9fa2a57d7 Merge branch 'release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
#	src/Umbraco.Core/Extensions/PublishedContentExtensions.cs
#	src/Umbraco.Web.UI.Client/src/packages/tiptap/components/input-tiptap/tiptap-toolbar.element.ts
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/ContentNavigationServiceBaseTests.cs
2024-11-12 12:14:50 +01:00
Ronald Barendse
b002ac8600 Add UDI support for relation, user and user-group entity types (#17490)
* Add GetUdi() extension methods for IUser and IUserGroup

* Add GetUdi() extension methods for IRelation

* Move GetUdi() extension methods for Script and Stylesheet to interface types

* Register relation, user and user-group as known UDI types

* Obsolete UdiParserServiceConnectors

* Use IScript and IStylesheet types in tests

* Fix UdiTests after obsoleting UdiParserServiceConnectors
2024-11-12 08:45:58 +01:00
Sven Geusens
e7eb14d310 V15/task/cleanup obsolete (#17433)
* Replace obsolete UserGroup Alias consts to key equivalent in tests

* Update use of usergroup alias consts to key equivalent in IsSystemgroup extension method

* Obsolete (internally) unused helper function which purpose doesn't even seem true

* Prepped EmbedProviders for proper removal of non async methods and unneeded proxy methods

* Remove obsoleted UmbracoPath and updated internal references

* Corrected  mistake and updated unittets

* Update usergroup tests that use aliases for "system" groups

* Replace more uses of globalsettings.UmbracoPath

* Remove GetDateType by key non async

* Cleanup some usages of hostingEnvironment.MapPathContentRoot

* More easy obsoletion cleanup

* Small Typeload cleanup

* More obsolete removal

* Deploy obsoletion cleanup

* Remove obsolete methods from OEmbedProviderBase.cs

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-11-07 12:20:22 +01:00
Jason Prothero
421a3a9bd9 Fix stylecop errors with function parameters on separate lines 2024-10-12 02:00:56 +02:00
Bjarke Berg
6e0ab4dd7e Merge remote-tracking branch 'origin/release/15.0' into v15/dev
# Conflicts:
#	src/Umbraco.PublishedCache.HybridCache/Services/MediaCacheService.cs
#	src/Umbraco.Web.UI.Client
#	tests/Umbraco.Tests.Common/Builders/MediaTypeEditingBuilder.cs
2024-10-10 11:11:08 +02:00
Mole
2b12eadd3d V15: Refresh IPublishedContentTypeFactory when data types change (#17180)
* Refresh IPublishedContentTypeFactory when data types change

* Add tests
2024-10-07 11:11:22 +02:00
Sebastiaan Janssen
4090829fe8 Ignore warnings about using obsolete members in Umbraco.Tests.UnitTests.csproj and Umbraco.Core.csproj
LocalFileSystemTemporaryFileRepository: Don't pass null to a method that doesn't accept nullable argument - https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2264
Fix remaining warning marked as error
2024-10-02 14:37:37 +02:00
Andreas Zerbst
04d71fa354 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:37:50 +02:00
Andreas Zerbst
e15b7f2214 V15 QA Hybrid Cache added media cache tests and fixed media cache refreshing (#17160)
* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

* Builder testing

* Media test

* Created builders

* Updated method from getByKey to getById to maintain structure

* Updated test setup

* Moved tests

* Cleaned up builders

* Added helper

* Added space

* Updated tests

* Cleaned setup

* Added tests for MediaTypes

* Uncommented tests

* Added builder extensions

* Added interfaces for builder methods

* Updated to use interface pattern

* Updated name

* Used builder

* Cleaned up test

* Added media scope tests

* Removed PropertyTypeModelBuilder

* moved PropertyTypeModelBuilder to PropertyTypeEditingBuilder

* Removed constructor

* Removed duplicate

* Fixed naming

* Reverted

* Removed space

* Added todos

* Removed only run on linux

* Commet out DocumentHybridCacheTemplateTest again

This is fixed in the release branch

* Update media when media type is updated

* Remove todo the tests pass now

---------

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-10-02 13:36:26 +02:00
Sebastiaan Janssen
c2d83808a5 Merge branch 'v15/dev' into contrib
# Conflicts:
#	src/Umbraco.Infrastructure/PublishedContentQuery.cs
#	src/Umbraco.PublishedCache.NuCache/ContentCache.cs
#	src/Umbraco.PublishedCache.NuCache/ContentStore.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.ContentNodeKitSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/DataSource/BTree.DictionaryOfPropertyDataSerializer.cs
#	src/Umbraco.PublishedCache.NuCache/Umbraco.PublishedCache.NuCache.csproj
#	tests/Umbraco.Tests.UnitTests/Umbraco.Infrastructure/PublishedCache/PublishContentCacheTests.cs
2024-10-02 12:35:20 +02:00
Mole
1258962429 V15: Remove Nucache (#17166)
* Remove nucache reference from Web.Common

* Get tests building-ish

* Move ReservedFieldNamesService to the right project

* Remove IPublishedSnapshotStatus

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Remove IPublishedSnapshot from UmbracoContext

* Begin removing usage of IPublishedSnapshot from PublishedContentExtensions

* Fix PublishedContentExtensions.cs

* Don't use snapshots in delivery media api

* Use IPublishedMediaCache in QueryMediaApiController

* Remove more usages of IPublishedSnapshotAccessor

* Comment out tests

* Remove more usages of PublishedSnapshotAccessor

* Remove PublishedSnapshot from property

* Fixed test build

* Fix errors

* Fix some tests

* Delete NuCache 🎉

* Implement DatabaseCacheRebuilder

* Remove usage of IPublishedSnapshotService

* Remove IPublishedSnapshotService

* Remove TestPublishedSnapshotAccessor and make tests build

* Don't test Snapshot cachelevel

It's no longer supported

* Fix BlockEditorConverter

Element != Element document type

* Remember to set cachemanager

* Fix RichTextParserTests

* Implement TryGetLevel on INavigationQueryService

* Fake level and obsolete it in PublishedContent

* Remove ChildrenForAllCultures

* Hack Path property on PublishedContent

* Remove usages of IPublishedSnapshot in tests

* More ConvertersTests

* Add hybrid cache to integration tests

We can actually do this now because we no longer save files on disk

* Rename IPublishedSnapshotRebuilder to ICacheRebuilder

* Comment out tests

* V15: Replacing the usages of Parent (navigation data) from IPublishedContent (#17125)

* Fix .Parent references in PublishedContentExtensions

* Add missing methods to FriendlyPublishedContentExtensions (ones that you were able to call on the content directly as they now require extra params)

* Fix references from the extension methods

* Fix dependencies in tests

* Replace IPublishedSnapshotAccessor with the content cache in tests

* Resolving more .Parent references

* Fix unit tests

* Obsolete and use extension methods

* Remove private method and use extension instead

* Moving code around

* Fix tests

* Fix more references

* Cleanup

* Fix more usages

* Resolve merge conflict

* Fix tests

* Cleanup

* Fix more tests

* Fixed unit tests

* Cleanup

* Replace last usages

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>

* Remove usage of IPublishedSnapshotAccessor from IRequestItemProvider

* Post merge fixup

* Remo IPublishedSnapshot

* Add HasAny to IDocumentUrlService

* Fix TextBuilder

* Fix modelsbuilder tests

* Use explicit types

* Implement GetByContentType

* Support element types in PublishedContentTypeCache

* Run enlistments before publishing notifications

* Fix elements cache refreshing

* Implement GetByUdi

* Implement GetAtRoot

* Implement GetByRoute

* Reimplement GetRouteById

* Fix blocks unit tests

* Initialize domain cache on boot

* Only return routes with domains on non default lanauges

* V15: Replacing the usages of `Children` (navigation data) from `IPublishedContent` (#17159)

* Update params in PublishedContentExtensions to the general interfaces for the published cache and navigation service, so that we can use the extension methods on both documents and media

* Introduce GetParent() which uses the right services

* Fix obsolete message on .Parent

* Obsolete .Children

* Fix usages of Children for ApiMediaQueryService

* Fix usage in internal

* Fix usages in views

* Fix indentation

* Fix issue with delete language

* Update nuget pacakges

* Clear elements cache when content is deleted

instead of trying to update it

* Reset publishedModelFactory

* Fixed publishing

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: kjac <kja@umbraco.dk>
2024-10-01 15:03:02 +02:00
Sebastiaan Janssen
8d81c70391 Merge branch 'v15/dev' into contrib 2024-09-27 09:09:18 +02:00
Sebastiaan Janssen
910fb4062b Merge branch 'v14/dev' into contrib 2024-09-27 08:54:49 +02:00
Bjarke Berg
1951ac96b5 Merge remote-tracking branch 'refs/remotes/origin/v14/dev' into v15/dev
# Conflicts:
#	build/nightly-E2E-test-pipelines.yml
#	src/Umbraco.Infrastructure/Security/BackOfficeIdentityUser.cs
#	src/Umbraco.Web.UI.Client
#	src/Umbraco.Web.UI.Login/package-lock.json
#	version.json
2024-09-26 08:56:43 +02:00
Ronald Barendse
609b5f76d4 Fix IContentBase.GetUdi() extension method to support document-blueprint entity type (#16939)
* Add tests for all UDI entity types

* Fix IContentBase UDI entity type for blueprints

* Remove redundant switch statements and reorder methods
2024-09-26 07:47:33 +02:00
Emma L Garland
ac575662ac Resolved more warnings, and marked more warning types as errors (#16991)
* Fix warnings SA1111, SA1028, SA1500, IDE1270  in Umbraco.Web.Website, and updated rules.

* Remove warnings: IDE0270: Null check can be simplified

* More SqlServer project warnings resolved

* CS0105 namespace appeared already

* Suppress warning until implementation:

#pragma warning disable CS0162 // Unreachable code detected
#pragma warning disable CS0618 // Type or member is obsolete

CS0162 remove unreachable code
SA1028 remove trailing whitespace
SA1106 no empty statements
CS1570 malformed XML
CS1572 corrected xml parameter
CS1573 param tag added
IDE0007 var not explicit
IDE0008 explicit not var
IDE0057 simplify substring
IDE0074 compound assignment
CA1825 array.empty

Down to 3479 warnings

* - SA1116, SA117 params on same line
- IDE0057 substring simplified

Specific warnings for Umbraco.Tests.Benchmarks

* Fixed IDE0074 compound assignment and added specific warnings for Umbraco.Tests.Common

* Specific warnings for Umbraco.Tests.Integration and Umbraco.Tests.Common

Fixed:

- SA1111, SA1116, SA117 params and line formatting (not all as there are many)
- SA1122 string.Empty
- IDE0057 simplify substring
- IDE0044,IDE0044 make field readonly
- IDE1006 naming rule violation (add _)
- SA1111 closing parenthesis on line of last parameter
- SA1649 filename match type name
- SA1312,SA1306 lowercase variable and field names

* Fixed various warnings where they are more straight-forward, including:

- SA1649 file name match type name
- SA111 parenthesis on line of last parameter
- IDE0028 simplify collection initializer
- SA1306 lower-case letter field
- IDE044 readonly field
- SA1122 string.Empty
- SA1116 params same line
- IDE1006 upper casing
- IDE0041 simplify null check

Updated the following projects to only list their remaining specific warning codes:

- Umbraco.Tests.UnitTests

Typo in `Umbraco.Web.Website` project

* Reverted test change

* Now 1556 warnings.

Fixed various warnings where they are more straight-forward, including:

- SA1111/SA1116/SA1119 parenthesis
- SA1117 params
- SA1312 lowercase variable
- SA1121 built-in type
- SA1500/SA1513/SA1503 formatting braces
- SA1400 declare access modifier
- SA1122 string.Empty
- SA1310 no underscore
- IDE0049 name simplified
- IDE0057 simplify substring
- IDE0074 compound assignment
- IDE0032 use auto-property
- IDE0037 simplify member name
- IDE0008 explicit type not var
- IDE0016/IDE0270/IDE0041 simplify null checks
- IDE0048/SA1407 clarity in arithmetic
- IDE1006 correct param names
- IDE0042 deconstruct variable
- IDE0044 readonly
- IDE0018 inline variable declarations
- IDE0074/IDE0054 compound assignment
- IDE1006 naming
- CS1573 param XML
- CS0168 unused variable

Comment formatting in project files for consistency.

Updated all projects to only list remaining specific warning codes as warnings instead of errors (errors is now default).

* Type not var, and more warning exceptions

* Tweaked merge issue, readded comment about rollback

* Readded comment re rollback.

* Readded comments

* Comment tweak

* Comment tweak
2024-09-24 12:56:28 +01:00
Mole
e342e795dd V15: Cache Seeding (#17102)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Updated tests

* Added tests

* Use init for ContentSourceDto

* Startup of setup

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Created interfaces for the builder with the necessary properties

* Created builder for PropertyTypeContainer

* Created builder for PropertyTypeEditing

* Created builder for PropertyTypeValidationEditing

* Made adjustments to the builder

* Updated name of usage

* Commented out to test

* Cleaned up builders

* Updated integration test setup

* Moved tests

* Added interface

* Add IDocumentSeedKeyProvider and migrate existing logic to seed key provider

* Added functionality to the INavigationQueryService to get root keys

* Fixed issue with navigation

* Created helper to Convert a IContentType to ContentTypeUpdateModel

* Added interfaces

* Added builder

* Cleaned up builders and added fixes

* Added tests for PublishedContentTypeCache

* Applied changes in builder

* Add BreadthFirstKeyProvider

* Use ISet for seedkey providers

* Implement GetContentSource by key

* Seed the cache with keys provided by seed key providers

* Builder updates

* Test setup updates

* Updated tests

* Dont require contenttype keys for seeding

* Fix cache settings

* Don't inject cache settings into SeedingNotificationHandler

* Fix tests

* Use enlistment for setting updated cache item

* Pin seeded nodes for longer

* Fix BreadthFirstKeyProvider

* Fix ContentTypeSeedKeyProvider

* Fix tests

* Only seed published documents

* Only cache published if contentCacheNode is not draft

* Fix incorrect templateId

* Removed unnecessary setup

* initialized value

* Fixed template test

* Removed test

* Updated tests

* Removed code that was not used

* Removed unused cacheSettings

* Re-organize to support media cache seeding

* Add MediaBreadthFirstKeyProvider

* Seed media

* Don't use IdKeyMap when removing content from cache

* Don't clear IdKeyMap in DocumentCacheService

* Add unit tests

* Don't use IdKeyMap when deleting media

* Add default value to timespan

* Use cancellation tokens when doing loop

* Fixed Models Builder error

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-09-24 09:39:23 +02:00
Nikolaj Geisle
2704d4a34a V15: Hybrid Caching (#16938)
* Update to dotnet 9 and update nuget packages

* Update umbraco code version

* Update Directory.Build.props

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Include preview version in pipeline

* update template projects

* update global json with specific version

* Update version.json to v15

* Rename TrimStart and TrimEnd to string specific

* Rename to Exact

* Update global.json

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Remove includePreviewVersion

* Rename to trim exact

* Add new Hybridcache project

* Add tests

* Start implementing PublishedContent.cs

* Implement repository for content

* Refactor to use async everywhere

* Add cache refresher

* make public as needed for serialization

* Use content type cache to get content type out

* Refactor to use ContentCacheNode model, that goes in the memory cache

* Remove content node kit as its not needed

* Implement tests for ensuring caching

* Implement better asserts

* Implement published property

* Refactor to use mapping

* Rename to document tests

* Update to test properties

* Create more tests

* Refactor mock tests into own file

* Update property test

* Fix published version of content

* Change default cache level to elements

* Refactor to always have draft

* Refactor to not use PublishedModelFactory

* Added tests

* Added and updated tests

* Fixed tests

* Don't return empty object with id

* More tests

* Added key

* Another key

* Refactor CacheService to be responsible for using the hybrid cache

* Use notification handler to remove deleted content from cache

* Add more tests for missing functions

* Implement missing methods

* Remove HasContent as it pertains to routing

* Fik up test

* formatting

* refactor variable names

* Implement variant tests

* Map all the published content properties

* Get item out of cache first, to assert updated

* Implement member cache

* Add member test

* Implement media cache

* Implement property tests for media tests

* Refactor tests to use extension method

* Add more media tests

* Refactor properties to no longer have element caching

* Don't use property cache level

* Start implementing seeding

* Only seed when main

* Add Immutable for performance

* Implement permanent seeding of content

* Implement cache settings

* Implement tests for seeding

* Update package version

* start refactoring nurepo

* Refactor so draft & published nodes are cached individually

* Refactor RefreshContent to take node instead of IContent

* Refactor media to also use cache nodes

* Remove member from repo as it isn't cached

* Refactor media to not include preview, as media has no draft

* create new benchmark project

* POC Integration benchmarks with custom api controllers

* Start implementing content picker tests

* Implement domain cache

* Rework content cache to implement interface

* Start implementing elements cache

* Implement published snapshot service

* Publish snapshot tests

* Use snapshot for elements cache

* Create test proving we don't clear cache when updating content picker

* Clear entire elements cache

* Remove properties from element cache, when content gets updated.

* Rename methods to async

* Refactor to use old cache interfaces instead of new ones

* Remove snapshot, as it is no longer needed

* Fix tests building

* Refactor domaincache to not have snapshots

* Delete benchmarks

* Delete benchmarks

* Add HybridCacheProject to Umbraco

* Add comment to route value transformer

* Implement is draft

* remove snapshot from property

* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)

* Added builder extension withParentKey

* Created builder with ContentEditingService

* Added usage of the ContentEditingService to SETUP

* Started using ContentEditingService builder in tests

* Updated builder extensions

* Fixed builder

* Clean up

* Clean up, not done

* Added Ids

* Remove entries from cache on delete

* Fix up seeding logic

* Don't register hybrid cache twice

* Change seeded entry options

* Update hybrid cache package

* Fix up published property to work with delivery api again

* Fix dependency injection to work with tests

* Fix naming

* Dont make caches nullable

* Make content node sealed

* Remove path and other unused from content node

* Remove hacky 2 phase ctor

* Refactor to actually set content templates

* Remove umbraco context

* Remove "HasBy" methods

* rename property data

* Delete obsolete legacy stuff

* Add todo for making expiration configurable

* Add todo in UmbracoContext

* Add clarifying comment in content factory

* Remove xml stuff from published property

* Fix according to review

* Make content type cache injectible

* Make content type cache injectible

* Rename to database cache repository

* Rename to document cache

* Add TODO

* Refactor to async

* Rename to async

* Make everything async

* Remove duplicate line from json schema

* Move Hybrid cache project

* Remove leftover file

* Refactor to use keys

* Refactor published content to no longer have content data, as it is on the node itself

* Refactor to member to use proper content node ctor

* Move tests to own folder

* Add immutable objects to property and content data for performance

* Make property data public

* Fix member caching to be singleton

* Obsolete GetContentType

* Remove todo

* Fix naming

* Fix lots of exposed errors due to scope test

* Add final scope tests

* Rename to document cache service

* Rename test files

* Create new doc type tests

* Add ignore to tests

* Start implementing refresh for content type save

* Clear contenttype cache when contenttype is updated

* Fix test

Teh contenttype is not upated unless the property is dirty

* Use init for ContentSourceDto

* Fix get by key in PublishedContentTypeCache

* Remove ContentType from PublishedContentTypeCache when contenttype is deleted

* Update to preview 7

* Fix versions

* Increase timeout for sqlite integration tests

* Undo timeout increase

* Try and undo init change to ContentSourceDto

* That wasn't it chief

* Try and make DomainAndUrlsTests non NonParallelizable

* Update versions

* Only run cache tests on linux for now

---------

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2024-09-09 17:49:18 +02:00
Emma L Garland
98e9a300bc Treat warnings as errors for initial projects (#15019)
* Treat warnings as errors for Umbraco.Cms, Umbraco.Cms.Api.Common and Umbraco.Cms.Api.Delivery projects.

* Treat warnings as errors for Umbraco.Cms.Imaging.ImageSharp and Umbraco.Cms.Imaging.ImageSharp2 projects

* Treat warnings as errors for Umbraco.Cms.Persistence.EFCore.Sqlite, Umbraco.Cms.Persistence.EFCore.SqlServer and Umbraco.Cms.StaticAssets

* Treat warnings as errors for Umbraco.Cms.Targets

* Treat warnings as errors for templates/Umbraco.Templates and Umbraco.JsonSchema

* More projects with warnings as errors.

* Reverted warnings as errors due to some file changes since the update.

* Remove unwanted tag

* Removed warnings as errors on TestData project

* Implement warnings as errors in props file, and exclude specific warnings where appropriate as per PR review suggestions.

* Reverted spaces

* Revert "Reverted spaces"

This reverts commit 3734c45e2270c3324768b33e459aefcc6a8c4739.

* Update sdk version to fully support Umbraco.code 2.2.0 dependency on Microsoft.CodeAnalysis.CSharp.Workspaces 4.10.0

* Ignore PathToLongErrors for now in static assets

---------

Co-authored-by: Emma Garland <emma.garland@rocksolidknowledge.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
2024-08-27 18:56:09 +02:00
Elitsa Marinovska
c8180d508b V14: Test new content and media list view service (#16193)
* Fix operation status msg

* Cleanup

* Removing unused status

* Adding WithConfigurationEditor on DataEditorBuilder to be able to create new list views with custom configuration

* Adding list view service tests for content and media

* Adding list view service tests base

* Consistency

* Clean up

* More cleanup

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-05-01 10:54:26 +02:00
Kenn Jacobsen
acae5f2d57 Remove "additional data" from entities (#16024)
* Remove "additional data" from entities

* Fix merge issue

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-04-11 10:55:21 +02:00
Ronald Barendse
1c5e27397b Remove Newtonsoft.Json from tests (#16029) 2024-04-10 19:30:32 +02:00
Elitsa Marinovska
9c18cd22e0 V14: Deleted code marked as obsolete for V14 (#15998)
* Obsoletions related to Delivery API

* Fix TypeLoader and TypeFinder tests

* Remove obsolete and default implementations of IFileSource and IFileTypeCollection

* More Delivery API related obsoletions

* VariationContextAccessor related

* ValueFactories obsoletion and fix references

* ValueSetBuilders obsoletions

* ValueConverters obsoletions

* Other obsolete ctors and methods

* Forgotten VariationContextAccessor obsoletion

* More obsoletions

* XPath related obsoletions

* Revert XmlHelper changes

* Delete RenamedRootNavigator and its tests

* Fix test

* XmlHelper obsoletion

* Return null instead of GetXPathValue

* Obsolete entire class instead

* Remove XPath obsoletions from IPublishedCache

* Remove XPath-related if-block that is no longer needed

* Change obsolete msg for classes needed for NuCache

* Moving classes to NuCache and making them internal

* Remove more XPath-related obsoletions

* Remove NavigableNavigator and its tests

* Cleanup

* Remove Xpath references from tests

* Revert interface deletion in MediaCache

* Using XOR operation

Co-authored-by: Nuklon <Nuklon@users.noreply.github.com>

---------

Co-authored-by: Nuklon <Nuklon@users.noreply.github.com>
2024-04-09 09:06:48 +02:00
Sven Geusens
67d80da9ed Merge branch 'v13/dev' into v14/dev
# Conflicts:
#	src/Umbraco.Infrastructure/PropertyEditors/MultiUrlPickerValueEditor.cs
#	src/Umbraco.Web.Common/Profiler/WebProfiler.cs
#	src/Umbraco.Web.UI.Client/package-lock.json
#	src/Umbraco.Web.UI.Client/src/common/directives/validation/valpropertymsg.directive.js
#	src/Umbraco.Web.UI.Client/src/views/common/infiniteeditors/linkpicker/linkpicker.controller.js
#	src/Umbraco.Web.UI.Client/src/views/common/infiniteeditors/linkpicker/linkpicker.html
#	src/Umbraco.Web.UI.Client/src/views/content/content.create.controller.js
#	src/Umbraco.Web.UI.Client/src/views/content/create.html
#	src/Umbraco.Web.UI.Client/src/views/propertyeditors/blockgrid/umbBlockGridPropertyEditor.component.js
#	src/Umbraco.Web.UI.Client/src/views/propertyeditors/multiurlpicker/multiurlpicker.controller.js
#	src/Umbraco.Web.UI.Client/src/views/propertyeditors/multiurlpicker/multiurlpicker.html
#	src/Umbraco.Web.UI.Client~HEAD

# Ignored changes
#  https://github.com/umbraco/Umbraco-CMS/pull/14787
#  https://github.com/umbraco/Umbraco-CMS/pull/15853
#  https://github.com/umbraco/Umbraco-CMS/pull/15841
2024-03-27 16:13:09 +01:00
nzdev
ac5539a8be Avoid string.Format allocations when profiler is not enabled. 2024-03-13 14:39:57 +00:00
Kenn Jacobsen
69a691f315 Removed unused content editing models (#15855) 2024-03-11 13:15:25 +01:00
Mole
9cb1d66b10 V14: Remove Newtonsoft from packed projects (#15811)
* Migrate ObjectJsonExtensions

* Use more generic exception to not use Newtonsoft

It should matter if it's a JsonReaderException, if we can't read we can't read

* Remove obsoleted constructors

* Use more generic exception in ContentValueSetBuilder

* Fix constructors

* Remove UdiRangeJsonConverter

* Remove more legacy newtonsoft stuff

* Migrate away from newtonsoft in CacheInstructionService

* Remove unused model binders

* Remove more newtonsoft

* Remove newtonsoft from DatabaseServerMessenger

* Remove now irrelevant benchmark

* Remove the usage of Newtonsoft from ImageCropperTemplateCoreExtensions

The value converter will never return JObject, JsonDocument, or JsonNode

* Remove usages of newtonsoft in ComplexPropertyEditorContentNotificationHandler

JTokens are no longer returned, so we don't need to check for it

* Remove newtonsoft references

* Re-add newtonsoft dependency to Umbraco.Tests.Common

* Fix package references

* move dependency

---------

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2024-03-01 12:51:21 +01:00
Kenn Jacobsen
976bca456e Remove macros (#15794)
* Remove macros!

* Clean up snippets

* Add migration that deletes all macros

* Review comments
2024-02-29 15:11:06 +01:00
Kenn Jacobsen
d7b77b0163 Remove Newtonsoft.Json from ImageCropperPropertyEditor (#15784)
* Remove Newtonsoft.Json from ImageCropperPropertyEditor.cs

* Make the media JSON valid
2024-02-29 13:04:41 +01:00
Bjarke Berg
2494d8c5aa Granular permissions in Management API (#15734)
* It builds..

* Added granular permissions

* Added granular permissions

* Rename content to document

* Added migration

* Fixed issues causing the migration from v13 was not possible.

* Merged Permissions and Granular Permissions in viewmodel

* Prepared the viewmodel to a future where permissions can be more types.

* OpenApi

* Allow to translate a single char to many strings

* Use frontend friendly values for known permissions

* Validate the documents exist

* Allow setting non-document settings

* Add "$type" when required

* Rename to presentation model and update OpenApi.json

* OpenApi.json

* Fix tests

* OpenAPI

* Fixed issues with upgrades

* Add the discriminator name

* Fixed issues that only happended on SqlServer

* Fixed queries for SqlServer

* Clean up

* More cleanup

* Fix issue when migrating sqlserver

* Split fallback permissions into own concept in view model

* Also split on current user

* Added a extenable pattern for mappers between DTO => Granular Permission => ViewModel and ViewModel => Granular Permission

* Fixed issue with new exists method, that did not take duplicate keys into account.

* Added sections to current user response model

* Formatting fixes

* Move class to its own file

* xml comment

---------

Co-authored-by: Zeegaan <skrivdetud@gmail.com>
2024-02-27 21:57:02 +01:00
Mole
7735552877 V14: Migrate Newtonsoft to System.Text in value converters (#15728)
* Force system text json for IJSonSerializer

* Migrate ColorPickerValueConverter

* Move ColorPickerValueConverter

* Clean up ColorPickerValueConverter

* Remove obsoleted property editors

* Migrate FlexibleDropdownPropertyValueConverter to System.Text.Json

* Use IJsonSerializer instead and move the value converter to Core

* Migrate ImageCropperValueConverter to System.Text.Json

* Inject jsonserializer in test and obsolete old constructor

* Migrate JsonValueConverter to System.Text.Json

* Remove ContextualConfigurationEditorJsonSerializer

* Remove JsonNetSerializer

* Remove obsolete DeserializeSubset from JsonSerializer interface

* Fix FlexibleDropdownPropertyValueConverter

* Update test JSON to be actual valid json

* Update more test json

* Update time format to be valid

* Add JsonPropertyName to models
2024-02-22 11:22:57 +01:00