* Added request caching to media picker media retrieval, to improve performance in save operations.
* WIP: Update or insert in bulk when updating property data.
* Add tests verifying UpdateBatch.
* Fixed issue with UpdateBatch and SQL Server.
* Removed stopwatch.
* Fix test on SQLite (failing on SQLServer).
* Added temporary test for direct call to NPoco UpdateBatch.
* Fixed test on SQLServer.
* Add integration test verifying the same property data is persisted as before the performance refactor.
* Log expected warning in DocumentUrlService as debug.
Fix failing SQLServer integration tests
Adjusted the tests so that the created content is retrieved again after creation, instead of using the returned IContent.
This is needed because SQLServer, when using datetime, rounds to the closest .000, .003, or .007, which would cause the comparisons to fail.
We should consider moving away from datetime to datetime2, as the former should be avoided according to Microsoft.
https://learn.microsoft.com/en-us/sql/t-sql/data-types/datetime-transact-sql?view=sql-server-ver17
* Ports fix to regression of the caching of null representations for missing dictionary items.
* Fixed error raised in code review.
---------
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
* Retrieve only ISO codes from the database rather than full language objects if that's all we need.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Removed repository updates and migrated the new service method to an extension method.
* Fixed issue after merge.
* Removed left-over using
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
* Add migration to create missing tabs
In v13, if a tab had groups in both a composition and the content type, the tab might not exist on the content type itself.
Newer versions require such tabs to also exist directly on the content type. This migration ensures those tabs are created.
Also fixes an issue in LeftJoin where nested sql arguments were being discarded.
* Small fixes
* WIP: Integration test.
* Added asserts to show the current issue with the integration test.
* Adjusted the integration test
* Added logging of result. Minor re-order and extraction refactoring in integration test.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Improve GetManagementApiUrl to use the globally defined default version if not specified on the controller
* Add a test to check logic introduced in #20083
* Update tests/Umbraco.Tests.Integration/ManagementApi/Trees/DocumentTypeSiblingControllerTests.cs
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Update tests/Umbraco.Tests.Integration/ManagementApi/Trees/DocumentTypeSiblingControllerTests.cs
Co-authored-by: Andy Butland <abutland73@gmail.com>
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Adding member types sibling endpoints
* Introducing sibling endpoint for Partial Views and logic.
* Introducing sibling endpoint for stylesheets
* Introducing sibling endpoint for scripts
* Introducing FileSystemTreeServiceBase.cs
* Introducing interfaces for implementation specific services
* Introducing services for specific trees
* Modifying controller bases to fit new interface and logic.
* Obsoleting old constructors related to PartialView
* Obsoleting ctors related to Stylesheets
* Obsoleting ctors related to scripts
* Adding tests for scriptsTreeService
* Adding tests for siblings
* Removing unused dependencies
* Removing signs and replacing it with flags
* Fixing breaking changes by obsoletion
* Fixing more breaking changes
* Registering missing service
* Fixing breaking changes again
* Changing name of method GetSiblingsViewModels
* Rewritten tests for less bloat and less duplicate code
* Expanding tests to include other methods from service
* Test refactoring: avoided populating file systems that weren't under test, updated encapsulation, renaming, further re-use.
* Management API: Expanding the existing sibling endpoints to support trashed entities (#20154)
* Refactoring existing logic to include trashed items
* Including tests for trashed entities
* Groundwork for trashed siblings
* Documents trashed siblings endpoint
* Controller for Media trashed items
* Expanding tests to include a test for trashed siblings
* Code review corrections
* Resolving code review
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
Reworks update of user groups on a user by updating in place rather than deleting and re-adding.
Ensure user groups affected by the update are invalidated in the repository cache.
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
* Support querystring and anchor for local links in Delivery API output (#20142)
* Support querystring and anchor for local links in Delivery API output
* Add default implementation for backwards compat
* Add default implementation for backwards compat (also on the interface)
* Fix default implementation
* Add extra tests proving that querystring/postfix can be handled for local links in both legacy and current format.
* Adding a check to see if the posted value's source path isn't null or empty.
* Moving validation logic to proper files
* Moved logic to a required validator
* Adding tests to ensure validation works
* Minor tidy up: XML header comments, re-use in tests, clarified test names.
* Adding unit tests for file upload validation
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Initial implementation of non existing property editor
* Adjust `MissingPropertyEditor` to not require registering in PropertyEditorCollection
* Add `MissingPropertyEditor.name` back
* Remove unused dependencies from DataTypeService
* Removed reference to non existing property
* Add parameterless constructor back to MissingPropertyEditor
* Add validation error on document open to property with missing editor
* Update labels
* Removed public editor alias const
* Update src/Umbraco.Web.UI.Client/src/packages/property-editors/missing/manifests.ts
* Add test that checks whether the new MissingPropertyEditor is returned when an editor is not found
* Also check if the editor UI alias is correct in the test
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Share property editor instances between properties
* Only store missing property editors in memory in `ContentMapDefinition.MapValueViewModels()`
* Add value converter for the missing property editor to always return a string (same as the Label did previously)
* Small improvements to code block
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Adding signs to variants and adjusting HasPendingChangesSignProvider.cs
* HasPendingChangesSignProvider.cs now populates variants & refactoring to move logic to DocumentPresentationFactory.cs
* Working HasScheduleSignProvider.cs to provide variant signs
* Refactoring ISignProvider.cs to take an IEnumerable again
* Moving code from controllers to factories
* Refactoring HasPendingChangesSignProvider.cs to use the right Interface method
* Refactoring HasScheduleSignProvider.cs to be less bloated, and more readable (hopefully)
* Refactoring tests to look at variants and include a list
* Changing instantiation to be better
* Fixed minor logic issue in HasScheduleSignProvider.cs
* Refactoring to include just 1 database call.
* Adjusting tests to use the new methods.
* Reverted breaking changes
* Prevents the removal of all user groups from a user.
* Add additional user group when removing
---------
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>
* Create sign provider collection and call registered providers on rendering a page of tree item view models.
Re-work tree controller constructors to provide registered providers as a collection.
* Stub implementation of sign provider for documents with a scheduled publish pending.
* Complete implementation of tree sign for pending scheduled publish.
* Added integration test for new method on IContentService.
* Added unit test for HasScheduleSignProvider.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Tidied usings and clarified method header comments.
* Adding a fixed prefix to all future signs, and removing the provider property
* Adding a sign for protected tree documents.
* Adding IsProtectedSignProviderTest.cs & correcting HasScheduleSignProviderTests.cs to no longer assert the provider
* Fixing minor things in accordance with CR
* Adding collection items compatibility
* Introduced IHasSigns interface to provide more re-use across trees and collections.
Fixed updates to base content controllers (no need to introduce a new type variable).
Removed passing entities for populating tree signs (we aren't using it, so simplifies things).
* Refactoring a bit to make existing code less duplicated and fixing some constructor obsoletion
* Introducing a has pending changes sign.
* Applying changes based on CR
* Introducing tests for HasPendingChangesSignProvider.cs and stopped the use of contentService
* Introducing tests for HasPendingChangesSignProvider.cs and slight logic change
* Introduced HasCollectionSignProvider.cs and tests.
* Introducing collection signs to Media Tree & Media Collection items
* Introducing Plain Items and tests. Refactoring tests as well
* Introduced alternative CanProvideSigns() implementation on IsProtectedSignProvider.cs
* Slight refactoring to reduce bloating.
* Adding [ActivatorUtilitiesConstructor] since it threw an error otherwise
* Minor cleanup.
* Updated OpenApi.json.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: NillasKA <kramernicklas@gmail.com>
* fix some test which are not running
* resolve code review comments
* Moved cleaned up tests to unit tests (as they are unit tests, not integration tests).
Removed tests marked as no longer necessary.
Update tests name to better reflect test case.
* Made explict test faster so it could run on the pipeline.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Optimize document and media seeding by looking up from database in batches.
* Ensure null values aren't stored in the cache when checking existance.
* Fixed failing integration tests.
* Resolved issue with not writing to the L1 cache on an L2 hit.
* Tidied up and populated XML header comments.
* Address issue raised in code review.
* Content picker search with start node configured not taking user start nodes into account (#19800)
* Fix users being able to see nodes they don't have access to when using the picker search
* Readability and naming improvements
* Additional fixes
* Adjust tests
* Additional fixes
* Small improvement
* Replaced the root ids with constants
* Update src/Umbraco.Web.BackOffice/Trees/MemberTreeController.cs
Co-authored-by: Andy Butland <abutland73@gmail.com>
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
# Conflicts:
# src/Umbraco.Examine.Lucene/BackOfficeExamineSearcher.cs
# src/Umbraco.Web.BackOffice/Trees/ContentTreeController.cs
# src/Umbraco.Web.BackOffice/Trees/MediaTreeController.cs
# src/Umbraco.Web.BackOffice/Trees/MemberTreeController.cs
# tests/Umbraco.Tests.Integration/Umbraco.Examine.Lucene/UmbracoExamine/BackOfficeExamineSearcherTests.cs
* Add new constructor without unused and obsolete parameters
* Use non obsolete constructor in tests
* Add `dataTypeId` as parameter in document and media search endpoints to get `ignoreUserStartNodes` value
* Update backend API generated typed client
* Updated picker search to pass in data type unique
* Move data type retrieval to UmbPickerContext
* Adjust the controller constructors to make it non breaking
* Adjust controller methods to make non-breaking.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Fix moving properties between groups sometimes clearing their values
* Small adjustment
* Fix failing integration test
The mapping method was only setting the property group when it was not null, but for orphaned properties we want to specifically set it to null.
* Adjust 'Can_Move_Properties_To_Another_Container' integration test to check more scenarios and that values are kept
* Adjust to add isElement variable in test (as previously)
* Removed two unnecessary delete clauses when removing content types (they are looking for user group Ids, but we are deleting a content type).
* Renamed table name constant with obsoletion to better reflect name and contents of table.
* Added granular permission for property value records to delete clauses when deleting a document type.
* Delete property value permissions for removed property types.
* Added integration tests to verify behaviour.
* Adds support for the "folders only" flag on retrieving siblings of a node.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Updated test code.
* Removed double secondary ordering by node Id and ensured we include this clause for all sort orders.
* Ensure that ordering by node Id is always added only once and last, and only if it's not already been included in the order by clause.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Provides an abstraction for creating the JavaScriptEncoder used in SystemTextConfigurationEditorJsonSerializer.
* Generalised JSON serialization encoder factory to work for all System.Tex.Json serializers.
Added the serializer's name as a parameter to allow for different encodings per serializer if required.
* Fixed tests by removing use of obsolete constructors.
* Removed name parameter and used a generic type instead.
* Added user start node restrictions to sibling endpoints.
* Further integration tests.
* Tidy up.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Revert previous update.
* Retrieves item counts before and after the target for sibling endpoints and returns in API response.
* Applied previous update correctly.
* Removed blank line.
* Fix build and test asserts following merge.
* Update OpenApi.json.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: mole <nikolajlauridsen@protonmail.ch>
* Add integration tests that shows the problem
* Fix the problem and add explenation
* Improved comments slightly to help when we come back here!
Moved tests alongside existing ones related to scopes.
Removed long running attribute from tests (they are quite fast).
* Fixed casing in comment.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Started implementing new LongRunningOperationService and adjusting tasks to use this service
This service will manage operations that require status to be synced between servers (load balanced setup).
* Missing migration to add new lock. Other simplifications.
* Add job to cleanup the LongRunningOperations entries
* Add new DatabaseCacheRebuilder.RebuildAsync method
This is both async and returns an attempt, which will fail if a rebuild operation is already running.
* Missing LongRunningOperation database table creation on clean install
* Store expire date in the long running operation. Better handling of non-background operations.
Storing an expiration date allows setting different expiration times depending on the type of operation, and whether it is running in the background or not.
* Added integration tests for LongRunningOperationRepository
* Added unit tests for LongRunningOperationService
* Add type as a parameter to more repository calls. Distinguish between expiration and deletion in `LongRunningOperationRepository.CleanOperations`.
* Fix failing unit test
* Fixed `PerformPublishBranchAsync` result not being deserialized correctly
* Remove unnecessary DatabaseCacheRebuildResult value
* Add status to `LongRunningOperationService.GetResult` attempt to inform on why a result could not be retrieved
* General improvements
* Missing rename
* Improve the handling of long running operations that are not in background and stale operations
* Fix failing unit tests
* Fixed small mismatch between interface and implementation
* Use a fire and forget task instead of the background queue
* Apply suggestions from code review
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Make sure exceptions are caught when running in the background
* Alignment with other repositories (async + pagination)
* Additional fixes
* Add Async suffix to service methods
* Missing adjustment
* Moved hardcoded settings to IOptions
* Fix issue in SQL Server where 0 is not accepted as requested number of rows
* Fix issue in SQL Server where query provided to count cannot contain orderby
* Additional SQL Server fixes
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Add support for programmatic creation of property types providing the data type key (#19720)
* Add support for programmatic creation of property types providing the data type key.
* Add integration tests
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Don't use Lazy
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* PoC implementation
* Move to controller base
* Implement solution that seems worse, but works better
* Don't require parent key in repository method
* Fix typos
* Add siblings for data type, media type and media
* Add endpoint for template
* Add DocumentType and DocumentBlueprint controllers
* Fix naming
* Fix case if siblings are under root
* Take item ordering into account
not all entities are ordered by sort order
* Add default implementation
* Fix parentkey
* Add tests
* Format optimizations for split view
* Add test covered requirement to description
* Cover positive case and make test case output more readable
* reduce allocations
* Clarify test
---------
Co-authored-by: Migaroez <geusens@gmail.com>
* Fixes issue where content created from blueprint would not persist file upload property values.
* Ensure a copy of a file upload is created when scaffolding content from a blueprint, like we do when copying content.
* Clarified comment.
* Removed unneeded usings.
* Fixed spelling.
* Handle create of blueprint from content to create a new uploaded file.
Handle delete of blueprint to delete uploaded files.
* Added abstraction for aggregation of granular permissions to support custom permissions.
* Refactor to move responsibility for aggregating granular permissions to the respective mappers.
* Added XML header comments for permission mappers.
* Tidied up/removed warnings in UserPresentationFactory interface and implementation.
* Optimized retrieval of documents in DocumentPermissionMapper.
* Fixed method header comment.
* Use entity service rather than content service to retrieve key and path.
* Refactor descendant enumeration in DeliveryApiContentIndexHelper
Improved loop condition to allow for processing of more than 10.000 descendants for indexing.
* Add failing test for original issue.
* Renamed variable for clarity.
---------
Co-authored-by: Brynjar Þorsteinsson <brynjar@vettvangur.is>
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Fix for https://github.com/umbraco/Umbraco-CMS/issues/18872
* Parsing added for current value
* Build fix.
* Cyclomatic complexity fix
* Resolved breaking change.
* Pass content key.
* Simplified collections.
* Added unit tests to verify behaviour.
* Allow file upload on block list.
* Added unit test verifying added property.
* Added unit test verifying removed property.
* Restored null return for null value fixing failing integration tests.
* Logic has been updated according edge cases
* Logic to copy files from block list items has been added.
* Logic to delete files from block list items on content deletion has been added
* Test fix.
* Refactoring.
* WIP: Resolved breaking changes, minor refactoring.
* Consistently return null over empty, resolving failure in integration test.
* Removed unnecessary code nesting.
* Handle distinct paths.
* Handles clean up of files added via file upload in rich text blocks on delete of the content.
* Update src/Umbraco.Infrastructure/PropertyEditors/FileUploadPropertyEditor.cs
Co-authored-by: Sven Geusens <geusens@gmail.com>
* Fixed build of integration tests project.
* Handled delete of file uploads when deleting a block from an RTE using a file upload property.
* Refactored ensure of property type property populated on rich text values to a common helper extension method.
* Fixed integration tests build.
* Handle create of new file from file upload block in an RTE when the document is copied.
* Fixed failing integration tests.
* Refactored notification handlers relating to file uploads into separate classes.
* Handle nested rich text editor block with file upload when copying content.
* Handle nested rich text editor block with file upload when deleting content.
* Minor refactor.
* Integration test compatibility supressions.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: Sven Geusens <geusens@gmail.com>
* Fix check for pending package migration to use the package not plan name.
* Cover all package name/identifier permutations and fix the API output for multiple plans
* Adjusted log message to not refer to unattended migrations as migrations may be being run attended.
---------
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
* Assert dates in content editing integration tests to millisecond only.
* Add date time extension unit tests and refactor to switch statement.
* Removed whitespace.
* Rename `IContentService.CreateContentFromBlueprint` to `CreateBlueprintFromContent`
In reality, this method is used by the core to create a blueprint from content, and not the other way around, which doesn't need new ids. This was causing confusion, so the old name has been marked as deprecated in favor of the new name. If developers want to create content from blueprints they should use `IContentBlueprintEditingService.GetScaffoldedAsync()` instead, which is what is used by the management api.
* Added integration tests to verify that new block ids are generated when creating content from a blueprint
* Return copy of the blueprint in `ContentBlueprintEditingService.GetScaffoldedAsync` instead of the blueprint itself
* Update CreateContentFromBlueprint xml docs to mention both replacement methods
* Fix tests for rich text blocks
* Small re-organization
* Adjusted tests that were still referencing `ContentService.CreateContentFromBlueprint`
* Add default implementation to new CreateBlueprintFromContent method
* Update tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Services/ContentBlueprintEditingServiceTests.GetScaffold.cs
Co-authored-by: Andy Butland <abutland73@gmail.com>
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* v16 cherry pick of member partial cache invalidator see #19314
# Resolved merge conflic in src/Umbraco.Core/Cache/Refreshers/Implement/MemberCacheRefresher.cs
* Take nullmember cacheitems into account
* Ensure tag operations are case insensitve on insert across database types.
* Ensure tags provided in a single property are case insensitively distinct when saving the tags and relationships.
* Update src/Umbraco.Infrastructure/Persistence/Repositories/Implement/TagRepository.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Handle case sensitivity on insert with tag groups too.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
